7cb74b158f
Replace storage.IncompletePieceToWriter with io.Writer
...
It was incorrect to assume piece hashing only operates on incomplete chunk data. This actually uncovered a bug where duplicate hash checks occurred, and the redundant checks would fail due to not reading the completed data.
2020-11-06 16:23:38 +11:00
96b640065a
sqlite storage: Add capacity management
2020-11-03 15:17:12 +11:00
8afb05a8a5
sqlite storage: Batch writes synchronously instead of buffering them first
2020-11-03 15:16:57 +11:00
3c102393d1
Comment on storage.IncompletePieceToWriter
2020-11-03 14:28:16 +11:00
784345e9f7
sqlite storage: Init schema in NewPool instead of NewProvider and add an option to disable
2020-11-03 13:11:44 +11:00
d04622e4b3
sqlite storage: Track total blob data size manually
...
Works around possible full table scan at startup, and possible lack of caching around cast(data as blob).
2020-11-03 13:10:17 +11:00
636b20b860
Add write incomplete/consecutive chunks interfaces
2020-11-02 15:35:07 +11:00
17e22516ff
sqlite storage: Add batched writes
2020-10-30 19:46:51 +11:00
ba70ad9b41
sqlite storage: Set mmap_size to a very large size
...
Seems to fall back to ~2GiB on my system.
2020-10-30 18:40:47 +11:00
b75ebbf9e7
sqlite storage: Provide helpers and reasonable defaults
2020-10-30 12:20:54 +11:00
fc039262d9
sqlite storage: Force data to be used as a blob
...
There's a bug in crawshaw.io/sqlite, and some forks where inserting []byte results in a text type instead of blob. To ensure things work correctly, we coerce data to blob wherever we can. See https://github.com/crawshaw/sqlite/issues/94 and the fork that fixes it.
2020-10-30 10:47:50 +11:00
8e1a8440bf
Performance fiddling on sqlite storage
2020-10-27 17:07:49 +11:00
e30084223d
sqlite storage: Include capacity management
2020-10-27 11:08:37 +11:00
55d4bcaf26
sqlite storage: Do ReadAt without using incremental I/O
2020-10-27 11:08:08 +11:00
7fa8c604ab
sqlite storage: Working recursive CTE for excess data
2020-10-23 11:03:42 +11:00
1fc63bdded
sqlite storage: Add NewProviderPool
2020-10-23 09:03:44 +11:00
bb2ce15114
sqlite storage: Rename table to blob
...
Avoiding plural table names.
2020-10-23 09:01:15 +11:00
8088002ba6
Fix error handling for bad torrent adds with safe file handling
2020-10-15 16:03:43 +11:00
89235e180f
Sanitize metainfo file paths for file-based storage
...
Fixes exploit where specially crafted infos can cause the client to write files to arbitrary locations on local storage when using file-based storages like mmap and file.
2020-10-15 15:45:08 +11:00
3820017a0c
Add last_used to sqlite storage
2020-10-13 09:36:58 +11:00
d820f7861c
Add a sqlite data storage implementation
2020-10-11 12:58:27 +11:00
00ac333ea2
Panic on bad piece per resource chunk names
...
This was masking a bug in a coming sqlite storage implementation. Maybe it could be relaxed to checking for no filepath.Dir in the future.
2020-10-11 12:57:33 +11:00
417d7d1d48
Tweaks to storage error and completion handling
2020-10-11 12:54:03 +11:00
898260c177
Tidy up some storage close handling
2020-10-11 12:40:43 +11:00
7455b98a0f
Switch to crawshaw.io/sqlite
...
I'm using it elsewhere, and it seems to conflict when linking with github.com/mattn/go-sqlite3.
2020-07-15 16:16:14 +10:00
235c67a6ec
Remove debug logging from storage file implementation
2020-06-01 11:12:51 +10:00
604af2be82
storage file implementation: Error on short writes
2020-06-01 11:12:33 +10:00
69a0f2f1e4
Got file storage working with segment index
2020-06-01 00:14:51 +10:00
7fec6785bf
Abstract out segments mapping and use it in mmap storage
2020-05-31 21:00:19 +10:00
c04f09ee42
Fix panic closing torrent that failed to open storage
2020-05-21 12:17:10 +10:00
ae9aaaaa20
Use anacrolix/log in storage completion
2020-05-03 14:10:46 +10:00
2ea04e9083
Use default bbolt package alias
2020-03-24 12:55:03 +11:00
7175dba614
Fix bbolt compatibility with go1.14
2020-03-24 12:55:03 +11:00
19ce53e69f
Make io.EOF an expected error from storage.Piece.ReadAt
...
Fixes #381 .
2020-02-27 16:45:57 +11:00
c855c7a366
Differentiate between storage.ClientImpl{,Closer}
2020-02-21 14:12:44 +11:00
72e54fb353
Resource per piece storage: Store incomplete chunks separately
2020-01-14 10:51:09 +11:00
83867dbbe9
Move entirely to etcd-io/bbolt
2019-11-04 09:19:07 +11:00
e7bd5c2991
Replace github.com/boltdb/bolt with github.com/etcd-io/bbolt
2019-10-12 11:07:49 +11:00
e3420e0b60
Don't close shared client piece completion in mmap storage
...
Fixes #335 .
2019-10-11 17:37:22 +11:00
cecd83f966
Don't ignore error getting mmap piece completion
2019-10-11 17:36:46 +11:00
cb1bf0f413
goimports -local
2019-08-21 20:58:40 +10:00
fc4fab91f5
Switch to goimports import sorting
...
Used to use sortimports, but it's old, and goimports seems to have an opinion now.
2018-11-02 23:12:01 +11:00
bd5ae2cb6d
Fix doc comment
2018-07-15 12:56:28 +10:00
ffd41230a6
Fix file permissions
2018-06-09 15:11:28 +02:00
a101ebb07e
package assert->expect
2018-04-12 11:34:24 +10:00
6441e98f62
Implement go vet recommendations
2018-02-02 19:28:09 +11:00
e686523178
storage: Export NewSqlitePieceCompletion
2018-01-12 11:09:33 +11:00
d5e2d7ce99
storage: Don't check completion state on every write
2018-01-12 10:45:19 +11:00
f3ff3821ec
file storage: Report errors getting completion
2018-01-12 10:41:42 +11:00
78e23734d4
Clarify the PieceCompletion.Set method bool argument
2018-01-11 22:10:52 +11:00
Matt Joiner