Commit Graph

236 Commits

Author SHA1 Message Date
Matt Joiner 69a0f2f1e4 Got file storage working with segment index 2020-06-01 00:14:51 +10:00
Matt Joiner 7fec6785bf Abstract out segments mapping and use it in mmap storage 2020-05-31 21:00:19 +10:00
Matt Joiner c04f09ee42 Fix panic closing torrent that failed to open storage 2020-05-21 12:17:10 +10:00
Matt Joiner ae9aaaaa20 Use anacrolix/log in storage completion 2020-05-03 14:10:46 +10:00
Matt Joiner 2ea04e9083 Use default bbolt package alias 2020-03-24 12:55:03 +11:00
Matt Joiner 7175dba614 Fix bbolt compatibility with go1.14 2020-03-24 12:55:03 +11:00
Matt Joiner 19ce53e69f Make io.EOF an expected error from storage.Piece.ReadAt
Fixes #381.
2020-02-27 16:45:57 +11:00
Matt Joiner c855c7a366 Differentiate between storage.ClientImpl{,Closer} 2020-02-21 14:12:44 +11:00
Matt Joiner 72e54fb353 Resource per piece storage: Store incomplete chunks separately 2020-01-14 10:51:09 +11:00
Matt Joiner 83867dbbe9 Move entirely to etcd-io/bbolt 2019-11-04 09:19:07 +11:00
Matt Joiner e7bd5c2991 Replace github.com/boltdb/bolt with github.com/etcd-io/bbolt 2019-10-12 11:07:49 +11:00
Matt Joiner e3420e0b60 Don't close shared client piece completion in mmap storage
Fixes #335.
2019-10-11 17:37:22 +11:00
Matt Joiner cecd83f966 Don't ignore error getting mmap piece completion 2019-10-11 17:36:46 +11:00
Matt Joiner cb1bf0f413 goimports -local 2019-08-21 20:58:40 +10:00
Matt Joiner fc4fab91f5 Switch to goimports import sorting
Used to use sortimports, but it's old, and goimports seems to have an opinion now.
2018-11-02 23:12:01 +11:00
Matt Joiner bd5ae2cb6d Fix doc comment 2018-07-15 12:56:28 +10:00
Ian Bishop ffd41230a6 Fix file permissions 2018-06-09 15:11:28 +02:00
Matt Joiner a101ebb07e package assert->expect 2018-04-12 11:34:24 +10:00
Matt Joiner 6441e98f62 Implement go vet recommendations 2018-02-02 19:28:09 +11:00
Matt Joiner e686523178 storage: Export NewSqlitePieceCompletion 2018-01-12 11:09:33 +11:00
Matt Joiner d5e2d7ce99 storage: Don't check completion state on every write 2018-01-12 10:45:19 +11:00
Matt Joiner f3ff3821ec file storage: Report errors getting completion 2018-01-12 10:41:42 +11:00
Matt Joiner 78e23734d4 Clarify the PieceCompletion.Set method bool argument 2018-01-11 22:10:52 +11:00
Matt Joiner a82c76defb Update sqlite3 piece completion 2018-01-11 22:10:00 +11:00
Matt Joiner 09218898e9 Use NoSync with bolt storage and piece completion
Fixes terrible slowness on Linux.
2018-01-09 23:11:34 +11:00
Matt Joiner 57fec52371 mmap_span: Synchronize access to memory maps to avoid race when unmapping 2018-01-06 16:39:52 +11:00
Matt Joiner 6bcf2a58b2 storage: Rename mmapStorage->mmapClientImpl 2018-01-06 16:38:41 +11:00
Matt Joiner b9a1cb5b8b missinggo.LimitLen changed 2017-12-03 13:44:08 +11:00
Matt Joiner 520da249a1 Test that mmap size doesn't exceed system memory address width
Should fix http://paste.ubuntu.com/26074620/
2017-12-02 10:26:09 +11:00
Matt Joiner 4190856b6c sortimports 2017-12-01 18:12:29 +11:00
Matt Joiner cecc70f4cc More megacheck 2017-11-08 00:12:34 +11:00
Matt Joiner dd083a4e11 Track completion known to implementation state
Addresses #193
2017-10-12 16:09:32 +11:00
Matt Joiner 018afed5a7 storage: Remove incorrect comment 2017-09-23 15:27:26 +10:00
Matt Joiner 9e6cdff175 storage: Don't add empty mmaps to the mmap span 2017-09-18 13:41:19 +10:00
Matt Joiner 19adb2cac3 Improve mmap storage error messages
Trying to map unusual Info's gave unhelpful messages.
2017-09-18 12:19:34 +10:00
Matt Joiner cee24fe21b storage: Add TODO about handling Close errors 2017-09-12 18:31:56 +10:00
mlmhl edcb9deb95 bug fix: initial infoHash for mmapStoragePiece 2017-07-10 19:33:39 +08:00
Matt Joiner 49648b9ae1 Close implicit Client default storage on Client.Close
Fixes #158
2017-06-01 22:57:08 +10:00
Matt Joiner c2f77fc84a Make bolt completion DB directory if necessary 2017-06-01 11:20:50 +10:00
mlmhl d8c7f37d2b make pieceCompletion configurable 2017-05-22 10:23:37 +08:00
Matt Joiner 65a7c3daa7 storage: Fix race condition in mapPieceCompletion 2017-03-30 21:38:44 +11:00
Matt Joiner 9bd463ffc0 storage: Add package doc 2017-03-18 13:03:02 +11:00
Chris Walker 9d96cd659f fix `Spec.Storage` and allow per-torrent dir
`TorrentSpec.Storage` was not honored when calling `Client.AddTorrentSpec`
and the configured `cfg.DefaultStorage` was always used. Now if you construct
your `TorrentSpec` you can specify any `StorageImpl`

Also, the most common use case for custom storage being per-torrent paths for
FileStorage, this adds a `pathMaker` function to the File implementation that
allows customization, along with the default (always use base path) and my use
case (which seemed common enough from the Gitter chat) which is infohash based
subdirectories.

All Public methods have not changed signature, but 1 private method did, hence
the test update.
2017-03-16 14:40:21 +00:00
Matt Joiner 1c37903a74 Remove FileStorePieces storage backend
ResourcePIeces is now preferred.
2017-01-05 17:00:59 +11:00
Matt Joiner 5e83287890 storage: ClientImpl interface now includes Close
This is now necessary because bolt doesn't allow multiple DB handles, so the bolt piece completion DB instance must be at the client level. The new method is not currently used from the Client however, this maybe necessary to avoid leaks.
2016-10-25 20:00:09 +11:00
Matt Joiner 3814a50fa0 storage: Fix spelling mistake 2016-10-25 19:57:35 +11:00
Matt Joiner 27893acf34 storage: Only include sqlite3 piece completion implementation if cgo is enabled
Issue #124.
2016-10-25 19:54:09 +11:00
Matt Joiner 18309b866b storage: Remove sqlite3 import from bolt implementation
#124
2016-10-25 19:52:44 +11:00
Matt Joiner 5c5a26afed Add bolt piece completion DB
This means it can be persistent without needing cgo. Fixes issues #115 and #124.
2016-10-25 19:07:26 +11:00
Matt Joiner 69ffbe6a48 storage: Move sqlite3 import to appropriate file 2016-09-16 11:19:26 +10:00
Matt Joiner d4e30f84f2 Rename some of the unexported file storage types 2016-09-12 17:26:39 +10:00
Matt Joiner b5ed171ac5 Create zero-length files in the file storage when the torrent storage is opened
Fixes #111.
2016-09-12 17:01:00 +10:00
Matt Joiner 1e919dd6b1 Rework storage interfaces to make them simpler to implement
This allows lots of behaviour to be baked into the new Client, Torrent and Piece wrappers, rather than duplicating (badly) them in all the backend implementations.
2016-09-02 15:10:57 +10:00
Matt Joiner db3be3441f storage.boltDB: Some comments, and use global completedValue 2016-08-31 21:00:44 +10:00
Matt Joiner c121615fab Extract chunkSize as a const 2016-08-31 20:11:04 +10:00
Matt Joiner e51cb07f6e Uncomment the error handling in storage.boltDBPiece.ReadAt 2016-08-31 18:02:45 +10:00
Matt Joiner 3c2db92a83 Add boltdb storage backend 2016-08-31 17:48:50 +10:00
Matt Joiner 2a1cef7c9e Remove the InfoEx type, and don't generate its infohash on the fly
Fixes #106.
2016-08-26 20:29:05 +10:00
Matt Joiner a1ba463750 Fix issue #97 2016-07-12 16:45:22 +10:00
Matt Joiner f055abe2fc Fix issue #96
In the native file-based storage, mark pieces incomplete if the necessary file data is missing, or there's a read error on a piece.
2016-07-10 23:03:59 +10:00
Matt Joiner 9c837a03d0 Move issue #95 tests into their own file 2016-07-09 00:36:53 +10:00
Matt Joiner 11a53fa732 Fix issue #95: Closing torrent storage also closed client storage for some storage types 2016-07-09 00:36:32 +10:00
Matt Joiner 408686b440 Add a test for issue #95 2016-07-09 00:28:12 +10:00
Matt Joiner cd1c1a9fed mmap storage: Close pieceCompletion on Close 2016-07-06 14:02:15 +10:00
Matt Joiner dbb3c25b1a storage: Move pieceCompletion interface into correct file 2016-07-06 13:38:16 +10:00
Matt Joiner ce0a3d56f1 storage: Remove debug log 2016-06-20 18:21:16 +10:00
Matt Joiner d80d0db9d5 storage: Also use completion DB in mmap implementation 2016-06-20 18:20:59 +10:00
Matt Joiner cee200a5a2 Add piece completion storage
Toward fixing https://github.com/anacrolix/torrent/issues/50.
2016-06-20 17:51:05 +10:00
Matt Joiner b75e85d187 storage.MMapTorrent should not be exported 2016-06-20 16:11:43 +10:00
Matt Joiner 869fb25cbc Changes to missinggo resource package 2016-05-16 22:32:00 +10:00
Matt Joiner c1e0d37266 Rename some storage funcs 2016-05-16 22:02:03 +10:00
Matt Joiner 02ea8b1081 Rename storage.I->Client 2016-05-16 21:50:43 +10:00
Matt Joiner 61e43f4261 Add piece-per-resource storage 2016-05-16 18:44:58 +10:00
Matt Joiner 0aabd37e6e Add some comments to file storage 2016-05-09 15:46:48 +10:00
Seth Hoenig 9f4bf70e17 refactor struct identifiers to follow conventional go names 2016-04-18 23:11:11 -05:00
Matt Joiner 86e34977de piece-file storage: Use the receiver-local FileStore field
Because aliens, mainly.
2016-04-14 17:28:30 +10:00
Matt Joiner b80bb6393a Use metainfo.Hash for piece and info hashes 2016-04-04 13:01:31 +10:00
Matt Joiner e3a7433154 piece file storage: Don't write to completed pieces 2016-04-03 16:37:50 +10:00
Matt Joiner f19e8b32aa storage: Don't read bad completed piece files 2016-04-03 16:33:31 +10:00
Matt Joiner 7638d678a8 storage: Fix io.EOF on short piece files 2016-04-02 16:57:22 +11:00
Matt Joiner 7790e72832 Move storage.FileStore interface to missinggo 2016-03-30 19:12:33 +11:00
Matt Joiner baa8c1aed2 Few fixes for downstream projects 2016-03-29 12:13:03 +11:00
Matt Joiner ee22446440 Reimplement piece storage 2016-03-29 11:14:34 +11:00
Matt Joiner a5b54f21a1 Make opening a torrent in storage an explicit method
This is storage types where opening can fail, like mmap
2016-03-28 22:40:29 +11:00
Matt Joiner 775cf53809 Get mmap storage working 2016-03-28 21:57:04 +11:00
Matt Joiner b97b50aca9 New storage interface 2016-03-28 20:38:30 +11:00