Commit Graph

588 Commits

Author SHA1 Message Date
Matt Joiner 2a5488f723 Add missing debug level to log statement 2019-08-22 10:20:13 +10:00
Matt Joiner 4850ce6ab3 Restrict the number of concurrent piece hashes
This fixes bad behaviour where running out of file descriptors, and overloading the system with goroutines and concurrent I/O may cause newly started torrents to fail to hash data.
2019-08-22 10:17:06 +10:00
Matt Joiner ee5e459ea3 Tidy up piece completion change handling 2019-08-21 21:06:39 +10:00
Matt Joiner 14fbacd024 Include the piece number when logging piece hash errors 2019-08-21 21:00:47 +10:00
Matt Joiner cb1bf0f413 goimports -local 2019-08-21 20:58:40 +10:00
Matt Joiner 5a2c1c9b76 Upgrade to simplified logger 2019-08-21 20:44:12 +10:00
Matt Joiner 9224303c1e Update all imports of dht to v2 2019-08-10 18:46:07 +10:00
Matt Joiner e9b10f774d Comments on the VerifyData methods 2019-07-25 14:13:42 +10:00
Matt Joiner bbeb69bb5f Ignore cached piece completion state when verifying data
Forcing data to be checked via VerifyData, and probably other places where piece checks are queued were being short-circuited due to cached piece completion. If a piece check is queued, or verifyData called, the cached completion state should be ignored.
2019-07-25 14:10:21 +10:00
Matt Joiner 27b7fbe3f3 Fix announcing to S3 HTTP trackers 2019-07-17 18:12:11 +10:00
Matt Joiner 5b44954223 Send tracker stopped event from the tracker scraper routine
Fixes potential blocking on the announce while the client lock is held, as well as differing arguments to the other announces introduced by #321.
2019-07-17 11:56:25 +10:00
Matt Joiner d27b304dfc Remove "Sending stopped event to trackers" log 2019-06-18 10:27:17 +10:00
Leandro Martelli 7ce45366f5 Adding Started and Stopped events 2019-06-03 09:45:34 +10:00
Matt Joiner d478888231 When failing to read stored data, try updating only the completion state for the failed piece
On rare occasions, reads are failing in a loop, exhausting all the available file descriptors. It's not clear why, it could be an error in the filecache storage backend I'm using, or some logic error regarding when it's okay to try to read.
2019-04-09 12:57:54 +10:00
Matt Joiner 499b2d8725 Fix race condition in Torrent.SetDisplayName 2019-03-20 11:01:56 +11:00
Matt Joiner 3920c6c325 Fix race in Torrent.String 2019-03-12 11:22:25 +11:00
Matt Joiner b102ce901a Reduce allocations for Piece.hash 2019-01-30 17:54:02 +11:00
Matt Joiner 1749152065 Reorder some code in dht announcer 2019-01-22 01:41:07 +01:00
Matt Joiner a65784374b Fix cancellation of dht announce when peers are wanted 2019-01-22 01:16:41 +01:00
Matt Joiner 69c39f0036 Restart DHT announces at regular intervals
The existing implementation would crawl the DHT until the address bloom filter was full. This could take ages unless enough peers were found to fill the pending nodes for the torrent to the high water mark. Fixes #301.
2019-01-21 22:54:03 +01:00
Matt Joiner 7fa29b9e4d Fix build error 2019-01-16 10:11:47 +00:00
Matt Joiner ee5a941203 Switch entirely to anacrolix/log 2019-01-15 18:18:30 +00:00
Matt Joiner 3ba991e38a Remove unused code 2018-11-27 23:57:17 +11:00
Matt Joiner 9bb8a327c0 Remove log that used missinggo.CryHeard 2018-11-21 17:05:30 +11:00
Matt Joiner 1dc406c81c Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
Matt Joiner 0032b45a02 Store remoteAddr with each connection
It would appear net.Conns returned from proxies don't have a RemoteAddr the client expects.
2018-11-04 21:15:51 +11:00
Matt Joiner fc4fab91f5 Switch to goimports import sorting
Used to use sortimports, but it's old, and goimports seems to have an opinion now.
2018-11-02 23:12:01 +11:00
Matt Joiner 6dd3b9c12c Law of Demeter Client.mu 2018-07-25 13:42:28 +10:00
Matt Joiner 631e18d231 Set Torrent.pieces cap too
We know exactly how many we want, and don't want to use any more memory.
2018-07-23 10:28:11 +10:00
Matt Joiner 6d6197b0a1 Switch pieceIndex back to an int
I suspect that interface conversions using packet iter are causing a lot of allocation. Either way, with the casting this adds, we should be able to change pieceIndex's type alias now with minimal code change.
2018-07-17 21:28:01 +10:00
Matt Joiner f5bd377941 Change pieceIndex to peer_protocol.Integer 2018-07-12 09:15:15 +10:00
Matt Joiner 2685ef6aa9 Use half open limit directly from Config 2018-07-10 12:23:00 +10:00
Matt Joiner 1f6ba94882 Make extended handshake a struct, and move a bunch of extended stuff into peer_protocol 2018-07-10 12:20:36 +10:00
Matt Joiner e3cb94006c Reinstate the reduce dial timeout and update some values
Fixes #263
2018-07-10 11:19:14 +10:00
Matt Joiner 316640f921 Fix panic when failing to parse tracker URL 2018-07-07 11:35:47 +10:00
Matt Joiner b2117bc571 Use timers for duplicate requests
Nothing was triggering request updates when timeouts expired. This is the simplest fix, no performance considered.
2018-06-30 00:10:31 +10:00
Matt Joiner 8255194c64 Fix racy Torrent.wantConns check when adding connections 2018-06-26 20:47:21 +10:00
Matt Joiner c921242f30 Do requests synchronously, and don't request from hashing or queued pieces
Calculating the desired state was a nice idea, but too hard to debug. This way should also be faster.
2018-06-26 14:51:55 +10:00
Matt Joiner 7dd532d46f Check that chunks we request aren't being hashed or queued for hash 2018-06-26 13:04:15 +10:00
Matt Joiner 85dc3c204a Rework stats for receiving chunks
Related to #253.
2018-06-25 14:06:30 +10:00
Matt Joiner 195695042d Add requestStrategy 3, which duplicates requests only after a timeout
Possible solution for #253.
2018-06-24 20:04:31 +10:00
Matt Joiner 159c4a0e28 Ensure 64-bit alignment of ConnStats fields
Fixes #262.
2018-06-23 18:33:56 +10:00
Matt Joiner 96050948c2 Use scope timer in Torrent.writeChunk 2018-06-21 23:22:13 +10:00
Matt Joiner 4fafad46b2 Log and handle error with bad tracker URLs 2018-06-17 16:19:24 +10:00
Matt Joiner 174120e1e8 Rework Torrent pending requests assertions
Should provide more info, and sooner for #249.
2018-06-16 17:00:50 +10:00
Matt Joiner 0caafd8e87 Allow dropping connections to the same peer ID
Necessary for a test that expects one connection to each other Client.
2018-06-16 16:40:37 +10:00
Matt Joiner 0f90af85f1 Torrent.Stats: RLock instead of Lock 2018-06-16 16:38:14 +10:00
Matt Joiner ad5e44eaf5 Add lots of new expvars and perf timers 2018-06-15 22:42:05 +10:00
Matt Joiner 90f84bda71 Allow adding duplicate connections
I believe this fixes a long-standing race condition when dropping connections to the same peer ID where Clients assume the peer has added connections in the same order. Further commits may address duplicate connections further.
2018-06-13 10:53:11 +10:00
Matt Joiner 224522545b Fix build error with recent go version 2018-06-12 21:51:34 +10:00
Matt Joiner 360e1dbdd1 Track ConnStats with atomics 2018-06-12 20:21:53 +10:00
Matt Joiner a287d069a4 Fix crash adding peers when Torrent is closed 2018-06-12 20:17:15 +10:00
Matt Joiner cea5584d6b Track outgoing through a new field on connection, and rework duplicate connection preferencing 2018-06-12 20:14:00 +10:00
Matt Joiner ac6ba9f021 Add ConnStats at Client level
ConnStats management is refactored to make this less tedious.
2018-06-10 09:20:33 +10:00
Matt Joiner df38775630 A few adjustments to prioritized peers usage 2018-04-14 21:44:41 +10:00
Matt Joiner ed75065f76 Reinstate Uploaded and Downloaded values in tracker announces
Fixes followup comment in #71.
2018-04-12 16:12:14 +10:00
Matt Joiner 319e57d1c6 Rework conns to/and allow multiple DHT servers
This will help with #229, and IPv6 support.
2018-04-12 11:41:07 +10:00
Matt Joiner 92f6209c5f Prioritize pending nodes with BEP 40 2018-04-04 17:59:28 +10:00
Matt Joiner 8f7408b1f9 IPv6 tracker support 2018-02-19 16:19:18 +11:00
Matt Joiner 9513270c06 Trackers shown are just the enabled ones 2018-02-17 11:13:48 +11:00
Matt Joiner 377cb62da9 Make half-open connection limit more dynamic 2018-02-16 12:15:07 +11:00
Matt Joiner dbb2936cdf TODO 2018-02-16 11:03:21 +11:00
Matt Joiner 70010ce691 Support IPv6 peers over PEX 2018-02-13 00:50:32 +11:00
Matt Joiner fadbd68d99 Count peers not added because of bad addresses and rearrange method 2018-02-08 23:55:28 +11:00
Matt Joiner 898e878c69 Aggressively wake Readers 2018-02-04 22:47:01 +11:00
Matt Joiner 44e44637bf TODOs and comments 2018-02-04 19:14:07 +11:00
Matt Joiner e0c2265cb0 Support sending HaveAll 2018-02-04 19:10:25 +11:00
Matt Joiner ec7b90db45 connections should not be added if the Torrent is closed 2018-02-04 13:00:08 +11:00
Matt Joiner ea8659c951 Move initiateConn and openNewConns onto Torrent 2018-02-04 12:59:23 +11:00
Matt Joiner e67b85215c Piece.everHashed is replaced by storageCompletionOk 2018-02-03 12:14:39 +11:00
Matt Joiner f23d097502 Include closed connections in dirtiers 2018-02-03 12:08:16 +11:00
Matt Joiner 9567aa901c Add missing increments to some stats 2018-02-03 11:53:11 +11:00
Matt Joiner 5a4e8cd4c5 Rename and rearrange some conn stats 2018-02-03 00:41:13 +11:00
Matt Joiner fe991128ed Rework connection stat reconciliation with Torrent and refactor doppleganger handling 2018-02-02 19:07:20 +11:00
Matt Joiner b0c1f99509 Merge a bunch of stuff into ConnStats and refactor connection.upload 2018-02-02 19:04:56 +11:00
Matt Joiner 8a43910e2d connection.writer wasn't checking closed state 2018-02-02 13:36:18 +11:00
Matt Joiner 2166bf1ec0 Some changes to status output 2018-02-01 18:45:58 +11:00
Matt Joiner 9387e60999 Track concurrent requests across connections
In preparation for some more request strategy options
2018-02-01 14:46:03 +11:00
Matt Joiner 1a7708b484 sortimports 2018-01-31 16:42:40 +11:00
Matt Joiner ceaf5a2360 Add some sanity checks to priorities extracted from Torrent.pendingPieces 2018-01-29 18:22:21 +11:00
Matt Joiner 77806c1f49 More development of the new logging interface 2018-01-29 18:22:21 +11:00
Matt Joiner ef4c0f1413 Add Torrent.Stats.ConnectedSeeders
Fixes #231
2018-01-29 18:22:21 +11:00
Matt Joiner 53e32ca9dd Start using new log package 2018-01-28 16:07:11 +11:00
Matt Joiner 5ef22a978b Synchronize piece completion on piece hash failure
Pieces that were unknown, remained unknown despite hashing for that very reason.
2018-01-28 16:02:14 +11:00
Matt Joiner 906d3bc5bb Include completion known in PieceState 2018-01-28 15:58:55 +11:00
Matt Joiner de928be582 Remove premature update to piece priority after piece is completed
This should have prevented Torrent.piecePriorityChanged from being called, meaning requests for the completed piece were not canceled, and the piece remained in connection's piece request queue, which meant wasted effort downloading chunks for an already acquired piece.

Thanks to @elgatito for discovering the issue.
2018-01-28 15:52:14 +11:00
Matt Joiner 1725fc9a36 Include torrent stats in status 2018-01-27 14:31:46 +11:00
Matt Joiner 52524925d2 Add separate piece priorities, and cache pending pieces
Should fix a bug where prioritizing files is not alone sufficient to trigger downloading.
2018-01-25 17:18:36 +11:00
Matt Joiner e3c098441a Add Torrent.setInfo 2018-01-25 17:10:37 +11:00
Matt Joiner 4807d05018 Improve some comments 2018-01-25 17:02:52 +11:00
Matt Joiner 21108bf6ec Fix file piece index offset calculations 2018-01-25 17:01:29 +11:00
Matt Joiner 0b553b296f Add File priorities
Fixes #220.
2018-01-21 22:49:12 +11:00
Matt Joiner 696595a76e Add comment explaining Torrent.requestStrategy 2018-01-12 12:24:51 +11:00
Matt Joiner ed7e3f6ec0 Make Torrent.pendingPieces a priority bitmap in preparation for #220 2018-01-12 12:24:37 +11:00
Matt Joiner 87a033e074 Readers obtained from File.NewReader should not readahead into other Files
Fixes #221. Additionally Torrent.length is a pointer and isn't set until the info is available to avoid it defaulting to zero.
2018-01-06 23:15:41 +11:00
Matt Joiner 9b718566ba Add File.NewReader
Fixes #222
2018-01-06 16:37:13 +11:00
Matt Joiner 25cf92502d Improve a log message 2017-12-28 23:58:07 +11:00
Matt Joiner 54a7d83c45 Log dropping of bad peers in debug mode only 2017-12-03 13:44:37 +11:00
Matt Joiner 27253eb0d0 Fix panic in Torrent.addrActive if there are closed connections
Fixes #214
2017-12-02 10:01:27 +11:00
Matt Joiner 47622fad36 Fix the Stats panic in #214 2017-12-02 09:58:08 +11:00
Matt Joiner d77e939944 Synchronize access to storage 2017-12-01 23:09:07 +11:00
Matt Joiner afa9d2ba28 Torrent.bytesLeft was taking a copy instead of pointer to Piece 2017-12-01 17:58:42 +11:00
Matt Joiner 2daa604eb2 Publish piece state changes after verifying piece 2017-12-01 14:03:05 +11:00
Matt Joiner 8f7720edb2 Merge commit '6ab65a49a8a72dea1a28968b2ab42a85fd4566ec'
# Conflicts:
#	config.go
2017-11-08 20:23:15 +11:00
Matt Joiner 163a63f9a7 Torrent.Metainfo: Don't return incomplete metadata bytes
Fixes #208
2017-11-08 19:56:20 +11:00
Matt Joiner 3aa1e8f3ef Add a test for behaviour after getting metadata_size and before GotInfo
#208
2017-11-08 19:31:10 +11:00
themihai 6ab65a49a8 feat(config): Allows the torrent client to customise the client identity and connection config 2017-11-07 20:14:13 +02:00
Matt Joiner ff1bd84e7e Fix #205 2017-11-08 00:34:59 +11:00
Matt Joiner 9757f04cf8 Apply megacheck to torrent package 2017-11-07 16:11:59 +11:00
Matt Joiner 2c6b842bbf Remove noisy log message 2017-11-05 15:39:36 +11:00
Matt Joiner 361c8654c7 Add some TODOs 2017-11-05 15:39:35 +11:00
Samuel 3e74396c66 Aggressive Upload (#203)
* Aggressive Upload Config

* Aggressive Upload

* Flip config option.

* Flip Aggressive Upload Config Option
2017-11-05 14:04:33 +11:00
Matt Joiner 5d133ea387 Remove debug log message 2017-10-12 17:52:26 +11:00
Matt Joiner dd083a4e11 Track completion known to implementation state
Addresses #193
2017-10-12 16:09:32 +11:00
Matt Joiner 17ea4f7fdd Performance optimizations to calculating bytes left in Torrent
Was generating significant overhead in cmd/torrent
2017-09-23 15:28:13 +10:00
Matt Joiner 2c91d1e3fb Changes to Bitmap referencing 2017-09-23 15:27:10 +10:00
Matt Joiner 6aad8041ab Implement new request strategy
The new strategy, 2, has the fastest connection download by priority in order, and all other pieces stick to a randomized ordering that's stable per connection.
2017-09-23 15:25:47 +10:00
Matt Joiner 1f6a7eb810 It looks like Torrent.worstBadConn was returning an arbitrary bad connection, it wasn't the worst
heap.Init got lost somewhere along the way? Need a unit test for this.
2017-09-21 19:32:03 +10:00
Matt Joiner bb53c97d38 Move uploading to the connection writer 2017-09-18 12:09:08 +10:00
Matt Joiner ed0fa62340 Comment and formatting 2017-09-17 00:45:12 +10:00
Matt Joiner 69ffb9bc8d Fix build error 2017-09-17 00:44:09 +10:00
Matt Joiner 846da66103 Merge pull request #188 from boramalper/KnownSwarm
added the initial version of the (t *Torrent) KnownSwarm() function
2017-09-17 00:20:00 +10:00
Bora M. Alper a66b0e83f3 added support for half-open peers in KnownSwarm() function 2017-09-16 11:48:16 +01:00
Matt Joiner b3137b1ede Make type piece public 2017-09-15 19:35:16 +10:00
Matt Joiner 2aa20b3e22 Don't verify data at startup, add Torrent.Piece.VerifyData and Torrent.VerifyData for this purpose
This has the side effect of deflaking a lot of tests that race to verify data when a torrent is added.
2017-09-15 19:22:32 +10:00
Matt Joiner b39df82fbc Include peer data with half open connections 2017-09-15 19:10:09 +10:00
Bora M. Alper f87b732959 added the initial version of the (t *Torrent) KnownSwarm() function 2017-09-12 15:22:53 +01:00
Matt Joiner bad6f07f5e Wrap writerCond in tickleWriter and be more selective about using it 2017-09-01 15:26:50 +10:00
Matt Joiner 13e79039f2 Generate requests, cancels and interest state in the connection writer 2017-08-31 23:48:52 +10:00
Matt Joiner 0388ec1eee Synchronize Torrent.Info 2017-08-29 15:16:53 +10:00
Matt Joiner a8a1ea755d Remove unused functions 2017-08-26 13:23:04 +10:00
Matt Joiner 76c60ffa77 Try a state-delta function for updating request state
Also adds Torrent.networkingEnabled, though it isn't yet useful.
2017-08-18 01:51:02 +10:00
Matt Joiner 493916c279 Rename Torrent.connHasWantedPieces->connection.peerHasWantedPieces 2017-08-18 01:48:19 +10:00
Matt Joiner 824441efa0 Move logging inside client lock, for Torrent.String
Fixes #163
2017-07-01 16:01:38 +10:00
mlmhl a3d231cb92 signal an event when torrent closed 2017-06-05 08:46:50 +08:00
Matt Joiner 296ef76c36 Tidy up the duplicate conn selection code 2017-05-26 13:00:28 +10:00
Matt Joiner 5bde8de95d perf.Timer.Stop renamed 2017-05-26 13:00:28 +10:00
luffyma(马林) d8023a5e39 bug fix: race condition between outgoing connection and incoming connection 2017-05-10 18:31:19 +08:00
Joe Lanford 0d19c72ea5 Added peer count stats to TorrentStats struct 2017-04-22 23:29:10 -04:00
Matt Joiner ba3e798b5f Fix a logging of metainfo.Hash 2017-02-24 21:56:23 +11:00
Matt Joiner aa74d992d3 Don’t track the empty string, and ditch Torrent.announceList 2017-02-19 15:57:30 +11:00
Matt Joiner 350dd01ab9 Fix torrent status info hash hex 2017-02-16 20:10:32 +11:00
Matt Joiner a959bdd0f8 Revert "Quick fix for missing MetaInfo.Announce everywhere with trackers"
This reverts commit b7a8d08bbf.
2017-02-15 18:40:30 +11:00
Matt Joiner b7a8d08bbf Quick fix for missing MetaInfo.Announce everywhere with trackers 2017-02-13 22:28:42 +11:00
Matt Joiner b868a6b9e2 dht.Server.Announce now takes [20]byte 2017-02-07 15:01:02 +11:00
Matt Joiner 79d800fe92 Trivial logging, comment, Stringer changes 2017-02-02 16:53:19 +11:00
Matt Joiner 55b6025f80 Log the storage TorrentImpl type 2017-01-04 18:15:49 +11:00
Matt Joiner 9cf09dba8a Remove unused funcs and idents 2017-01-01 11:03:02 +11:00
Matt Joiner 186b5073d2 Move method queuePieceCheck 2017-01-01 11:02:37 +11:00
Matt Joiner d48c7c8e36 sort imports 2017-01-01 11:01:41 +11:00
Matt Joiner b68d7cd08e dht/… moved to github.com/anacrolix/dht 2016-12-14 11:43:37 +11:00
Matt Joiner f510c5cf22 Fix piece failures not clearing dirty chunks
I don’t like how complicated piece completion/hashing management is, but it works.
2016-12-06 16:10:10 +11:00
Matt Joiner af7a0d3464 Add Torrent.Closed 2016-11-30 18:02:39 +11:00
Matt Joiner 1f2f037209 Fix logging of trust slice 2016-11-30 18:01:39 +11:00
Matt Joiner f451a39b69 Fix harmless race conditions in Client.WriteStatus introduced way back 2016-11-27 14:26:45 +11:00
Matt Joiner 4a7fbf6170 Add peers received from received announce_peer DHT messages to the Client
Addresses #133
2016-11-27 00:05:19 +11:00
Matt Joiner fdf75d59c8 Kick only the worst connection when a piece fails a check 2016-11-23 12:59:23 +11:00
Matt Joiner 57b679ffb0 Add a note about dropping connections that we sent a HAVE for incomplete pieces 2016-11-23 11:52:41 +11:00
Matt Joiner 4807c9e29a Make Torrent.conns a map
Can't remember if I've tried this before. But dropping arbitrary connections while iterating established conns will become much simpler.
2016-11-23 11:48:44 +11:00
Matt Joiner ebbd555e7b Move a bunch of Client methods onto more appropriate types 2016-11-22 21:12:53 +11:00
Matt Joiner 90ca45dd2d Don't accept pieces with bad indexes
Fixes #132
2016-11-22 14:20:48 +11:00
Matt Joiner df60a765a7 Torrent.unpendPieces: Only update unpended pieces
Updating all pieces at once is no longer faster. (There was an optimization for this a while ago that no longer exists.)
2016-10-31 19:05:08 +11:00
Matt Joiner a8e96ce996 When Reader position changes only update piece priorities that may be affected
Seems to significantly reduce CPU load reading from torrents with huge piece counts.
2016-10-31 19:00:08 +11:00
Matt Joiner 95f73db7e4 Ditch the use of generic slices.HeapInterface for finding the worst "bad" connection
It's become a bottleneck. Implement it directly using heap.Interface.
2016-10-31 16:24:48 +11:00
Matt Joiner 1725133111 Move the implementation of connection.requestPiecePendingChunks out of Torrent 2016-10-25 14:58:17 +11:00
Matt Joiner 0e221dbdcd Optimize piece priorities when reader position changes
Gives a decent boost to throughput and reduces a lot of CPU when reading very quickly from Reader.
2016-10-23 16:33:26 +11:00
Matt Joiner de761fb506 Do chunk pooling at Torrent instead of connection level 2016-10-05 15:57:00 +11:00
Matt Joiner 1e919dd6b1 Rework storage interfaces to make them simpler to implement
This allows lots of behaviour to be baked into the new Client, Torrent and Piece wrappers, rather than duplicating (badly) them in all the backend implementations.
2016-09-02 15:10:57 +10:00
Matt Joiner 8b17d2a63a Rework Reader position changes affecting piece priorities
Torrent.updatePiecePriorities is always followed by Torrent.piecePriorityChanged, so move it into the former function. Also drop the separate mutex for Reader, and add a missing Reader.posChanged call.
2016-08-30 15:41:26 +10:00
Matt Joiner 6c15a030d6 Improve comments 2016-08-30 15:07:59 +10:00
Matt Joiner 2a1cef7c9e Remove the InfoEx type, and don't generate its infohash on the fly
Fixes #106.
2016-08-26 20:29:05 +10:00
Matt Joiner 6065a55818 Fix some more races in the status writing 2016-08-02 11:18:33 +10:00
Matt Joiner 1ca2e7e95a Add some extra conditions to prevent network activity when a torrent is closed
From suggestions by @axet, this might be a thing sometimes.
2016-07-30 00:41:45 +10:00
Matt Joiner 4143b7470d Begin exposing all the magic that Client.WriteStatus has so people can parse Client state their own way
It wasn't possible to move Client.WriteStatus to an external package to verify it doesn't depend on same-package access to data because then it can't be used in the tests, and it's extremely useful there. So I've settled for not locking the Client, and trying to use all the public methods. It's a work in progress.
2016-07-30 00:37:52 +10:00
Alexey Kuznetsov 49315045a0 it is actually sha1 hash size 2016-07-29 22:50:58 +10:00
Matt Joiner 3c75f68413 Kick connections that contribute the final piece of missing metadata that fails to validate
Failing to get metainfo was resetting Torrent.metadataBytes, which caused arriving metadata data to appear to belong to nonexistent pieces, despite passing the check that we'd in fact requested them from the sending connection. This was unnecessarily noisy. Instead return an error if we got all the data, but couldn't set the Torrent's info bytes, that should propagate out and cause the connection that contributed the final missing piece to be dropped. This will at least provide some improved resistance to unfriendly behaviour on the network.
2016-07-24 00:34:40 +10:00
Matt Joiner 980cd69ab2 Break up the DHT announcer code into smaller functions, and add a 5 minute delay between announces 2016-07-23 22:38:31 +10:00
Matt Joiner f5d58b3dc6 Unused *Client argument to Torrent.writeStatus 2016-07-12 21:23:20 +10:00
Matt Joiner 29844b8b6c Few slices name changes 2016-07-12 17:49:00 +10:00
Matt Joiner 507daba528 Rework tracker scraper code to allow tracker stats
Inspired by @axet's work.
2016-07-12 16:44:06 +10:00
Matt Joiner 6e399e8f50 Add test for issue #97 2016-07-12 16:42:54 +10:00
Matt Joiner fbe0ded844 Add connection read stats 2016-07-12 16:42:04 +10:00
Matt Joiner a4e140b939 New slices package 2016-07-12 16:40:14 +10:00
Matt Joiner a6cecc6b5f missinggo.Sort->SortSlice 2016-07-07 14:14:59 +10:00
Matt Joiner 240abaf512 Add Torrent.SetMaxEstablishedConns 2016-07-06 00:42:16 +10:00
Matt Joiner 326b36545b Prepare to allow max conns per torrent to be configured 2016-07-05 16:23:17 +10:00
Matt Joiner 12191dbfa3 Add counters for uploads
Fixes #71.
2016-07-05 15:52:33 +10:00
Matt Joiner 12486412b3 Remove redundant check in Torrent.SetInfoBytes 2016-06-27 15:46:43 +10:00
Matt Joiner afc76861e0 Throw away Torrent display name when info becomes available 2016-06-27 15:46:27 +10:00
Matt Joiner b84298351a Move a few methods to Torrent, and track how many DHT announces occur
I suspect that DHT announces may occur too regularly, or without a break for torrents that aren't able to get over the lower water pending peer mark.
2016-06-15 15:29:47 +10:00
Matt Joiner c2bdb31bb3 Client.announceTorrentDHT -> Torrent.announceDHT 2016-06-15 15:13:53 +10:00
Matt Joiner d8dc3ad239 Torrent.MetaInfo: don't crash if info isn't available yet 2016-05-23 20:08:18 +10:00
Matt Joiner 56a89905b6 Update Torrent.wantPeersEvent when info is obtained 2016-05-23 11:11:56 +10:00
Matt Joiner c6ec935e53 Update Torrent.wantPeersEvent in several places 2016-05-23 10:19:14 +10:00
Matt Joiner 512aa7ede4 Don't start tracker scrapers if trackers are disabled 2016-05-22 23:44:08 +10:00
Matt Joiner 20870ec4ff Rework torrent tracker scraping
New scrapers are added anytime trackers are added to a torrent. In the future they will also be stopped as soon as they're removed. All trackers are concurrently scraped, the old-style of sticking to one tracker that works is abandoned for now.
2016-05-22 22:45:08 +10:00
Matt Joiner 8cdcbe9d40 Fix the tracker method receivers and names 2016-05-19 23:58:00 +10:00
Matt Joiner 7600ac70e3 Move torrent tracker methods into torrent.go 2016-05-19 22:49:37 +10:00
Matt Joiner 02ea8b1081 Rename storage.I->Client 2016-05-16 21:50:43 +10:00
Matt Joiner 6f470a2d02 Don't log missing files during hashing 2016-05-16 19:50:46 +10:00
Matt Joiner c0283d3314 Return an error from Torrent.setMetadataSize
This should mean connections sending ridiculous metadata sizes get dropped.
2016-05-16 18:48:56 +10:00
Matt Joiner d2951c6c4b Move requestPendingMetadata onto connection from Client 2016-05-16 18:46:38 +10:00
Matt Joiner 2b46a17b39 Bit of a tidy in the tracker code 2016-05-12 12:43:37 +10:00
Matt Joiner 528836ab4c Get rid of obsoleted Torrent.ceaseNetworking, and clean up Torrent.closing 2016-05-11 21:44:55 +10:00
Matt Joiner a45a09b877 Rework setting of info bytes 2016-05-09 15:47:39 +10:00
Matt Joiner 2d160b0419 metainfo.InfoEx.Hash becomes a function, UpdateBytes is added, and Bytes used in marshalling
Fixes #80.
2016-05-05 22:40:38 +10:00
Matt Joiner f6b83130c9 Move a bunch of Torrent methods from client.go 2016-05-03 16:47:11 +10:00
Matt Joiner 28c9ec2bd1 Some changes to metainfo.InfoEx and testutil 2016-04-30 11:07:29 +10:00
Seth Hoenig 9f4bf70e17 refactor struct identifiers to follow conventional go names 2016-04-18 23:11:11 -05:00
Matt Joiner da4d40ed7e Merge all our global consts and vars into global.go
For easier referral to other parties.
2016-04-14 17:19:07 +10:00
Matt Joiner 48871324f4 Comments and logs around bad metadata size 2016-04-07 19:13:51 +10:00
Matt Joiner 49a3c87a53 Fix possible deadlock 2016-04-04 20:39:57 +10:00
Matt Joiner 82ee860e8d Apply unconvert linter 2016-04-04 15:28:25 +10:00
Matt Joiner 1566e2afdc Remove deadcode 2016-04-04 15:18:51 +10:00
Matt Joiner b80bb6393a Use metainfo.Hash for piece and info hashes 2016-04-04 13:01:31 +10:00
Matt Joiner d3a1c79c79 Merge Torrent and torrent types 2016-04-03 18:40:43 +10:00
Matt Joiner 501ae0c98c Comments 2016-04-03 16:52:52 +10:00
Matt Joiner 08830b97a1 Make everything on type torrent private 2016-04-03 16:50:53 +10:00
Matt Joiner bc8fb2a165 If we fail to write to a piece, check its completion state 2016-04-03 16:36:57 +10:00
Matt Joiner a5b54f21a1 Make opening a torrent in storage an explicit method
This is storage types where opening can fail, like mmap
2016-03-28 22:40:29 +11:00
Matt Joiner 775cf53809 Get mmap storage working 2016-03-28 21:57:04 +11:00
Matt Joiner b97b50aca9 New storage interface 2016-03-28 20:38:30 +11:00
Matt Joiner bb04cb8843 Tidy up bytesLeft calculations
This should fix a bug where completion is shown as negative in WriteStatus
2016-03-22 13:11:36 +11:00
Matt Joiner e2ffcd008b Add t.bytesLeftAnnounce
Now it's safe to remove tracker specific bytesLeft contract
2016-03-22 09:06:48 +11:00
Matt Joiner 61af11fddc Off by one error in torrent.pieceLength? 2016-02-21 17:25:47 +11:00
Matt Joiner 5b443fb63d Refresh all piece completion states when data is missing 2016-02-21 17:24:59 +11:00
Matt Joiner e20858a1e8 Move dataReadAt to torrent.readAt, and do the waitNoPendingWrites check there 2016-02-21 03:32:59 +11:00
Matt Joiner dcdf85a474 Ditch the torrent stateMu for the client mutex 2016-02-21 03:31:50 +11:00
Matt Joiner f6472fc1fd Code comment 2016-02-20 14:41:04 +11:00
Matt Joiner d5f888069f Ditch Data.WriteSectionTo, and fix cmd/torrent-verify 2016-02-18 11:45:31 +11:00
Matt Joiner 589ab02dc4 Ensure piece completion state is correct at startup 2016-02-17 17:20:21 +11:00
Matt Joiner 73208b97ca Code comments 2016-02-17 17:09:43 +11:00
Matt Joiner 0f9f7ba01b Add torrent.completedPieces bitmap
Reduce load on data.PieceComplete for torrents with lots of pieces, when reader position changes. Not sure of the improvement yet.
2016-02-17 00:00:55 +11:00
Matt Joiner b514434581 Trigger piece request order changes on connections in a few spots
The client was requesting already obtained data in certain circumstances. This fixes it.
2016-02-11 17:57:57 +11:00
Matt Joiner 034c816934 torrent.piecePartiallyDownloaded incorrectly returned true when all the chunks were dirty 2016-02-10 00:46:54 +11:00
Matt Joiner 24b8b13d2d Fix pending of already completed pieces when pending ranges of pieces 2016-02-08 21:36:50 +11:00
Matt Joiner dab95b5431 Remove unreachable code 2016-02-08 03:01:54 +11:00
Matt Joiner 623d9e0e06 Rework requesting of shuffled pending pieces 2016-02-07 21:58:48 +11:00
Matt Joiner dc215d083e Export PieceStateChange through piece state change pubsub 2016-02-07 21:55:47 +11:00
Matt Joiner 0c28a0f1b1 Use new tracker package interface 2016-02-07 18:49:35 +11:00
Matt Joiner 77d6e9e5cb Use new missinggo iterator style, and speed up torrent.connHasWantedPieces() 2016-02-07 01:22:31 +11:00
Matt Joiner 1b8bc1498c Use missinggo.bitmap for tracking dirty chunks 2016-02-05 15:45:32 +11:00
Matt Joiner 6942f3e6d5 Changes to piece priority need to be published as a piece state change
This occurs because we're now able to deprioritize pieces after readers seek or close
2016-02-05 01:19:42 +11:00
Matt Joiner 63c73e18b3 Add File.Cancel 2016-02-05 01:18:54 +11:00
Matt Joiner 89f23c226c Fix bug in readahead 2016-02-02 04:45:57 +11:00
Matt Joiner 430f26f726 Recalculate all piece priorities more efficiently 2016-02-01 22:06:13 +11:00
Matt Joiner afa264e6c5 Reintroduce connection piece inclinations, and begin caching piece priorities 2016-02-01 21:11:41 +11:00
Matt Joiner 3ed628356b Use missinggo.Event for connection closing event 2016-02-01 21:08:52 +11:00
Matt Joiner b90dacd324 Abstract the pendingPieces as a bitmap 2016-02-01 01:46:28 +11:00
Matt Joiner ae9bba2611 Fix downloading of unwanted chunks, and write out downloaded chunks using the connection loop goroutine
Doing the writes with the connection loop goroutine prevents memory use blowing out when chunks aren't written out quickly enough.
2016-01-28 05:54:48 +11:00