Commit Graph

492 Commits

Author SHA1 Message Date
Matt Joiner 0ab6d108be Pool webtorrent tracker websockets at the Client level 2020-04-21 18:08:43 +10:00
Matt Joiner c074b30d94 Log webrtc connection errors and rework webtorrent.TrackerClient 2020-04-20 13:42:25 +10:00
Matt Joiner 4a2802a702 Logging fixes 2020-04-18 17:45:01 +10:00
Matt Joiner d05ce2d274 Update anacrolix/log 2020-04-16 17:20:58 +10:00
Matt Joiner e5f7765dfa Fix small cache transfer tests
Since hashing Peer Addrs in the Torrent pending peers, clients are getting more than one connection to the seeder in the tests and failing.
2020-04-16 14:12:17 +10:00
Matt Joiner c2210c058c Fix pending peer counts in transfer tests
Also expose number of peers successfully added in some method returns.
2020-04-16 14:09:45 +10:00
Matt Joiner 070e11a0dd Adjust recently added logging 2020-04-16 12:17:18 +10:00
Matt Joiner 730cebf5aa Only announce to DHT if we have dialers or listeners 2020-04-16 12:03:27 +10:00
Matt Joiner eec337d4bc Set tracker announce numWant to 0 if we're unable to dial 2020-04-16 12:02:34 +10:00
Matt Joiner 74986db9ed Don't try to do conventional dials if we have no dialers
Stops us from consuming all the peers we store for no reason.
2020-04-16 12:00:49 +10:00
Matt Joiner 6a86e9e999 Include hashed addr when comparing prioritized peers
In particular, fixes the issue where peers where can't calculate a priority and fallback on 0 clobber each other.
2020-04-16 11:59:31 +10:00
Matt Joiner 5f47e808fb Update the peersAddedBySource expvar 2020-04-16 11:56:58 +10:00
Matt Joiner c27e3f8de3 Merge branch 'webtorrent' 2020-04-15 17:26:46 +10:00
Matt Joiner 92162295af Clone the AnnounceList out of Torrent.Metainfo
Should fix a race with Torrent.AddTrackers
2020-04-15 17:26:02 +10:00
Yaroslav Kolomiiets 2b2c480706 PEX: add periodic deltas 2020-04-15 17:24:44 +10:00
Yaroslav Kolomiiets 93430aa01a PEX: add connection tracking 2020-04-15 17:24:44 +10:00
Yaroslav Kolomiiets 634edd2875 Share current connections with peers over PEX (anacrolix#341) 2020-04-15 17:24:44 +10:00
Matt Joiner f90cbcce20 Fix locks around Torrent.announceRequest 2020-04-13 19:13:59 +10:00
Matt Joiner 05e448d132 webtorrent logging improvements 2020-04-13 19:13:23 +10:00
Matt Joiner 93366bb63a Rename webtorrent.NewClient->NewTrackerClient 2020-04-13 19:12:54 +10:00
Matt Joiner e5fd9ede54 Tidy up the webtorrent package, remove buffer 2020-04-13 14:31:39 +10:00
Matt Joiner 1b3cbc1287 Improve webtorrent tracker client logging 2020-04-13 14:17:46 +10:00
Matt Joiner f45dac1e38 Use offer_id for webrtc conn string 2020-04-13 14:04:34 +10:00
Matt Joiner 6f2c65fe33 Big tidy up of webtorrent code 2020-04-07 14:30:27 +10:00
Matt Joiner c5bb913c9e Fix missing return when creating websocket tracker announcer 2020-04-07 12:16:53 +10:00
Matt Joiner 27bfdd0dbf Create peer connections when WebRTC connections obtain a data channel 2020-04-07 10:59:10 +10:00
Matt Joiner 77cbbec926 Start using the webtorrent package 2020-04-06 16:45:47 +10:00
Matt Joiner 4d516ddadd Create stub for websocket trackers 2020-04-06 15:38:01 +10:00
Matt Joiner 378ccd0932 Logging improvements for DHT 2020-03-17 13:48:09 +11:00
Matt Joiner 4a5b891a34 Add doc comments related to #383 2020-03-16 16:30:39 +11:00
Matt Joiner 19ce53e69f Make io.EOF an expected error from storage.Piece.ReadAt
Fixes #381.
2020-02-27 16:45:57 +11:00
Matt Joiner ea71bf770c Expose PieceStateRun formatting 2020-02-27 16:42:33 +11:00
Matt Joiner bae791a5a2 Disable data downloading on storage write errors
Also add a hook and Torrent methods to modify this.
2020-02-22 19:40:50 +11:00
Matt Joiner ddc61845ac Misc cleanup 2020-02-22 19:38:56 +11:00
Matt Joiner dff436f102 Rename connection->PeerConn and fix exports 2020-02-21 11:07:50 +11:00
Matt Joiner afe4d8795d Support custom DHT servers
Addresses #266.
2020-02-20 17:46:29 +11:00
Matt Joiner d24922dc09 Add support for non-IP-based networks
Includes a test with unix sockets. Exposes AddDialer, AddListener, and reworks Peer.
2020-02-20 16:47:37 +11:00
Matt Joiner 376ff763fe Apply staticcheck 2020-02-20 11:09:57 +11:00
Matt Joiner 913f6df80d Use CopyN to ensure early io.EOF isn't ignored when hashing pieces 2020-01-23 13:56:39 +11:00
Matt Joiner cf0d0118ea Finish fixing IP banning on storage errors 2020-01-23 13:54:37 +11:00
Matt Joiner 1d5b8f7451 Reflow some comments 2020-01-22 15:56:16 +11:00
Matt Joiner 2559af0f9c Extract the request timeout stuff into requestStrategyThree 2020-01-14 10:51:09 +11:00
Matt Joiner 4c989da21e Extract the request strategy logic
Bit messy. Now it'll be easier to clean-up what it depends on, and test.
2020-01-14 10:51:09 +11:00
Matt Joiner 381fb293a5 Fix typos and spellos in comments 2020-01-14 10:51:09 +11:00
Matt Joiner 1039e00955 When piece checks fail only ban untrusted peers and only when the entire piece is dirty
This should help with addressing https://github.com/anacrolix/torrent/issues/364.
2020-01-14 10:51:09 +11:00
Matt Joiner bf0498ed5a Improve hash piece error logging 2020-01-14 10:51:09 +11:00
Matt Joiner b9fb0bf2a2 Don't create piece hashers for closed Torrents 2020-01-14 10:51:09 +11:00
Matt Joiner 814a714459 Propagate back piece hashing errors
Attempting to solve https://github.com/anacrolix/torrent/issues/364.
2020-01-14 10:51:09 +11:00
Matt Joiner 4104880a66 Remove unnecessary locking in Torrent.Piece 2019-12-18 16:49:15 +11:00
Matt Joiner 5f1d937b62 Add connection trust flag, and more tests with small caches
Thanks to observations and feedback from @ccampbell.
2019-12-18 13:52:00 +11:00
Matt Joiner f448f55e88 Coalesce piece state change notifications on client unlock
Reported by Craig Campbell <iamcraigcampbell@gmail.com>.
2019-12-13 15:55:56 +11:00
Matt Joiner da2d174fc9 Include piece index in log message 2019-11-20 20:04:47 +11:00
Matt Joiner 289293f3cc Rename peer source constants 2019-10-17 17:46:35 +11:00
Matt Joiner 5af5e10640 In the torrent status, sort trackers by the scheme last 2019-10-15 14:59:23 +11:00
Matt Joiner 2a5488f723 Add missing debug level to log statement 2019-08-22 10:20:13 +10:00
Matt Joiner 4850ce6ab3 Restrict the number of concurrent piece hashes
This fixes bad behaviour where running out of file descriptors, and overloading the system with goroutines and concurrent I/O may cause newly started torrents to fail to hash data.
2019-08-22 10:17:06 +10:00
Matt Joiner ee5e459ea3 Tidy up piece completion change handling 2019-08-21 21:06:39 +10:00
Matt Joiner 14fbacd024 Include the piece number when logging piece hash errors 2019-08-21 21:00:47 +10:00
Matt Joiner cb1bf0f413 goimports -local 2019-08-21 20:58:40 +10:00
Matt Joiner 5a2c1c9b76 Upgrade to simplified logger 2019-08-21 20:44:12 +10:00
Matt Joiner 9224303c1e Update all imports of dht to v2 2019-08-10 18:46:07 +10:00
Matt Joiner e9b10f774d Comments on the VerifyData methods 2019-07-25 14:13:42 +10:00
Matt Joiner bbeb69bb5f Ignore cached piece completion state when verifying data
Forcing data to be checked via VerifyData, and probably other places where piece checks are queued were being short-circuited due to cached piece completion. If a piece check is queued, or verifyData called, the cached completion state should be ignored.
2019-07-25 14:10:21 +10:00
Matt Joiner 27b7fbe3f3 Fix announcing to S3 HTTP trackers 2019-07-17 18:12:11 +10:00
Matt Joiner 5b44954223 Send tracker stopped event from the tracker scraper routine
Fixes potential blocking on the announce while the client lock is held, as well as differing arguments to the other announces introduced by #321.
2019-07-17 11:56:25 +10:00
Matt Joiner d27b304dfc Remove "Sending stopped event to trackers" log 2019-06-18 10:27:17 +10:00
Leandro Martelli 7ce45366f5 Adding Started and Stopped events 2019-06-03 09:45:34 +10:00
Matt Joiner d478888231 When failing to read stored data, try updating only the completion state for the failed piece
On rare occasions, reads are failing in a loop, exhausting all the available file descriptors. It's not clear why, it could be an error in the filecache storage backend I'm using, or some logic error regarding when it's okay to try to read.
2019-04-09 12:57:54 +10:00
Matt Joiner 499b2d8725 Fix race condition in Torrent.SetDisplayName 2019-03-20 11:01:56 +11:00
Matt Joiner 3920c6c325 Fix race in Torrent.String 2019-03-12 11:22:25 +11:00
Matt Joiner b102ce901a Reduce allocations for Piece.hash 2019-01-30 17:54:02 +11:00
Matt Joiner 1749152065 Reorder some code in dht announcer 2019-01-22 01:41:07 +01:00
Matt Joiner a65784374b Fix cancellation of dht announce when peers are wanted 2019-01-22 01:16:41 +01:00
Matt Joiner 69c39f0036 Restart DHT announces at regular intervals
The existing implementation would crawl the DHT until the address bloom filter was full. This could take ages unless enough peers were found to fill the pending nodes for the torrent to the high water mark. Fixes #301.
2019-01-21 22:54:03 +01:00
Matt Joiner 7fa29b9e4d Fix build error 2019-01-16 10:11:47 +00:00
Matt Joiner ee5a941203 Switch entirely to anacrolix/log 2019-01-15 18:18:30 +00:00
Matt Joiner 3ba991e38a Remove unused code 2018-11-27 23:57:17 +11:00
Matt Joiner 9bb8a327c0 Remove log that used missinggo.CryHeard 2018-11-21 17:05:30 +11:00
Matt Joiner 1dc406c81c Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
Matt Joiner 0032b45a02 Store remoteAddr with each connection
It would appear net.Conns returned from proxies don't have a RemoteAddr the client expects.
2018-11-04 21:15:51 +11:00
Matt Joiner fc4fab91f5 Switch to goimports import sorting
Used to use sortimports, but it's old, and goimports seems to have an opinion now.
2018-11-02 23:12:01 +11:00
Matt Joiner 6dd3b9c12c Law of Demeter Client.mu 2018-07-25 13:42:28 +10:00
Matt Joiner 631e18d231 Set Torrent.pieces cap too
We know exactly how many we want, and don't want to use any more memory.
2018-07-23 10:28:11 +10:00
Matt Joiner 6d6197b0a1 Switch pieceIndex back to an int
I suspect that interface conversions using packet iter are causing a lot of allocation. Either way, with the casting this adds, we should be able to change pieceIndex's type alias now with minimal code change.
2018-07-17 21:28:01 +10:00
Matt Joiner f5bd377941 Change pieceIndex to peer_protocol.Integer 2018-07-12 09:15:15 +10:00
Matt Joiner 2685ef6aa9 Use half open limit directly from Config 2018-07-10 12:23:00 +10:00
Matt Joiner 1f6ba94882 Make extended handshake a struct, and move a bunch of extended stuff into peer_protocol 2018-07-10 12:20:36 +10:00
Matt Joiner e3cb94006c Reinstate the reduce dial timeout and update some values
Fixes #263
2018-07-10 11:19:14 +10:00
Matt Joiner 316640f921 Fix panic when failing to parse tracker URL 2018-07-07 11:35:47 +10:00
Matt Joiner b2117bc571 Use timers for duplicate requests
Nothing was triggering request updates when timeouts expired. This is the simplest fix, no performance considered.
2018-06-30 00:10:31 +10:00
Matt Joiner 8255194c64 Fix racy Torrent.wantConns check when adding connections 2018-06-26 20:47:21 +10:00
Matt Joiner c921242f30 Do requests synchronously, and don't request from hashing or queued pieces
Calculating the desired state was a nice idea, but too hard to debug. This way should also be faster.
2018-06-26 14:51:55 +10:00
Matt Joiner 7dd532d46f Check that chunks we request aren't being hashed or queued for hash 2018-06-26 13:04:15 +10:00
Matt Joiner 85dc3c204a Rework stats for receiving chunks
Related to #253.
2018-06-25 14:06:30 +10:00
Matt Joiner 195695042d Add requestStrategy 3, which duplicates requests only after a timeout
Possible solution for #253.
2018-06-24 20:04:31 +10:00
Matt Joiner 159c4a0e28 Ensure 64-bit alignment of ConnStats fields
Fixes #262.
2018-06-23 18:33:56 +10:00
Matt Joiner 96050948c2 Use scope timer in Torrent.writeChunk 2018-06-21 23:22:13 +10:00
Matt Joiner 4fafad46b2 Log and handle error with bad tracker URLs 2018-06-17 16:19:24 +10:00
Matt Joiner 174120e1e8 Rework Torrent pending requests assertions
Should provide more info, and sooner for #249.
2018-06-16 17:00:50 +10:00
Matt Joiner 0caafd8e87 Allow dropping connections to the same peer ID
Necessary for a test that expects one connection to each other Client.
2018-06-16 16:40:37 +10:00