Commit Graph

2727 Commits

Author SHA1 Message Date
Matt Joiner 927866e881 Limit conns per host across webseed clients 2021-02-09 19:22:38 +11:00
Matt Joiner 0cc655deed Fix closing of webseed peers 2021-02-09 19:21:54 +11:00
Matt Joiner 549ab3c160 Buff issue 465 test command 2021-02-09 19:17:22 +11:00
Matt Joiner ce44a4c3f2 Add minimal reproducing code for issue 465
From https://github.com/anacrolix/torrent/issues/465#issuecomment-774869977.
2021-02-09 17:05:02 +11:00
Matt Joiner 579925fa94 Fix panic in benchmark 2021-02-05 10:01:33 +11:00
Matt Joiner bc0936c44a Use locks on piece per resource pieces to prevent races in reading incomplete data 2021-02-04 19:18:59 +11:00
Matt Joiner 78c77c0b45 Expose mmap_size in sqlite storage, and change default to 8 MiB 2021-02-04 18:29:40 +11:00
Matt Joiner c714cb6992 Fix some go vet warnings 2021-02-03 19:42:46 +11:00
Matt Joiner 371d54ab76 Fix Close race in sqlite storage when batch writes disabled 2021-02-03 19:27:26 +11:00
Matt Joiner 6bc2a2a6ac Revert "Fix stalls for responsive transfer tests"
This reverts commit d48c6ae6dc.
2021-02-03 18:55:54 +11:00
Matt Joiner 6f9c2fed61 Add batch writes cases to sqlite storage benchmarks 2021-02-03 18:55:14 +11:00
Matt Joiner f3cd7007eb Update deps 2021-02-02 10:47:38 +11:00
Matt Joiner 48843c0b87 Fix for go<1.16 2021-02-02 10:41:39 +11:00
Matt Joiner 15ccbc95c6 Fix sqlite storage for numconns 1 2021-02-02 10:41:39 +11:00
Matt Joiner d48c6ae6dc Fix stalls for responsive transfer tests
Looks like there's a timing issue between marking things complete, and already having decided to read the data from incomplete chunks.
2021-02-02 10:41:39 +11:00
Matt Joiner 270a2ba1ae Switch to reading consecutive incomplete chunks
This fixes a race where a sqlite conn isn't reserved for the read part of a MarkComplete operation after the write has already begun.
2021-02-02 10:41:38 +11:00
Matt Joiner 4ae8a1bc52 Add tests for numconns 2 2021-02-02 10:41:38 +11:00
Matt Joiner ad2ea89961 Use go1.16 release branch in CI 2021-02-02 10:41:38 +11:00
Shubhendra Singh Chauhan 0cdb33ede0 fixed code quality issues using DeepSource
* Added .deepsource.toml

* Removed empty default in select

* Removed unnecessary use of slice

* Fixed check for empty string
2021-01-30 15:23:49 +11:00
Matt Joiner f5915ad12c Use iotest.TestReader 2021-01-29 23:32:01 +11:00
Matt Joiner 887db4a276 Some deps got tagged 2021-01-29 22:05:09 +11:00
Matt Joiner 840f2ff3e9 Update crawshaw.io/sqlite
Now includes the fix for the pool race. I've removed the replace, the only thing left in the getlantern fork is the linking fix, and I forget when it's an issue.
2021-01-29 17:59:56 +11:00
Matt Joiner 68d8604c20 We're not expecting chunks if we have no outstanding requests 2021-01-29 16:14:08 +11:00
Matt Joiner 43a2942b98 Set not interested when we have no pending requests 2021-01-29 16:13:46 +11:00
Matt Joiner 60e992ec97 Rework webseed peers to use a pool of requesters
This fixes the limitation that the max outstanding requests for a webseed peer must match the request concurrency. It should mean less recalculation, and more pipelining.
2021-01-29 16:01:35 +11:00
Matt Joiner 4078644db9 Pass callbacks to webseed peers 2021-01-28 18:25:06 +11:00
Matt Joiner 1ac5811990 Remove requests as soon as chunk data is received
Note that this breaks the backpressure on webseed responses again, and should be fixed shortly.
2021-01-28 16:36:35 +11:00
Matt Joiner ca1497ad58 Add peer upload order analysis 2021-01-28 16:31:13 +11:00
Matt Joiner 47d6282e54 Don't cancel request on current peer when receiving chunk 2021-01-28 14:26:10 +11:00
Matt Joiner b86c212d30 Update deps 2021-01-28 14:23:34 +11:00
Matt Joiner 456a2f7c5d Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
Matt Joiner f409daa93f PORT message belongs to BEP 5 2021-01-27 17:08:15 +11:00
Matt Joiner fbc9a77cd8 Escape URI path 2021-01-26 21:49:53 +11:00
Matt Joiner 7440ef92d2 Update deps 2021-01-25 15:54:37 +11:00
Matt Joiner 9d6bf7a4f0 Remove cast(data as blob) workaround
Upstream merged the fix. Good performance boost.
2021-01-25 15:54:37 +11:00
Matt Joiner c424a2510e Create index on blob(last_used) 2021-01-25 15:54:37 +11:00
Matt Joiner 3e34763678 Add sqlite-storage-cli 2021-01-25 15:54:37 +11:00
Matt Joiner 5920bcae89 Remove obsolete reference to piece storage opts 2021-01-25 15:54:37 +11:00
Matt Joiner 25c60b72eb Fix race in writes to closed provider 2021-01-25 15:54:37 +11:00
Matt Joiner beb9ec8ffe Fix transfer tests involving sqlite memory storage 2021-01-25 15:54:37 +11:00
Matt Joiner b69bb34eaf Rejig some storage options 2021-01-25 15:54:37 +11:00
Matt Joiner 28682f6ae6 Fix unused opts param 2021-01-25 15:54:37 +11:00
Matt Joiner 19f5984aec Include context.Context and pprof label in withConn 2021-01-25 15:54:37 +11:00
Matt Joiner 2ddd3169ef Move storage piece benchmarks to storage/test and add a lot more dials 2021-01-25 15:54:37 +11:00
Matt Joiner cb5f80ec11 Improve configurability and add PutSized to sqlite storage 2021-01-25 15:54:37 +11:00
Matt Joiner 434dfdf6e0 Add MarkComplete benchmark for sqlite 2021-01-25 15:54:37 +11:00
Matt Joiner b5215ea7e8 Expose Peer.Network to replace Peer.RemoteAddr.Network 2021-01-25 15:49:48 +11:00
Matt Joiner 9f9953be52 Don't expose Peer network through RemoteAddr 2021-01-25 15:49:48 +11:00
Matt Joiner 955e337b78 Generate a RemoteAddr for webseed Peers 2021-01-25 15:49:48 +11:00
Matt Joiner 669905e203 Add ReceivedUsefulData Callback 2021-01-25 15:49:48 +11:00