258585fd8b
sqlite storage: Buffer write requests
2020-11-16 16:37:11 +11:00
b0ebc856d9
sqlite storage: Add some expvars
2020-11-16 16:37:11 +11:00
e65fac26ac
sqlite storage: Add error return from withConn
2020-11-16 16:37:11 +11:00
7afa4a2de4
Fix panic in when bbolt storage has Closed
2020-11-16 16:37:11 +11:00
6cceb5749d
Mark piece complete without Client lock
2020-11-16 16:37:11 +11:00
0d40c4bac2
Read peer request data without Client lock
2020-11-16 16:37:11 +11:00
b020b8c2b6
Add support for the x.pe magnet link parameter
2020-11-16 16:37:11 +11:00
71b9718347
optimise PEX by avoiding intermediate storage while preparing PEX messages
2020-11-16 15:59:32 +11:00
0c1a506e04
optimise generation of the initial PEX
2020-11-12 22:25:17 +11:00
94055287b0
simplify pexMsgFactory
2020-11-12 22:25:17 +11:00
ebdfae97a2
add test covering a recently fixed regression in initial PEX
2020-11-12 22:25:17 +11:00
28b6c93d6e
Fix excessive logging from mmap-backed storage ( anacrolix/torrent#436 )
2020-11-12 08:57:42 +11:00
493ee9dfd1
Implement fmt.Formatter for metainfo.Hash
...
It's so easy to make mistakes by specifying %x when printing these.
2020-11-11 15:31:55 +11:00
6b5da26d38
Improve cmd/tracker-announce
...
Add -tracker flag, support specifying infohashes directly, spew announce responses so they're more readable.
2020-11-11 15:30:53 +11:00
e6da640bb2
Tidy up doc, file names, naming
2020-11-09 10:56:45 +11:00
20d9c63885
Update README.md
2020-11-09 09:20:05 +11:00
252d220709
Fix initial pex message regression
...
As pointed out by @yarikk c1d189ed31 (r43920706)
2020-11-08 23:13:59 +11:00
7cb74b158f
Replace storage.IncompletePieceToWriter with io.Writer
...
It was incorrect to assume piece hashing only operates on incomplete chunk data. This actually uncovered a bug where duplicate hash checks occurred, and the redundant checks would fail due to not reading the completed data.
2020-11-06 16:23:38 +11:00
81224de762
Fix PkgGoDev reference in README
2020-11-06 08:44:02 +11:00
dcb2c0bb41
Fix piece getting queued for hash multiple times
...
Pieces could get queued for hash multiple times when we receive chunks if the piece starts getting hashed before we're done writing all the chunks out. This was only found because piece hashing currently only checks the incomplete data, which is missing after the first piece hash passes, the data is marked complete, then the subsequently queued hash has nothing to read.
2020-11-06 08:39:56 +11:00
131037dd9f
Add the ReceiveEncryptedHandshakeSkeys callback
2020-11-05 13:28:45 +11:00
15c8846153
mse/cmd/mse: Fail on missing subcommand
2020-11-04 16:34:31 +11:00
7b2a7a667d
Remove old comment
2020-11-04 11:34:24 +11:00
96b640065a
sqlite storage: Add capacity management
2020-11-03 15:17:12 +11:00
8afb05a8a5
sqlite storage: Batch writes synchronously instead of buffering them first
2020-11-03 15:16:57 +11:00
3c102393d1
Comment on storage.IncompletePieceToWriter
2020-11-03 14:28:16 +11:00
784345e9f7
sqlite storage: Init schema in NewPool instead of NewProvider and add an option to disable
2020-11-03 13:11:44 +11:00
d04622e4b3
sqlite storage: Track total blob data size manually
...
Works around possible full table scan at startup, and possible lack of caching around cast(data as blob).
2020-11-03 13:10:17 +11:00
94efdbbded
Update the readahead window on read failure
2020-11-03 10:41:47 +11:00
636b20b860
Add write incomplete/consecutive chunks interfaces
2020-11-02 15:35:07 +11:00
17e22516ff
sqlite storage: Add batched writes
2020-10-30 19:46:51 +11:00
ba70ad9b41
sqlite storage: Set mmap_size to a very large size
...
Seems to fall back to ~2GiB on my system.
2020-10-30 18:40:47 +11:00
b75ebbf9e7
sqlite storage: Provide helpers and reasonable defaults
2020-10-30 12:20:54 +11:00
c28e9aaeae
Don't delete requests until after they're written to disk
...
This prevents too many pending writes building up. Webseed peers re-request synchronously, and the writes are done asynchronously, so they download too quickly and there was no backpressure. The backpressure now is provided by the upper limit on outstanding requests per connection.
2020-10-30 12:19:53 +11:00
fc039262d9
sqlite storage: Force data to be used as a blob
...
There's a bug in crawshaw.io/sqlite, and some forks where inserting []byte results in a text type instead of blob. To ensure things work correctly, we coerce data to blob wherever we can. See https://github.com/crawshaw/sqlite/issues/94 and the fork that fixes it.
2020-10-30 10:47:50 +11:00
6f619c39cb
Include a null byte in the middle of the Greeting test
2020-10-30 10:42:45 +11:00
7410e28329
Fix peer request sleepiness
...
New requests weren't being issued to the current peer when being deleted. For webseeds, this would cause them to not bother issuing new requests indefinitely.
(cherry picked from commit 146a16df4ea26d33b0ce0391c8220de14c9e18f4)
2020-10-30 09:13:39 +11:00
27b89f08c9
Don't fail on 386 tests in CI
...
Looks like crawshaw.io/sqlite fails to build, possibly due to cross-compiling.
2020-10-28 12:09:24 +11:00
520b28cc59
Switch CI to go1.15
2020-10-28 10:03:37 +11:00
845d3fbe8f
Remove some unused code
2020-10-27 17:56:55 +11:00
8e1a8440bf
Performance fiddling on sqlite storage
2020-10-27 17:07:49 +11:00
43891309ad
Return errors from Reader if data downloading won't occur
...
Chunk write errors to storage can disable data download. Previously Readers would wait indefinitely for the data to become available. This change returns an error instead of stalling.
2020-10-27 14:10:17 +11:00
2a7352aad2
Don't block trying to fill entire Reader.Read
...
This conforms more to the contract in io.Reader. It's possible the old behaviour was better in reducing overhead, but that can be iterated on (or added as comments next time).
2020-10-27 12:59:07 +11:00
1bfca56e94
Read from more than a single piece in each read to Torrent storage
2020-10-27 12:24:43 +11:00
c6018f9d90
Remove separate readable/available in Reader
2020-10-27 12:05:40 +11:00
e30084223d
sqlite storage: Include capacity management
2020-10-27 11:08:37 +11:00
55d4bcaf26
sqlite storage: Do ReadAt without using incremental I/O
2020-10-27 11:08:08 +11:00
7fa8c604ab
sqlite storage: Working recursive CTE for excess data
2020-10-23 11:03:42 +11:00
1fc63bdded
sqlite storage: Add NewProviderPool
2020-10-23 09:03:44 +11:00
bb2ce15114
sqlite storage: Rename table to blob
...
Avoiding plural table names.
2020-10-23 09:01:15 +11:00
Matt Joiner