d04622e4b3
sqlite storage: Track total blob data size manually
...
Works around possible full table scan at startup, and possible lack of caching around cast(data as blob).
2020-11-03 13:10:17 +11:00
94efdbbded
Update the readahead window on read failure
2020-11-03 10:41:47 +11:00
636b20b860
Add write incomplete/consecutive chunks interfaces
2020-11-02 15:35:07 +11:00
17e22516ff
sqlite storage: Add batched writes
2020-10-30 19:46:51 +11:00
ba70ad9b41
sqlite storage: Set mmap_size to a very large size
...
Seems to fall back to ~2GiB on my system.
2020-10-30 18:40:47 +11:00
b75ebbf9e7
sqlite storage: Provide helpers and reasonable defaults
2020-10-30 12:20:54 +11:00
c28e9aaeae
Don't delete requests until after they're written to disk
...
This prevents too many pending writes building up. Webseed peers re-request synchronously, and the writes are done asynchronously, so they download too quickly and there was no backpressure. The backpressure now is provided by the upper limit on outstanding requests per connection.
2020-10-30 12:19:53 +11:00
fc039262d9
sqlite storage: Force data to be used as a blob
...
There's a bug in crawshaw.io/sqlite, and some forks where inserting []byte results in a text type instead of blob. To ensure things work correctly, we coerce data to blob wherever we can. See https://github.com/crawshaw/sqlite/issues/94 and the fork that fixes it.
2020-10-30 10:47:50 +11:00
6f619c39cb
Include a null byte in the middle of the Greeting test
2020-10-30 10:42:45 +11:00
7410e28329
Fix peer request sleepiness
...
New requests weren't being issued to the current peer when being deleted. For webseeds, this would cause them to not bother issuing new requests indefinitely.
(cherry picked from commit 146a16df4ea26d33b0ce0391c8220de14c9e18f4)
2020-10-30 09:13:39 +11:00
27b89f08c9
Don't fail on 386 tests in CI
...
Looks like crawshaw.io/sqlite fails to build, possibly due to cross-compiling.
2020-10-28 12:09:24 +11:00
520b28cc59
Switch CI to go1.15
2020-10-28 10:03:37 +11:00
845d3fbe8f
Remove some unused code
2020-10-27 17:56:55 +11:00
8e1a8440bf
Performance fiddling on sqlite storage
2020-10-27 17:07:49 +11:00
43891309ad
Return errors from Reader if data downloading won't occur
...
Chunk write errors to storage can disable data download. Previously Readers would wait indefinitely for the data to become available. This change returns an error instead of stalling.
2020-10-27 14:10:17 +11:00
2a7352aad2
Don't block trying to fill entire Reader.Read
...
This conforms more to the contract in io.Reader. It's possible the old behaviour was better in reducing overhead, but that can be iterated on (or added as comments next time).
2020-10-27 12:59:07 +11:00
1bfca56e94
Read from more than a single piece in each read to Torrent storage
2020-10-27 12:24:43 +11:00
c6018f9d90
Remove separate readable/available in Reader
2020-10-27 12:05:40 +11:00
e30084223d
sqlite storage: Include capacity management
2020-10-27 11:08:37 +11:00
55d4bcaf26
sqlite storage: Do ReadAt without using incremental I/O
2020-10-27 11:08:08 +11:00
7fa8c604ab
sqlite storage: Working recursive CTE for excess data
2020-10-23 11:03:42 +11:00
1fc63bdded
sqlite storage: Add NewProviderPool
2020-10-23 09:03:44 +11:00
bb2ce15114
sqlite storage: Rename table to blob
...
Avoiding plural table names.
2020-10-23 09:01:15 +11:00
c1d189ed31
Performance improvements to PEX
2020-10-23 08:58:55 +11:00
8088002ba6
Fix error handling for bad torrent adds with safe file handling
2020-10-15 16:03:43 +11:00
89235e180f
Sanitize metainfo file paths for file-based storage
...
Fixes exploit where specially crafted infos can cause the client to write files to arbitrary locations on local storage when using file-based storages like mmap and file.
2020-10-15 15:45:08 +11:00
635ec27fb3
Don't update requests for the current connection first when a request is deleted
2020-10-15 12:56:06 +11:00
0b726a899b
We might want to close webseed peers on response errors
2020-10-15 12:45:19 +11:00
30d0b5b9c7
webseed: Handle http response status codes
2020-10-15 12:42:27 +11:00
4a4cb5dc58
Ensure PeerConn._close is called for incoming connections
...
This fixes missing calls to PeerConnClosed callback.
2020-10-14 15:11:45 +11:00
07bc79dbdc
Test both sqlite file and in-memory leecher storages
2020-10-13 13:34:05 +11:00
02f82131ea
Move pexConnState from peer to PeerConn
2020-10-13 13:33:22 +11:00
152d8715ff
Add PeerConnClosed callback
2020-10-13 13:02:39 +11:00
3820017a0c
Add last_used to sqlite storage
2020-10-13 09:36:58 +11:00
ad10a09982
Use bind-blob fix for crawshaw.io/sqlite
2020-10-12 17:13:05 +11:00
6ed1062176
Fix some log levels
2020-10-12 17:12:34 +11:00
d820f7861c
Add a sqlite data storage implementation
2020-10-11 12:58:27 +11:00
00ac333ea2
Panic on bad piece per resource chunk names
...
This was masking a bug in a coming sqlite storage implementation. Maybe it could be relaxed to checking for no filepath.Dir in the future.
2020-10-11 12:57:33 +11:00
417d7d1d48
Tweaks to storage error and completion handling
2020-10-11 12:54:03 +11:00
898260c177
Tidy up some storage close handling
2020-10-11 12:40:43 +11:00
3d7a95a65d
Use testing.T.Name instead of missinggo.GetTestName
...
This handles subtests correctly, and removes the need to specify when to export due to collisions with the old function.
2020-10-10 10:59:17 +11:00
a9102abf1d
Tidy up some logging
2020-10-10 10:24:58 +11:00
d3daaaf75a
Fix webseed requests for non-trivial path components
2020-10-07 10:22:55 +11:00
014cb3986b
Fix PeerConn conn status string
2020-10-06 09:50:47 +11:00
14f2e8087e
Fix unused import
2020-10-01 11:50:42 +10:00
01dfc28484
Set tracker announce timeout back to default
...
Add lots of comments about how this works and why.
2020-10-01 11:45:47 +10:00
3d7ac87e3b
Expose Client.ConnStats
2020-10-01 11:20:44 +10:00
93685a55fe
Include ip param in http announces
2020-10-01 10:46:27 +10:00
e1d59f791e
Pass key param in http announces
2020-10-01 10:45:29 +10:00
010362ec82
Rework HTTP announce timeouts
...
Use Request.Context to implement timeouts, set the default to 3s for announces from the Client.
2020-10-01 10:45:05 +10:00
Matt Joiner