Commit Graph

79 Commits

Author SHA1 Message Date
Matt Joiner cf6c20d306 Add function to wrap squirrel.Cache as storage.ClientImpl 2021-08-26 11:20:37 +10:00
Matt Joiner 91873addfa Update squirrel 2021-08-26 11:19:39 +10:00
Matt Joiner d6fcf7a32a Use separate squirrel module 2021-08-25 14:37:00 +10:00
Matt Joiner 19d5905b6c Removed unused sqlite "provider" storage 2021-08-25 12:35:07 +10:00
Matt Joiner 8df24008ea Embed SQL 2021-08-24 22:24:39 +10:00
Matt Joiner 08d2dea5b4 Implement {Set,Get}Tag on SquirrelBlob 2021-08-24 18:37:38 +10:00
Matt Joiner f2f541a46b Begin extracting 'squirrel' from storage/sqlite 2021-08-24 17:52:31 +10:00
Matt Joiner 8a65ef627d Set direct sqlite storage conn to nil on close
This might help catch the reason for SQLITE_MISUSE in getCapacity.
2021-08-19 13:37:00 +10:00
Bora M. Alper 16176b762e
Add linter CI (#542)
* Add linter CI

Signed-off-by: Bora M. Alper <bora@boramalper.org>

* Make gosec CI ignore SHA1 and upload sarif

Signed-off-by: Bora M. Alper <bora@boramalper.org>

* Fix formatting of source files

Signed-off-by: Bora M. Alper <bora@boramalper.org>

* Make go vet ignore unkeyed composite literals and fix other warnings

Signed-off-by: Bora M. Alper <bora@boramalper.org>

* Make staticcheck ignore unused methods and fix other warnings

Signed-off-by: Bora M. Alper <bora@boramalper.org>

* Use golangci-lint

Signed-off-by: Bora M. Alper <bora@boramalper.org>
2021-08-16 11:11:31 +10:00
Matt Joiner 87294355fb Fix race in sqlite direct storage init
There's a race between starting the blob flusher and assigning the timer to the storage client.
2021-08-11 09:44:09 +10:00
Matt Joiner 5fc42e8105 Remove sqlite piece-resource storage 2021-06-21 12:29:37 +10:00
Matt Joiner ebd19af795 Merge branch 'request-strategy-rewrite' 2021-06-21 12:04:06 +10:00
deepsource-autofix[bot] a68f040ea6 Fix nil context being passed to function 2021-06-07 19:32:02 +10:00
Matt Joiner 5f8471e21b Rework storage.TorrentImpl to support shared capacity key 2021-06-07 13:01:39 +10:00
Matt Joiner a60a4c9117 Progress testing without cgo a bit 2021-05-21 15:22:26 +10:00
Matt Joiner b21aebeaae Make wal the default for sqlite resource pieces
Fixes broken test as a result.
2021-05-17 11:56:10 +10:00
Matt Joiner 7798e2a306 Make synchronous=off the default 2021-05-17 11:56:10 +10:00
Matt Joiner c585b84126 Set smarter defaults 2021-05-17 11:56:10 +10:00
Matt Joiner fc0f2d146d Set page size before initializing connections
Setting page_size seems to be ignored if done after setting journal_mode, specifically to WAL I think. There's huge performance benefits to getting it right.
2021-05-17 11:56:10 +10:00
Matt Joiner 961cbfd765 Add CustomDirect BenchmarkMarkComplete sub-test 2021-05-14 15:42:54 +10:00
Matt Joiner 7b3e48b915 Expose SetSynchronous as an option 2021-05-14 15:41:56 +10:00
Matt Joiner 5030804b01 Fix race in MarkNotComplete 2021-05-14 15:41:28 +10:00
Matt Joiner e5d21dbf34 Don't create blobs when reading 2021-05-14 15:41:23 +10:00
Matt Joiner b151514154 Tidy up sqlite direct piece receiver name 2021-05-14 15:40:59 +10:00
Matt Joiner 9feb85d0d1 Remove unused method 2021-05-14 15:40:38 +10:00
Matt Joiner 8dcc52ccf9 Don't set the page size by default 2021-05-06 15:25:01 +10:00
Matt Joiner 96574468c5 Expose a variety of blob cleanup styles 2021-05-06 15:17:31 +10:00
Matt Joiner abe003b6b3 Benchmark different mmap sizes and journal modes 2021-05-06 12:00:20 +10:00
Matt Joiner 20c00dedc0 Rework lots of option handling 2021-05-05 21:36:36 +10:00
Matt Joiner acfe0ba87e Rename new.go 2021-05-05 15:55:08 +10:00
Matt Joiner 745a34b43e Handle expired blobs as they occur 2021-05-05 11:47:39 +10:00
Matt Joiner 675a0ab0dc Rework to use a pool of blobs 2021-05-05 10:02:15 +10:00
Matt Joiner 855144212c Get benchmarks working 2021-05-04 22:56:43 +10:00
Matt Joiner afea28091f Implement sqlite directly without using piece resources 2021-05-04 19:51:42 +10:00
Matt Joiner 0021c2a70c Use BenchmarkMarkComplete for non-sqlite storages too 2021-05-04 12:44:51 +10:00
Matt Joiner f00f513706 Add missing closeMu use in ReadConsecutiveChunks 2021-02-22 16:39:21 +11:00
Matt Joiner 86a2b71c47 Code comment 2021-02-19 12:37:26 +11:00
Matt Joiner 78c77c0b45 Expose mmap_size in sqlite storage, and change default to 8 MiB 2021-02-04 18:29:40 +11:00
Matt Joiner 371d54ab76 Fix Close race in sqlite storage when batch writes disabled 2021-02-03 19:27:26 +11:00
Matt Joiner 6f9c2fed61 Add batch writes cases to sqlite storage benchmarks 2021-02-03 18:55:14 +11:00
Matt Joiner 15ccbc95c6 Fix sqlite storage for numconns 1 2021-02-02 10:41:39 +11:00
Matt Joiner 270a2ba1ae Switch to reading consecutive incomplete chunks
This fixes a race where a sqlite conn isn't reserved for the read part of a MarkComplete operation after the write has already begun.
2021-02-02 10:41:38 +11:00
Matt Joiner fbc9a77cd8 Escape URI path 2021-01-26 21:49:53 +11:00
Matt Joiner 9d6bf7a4f0 Remove cast(data as blob) workaround
Upstream merged the fix. Good performance boost.
2021-01-25 15:54:37 +11:00
Matt Joiner c424a2510e Create index on blob(last_used) 2021-01-25 15:54:37 +11:00
Matt Joiner 3e34763678 Add sqlite-storage-cli 2021-01-25 15:54:37 +11:00
Matt Joiner 5920bcae89 Remove obsolete reference to piece storage opts 2021-01-25 15:54:37 +11:00
Matt Joiner 25c60b72eb Fix race in writes to closed provider 2021-01-25 15:54:37 +11:00
Matt Joiner beb9ec8ffe Fix transfer tests involving sqlite memory storage 2021-01-25 15:54:37 +11:00
Matt Joiner b69bb34eaf Rejig some storage options 2021-01-25 15:54:37 +11:00