Go to file
Matt Joiner ef75fa47f8
Create SECURITY.md
2021-05-14 10:17:19 +10:00
.circleci Don't bench with -race 2021-02-24 13:48:05 +11:00
.github/workflows Create codeql-analysis.yml 2021-02-18 14:48:15 +11:00
analysis Add peer upload order analysis 2021-01-28 16:31:13 +11:00
bencode Apply staticcheck 2020-02-20 11:09:57 +11:00
cmd cmd/torrent: Ensure progress is emitted at consistent intervals 2021-05-04 16:08:21 +10:00
common Got file storage working with segment index 2020-06-01 00:14:51 +10:00
fs Comment and log change on torrentfs behaviour change 2021-02-22 10:10:43 +11:00
internal Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
iplist goimports -local 2019-08-21 20:58:40 +10:00
logonce Add a prefix to logonce logger output 2018-09-03 11:48:31 +10:00
metainfo Fix DeepSource anti-pattern 2021-03-25 09:56:04 +11:00
mmap_span Fix excessive logging from mmap-backed storage (anacrolix/torrent#436) 2020-11-12 08:57:42 +11:00
mse Fix some go vet warnings 2021-02-03 19:42:46 +11:00
peer_protocol PORT message belongs to BEP 5 2021-01-27 17:08:15 +11:00
segments Further progress on webseeding 2020-06-01 18:25:45 +10:00
storage Allows one to use Custom PieceCompletion (#486) 2021-05-09 19:34:41 +10:00
test Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
testdata Merge branch 'fuse-email-issue' 2021-02-22 10:05:32 +11:00
tracker Fix some go vet warnings 2021-02-03 19:42:46 +11:00
util/dirwatch Big logging cleanup to improve experience from README 2021-05-04 12:44:51 +10:00
webseed Treat 404 responses from webseed peers as fatal 2021-02-18 14:36:08 +11:00
webtorrent Upgrade to pion/webrtc@v3 2021-02-21 13:46:37 -08:00
.deepsource.toml fixed code quality issues using DeepSource 2021-01-30 15:23:49 +11:00
.gitignore Fix PkgGoDev reference in README 2020-11-06 08:44:02 +11:00
LICENSE Update LICENSE 2018-01-27 11:21:33 +11:00
README.md Remove GoDoc badge 2021-05-04 12:43:50 +10:00
SECURITY.md Create SECURITY.md 2021-05-14 10:17:19 +10:00
TODO Update TODO file 2018-04-14 21:44:50 +10:00
bad_storage.go Use iotest.TestReader 2021-01-29 23:32:01 +11:00
bep40.go Move IpPort to missinggo 2018-11-16 10:35:30 +11:00
bep40_test.go fixes various vet errors 2020-01-14 09:09:09 +10:00
callbacks.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
client.go Big logging cleanup to improve experience from README 2021-05-04 12:44:51 +10:00
client_test.go Improve metainfo.MetaInfo.Magnet interface and add tests 2021-02-24 12:11:47 +11:00
closewrapper.go Add initial connection tracking 2018-11-21 17:02:22 +11:00
config.go Big logging cleanup to improve experience from README 2021-05-04 12:44:51 +10:00
conn_stats.go Add doc comments related to #383 2020-03-16 16:30:39 +11:00
deferrwl.go Switch back to anacrolix/sync for the Client lock 2019-12-17 20:49:40 +11:00
dht.go Fix dht Server Ping call 2021-03-11 11:32:31 +11:00
dialer.go Add support for non-IP-based networks 2020-02-20 16:47:37 +11:00
doc.go BEP27 isn't implemented 2020-05-24 17:47:16 +10:00
example_test.go Add File.NewReader 2018-01-06 16:37:13 +11:00
file.go Support disabling webtorrent 2020-06-01 18:24:46 +10:00
file_test.go Rewrite file.bytesLeft 2020-03-24 12:15:35 +11:00
global.go Update the peersAddedBySource expvar 2020-04-16 11:56:58 +10:00
go.mod cmd/torrent: Restore parsing of units for upload and download rates 2021-05-04 16:00:22 +10:00
go.sum cmd/torrent: Restore parsing of units for upload and download rates 2021-05-04 16:00:22 +10:00
handshake.go Add the ReceiveEncryptedHandshakeSkeys callback 2020-11-05 13:28:45 +11:00
handshake_test.go Support AllowedFast and enable fast extension 2018-02-05 00:18:38 +11:00
ipport.go Don't expose Peer network through RemoteAddr 2021-01-25 15:49:48 +11:00
issue97_test.go Propagate back piece hashing errors 2020-01-14 10:51:09 +11:00
issue211_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
listen.go Remove unused code 2018-11-27 23:57:17 +11:00
main_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
misc.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
misc_test.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
multiless.go Apply staticcheck 2020-02-20 11:09:57 +11:00
network_test.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
networks.go Improve network handling and only listen networks we will use 2018-11-28 10:30:21 +11:00
peer-impl.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
peer_info.go Don't expose Peer network through RemoteAddr 2021-01-25 15:49:48 +11:00
peer_infos.go Rename Peer to PeerInfo, and unexport PeerInfos 2020-05-29 19:44:48 +10:00
peerconn.go Fix closing of webseed peers 2021-02-09 19:21:54 +11:00
peerconn_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
peerid.go Rename Peer to PeerInfo, and unexport PeerInfos 2020-05-29 19:44:48 +10:00
peerid_test.go Change a bunch of Stringer and status stuff 2018-02-05 17:47:47 +11:00
pex.go PEX: impede full-meching in tracker-less swarms by adding a cooldown minute 2021-02-24 08:29:18 +11:00
pex_test.go Export Peer 2021-01-25 15:49:48 +11:00
pexconn.go PEX: impede full-meching in tracker-less swarms by adding a cooldown minute 2021-02-24 08:29:18 +11:00
pexconn_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
piece.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
piecestate.go Track piece marking state separately 2020-11-21 13:44:23 +11:00
portfwd.go Big logging cleanup to improve experience from README 2021-05-04 12:44:51 +10:00
prioritized_peers.go Rename Peer to PeerInfo, and unexport PeerInfos 2020-05-29 19:44:48 +10:00
prioritized_peers_test.go Rename Peer to PeerInfo, and unexport PeerInfos 2020-05-29 19:44:48 +10:00
protocol.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
ratelimitreader.go Fix download rate limiter and add test 2018-01-28 15:28:45 +11:00
reader.go Revert "Fix stalls for responsive transfer tests" 2021-02-03 18:55:54 +11:00
reader_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
request-strategy-defaults.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
request-strategy.go Expose more callbacks and Request and ChunkSpec 2021-01-28 14:23:22 +11:00
rlreader_test.go sortimports 2018-01-31 16:42:40 +11:00
socket.go Tidy up doc, file names, naming 2020-11-09 10:56:45 +11:00
spec.go Add support for the x.pe magnet link parameter 2020-11-16 16:37:11 +11:00
struct_test.go Export Peer 2021-01-25 15:49:48 +11:00
t.go Strictly adher to tracker announce response intervals if a torrent is private 2020-12-09 22:36:57 +11:00
testing.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
torrent.go Big logging cleanup to improve experience from README 2021-05-04 12:44:51 +10:00
torrent_pending_pieces.go Use stable bit set for pending pieces PriorityBitmap 2020-02-27 18:33:44 +11:00
torrent_stats.go Add doc comments related to #383 2020-03-16 16:30:39 +11:00
torrent_test.go Replace internal/tmproot with Testing.TB.TempDir 2021-02-22 14:30:09 +11:00
tracker_scraper.go Pass trackerScraper context to announces 2020-12-21 09:29:20 +11:00
url-net-addr.go Generate a RemoteAddr for webseed Peers 2021-01-25 15:49:48 +11:00
utp.go Add support for disabling IPv4 and IPv4 peers 2018-02-16 10:46:11 +11:00
utp_go.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
utp_libutp.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
utp_test.go Use the new firewall callback support in go-libutp 2018-07-25 17:11:09 +10:00
webrtc.go Upgrade to pion/webrtc@v3 2021-02-21 13:46:37 -08:00
webseed-peer.go Treat 404 responses from webseed peers as fatal 2021-02-18 14:36:08 +11:00
worst_conns.go Export Peer 2021-01-25 15:49:48 +11:00
wstracker.go Fix crash when announcing unloaded torrent to websocket 2020-05-03 14:10:46 +10:00

README.md

torrent

CircleCI Astronomer Rating GitHub Release PkgGoDev

This repository implements BitTorrent-related packages and command-line utilities in Go. The emphasis is on use as a library from other projects. It's been used 24/7 in production by downstream services since late 2014. The implementation was specifically created to explore Go's concurrency capabilities, and to include the ability to stream data directly from the BitTorrent network. To this end it supports seeking, readaheads and other features exposing torrents and their files with the various Go idiomatic io package interfaces. This is also demonstrated through torrentfs.

There is support for protocol encryption, DHT, PEX, uTP, and various extensions. There are several data storage backends provided: blob, file, bolt, mmap, and sqlite, to name a few. You can write your own to store data for example on S3, or in a database.

Some noteworthy package dependencies that can be used for other purposes include:

Installation

Install the library package with go get github.com/anacrolix/torrent, or the provided cmds with go get github.com/anacrolix/torrent/cmd/....

Library examples

There are some small examples in the package documentation.

Mentions

Downstream projects

There are several web-frontends and Android clients among the known public projects:

Help

Communication about the project is primarily through Discussions and the issue tracker.

Command packages

Here I'll describe what some of the packages in ./cmd do. Install them with go get github.com/anacrolix/torrent/cmd/....

torrent

Downloads torrents from the command-line.

$ torrent download 'magnet:?xt=urn:btih:KRWPCX3SJUM4IMM4YF5RPHL6ANPYTQPU'
... lots of jibba jabber ...
downloading "ubuntu-14.04.2-desktop-amd64.iso": 1.0 GB/1.0 GB, 1989/1992 pieces completed (1 partial)
2015/04/01 02:08:20 main.go:137: downloaded ALL the torrents
$ md5sum ubuntu-14.04.2-desktop-amd64.iso
1b305d585b1918f297164add46784116  ubuntu-14.04.2-desktop-amd64.iso
$ echo such amaze
wow

torrentfs

torrentfs mounts a FUSE filesystem at -mountDir. The contents are the torrents described by the torrent files and magnet links at -metainfoDir. Data for read requests is fetched only as required from the torrent network, and stored at -downloadDir.

$ mkdir mnt torrents
$ torrentfs -mountDir=mnt -metainfoDir=torrents &
$ cd torrents
$ wget http://releases.ubuntu.com/14.04.2/ubuntu-14.04.2-desktop-amd64.iso.torrent
$ cd ..
$ ls mnt
ubuntu-14.04.2-desktop-amd64.iso
$ pv mnt/ubuntu-14.04.2-desktop-amd64.iso | md5sum
996MB 0:04:40 [3.55MB/s] [========================================>] 100%
1b305d585b1918f297164add46784116  -

torrent-magnet

Creates a magnet link from a torrent file. Note the extracted trackers, display name, and info hash.

$ torrent-magnet < torrents/ubuntu-14.04.2-desktop-amd64.iso.torrent
magnet:?xt=urn:btih:546cf15f724d19c4319cc17b179d7e035f89c1f4&dn=ubuntu-14.04.2-desktop-amd64.iso&tr=http%3A%2F%2Ftorrent.ubuntu.com%3A6969%2Fannounce&tr=http%3A%2F%2Fipv6.torrent.ubuntu.com%3A6969%2Fannounce