]> Sergey Matveev's repositories - btrtrc.git/log
btrtrc.git
2 years agoShuffle duplicate requests
Matt Joiner [Sun, 12 Dec 2021 05:56:00 +0000 (16:56 +1100)]
Shuffle duplicate requests

Add missing import

2 years agoGenerate full sintel magnet link in fs test
Matt Joiner [Wed, 1 Dec 2021 23:50:02 +0000 (10:50 +1100)]
Generate full sintel magnet link in fs test

2 years agoAdd sintel.torrent
Matt Joiner [Wed, 1 Dec 2021 23:49:30 +0000 (10:49 +1100)]
Add sintel.torrent

2 years agoFix races using resources on Close
Matt Joiner [Wed, 1 Dec 2021 07:24:17 +0000 (18:24 +1100)]
Fix races using resources on Close

2 years agoDon't use the Client's IP blocklist after the Client is closed
Matt Joiner [Wed, 1 Dec 2021 04:02:36 +0000 (15:02 +1100)]
Don't use the Client's IP blocklist after the Client is closed

Fixes a segfault due to using a memory mapped blocklist in confluence.

2 years agoRemove unused torrent_pending_pieces.go
Matt Joiner [Wed, 1 Dec 2021 00:27:16 +0000 (11:27 +1100)]
Remove unused torrent_pending_pieces.go

Unfortunately github.com/elliotchance/orderedmap is still used in ./analysis, so no deps are removed.

2 years agoFilter update requests on piece priority change by peer choking and allowed fast
Matt Joiner [Tue, 30 Nov 2021 04:19:59 +0000 (15:19 +1100)]
Filter update requests on piece priority change by peer choking and allowed fast

2 years agoEnsure peerRequests.torrentStrategyInput is set
Matt Joiner [Mon, 29 Nov 2021 02:46:44 +0000 (13:46 +1100)]
Ensure peerRequests.torrentStrategyInput is set

2 years agoPEX: remove workaround for krpc marshaller
Yaroslav Kolomiiets [Wed, 8 Dec 2021 13:02:36 +0000 (13:02 +0000)]
PEX: remove workaround for krpc marshaller

2 years agoPEX: use new NodeAddr search methods in krpc
Yaroslav Kolomiiets [Wed, 8 Dec 2021 10:16:54 +0000 (10:16 +0000)]
PEX: use new NodeAddr search methods in krpc

2 years agoPEX: fluid event log
Yaroslav Kolomiiets [Mon, 6 Dec 2021 18:46:25 +0000 (18:46 +0000)]
PEX: fluid event log

2 years agoFix wasm build
Matt Joiner [Tue, 7 Dec 2021 02:07:58 +0000 (13:07 +1100)]
Fix wasm build

This was broken in the move to zombiezen sqlite, which still doesn't work on WASM/JS.

2 years agoDon't fail-fast on go versions
Matt Joiner [Tue, 7 Dec 2021 01:42:35 +0000 (12:42 +1100)]
Don't fail-fast on go versions

2 years agoHandle 503 returns from webseed peer endpoints
Matt Joiner [Mon, 6 Dec 2021 04:14:59 +0000 (15:14 +1100)]
Handle 503 returns from webseed peer endpoints

2 years agoRecord an observation about why webseed peers are doing most of the work in recent...
Matt Joiner [Thu, 2 Dec 2021 02:48:52 +0000 (13:48 +1100)]
Record an observation about why webseed peers are doing most of the work in recent testing

2 years agoDo webseed request parts sequentially
Matt Joiner [Thu, 2 Dec 2021 02:47:06 +0000 (13:47 +1100)]
Do webseed request parts sequentially

This means we can treat the number of bytes in the result with enough accuracy to decide if we should count it as a wasted chunk. Also I'm not sure why it was a good idea to do parts of a request in parallel anyway, it could just lead to spikes in outstanding requests to the webseed, rather than sticking to the predictable maxRequests limit.

2 years agoHandle cancelled webseed peer requests if they're not active
Matt Joiner [Thu, 2 Dec 2021 02:10:54 +0000 (13:10 +1100)]
Handle cancelled webseed peer requests if they're not active

2 years agoCheck if Torrent is closed before receiving webseed chunks
Matt Joiner [Fri, 3 Dec 2021 10:31:16 +0000 (21:31 +1100)]
Check if Torrent is closed before receiving webseed chunks

2 years agoFix minimum peer extensions error message
Matt Joiner [Fri, 3 Dec 2021 09:36:34 +0000 (20:36 +1100)]
Fix minimum peer extensions error message

2 years agoReduce idle request updates due to choke and unchoke messages
Matt Joiner [Mon, 29 Nov 2021 01:12:30 +0000 (12:12 +1100)]
Reduce idle request updates due to choke and unchoke messages

2 years agoMerge branch 'udp-tracker-no-dial' into te
Matt Joiner [Mon, 29 Nov 2021 00:19:54 +0000 (11:19 +1100)]
Merge branch 'udp-tracker-no-dial' into te

2 years agoRemove unused peer stuff in request strategy
Matt Joiner [Sun, 28 Nov 2021 23:14:16 +0000 (10:14 +1100)]
Remove unused peer stuff in request strategy

2 years agoOptimize the use of worstConnSlice again
Matt Joiner [Sun, 28 Nov 2021 23:07:48 +0000 (10:07 +1100)]
Optimize the use of worstConnSlice again

2 years agoOptimize Torrent.haveAnyPieces
Matt Joiner [Sun, 28 Nov 2021 13:44:24 +0000 (00:44 +1100)]
Optimize Torrent.haveAnyPieces

2 years agoAnnounce to DHT if we want conns, not just peers
Matt Joiner [Sun, 28 Nov 2021 11:46:45 +0000 (22:46 +1100)]
Announce to DHT if we want conns, not just peers

2 years agoWanting peers is a subset of wanting conns
Matt Joiner [Sun, 28 Nov 2021 11:37:56 +0000 (22:37 +1100)]
Wanting peers is a subset of wanting conns

2 years agoDon't want conns for seeding unless we have a piece
Matt Joiner [Sun, 28 Nov 2021 11:40:53 +0000 (22:40 +1100)]
Don't want conns for seeding unless we have a piece

2 years agoPeers with all pieces have wanted pieces only if we want any piece
Matt Joiner [Sun, 28 Nov 2021 11:36:19 +0000 (22:36 +1100)]
Peers with all pieces have wanted pieces only if we want any piece

2 years agoAbstract worseConn comparisons and add tests
Matt Joiner [Sun, 28 Nov 2021 06:06:10 +0000 (17:06 +1100)]
Abstract worseConn comparisons and add tests

2 years agoRename worst_conns.go
Matt Joiner [Sun, 28 Nov 2021 01:53:18 +0000 (12:53 +1100)]
Rename worst_conns.go

2 years agoMerge branch 'bsi-pending-requests' into te
Matt Joiner [Fri, 26 Nov 2021 22:38:58 +0000 (09:38 +1100)]
Merge branch 'bsi-pending-requests' into te

2 years agoUse modernc.org/sqlite version that fixes temp dir race
Matt Joiner [Fri, 26 Nov 2021 06:06:13 +0000 (17:06 +1100)]
Use modernc.org/sqlite version that fixes temp dir race

2 years agoComment out pending requests tests and asserts
Matt Joiner [Thu, 25 Nov 2021 11:59:20 +0000 (22:59 +1100)]
Comment out pending requests tests and asserts

These need to be updated. BSI for pending requests removes an enormous amount of memory use for very large torrents.

2 years agoRevert "Use a flat slice for pending request counts"
Matt Joiner [Thu, 25 Nov 2021 11:32:52 +0000 (22:32 +1100)]
Revert "Use a flat slice for pending request counts"

This reverts commit dfc421824ce6c146ff9d9062ad25748c0f8c9990.

2 years agotracker/udp: Panic on read errors if ConnClient not closed
Matt Joiner [Mon, 22 Nov 2021 21:35:45 +0000 (08:35 +1100)]
tracker/udp: Panic on read errors if ConnClient not closed

There's nothing exposed to handle this error currently, and if it occurs, it's better not to fail in silence. I'm currently debugging a situation where ConnClients stop working, this would be a condition we'd want to know about.

2 years agoImprove logging of UDP tracker packet dispatch error
Matt Joiner [Mon, 22 Nov 2021 21:33:29 +0000 (08:33 +1100)]
Improve logging of UDP tracker packet dispatch error

2 years agoWrap UDP tracker response error
Matt Joiner [Mon, 22 Nov 2021 21:32:25 +0000 (08:32 +1100)]
Wrap UDP tracker response error

2 years agoDon't dial in UDP tracking
Matt Joiner [Mon, 22 Nov 2021 07:05:50 +0000 (18:05 +1100)]
Don't dial in UDP tracking

This could fix an issue where tracker addresses change, but we're already bound to a particular address and so fail to receive any more responses.
It should also make it easier to share UDP sockets between UDP tracker clients, although that's not currently implemented.

2 years agoDoc comments
Matt Joiner [Mon, 22 Nov 2021 07:06:27 +0000 (18:06 +1100)]
Doc comments

2 years agoRemove CGO build flags for pure-Go squirrel branch
Matt Joiner [Fri, 19 Nov 2021 09:29:30 +0000 (20:29 +1100)]
Remove CGO build flags for pure-Go squirrel branch

2 years agoUse zombiezen branch of squirrel
Matt Joiner [Fri, 19 Nov 2021 09:10:07 +0000 (20:10 +1100)]
Use zombiezen branch of squirrel

2 years agogofumpt
Matt Joiner [Fri, 19 Nov 2021 05:19:32 +0000 (16:19 +1100)]
gofumpt

2 years agoUse zombiezen sqlite for piece completion
Matt Joiner [Fri, 19 Nov 2021 05:18:41 +0000 (16:18 +1100)]
Use zombiezen sqlite for piece completion

2 years agoQuote peer extension names to expvar
Matt Joiner [Fri, 19 Nov 2021 02:02:42 +0000 (13:02 +1100)]
Quote peer extension names to expvar

2 years agoPull synchronous firewall callback performance improvement
Matt Joiner [Wed, 17 Nov 2021 03:24:38 +0000 (14:24 +1100)]
Pull synchronous firewall callback performance improvement

Reduces CPU overhead when receiving lot of UDP traffic.

2 years agoOptimize PeerConn.peerHasAllPieces
Matt Joiner [Wed, 17 Nov 2021 03:01:11 +0000 (14:01 +1100)]
Optimize PeerConn.peerHasAllPieces

2 years agoAvoid calling net.Conn methods with the Client mutex locked
Matt Joiner [Wed, 17 Nov 2021 03:00:50 +0000 (14:00 +1100)]
Avoid calling net.Conn methods with the Client mutex locked

The utp firewall callback might want to use it. I can't think of a better way to determine with an incoming connection request should be blocked.

2 years agoUse tagged version of github.com/anacrolix/fuse v1.38.0
Matt Joiner [Tue, 16 Nov 2021 07:27:11 +0000 (18:27 +1100)]
Use tagged version of github.com/anacrolix/fuse

2 years agoSwitch to github.com/anacrolix/fuse
Matt Joiner [Tue, 16 Nov 2021 07:20:02 +0000 (18:20 +1100)]
Switch to github.com/anacrolix/fuse

2 years agoRemove dependency on and exclude of github.com/willf/bitset
Matt Joiner [Tue, 16 Nov 2021 06:36:48 +0000 (17:36 +1100)]
Remove dependency on and exclude of github.com/willf/bitset

2 years agoProvide context to readahead func
Matt Joiner [Sun, 14 Nov 2021 02:52:05 +0000 (13:52 +1100)]
Provide context to readahead func

2 years agoAdd Reader.SetReadaheadFunc
Matt Joiner [Wed, 10 Nov 2021 23:24:28 +0000 (10:24 +1100)]
Add Reader.SetReadaheadFunc

Fixes https://github.com/anacrolix/torrent/issues/687.

2 years agoAdd a comment about not discarding in webseed OK response bodies
Matt Joiner [Sun, 14 Nov 2021 03:01:46 +0000 (14:01 +1100)]
Add a comment about not discarding in webseed OK response bodies

2 years agoDon't log legitimate non-range requests v1.37.0
Matt Joiner [Fri, 12 Nov 2021 04:58:32 +0000 (15:58 +1100)]
Don't log legitimate non-range requests

2 years agoShould fix failing go1.16 CI build
Matt Joiner [Fri, 12 Nov 2021 04:18:20 +0000 (15:18 +1100)]
Should fix failing go1.16 CI build

2 years agoImprove error handling for bad webseeds
Matt Joiner [Fri, 12 Nov 2021 03:43:22 +0000 (14:43 +1100)]
Improve error handling for bad webseeds

DNS "no such host" errors didn't interrupt the stream of outbound requests.

2 years agoPass HTTP Proxy config into webseeding HTTP client
Matt Joiner [Fri, 12 Nov 2021 03:16:14 +0000 (14:16 +1100)]
Pass HTTP Proxy config into webseeding HTTP client

2 years agoRemove unused Client.updateRequests
Matt Joiner [Fri, 12 Nov 2021 02:55:43 +0000 (13:55 +1100)]
Remove unused Client.updateRequests

2 years agoBoost webseed max requests
Matt Joiner [Fri, 12 Nov 2021 02:51:58 +0000 (13:51 +1100)]
Boost webseed max requests

2 years agoIncrement webseed peer piece availability
Matt Joiner [Fri, 12 Nov 2021 02:41:55 +0000 (13:41 +1100)]
Increment webseed peer piece availability

2 years agoAllow non-partial webseed part responses for small files
Matt Joiner [Fri, 12 Nov 2021 02:30:13 +0000 (13:30 +1100)]
Allow non-partial webseed part responses for small files

2 years agowebseed: Close unused part responses after error
Matt Joiner [Fri, 12 Nov 2021 01:40:15 +0000 (12:40 +1100)]
webseed: Close unused part responses after error

Also don't bother to read their response bodies.

2 years agoDon't use non-directory webseed URLs for multi-file torrents
Matt Joiner [Fri, 12 Nov 2021 01:37:40 +0000 (12:37 +1100)]
Don't use non-directory webseed URLs for multi-file torrents

2 years agoMake TorrentSpecFromMetaInfoErr still usable on info error
Matt Joiner [Thu, 11 Nov 2021 23:46:35 +0000 (10:46 +1100)]
Make TorrentSpecFromMetaInfoErr still usable on info error

2 years agoComment that BEP 19 is WebSeeds
Matt Joiner [Thu, 11 Nov 2021 23:45:47 +0000 (10:45 +1100)]
Comment that BEP 19 is WebSeeds

2 years agocmd/torrentfs: Add missing envpprof.Stop
Matt Joiner [Thu, 11 Nov 2021 23:37:21 +0000 (10:37 +1100)]
cmd/torrentfs: Add missing envpprof.Stop

2 years agoPull uninitialized DHT store fix from anacrolix/dht v1.36.0
Matt Joiner [Tue, 9 Nov 2021 22:48:01 +0000 (09:48 +1100)]
Pull uninitialized DHT store fix from anacrolix/dht

Fixes https://github.com/anacrolix/torrent/issues/686.

2 years agogofumpt
Matt Joiner [Mon, 8 Nov 2021 03:47:01 +0000 (14:47 +1100)]
gofumpt

2 years agoImprove dht bootstrap stats logging
Matt Joiner [Thu, 4 Nov 2021 22:30:25 +0000 (09:30 +1100)]
Improve dht bootstrap stats logging

2 years agocmd/torrent: Add serve subcommand
Matt Joiner [Thu, 4 Nov 2021 09:02:55 +0000 (20:02 +1100)]
cmd/torrent: Add serve subcommand

2 years agobencode: Fix marshalling of unaddressable array of bytes
Matt Joiner [Thu, 4 Nov 2021 09:01:25 +0000 (20:01 +1100)]
bencode: Fix marshalling of unaddressable array of bytes

2 years agoExport addTorrentOpts
Matt Joiner [Thu, 4 Nov 2021 09:00:31 +0000 (20:00 +1100)]
Export addTorrentOpts

Now I see what the lint was complaining about.

2 years agoMove download subcmd code into separate file
Matt Joiner [Thu, 4 Nov 2021 00:36:34 +0000 (11:36 +1100)]
Move download subcmd code into separate file

2 years agocmd/torrent: Add bencode {json,spew} commands
Matt Joiner [Wed, 3 Nov 2021 02:49:06 +0000 (13:49 +1100)]
cmd/torrent: Add bencode {json,spew} commands

2 years agobencode.Bytes: Tests and stricter checks
Matt Joiner [Tue, 2 Nov 2021 06:28:05 +0000 (17:28 +1100)]
bencode.Bytes: Tests and stricter checks

2 years agoRun Go GitHub action with Go 1.16 as well as 1.17 v1.35.0
Matt Joiner [Mon, 1 Nov 2021 00:46:22 +0000 (11:46 +1100)]
Run Go GitHub action with Go 1.16 as well as 1.17

1.16 is still used by some downstream projects, and it might ensure support for pkg.go.dev (via old build constraint tags).

2 years agoRun Go GitHub action on all branches
Matt Joiner [Mon, 1 Nov 2021 00:45:08 +0000 (11:45 +1100)]
Run Go GitHub action on all branches

2 years agoRemove some unused parameter names
Matt Joiner [Mon, 1 Nov 2021 00:53:58 +0000 (11:53 +1100)]
Remove some unused parameter names

https://deepsource.io/gh/anacrolix/torrent/run/cfd6fa3d-c65b-4bf7-83f7-3bee87c21a4b/go/RVV-B0012

2 years agoAlter some directory permissioning in storage
Matt Joiner [Mon, 1 Nov 2021 00:50:12 +0000 (11:50 +1100)]
Alter some directory permissioning in storage

2 years agoRemove unused Torrent.numActivePeers
Matt Joiner [Mon, 1 Nov 2021 00:49:36 +0000 (11:49 +1100)]
Remove unused Torrent.numActivePeers

2 years agoRemove unused conn piece inclinations
Matt Joiner [Mon, 1 Nov 2021 00:49:27 +0000 (11:49 +1100)]
Remove unused conn piece inclinations

2 years agofixup! [trackerscraper] Add custom DNS lookup function
afjoseph [Fri, 29 Oct 2021 08:34:38 +0000 (10:34 +0200)]
fixup! [trackerscraper] Add custom DNS lookup function

2 years ago[fuzz] Put +build tags to support go1.16 as well
afjoseph [Thu, 28 Oct 2021 08:54:46 +0000 (10:54 +0200)]
[fuzz] Put +build tags to support go1.16 as well

2 years agoRevert "Remove old-style build tags"
afjoseph [Thu, 28 Oct 2021 08:53:04 +0000 (10:53 +0200)]
Revert "Remove old-style build tags"

This reverts commit 11ae25660ad3048ee5736694793fead3305f5c5b.

2 years ago[trackerscraper] Add custom DNS lookup function
afjoseph [Thu, 28 Oct 2021 08:13:21 +0000 (10:13 +0200)]
[trackerscraper] Add custom DNS lookup function

2 years agoLink to godocs.io for documentation
Matt Joiner [Fri, 29 Oct 2021 00:57:57 +0000 (11:57 +1100)]
Link to godocs.io for documentation

2 years agoFix unnecessary modification of Torrent.CancelPieces API
Matt Joiner [Fri, 29 Oct 2021 00:52:28 +0000 (11:52 +1100)]
Fix unnecessary modification of Torrent.CancelPieces API

https://github.com/anacrolix/torrent/issues/681

2 years agobencode: Encode arrays of bytes as strings
Matt Joiner [Thu, 28 Oct 2021 05:21:23 +0000 (16:21 +1100)]
bencode: Encode arrays of bytes as strings

This looks appropriate with the use of byte arrays in DHT for BEP44.

2 years agoRun all tests and benchmarks twice
Matt Joiner [Wed, 27 Oct 2021 04:15:27 +0000 (15:15 +1100)]
Run all tests and benchmarks twice

This helps detect bad cleanup in tests, which is very common.

2 years agoRemove unnecessary intermediate steps in Go workflow
Matt Joiner [Wed, 27 Oct 2021 04:14:48 +0000 (15:14 +1100)]
Remove unnecessary intermediate steps in Go workflow

2 years agoCheck if torrent is closed before handling peer request data read failures
Matt Joiner [Wed, 27 Oct 2021 04:14:11 +0000 (15:14 +1100)]
Check if torrent is closed before handling peer request data read failures

Should fix panics using storage after its been closed, like in https://github.com/anacrolix/torrent/runs/4017475163.

2 years agoFix panic in benchmark
Matt Joiner [Wed, 27 Oct 2021 02:57:19 +0000 (13:57 +1100)]
Fix panic in benchmark

2 years agoMerge branch 'peer-requesting' v1.34.0
Matt Joiner [Wed, 27 Oct 2021 01:42:26 +0000 (12:42 +1100)]
Merge branch 'peer-requesting'

2 years agoRetain the desired request ordering
Matt Joiner [Tue, 26 Oct 2021 23:13:39 +0000 (10:13 +1100)]
Retain the desired request ordering

This means we don't have to randomize the request order when we finally apply it to avoid favouring lower indices. The difference is very subtle but should be impactful with smaller connection counts and rarer torrents.

2 years agoAdd Peer.mustRequest
Matt Joiner [Tue, 26 Oct 2021 04:03:04 +0000 (15:03 +1100)]
Add Peer.mustRequest

2 years agoOnly assert pending requests when check is enabled
Matt Joiner [Tue, 26 Oct 2021 03:42:42 +0000 (14:42 +1100)]
Only assert pending requests when check is enabled

Uses too much CPU for production from its current callsites.

2 years agoAdd disabled code for limiting wasted chunks
Matt Joiner [Mon, 25 Oct 2021 10:37:25 +0000 (21:37 +1100)]
Add disabled code for limiting wasted chunks

2 years agoQuote webrtc offer ID as ASCII-only
Matt Joiner [Mon, 25 Oct 2021 10:03:51 +0000 (21:03 +1100)]
Quote webrtc offer ID as ASCII-only

2 years agoFix iter pending chunk request offsets
Matt Joiner [Mon, 25 Oct 2021 09:53:39 +0000 (20:53 +1100)]
Fix iter pending chunk request offsets

2 years agoSupport minimum peer extensions
Matt Joiner [Mon, 25 Oct 2021 09:52:44 +0000 (20:52 +1100)]
Support minimum peer extensions