]> Sergey Matveev's repositories - btrtrc.git/log
btrtrc.git
2 years agoQuote UDP tracker response error bodies with %#q v1.41.0
Matt Joiner [Sun, 20 Feb 2022 05:15:45 +0000 (16:15 +1100)]
Quote UDP tracker response error bodies with %#q

2 years agoPass client logger to anacrolix/go-libutp sockets (#722)
FIGBERT [Tue, 15 Feb 2022 05:18:32 +0000 (21:18 -0800)]
Pass client logger to anacrolix/go-libutp sockets (#722)

* Update anacrolix/go-libutp (v1.1.0 -> v1.2.0)

* Pass client logger to anacrolix/go-libutp

* Pass logger instead of option

The project now compiles properly when CGO is not enabled.

Additionally, the new argument (now log.Logger instead of
utp.NewSocketOpt) is now required. The tests have been updated to match
this change, and now pass logger.Default to NewUtpSocket.

* Correct function signature of NewUtpSocket

2 years agoFormat code with gofumpt (#724)
deepsource-autofix[bot] [Fri, 11 Feb 2022 11:45:12 +0000 (22:45 +1100)]
Format code with gofumpt (#724)

This commit fixes the style issues introduced in b81470d according to the output
from gofumpt.

Details: https://deepsource.io/gh/anacrolix/torrent/transform/ccafd976-fc9a-4c8a-bbfe-bc36426e79cb/

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2 years agoUpdate .deepsource.toml
DeepSource Bot [Fri, 11 Feb 2022 06:33:32 +0000 (06:33 +0000)]
Update .deepsource.toml

2 years agoCheckout go from github
Matt Joiner [Thu, 10 Feb 2022 04:08:22 +0000 (15:08 +1100)]
Checkout go from github

2 years agoRun tests on tip
Matt Joiner [Thu, 10 Feb 2022 04:03:43 +0000 (15:03 +1100)]
Run tests on tip

2 years agoFix and add caching to gotip CI actions
Matt Joiner [Thu, 10 Feb 2022 04:04:53 +0000 (15:04 +1100)]
Fix and add caching to gotip CI actions

Fix and add caching to gotip CI actions

2 years agogo CI: Use common action and parallel jobs
Matt Joiner [Thu, 10 Feb 2022 04:31:43 +0000 (15:31 +1100)]
go CI: Use common action and parallel jobs

2 years agoAlign Peer stats for webseed peers
Matt Joiner [Mon, 31 Jan 2022 01:53:44 +0000 (12:53 +1100)]
Align Peer stats for webseed peers

2 years agofs/test.sh: Fail if hash does not match
Matt Joiner [Wed, 9 Feb 2022 22:20:23 +0000 (09:20 +1100)]
fs/test.sh: Fail if hash does not match

2 years agoRe-enable CI wasm/js package build checks
Matt Joiner [Wed, 9 Feb 2022 01:47:05 +0000 (12:47 +1100)]
Re-enable CI wasm/js package build checks

2 years agoPull webrtc SCTP Transport wasm support
Matt Joiner [Wed, 9 Feb 2022 01:45:40 +0000 (12:45 +1100)]
Pull webrtc SCTP Transport wasm support

2 years agotest: use `T.TempDir` to create temporary test directory (#718)
Eng Zer Jun [Wed, 2 Feb 2022 07:26:46 +0000 (15:26 +0800)]
test: use `T.TempDir` to create temporary test directory (#718)

The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2 years agofs/test.sh: Add torrents before mounting
Matt Joiner [Mon, 31 Jan 2022 06:35:20 +0000 (17:35 +1100)]
fs/test.sh: Add torrents before mounting

2 years agoMake use of the lazylog branch of anacrolix/log
Matt Joiner [Sat, 22 Jan 2022 22:37:11 +0000 (09:37 +1100)]
Make use of the lazylog branch of anacrolix/log

This avoids significant allocations and overhead.

2 years agoDebug CI godo installation
Matt Joiner [Mon, 31 Jan 2022 01:57:14 +0000 (12:57 +1100)]
Debug CI godo installation

2 years agoContinue on wasm build error
Matt Joiner [Mon, 31 Jan 2022 00:55:07 +0000 (11:55 +1100)]
Continue on wasm build error

2 years agoRun fs/test.sh on 32 bit
Matt Joiner [Mon, 31 Jan 2022 00:52:57 +0000 (11:52 +1100)]
Run fs/test.sh on 32 bit

2 years agofs/test.sh: Use Sintel file so webrtc is tested
Matt Joiner [Mon, 31 Jan 2022 00:48:39 +0000 (11:48 +1100)]
fs/test.sh: Use Sintel file so webrtc is tested

2 years agoRevert "fs/test.sh: Use go run instead of godo"
Matt Joiner [Mon, 31 Jan 2022 00:47:40 +0000 (11:47 +1100)]
Revert "fs/test.sh: Use go run instead of godo"

This reverts commit e8bd16257c40a5b53ac19c4f8b7669eb41b699e2.

2 years agoInclude peer net.Conn endpoint addresses in webrtc conn string
Matt Joiner [Thu, 27 Jan 2022 02:45:36 +0000 (13:45 +1100)]
Include peer net.Conn endpoint addresses in webrtc conn string

2 years agoUse webrtc selected ICE candidate for peer addresses
Matt Joiner [Wed, 12 Jan 2022 03:22:42 +0000 (14:22 +1100)]
Use webrtc selected ICE candidate for peer addresses

2 years agoRename webtorrent source files
Matt Joiner [Wed, 12 Jan 2022 00:46:13 +0000 (11:46 +1100)]
Rename webtorrent source files

2 years agoUse tagged anacrolix/squirrel
Matt Joiner [Thu, 13 Jan 2022 22:22:08 +0000 (09:22 +1100)]
Use tagged anacrolix/squirrel

2 years agoSwitch from missinggo/MultiLess to anacrolix/multiless
Matt Joiner [Wed, 12 Jan 2022 04:07:26 +0000 (15:07 +1100)]
Switch from missinggo/MultiLess to anacrolix/multiless

2 years agoAdd tests for preferred network direction
Matt Joiner [Wed, 12 Jan 2022 04:01:33 +0000 (15:01 +1100)]
Add tests for preferred network direction

2 years agoAdd const noCacheBlobs in sqlite storage benchmarks
Matt Joiner [Mon, 10 Jan 2022 05:20:03 +0000 (16:20 +1100)]
Add const noCacheBlobs in sqlite storage benchmarks

2 years agogo mod tidy
Matt Joiner [Fri, 7 Jan 2022 08:15:59 +0000 (19:15 +1100)]
go mod tidy

2 years agobencode: Support decoding "" as dict key
Matt Joiner [Fri, 7 Jan 2022 08:11:37 +0000 (19:11 +1100)]
bencode: Support decoding "" as dict key

2 years agobencode: Return ErrUnexpectedEOF instead of EOF in the middle of values
Matt Joiner [Fri, 7 Jan 2022 08:06:33 +0000 (19:06 +1100)]
bencode: Return ErrUnexpectedEOF instead of EOF in the middle of values

2 years agoAdd bencode.Decoder.MaxStrLen
Matt Joiner [Fri, 7 Jan 2022 08:05:03 +0000 (19:05 +1100)]
Add bencode.Decoder.MaxStrLen

2 years agofs/test.sh: Use go run instead of godo
Matt Joiner [Mon, 27 Dec 2021 09:27:25 +0000 (20:27 +1100)]
fs/test.sh: Use go run instead of godo

2 years agotracker/udp.ConnClient: Make logging configurable
Matt Joiner [Tue, 28 Dec 2021 00:02:14 +0000 (11:02 +1100)]
tracker/udp.ConnClient: Make logging configurable

Should fix https://github.com/anacrolix/torrent/issues/701#issue-1088130173

2 years agoReject peer requests on data read failures v1.40.1
Matt Joiner [Mon, 27 Dec 2021 11:19:04 +0000 (22:19 +1100)]
Reject peer requests on data read failures

2 years agoReduce some logging
Matt Joiner [Mon, 27 Dec 2021 11:06:52 +0000 (22:06 +1100)]
Reduce some logging

2 years agoFix TestTorrentDroppedDuringResponsiveRead flakiness v1.40.0
Matt Joiner [Mon, 27 Dec 2021 10:16:51 +0000 (21:16 +1100)]
Fix TestTorrentDroppedDuringResponsiveRead flakiness

2 years agoAllow stealing from slower connections within priority classes
Matt Joiner [Thu, 23 Dec 2021 07:44:07 +0000 (18:44 +1100)]
Allow stealing from slower connections within priority classes

2 years agoDynamic outbound max requests
Matt Joiner [Thu, 23 Dec 2021 21:55:57 +0000 (08:55 +1100)]
Dynamic outbound max requests

This might be one solution to https://github.com/anacrolix/torrent/issues/698.

2 years agoInclude requests pending cancel in current request count
Matt Joiner [Thu, 23 Dec 2021 07:55:57 +0000 (18:55 +1100)]
Include requests pending cancel in current request count

This fix a situation where peers might be dropping our requests, and since we depend on all requests being satisfied before re-requesting, we get stuck waiting for the request to be filled.

2 years agoReplace piece availability with frequencies in Torrent status
Matt Joiner [Thu, 23 Dec 2021 06:01:39 +0000 (17:01 +1100)]
Replace piece availability with frequencies in Torrent status

2 years agoRevert go directive to 1.16
Matt Joiner [Thu, 23 Dec 2021 02:45:51 +0000 (13:45 +1100)]
Revert go directive to 1.16

2 years agocmd/torrent: Add --linear-discard
Matt Joiner [Thu, 23 Dec 2021 01:36:53 +0000 (12:36 +1100)]
cmd/torrent: Add --linear-discard

2 years agoFix panic in webseeds updating requests when Torrent is closed
Matt Joiner [Mon, 20 Dec 2021 03:32:37 +0000 (14:32 +1100)]
Fix panic in webseeds updating requests when Torrent is closed

I think this was caused by PieceRequestOrder being removed if all pieces in a storage capacity key are deleted.

2 years agocmd/torrent: Lower burst when there's a download rate limit
Matt Joiner [Mon, 20 Dec 2021 03:30:22 +0000 (14:30 +1100)]
cmd/torrent: Lower burst when there's a download rate limit

2 years agoApply download rate limiter to webseeds
Matt Joiner [Mon, 20 Dec 2021 03:29:43 +0000 (14:29 +1100)]
Apply download rate limiter to webseeds

2 years agoDon't wait for DHT pings
Matt Joiner [Mon, 20 Dec 2021 01:09:28 +0000 (12:09 +1100)]
Don't wait for DHT pings

Fixes a growing backlog of pings if DHT servers are rate-limited in their queries.

2 years agoMake Torrent.cancelRequestsForPiece more efficient
Matt Joiner [Fri, 17 Dec 2021 22:22:18 +0000 (09:22 +1100)]
Make Torrent.cancelRequestsForPiece more efficient

2 years agoUse relative availabilities to determine piece request order
Matt Joiner [Fri, 17 Dec 2021 11:06:21 +0000 (22:06 +1100)]
Use relative availabilities to determine piece request order

Most overhead comes from peers that connect that have everything, and we just increment every single piece's availability. There may be some unresolved determinism with torrents that share the same ordering.

2 years agogofumpt -extra
Matt Joiner [Fri, 17 Dec 2021 21:58:56 +0000 (08:58 +1100)]
gofumpt -extra

2 years agoAdd TestHaveAllThenBitfield
Matt Joiner [Fri, 17 Dec 2021 08:12:10 +0000 (19:12 +1100)]
Add TestHaveAllThenBitfield

The result of a misguided attempt to reduce piece peer availability increment and decrement overhead for have all/full-bitfield and conn closes.

2 years agofix http tracker arg position (#700)
Preston [Thu, 23 Dec 2021 01:34:16 +0000 (09:34 +0800)]
fix http tracker arg position (#700)

Co-authored-by: Bot Git <bot@example.com>
2 years ago* set proxy of websocket tracker to the proxy used by client config
martin [Tue, 14 Dec 2021 15:57:19 +0000 (23:57 +0800)]
* set proxy of websocket tracker to the proxy used by client config

2 years agoFix leaked mmap storage in test v1.39.2
Matt Joiner [Thu, 16 Dec 2021 00:41:31 +0000 (11:41 +1100)]
Fix leaked mmap storage in test

2 years agoFix race in TextPexConnState
Matt Joiner [Thu, 16 Dec 2021 00:07:19 +0000 (11:07 +1100)]
Fix race in TextPexConnState

2 years agoFix go-libutp import when CGO is disabled
Matt Joiner [Wed, 15 Dec 2021 22:44:59 +0000 (09:44 +1100)]
Fix go-libutp import when CGO is disabled

2 years agoTest benchmarks in a separate step and limit executions
Matt Joiner [Thu, 16 Dec 2021 03:19:19 +0000 (14:19 +1100)]
Test benchmarks in a separate step and limit executions

2 years agoFix BenchmarkUpdatePiecePriorities
Matt Joiner [Wed, 15 Dec 2021 08:09:22 +0000 (19:09 +1100)]
Fix BenchmarkUpdatePiecePriorities

2 years agoFix BenchmarkConnectionMainReadLoop
Matt Joiner [Wed, 15 Dec 2021 07:54:47 +0000 (18:54 +1100)]
Fix BenchmarkConnectionMainReadLoop

Broken by piece request order changes.

2 years agoSome optimizations in PieceRequestOrder.Update and item comparisons
Matt Joiner [Wed, 15 Dec 2021 07:07:17 +0000 (18:07 +1100)]
Some optimizations in PieceRequestOrder.Update and item comparisons

2 years agoRetract v1.39.0
Matt Joiner [Mon, 13 Dec 2021 01:16:43 +0000 (12:16 +1100)]
Retract v1.39.0

2 years agoTidy up request heaping comments v1.39.1
Matt Joiner [Mon, 13 Dec 2021 01:12:30 +0000 (12:12 +1100)]
Tidy up request heaping comments

2 years agoAdd Peer.cancelAllRequests for webseedPeer
Matt Joiner [Mon, 13 Dec 2021 01:11:38 +0000 (12:11 +1100)]
Add Peer.cancelAllRequests for webseedPeer

2 years agoUpdate requests after deleting all in some corner cases
Matt Joiner [Mon, 13 Dec 2021 01:09:12 +0000 (12:09 +1100)]
Update requests after deleting all in some corner cases

Choked by non-fast PeerConn, deleted PeerConn. They're not exactly guarded as strictly as they could be, so there's plenty of room for performance improvements here.

2 years agoAssert that a request isn't cancelled before requesting it again
Matt Joiner [Mon, 13 Dec 2021 00:56:57 +0000 (11:56 +1100)]
Assert that a request isn't cancelled before requesting it again

2 years agoUpdate peer requests if not interested when piece priorities change
Matt Joiner [Mon, 13 Dec 2021 00:55:47 +0000 (11:55 +1100)]
Update peer requests if not interested when piece priorities change

This should fix a starvation issue where we don't signal interest to a peer that has us choked.

2 years agoExplicitly use go-libutp synchronous firewall callback
Matt Joiner [Sun, 12 Dec 2021 10:32:29 +0000 (21:32 +1100)]
Explicitly use go-libutp synchronous firewall callback

2 years agoMerge branch 'request-strategy-experiments' v1.39.0
Matt Joiner [Sun, 12 Dec 2021 07:38:33 +0000 (18:38 +1100)]
Merge branch 'request-strategy-experiments'

This should bring in significant performance improvements that fix issues with the peer-requesting that existed from v1.34.0.

2 years agoMore consistent cancellation management
Matt Joiner [Sat, 11 Dec 2021 13:04:06 +0000 (00:04 +1100)]
More consistent cancellation management

2 years agoRemove unused request strategy code
Matt Joiner [Sat, 11 Dec 2021 03:14:24 +0000 (14:14 +1100)]
Remove unused request strategy code

2 years agoDon't group Torrent piece request orders without a storage capacity together
Matt Joiner [Sat, 11 Dec 2021 02:46:29 +0000 (13:46 +1100)]
Don't group Torrent piece request orders without a storage capacity together

2 years agoMinimize allocations due to heap escapes
Matt Joiner [Thu, 9 Dec 2021 10:49:29 +0000 (21:49 +1100)]
Minimize allocations due to heap escapes

2 years agoOnly steal an odd request if the stealer more recently received a chunk
Matt Joiner [Tue, 7 Dec 2021 03:19:44 +0000 (14:19 +1100)]
Only steal an odd request if the stealer more recently received a chunk

This helps break the stealing cycle during endgame, and lets us trickle the request to the peer conn with the best record. It might not be sufficient but works nice in testing so far.

2 years agoOnly use last requested when both requests are already pending
Matt Joiner [Tue, 7 Dec 2021 03:17:43 +0000 (14:17 +1100)]
Only use last requested when both requests are already pending

2 years agoAdd Torrent-level request cancel for consistency
Matt Joiner [Fri, 10 Dec 2021 07:04:45 +0000 (18:04 +1100)]
Add Torrent-level request cancel for consistency

2 years agoUpdate peer cancel assumptions and return
Matt Joiner [Mon, 6 Dec 2021 08:02:29 +0000 (19:02 +1100)]
Update peer cancel assumptions and return

2 years agoImprove panic message
Matt Joiner [Fri, 3 Dec 2021 10:31:40 +0000 (21:31 +1100)]
Improve panic message

2 years agoTry request stealing
Matt Joiner [Fri, 3 Dec 2021 10:30:41 +0000 (21:30 +1100)]
Try request stealing

2 years agoDon't try to request anything without the torrent info
Matt Joiner [Fri, 3 Dec 2021 05:41:52 +0000 (16:41 +1100)]
Don't try to request anything without the torrent info

2 years agoWait between duplicate requests
Matt Joiner [Thu, 2 Dec 2021 02:10:02 +0000 (13:10 +1100)]
Wait between duplicate requests

2 years agoStop iterating pieces when storage is exhausted
Matt Joiner [Wed, 1 Dec 2021 08:31:29 +0000 (19:31 +1100)]
Stop iterating pieces when storage is exhausted

2 years agoUse interfaces to lazily expose the bare minimum inputs to GetRequestablePieces
Matt Joiner [Wed, 1 Dec 2021 08:21:25 +0000 (19:21 +1100)]
Use interfaces to lazily expose the bare minimum inputs to GetRequestablePieces

2 years agoProvide mapping from infohash to Torrent in Input
Matt Joiner [Wed, 1 Dec 2021 04:20:42 +0000 (15:20 +1100)]
Provide mapping from infohash to Torrent in Input

2 years agoImplement piece request ordering with retained state
Matt Joiner [Wed, 1 Dec 2021 03:38:47 +0000 (14:38 +1100)]
Implement piece request ordering with retained state

2 years agoAvoid reflection sorting request pieces
Matt Joiner [Tue, 30 Nov 2021 10:31:32 +0000 (21:31 +1100)]
Avoid reflection sorting request pieces

2 years agoUse indices to lookup pieces for comparison
Matt Joiner [Tue, 30 Nov 2021 04:26:05 +0000 (15:26 +1100)]
Use indices to lookup pieces for comparison

2 years agoCache piece request orderings
Matt Joiner [Tue, 30 Nov 2021 04:18:38 +0000 (15:18 +1100)]
Cache piece request orderings

2 years agoRetract last few minor versions with peer-requesting issues
Matt Joiner [Sun, 12 Dec 2021 07:24:27 +0000 (18:24 +1100)]
Retract last few minor versions with peer-requesting issues

(cherry picked from commit 85a436a5ca75a47a52dcab9648d85f3c3fb53e35)

2 years agoRevert "Link to godocs.io for documentation"
Matt Joiner [Sun, 12 Dec 2021 06:20:02 +0000 (17:20 +1100)]
Revert "Link to godocs.io for documentation"

This reverts commit c2ed60f9f2d100ddd15a38a6c260121a9ad6e893.

2 years agoPartition piece request strategy by storage capacity key
Matt Joiner [Mon, 29 Nov 2021 02:07:18 +0000 (13:07 +1100)]
Partition piece request strategy by storage capacity key

2 years agoDo webseed request updates asynchronously
Matt Joiner [Mon, 6 Dec 2021 08:24:04 +0000 (19:24 +1100)]
Do webseed request updates asynchronously

Fixes a bug where received chunks are immediately requested while still being processed.

2 years agoAdd more identifying information to PeerConn.String
Matt Joiner [Mon, 6 Dec 2021 08:02:55 +0000 (19:02 +1100)]
Add more identifying information to PeerConn.String

Some logs for unexpected protocol violations could be helped by them.

2 years agoAdd generalized decodeJsonByteString and a fuzz target for it
Matt Joiner [Sat, 11 Dec 2021 02:43:25 +0000 (13:43 +1100)]
Add generalized decodeJsonByteString and a fuzz target for it

2 years agoAdd failing bencode fuzz input
Matt Joiner [Fri, 3 Dec 2021 06:15:59 +0000 (17:15 +1100)]
Add failing bencode fuzz input

2 years agoTransmission does not reject cancelled requests
Matt Joiner [Tue, 7 Dec 2021 03:16:02 +0000 (14:16 +1100)]
Transmission does not reject cancelled requests

2 years agoDon't log webseed request context cancellation results
Matt Joiner [Mon, 6 Dec 2021 08:24:48 +0000 (19:24 +1100)]
Don't log webseed request context cancellation results

2 years agobencode: Avoid allocating interface string up front
Matt Joiner [Fri, 10 Dec 2021 04:10:26 +0000 (15:10 +1100)]
bencode: Avoid allocating interface string up front

This should fix crashes during fuzzing. It's not a complete fix, we really want to limit the amount a given Decode can allocate. Maybe Go isn't the right language for this.

2 years agobencode: Fix decoding integer with leading +
Matt Joiner [Fri, 10 Dec 2021 03:29:35 +0000 (14:29 +1100)]
bencode: Fix decoding integer with leading +

2 years agobencode: Enforce dict key ordering
Matt Joiner [Fri, 10 Dec 2021 03:05:00 +0000 (14:05 +1100)]
bencode: Enforce dict key ordering

Fix bencode dict key ordering in HTTP tracker test

2 years agobencode incorrectly parsed integers with leading zeroes
Matt Joiner [Fri, 10 Dec 2021 02:36:53 +0000 (13:36 +1100)]
bencode incorrectly parsed integers with leading zeroes

2 years agoAdd bencode FuzzInterfaceRoundTrip
Matt Joiner [Fri, 10 Dec 2021 02:35:57 +0000 (13:35 +1100)]
Add bencode FuzzInterfaceRoundTrip