]> Sergey Matveev's repositories - btrtrc.git/log
btrtrc.git
2 years agoWrap error with %w in mainReadLoop v1.32.0
Matt Joiner [Thu, 30 Sep 2021 02:01:56 +0000 (12:01 +1000)]
Wrap error with %w in mainReadLoop

2 years agoFix mainReadLoop benchmark
Matt Joiner [Thu, 30 Sep 2021 02:01:34 +0000 (12:01 +1000)]
Fix mainReadLoop benchmark

2 years agoFix scheduling overhead in decoding benchmark
Matt Joiner [Thu, 30 Sep 2021 01:52:50 +0000 (11:52 +1000)]
Fix scheduling overhead in decoding benchmark

2 years agoCombine peer protocol fuzz targets into one file
Matt Joiner [Thu, 30 Sep 2021 01:19:50 +0000 (11:19 +1000)]
Combine peer protocol fuzz targets into one file

2 years agoMore optimizations in peer protocol message decoding
Matt Joiner [Thu, 30 Sep 2021 01:05:01 +0000 (11:05 +1000)]
More optimizations in peer protocol message decoding

2 years agopeer_protocol: Use faster form for Integer.{UnmarshalBinary,Read}
Matt Joiner [Thu, 30 Sep 2021 00:24:03 +0000 (10:24 +1000)]
peer_protocol: Use faster form for Integer.{UnmarshalBinary,Read}

2 years agoRemove an allocation reading message length
Matt Joiner [Wed, 29 Sep 2021 23:59:18 +0000 (09:59 +1000)]
Remove an allocation reading message length

2 years agoTry to avoid allocating buffer
Matt Joiner [Wed, 29 Sep 2021 23:45:50 +0000 (09:45 +1000)]
Try to avoid allocating buffer

2 years agoImprovements to decoder fuzzing
Matt Joiner [Wed, 29 Sep 2021 23:42:54 +0000 (09:42 +1000)]
Improvements to decoder fuzzing

2 years agoAdd some fuzzing in peer_protocol
Matt Joiner [Wed, 29 Sep 2021 23:01:10 +0000 (09:01 +1000)]
Add some fuzzing in peer_protocol

2 years agoLimit decoded bencode string lengths to 32 bits
Matt Joiner [Wed, 29 Sep 2021 06:56:01 +0000 (16:56 +1000)]
Limit decoded bencode string lengths to 32 bits

Found in fuzzing

2 years agoAdd some fuzzing seeds
Matt Joiner [Wed, 29 Sep 2021 02:12:16 +0000 (12:12 +1000)]
Add some fuzzing seeds

2 years agoFix parsing of strings that don't fit in memory
Matt Joiner [Wed, 29 Sep 2021 02:11:58 +0000 (12:11 +1000)]
Fix parsing of strings that don't fit in memory

2 years agoFix fuzz comparison of big.Int
Matt Joiner [Wed, 29 Sep 2021 02:11:17 +0000 (12:11 +1000)]
Fix fuzz comparison of big.Int

2 years agoUpdate fuzz to use new standard library support
Matt Joiner [Wed, 29 Sep 2021 00:13:32 +0000 (10:13 +1000)]
Update fuzz to use new standard library support

2 years agoSimplify bencode.Decoder.parseStringInterface
Matt Joiner [Tue, 28 Sep 2021 23:50:01 +0000 (09:50 +1000)]
Simplify bencode.Decoder.parseStringInterface

2 years agobencode: optimize `(*Decoder).parseStringInterface()` (#659)
YenForYang [Tue, 28 Sep 2021 23:30:35 +0000 (18:30 -0500)]
bencode: optimize `(*Decoder).parseStringInterface()` (#659)

2 years agoInlineable `(*Torrent).BytesMissing()` (#633)
YenForYang [Mon, 27 Sep 2021 02:43:09 +0000 (21:43 -0500)]
Inlineable `(*Torrent).BytesMissing()` (#633)

Honestly a name like `BytesLeft` would have been more suitable, but it's too late for that I guess.

2 years agoInlineable `addrIndex` (#673)
YenForYang [Mon, 27 Sep 2021 02:41:20 +0000 (21:41 -0500)]
Inlineable `addrIndex` (#673)

2 years agoEliminate `expect` dependency (#667)
YenForYang [Thu, 23 Sep 2021 14:10:30 +0000 (09:10 -0500)]
Eliminate `expect` dependency (#667)

2 years agoDefault to peer requesting disabled
Matt Joiner [Tue, 21 Sep 2021 06:07:08 +0000 (16:07 +1000)]
Default to peer requesting disabled

There are some flaws that need to be fixed up before it might be considered the default.

2 years agocmd/torrent: Don't make --stats default to --debug's value
Matt Joiner [Tue, 21 Sep 2021 03:06:06 +0000 (13:06 +1000)]
cmd/torrent: Don't make --stats default to --debug's value

2 years agoRemove unused function
Matt Joiner [Tue, 21 Sep 2021 03:05:16 +0000 (13:05 +1000)]
Remove unused function

2 years agoAdd ClientConfig.AlwaysWantConns
Matt Joiner [Tue, 21 Sep 2021 03:04:51 +0000 (13:04 +1000)]
Add ClientConfig.AlwaysWantConns

2 years agocmd/torrent: Refactor and set progress interval to 3s
Matt Joiner [Mon, 20 Sep 2021 12:01:18 +0000 (22:01 +1000)]
cmd/torrent: Refactor and set progress interval to 3s

2 years agoUse an iterator to skip through dirty chunks
Matt Joiner [Tue, 21 Sep 2021 00:48:15 +0000 (10:48 +1000)]
Use an iterator to skip through dirty chunks

2 years agocmd/torrent: Include download stats on interrupt
Matt Joiner [Mon, 20 Sep 2021 11:33:46 +0000 (21:33 +1000)]
cmd/torrent: Include download stats on interrupt

2 years agoTrack dirty chunks in a single bitmap on Torrent
Matt Joiner [Mon, 20 Sep 2021 08:52:54 +0000 (18:52 +1000)]
Track dirty chunks in a single bitmap on Torrent

2 years agoFix cancellation for global requesting
Matt Joiner [Mon, 20 Sep 2021 05:24:24 +0000 (15:24 +1000)]
Fix cancellation for global requesting

2 years agoMerge branch 'master' into peer-requesting
Matt Joiner [Mon, 20 Sep 2021 05:10:19 +0000 (15:10 +1000)]
Merge branch 'master' into peer-requesting

2 years agoReduce the diff to master and add peerRequesting feature const
Matt Joiner [Mon, 20 Sep 2021 05:09:28 +0000 (15:09 +1000)]
Reduce the diff to master and add peerRequesting feature const

2 years agocmd/torrent: Dump download statistics
Matt Joiner [Mon, 20 Sep 2021 04:17:58 +0000 (14:17 +1000)]
cmd/torrent: Dump download statistics

2 years agoFilter next requests application for peer state changes
Matt Joiner [Mon, 20 Sep 2021 04:16:55 +0000 (14:16 +1000)]
Filter next requests application for peer state changes

2 years agoAdd NOTES
Matt Joiner [Mon, 20 Sep 2021 02:31:44 +0000 (12:31 +1000)]
Add NOTES

2 years agoStore peer requests in a bitmap
Matt Joiner [Sun, 19 Sep 2021 05:16:37 +0000 (15:16 +1000)]
Store peer requests in a bitmap

2 years agoWhen updating requests, only tickle writer if there are no requests pending
Matt Joiner [Sat, 18 Sep 2021 11:01:06 +0000 (21:01 +1000)]
When updating requests, only tickle writer if there are no requests pending

Just a quick attempt to reduce load for now. There'll be a much better way to do this.

2 years agoDo peer requests separately for each peer
Matt Joiner [Sat, 18 Sep 2021 10:34:14 +0000 (20:34 +1000)]
Do peer requests separately for each peer

2 years agoAdd choking/allowed fast checks to Peer.shouldRequest
Matt Joiner [Sat, 18 Sep 2021 09:00:21 +0000 (19:00 +1000)]
Add choking/allowed fast checks to Peer.shouldRequest

2 years agoExport request_strategy.GetRequestablePieces
Matt Joiner [Sat, 18 Sep 2021 08:57:50 +0000 (18:57 +1000)]
Export request_strategy.GetRequestablePieces

2 years agogoimports doing its thing
Matt Joiner [Sat, 18 Sep 2021 08:54:54 +0000 (18:54 +1000)]
goimports doing its thing

2 years agoAdd some tests verifying request map ordering
Matt Joiner [Sat, 18 Sep 2021 03:50:55 +0000 (13:50 +1000)]
Add some tests verifying request map ordering

2 years agoUpdate anacrolix/args for bool Flag fix
Matt Joiner [Sat, 18 Sep 2021 03:23:36 +0000 (13:23 +1000)]
Update anacrolix/args for bool Flag fix

2 years agorequesting: Remove some obsoleted condition checks
Matt Joiner [Thu, 16 Sep 2021 07:10:07 +0000 (17:10 +1000)]
requesting: Remove some obsoleted condition checks

These shouldn't be necessary since peers were pruned for the ability to allocate chunks to the current piece.

2 years agorequest_strategy.Peer.canRequestPiece performance: Reorder conditions
Matt Joiner [Thu, 16 Sep 2021 07:08:59 +0000 (17:08 +1000)]
request_strategy.Peer.canRequestPiece performance: Reorder conditions

2 years agobencode: get type of `big.Int` without creating instance (#651)
YenForYang [Sat, 18 Sep 2021 02:44:47 +0000 (21:44 -0500)]
bencode: get type of `big.Int` without creating instance (#651)

Nope, it doesn't really matter. But anyway, see https://github.com/golang/crypto/blob/c084706c2272f3d44b722e988e70d4a58e60e7f4/cryptobyte/asn1.go#L267

2 years agobencode: simplify getting `marshalerType` and `unmarshalerType` (#652)
YenForYang [Sat, 18 Sep 2021 02:43:53 +0000 (21:43 -0500)]
bencode: simplify getting `marshalerType` and `unmarshalerType` (#652)

2 years agobencode: Simplify `(*Decoder).parseListInterface()` (#656)
YenForYang [Sat, 18 Sep 2021 02:42:20 +0000 (21:42 -0500)]
bencode: Simplify `(*Decoder).parseListInterface()` (#656)

Preserve as much type as possible (it'll be converted to an `interface{}` anyway, but we can return `[]interface{}` instead of `{}interface` here).

2 years agoPrevent allocation when checking interface (#649)
YenForYang [Sat, 18 Sep 2021 02:36:25 +0000 (21:36 -0500)]
Prevent allocation when checking interface (#649)

2 years agometainfo: Remove reflection dependency for `Piece.Hash()` (#661)
YenForYang [Sat, 18 Sep 2021 01:35:21 +0000 (20:35 -0500)]
metainfo: Remove reflection dependency for `Piece.Hash()` (#661)

`copy`  will copy exactly `HashSize` bytes here.

2 years agolockWithDeferreds: inlineable Unlock() (#603)
YenForYang [Fri, 17 Sep 2021 03:08:00 +0000 (22:08 -0500)]
lockWithDeferreds: inlineable Unlock() (#603)

2 years agorequesting: Filter peers that can't be allocated chunks
Matt Joiner [Thu, 16 Sep 2021 00:47:27 +0000 (10:47 +1000)]
requesting: Filter peers that can't be allocated chunks

2 years agoRejig waits between doRequests
Matt Joiner [Wed, 15 Sep 2021 14:12:58 +0000 (00:12 +1000)]
Rejig waits between doRequests

2 years agoPerformance improvements in piece requesting
Matt Joiner [Wed, 15 Sep 2021 14:12:24 +0000 (00:12 +1000)]
Performance improvements in piece requesting

2 years agoChange torrent capacity to not return a pointer
Matt Joiner [Wed, 15 Sep 2021 00:30:37 +0000 (10:30 +1000)]
Change torrent capacity to not return a pointer

It's an unnecessary complication for a storage implementer.

2 years agoSimplify PIece.iterUndirtiedChunks
Matt Joiner [Sat, 11 Sep 2021 11:17:47 +0000 (21:17 +1000)]
Simplify PIece.iterUndirtiedChunks

2 years agoPre-allocate request strategy pieces
Matt Joiner [Sat, 11 Sep 2021 11:17:31 +0000 (21:17 +1000)]
Pre-allocate request strategy pieces

2 years agocmd/torrent: Use anacrolix/args and merge several other cmds in as subcommands
Matt Joiner [Tue, 7 Sep 2021 02:08:31 +0000 (12:08 +1000)]
cmd/torrent: Use anacrolix/args and merge several other cmds in as subcommands

2 years agoInlineable `(*Torrent).Seeding()` (#626)
YenForYang [Wed, 15 Sep 2021 00:28:14 +0000 (19:28 -0500)]
Inlineable `(*Torrent).Seeding()` (#626)

2 years agoInlineable `(*Torrent).SetDisplayName()` (#625)
YenForYang [Wed, 15 Sep 2021 00:27:52 +0000 (19:27 -0500)]
Inlineable `(*Torrent).SetDisplayName()` (#625)

2 years agoInlineable `(*Torrent).PieceState()` (#621)
YenForYang [Wed, 15 Sep 2021 00:14:28 +0000 (19:14 -0500)]
Inlineable `(*Torrent).PieceState()` (#621)

2 years agoInlineable `(*Torrent).AddPeers()` (#622)
YenForYang [Wed, 15 Sep 2021 00:14:07 +0000 (19:14 -0500)]
Inlineable `(*Torrent).AddPeers()` (#622)

2 years agoInlineable `DownloadPieces()` and `CancelPieces()` (#623)
YenForYang [Wed, 15 Sep 2021 00:13:46 +0000 (19:13 -0500)]
Inlineable `DownloadPieces()` and `CancelPieces()` (#623)

2 years agoCreate default constructor for Client (#567)
YenForYang [Tue, 14 Sep 2021 13:01:20 +0000 (08:01 -0500)]
Create default constructor for Client (#567)

Allow for certain values to always be properly initialized on construction -- namely the maps for now. I'm currently working on a change that requires a baseline constructor; this change would make the use of `chansync.BroadcastCond` and `chansync.SetOnce` obsolete -- i.e. one can have channel members without worrying about proper initialization/destruction of a `chan struct{}`.

As for why `makeClient` returns a value instead of a pointer: returning a value gives us more options -- you can always take a pointer from a value later on cheaply, and have things moved to the heap if they weren't already. The same can't be said about getting a value back from a pointer. GC also could potentially have less work to do. Plus I personally find ownership to be an important concept (semi-borrowed from rust) -- use of values make ownership clear.

2 years agoClarify maximum value of "metadata_size" (#609)
YenForYang [Tue, 14 Sep 2021 12:36:19 +0000 (07:36 -0500)]
Clarify maximum value of "metadata_size" (#609)

2 years agoInlineable `(*File).BytesCompleted()` (#612)
YenForYang [Tue, 14 Sep 2021 12:11:35 +0000 (07:11 -0500)]
Inlineable `(*File).BytesCompleted()` (#612)

2 years agoOptimize `LoopbackListenHost()` (#615)
YenForYang [Tue, 14 Sep 2021 11:54:37 +0000 (06:54 -0500)]
Optimize `LoopbackListenHost()` (#615)

2 years agoInlineable (*reader).SetReadAhead and (*reader).Close (#594)
YenForYang [Tue, 14 Sep 2021 04:05:37 +0000 (23:05 -0500)]
Inlineable (*reader).SetReadAhead and (*reader).Close (#594)

Small fixes that prevent inlining of public functions

2 years agoDrop bradfitz/iter dependency (#605)
YenForYang [Tue, 14 Sep 2021 03:46:50 +0000 (22:46 -0500)]
Drop bradfitz/iter dependency (#605)

* Drop bradfitz/iter dependency

`range iter.N` looks nice and doesn't allocate, but unfortunately using a `range` expression blocks a function from being inlined wherever it's used (for now). It's not that we need inlining in all cases, but I do think a C-style for loop looks just as nice and is probably clearer to the majority. There also aren't any clear disadvantages to changing (unless you just happen to dislike the look of C)

* Update misc_test.go

* Update rlreader_test.go

* Update torrent_test.go

* Update bench_test.go

* Update client_test.go

* Update iplist_test.go

* Update mse_test.go

* Update peerconn_test.go

* Update peerconn.go

* Update order_test.go

* Update decoder_test.go

* Update main.go

* Update bench-piece-mark-complete.go

* Update main.go

* Update torrent.go

* Update iplist_test.go

* Update main.go

2 years agoCheaper byte to string conversion (#602)
YenForYang [Tue, 14 Sep 2021 00:41:04 +0000 (19:41 -0500)]
Cheaper byte to string conversion (#602)

Revamped https://github.com/anacrolix/torrent/pull/587

2 years agoInlineable reader.Seek and no-lock return for bad whence (#577)
YenForYang [Mon, 13 Sep 2021 12:52:58 +0000 (07:52 -0500)]
Inlineable reader.Seek and no-lock return for bad whence (#577)

* Inlineable reader.Seek and no-lock return for bad whence

Couldn't find an elegant way to early exit on bad whence. Using a closure could reduce code duplication slightly, but it's overkill for only 3 cases.

Note that returning 0 on an invalid whence is the behavior of `strings.Reader` and `bytes.Reader`.

* Update reader.go

2 years agoFix usages of eachListener() and make inlineable (#599)
YenForYang [Mon, 13 Sep 2021 11:35:51 +0000 (06:35 -0500)]
Fix usages of eachListener() and make inlineable (#599)

2 years agoInlineable methods to access File piecePriority (#601)
YenForYang [Mon, 13 Sep 2021 11:17:54 +0000 (06:17 -0500)]
Inlineable methods to access File piecePriority (#601)

2 years agoSimplify (*Torrent).gotMetainfo (#581)
YenForYang [Mon, 13 Sep 2021 01:41:11 +0000 (20:41 -0500)]
Simplify (*Torrent).gotMetainfo (#581)

2 years agoRemove xerrors dependency (#600)
YenForYang [Mon, 13 Sep 2021 01:38:08 +0000 (20:38 -0500)]
Remove xerrors dependency (#600)

* Remove xerrors dependency

* Update main.go

2 years agoReorder fields to reduce padding in reader (#597)
YenForYang [Mon, 13 Sep 2021 01:36:35 +0000 (20:36 -0500)]
Reorder fields to reduce padding in reader (#597)

Reorder some of the fields to reduce the size of `reader`. Not much of a change -- original size was 104 bytes, with changes (moving bool's to the end) the size is 96 bytes. There's also the whole cache-friendly bit -- I tried reordering fields to help here, but I doubt it makes much of a difference.

Some other notes: `sync.Locker` is a 16 byte field. I suggest changing this to something like `*sync.RWMutex` or `*lockWithDeferreds`. This would not only reduce size but would allow access to `RLock`/`RUnlock`.
`pieceRange` could also be shrunk to 8 bytes, but I feel like I'm probably asking for too much lol.

2 years agoUpdate ReadCloser interface check to ReadSeekCloser (#595)
YenForYang [Sun, 12 Sep 2021 04:14:07 +0000 (23:14 -0500)]
Update ReadCloser interface check to ReadSeekCloser (#595)

2 years agoFix http announce of infohash containing ' ' bytes (#591)
Matt Joiner [Sun, 12 Sep 2021 04:11:59 +0000 (14:11 +1000)]
Fix http announce of infohash containing ' ' bytes (#591)

Fixes https://github.com/anacrolix/torrent/issues/534.

2 years agoDon't call Client onClose handlers until torrent storages are closed
Matt Joiner [Fri, 10 Sep 2021 04:15:45 +0000 (14:15 +1000)]
Don't call Client onClose handlers until torrent storages are closed

2 years agoFix Torrent.Drop to wait for storage close without client lock
Matt Joiner [Fri, 10 Sep 2021 04:14:57 +0000 (14:14 +1000)]
Fix Torrent.Drop to wait for storage close without client lock

2 years agoAvoid allocating memory when checking interface (#588)
YenForYang [Fri, 10 Sep 2021 22:26:43 +0000 (17:26 -0500)]
Avoid allocating memory when checking interface (#588)

Not a big deal, but there isn't a need to allocate an object to check that `*Node` satisfies `bencode.Unmarshaler`.

2 years agoOptimize peersForPieces allocations for make and sorting
Matt Joiner [Fri, 10 Sep 2021 13:06:23 +0000 (23:06 +1000)]
Optimize peersForPieces allocations for make and sorting

2 years agoOptimize requesting peerId Uintptr allocation
Matt Joiner [Fri, 10 Sep 2021 13:02:20 +0000 (23:02 +1000)]
Optimize requesting peerId Uintptr allocation

2 years agoDisable ensureValidSortedPeersForPieceRequests
Matt Joiner [Fri, 10 Sep 2021 12:13:39 +0000 (22:13 +1000)]
Disable ensureValidSortedPeersForPieceRequests

Should approximately half the contribution of the parent function to overhead.

2 years agoOptimize sortPeersForPiece in allocatePendingChunks
Matt Joiner [Fri, 10 Sep 2021 12:12:08 +0000 (22:12 +1000)]
Optimize sortPeersForPiece in allocatePendingChunks

2 years agoOptimize Torrent.worstBadConn
Matt Joiner [Fri, 10 Sep 2021 11:48:44 +0000 (21:48 +1000)]
Optimize Torrent.worstBadConn

2 years agoRemove reader.opMu
Matt Joiner [Fri, 10 Sep 2021 10:58:11 +0000 (20:58 +1000)]
Remove reader.opMu

https://github.com/anacrolix/torrent/issues/553

2 years agogo fmt ./...
Matt Joiner [Thu, 9 Sep 2021 12:51:24 +0000 (22:51 +1000)]
go fmt ./...

2 years agoFix possible misuse of reflect.StringHeader
Matt Joiner [Thu, 9 Sep 2021 12:49:22 +0000 (22:49 +1000)]
Fix possible misuse of reflect.StringHeader

2 years agouseTorrentSources: fix http request resource leak (#562)
YenForYang [Thu, 9 Sep 2021 12:39:12 +0000 (07:39 -0500)]
useTorrentSources: fix http request resource leak (#562)

2 years agoClarify ownership of (*Torrent).chunkPool (#583)
YenForYang [Thu, 9 Sep 2021 12:27:16 +0000 (07:27 -0500)]
Clarify ownership of (*Torrent).chunkPool (#583)

Basically bind the lifetime of chunkPool to the torrent by using `sync.Pool` in lieu of `*sync.Pool`. Gives the GC ever so slightly less work to do.

2 years agoDon't readahead until a read occurs
Matt Joiner [Thu, 9 Sep 2021 12:19:29 +0000 (22:19 +1000)]
Don't readahead until a read occurs

2 years agoMake readahead algorithm linear
Matt Joiner [Thu, 9 Sep 2021 12:16:24 +0000 (22:16 +1000)]
Make readahead algorithm linear

2 years agoAdd default sqrt readahead algorithm
Matt Joiner [Thu, 9 Sep 2021 10:55:09 +0000 (20:55 +1000)]
Add default sqrt readahead algorithm

2 years agoExtract common Torrent.newReader
Matt Joiner [Thu, 9 Sep 2021 07:41:12 +0000 (17:41 +1000)]
Extract common Torrent.newReader

2 years agoAdd Webtor to downstream projects
Matt Joiner [Thu, 9 Sep 2021 12:12:29 +0000 (22:12 +1000)]
Add Webtor to downstream projects

2 years agoFix unit tests littering working directory
Matt Joiner [Thu, 9 Sep 2021 02:58:07 +0000 (12:58 +1000)]
Fix unit tests littering working directory

2 years agoUpdate README.md
Matt Joiner [Wed, 8 Sep 2021 03:46:03 +0000 (13:46 +1000)]
Update README.md

2 years agoUpdate README.md
Matt Joiner [Wed, 8 Sep 2021 03:45:25 +0000 (13:45 +1000)]
Update README.md

2 years agoDrop xerrors and missinggo/slices dependency for Client (#572)
YenForYang [Tue, 7 Sep 2021 23:34:06 +0000 (18:34 -0500)]
Drop xerrors and missinggo/slices dependency for Client (#572)

* Drop xerrors and missinggo/slices dependency for Client

Made `BadPeerIPs` inlineable since I changed `badPeerIPsLocked` anyway. Also eliminated an extra torrentsAsSlice copy since Go wasn't smart enough to figure it out.

2 years agoImprove Closing (#559)
Zilog8 [Sun, 5 Sep 2021 01:33:41 +0000 (21:33 -0400)]
Improve Closing (#559)

- Prevent double-closing in cmd/torrent
- Move async closing from storage to torrents, and wait on them to finish tidying up before returning/exiting.