]> Sergey Matveev's repositories - btrtrc.git/log
btrtrc.git
2 years agoExtract pendingRequests
Matt Joiner [Sat, 9 Oct 2021 08:00:58 +0000 (19:00 +1100)]
Extract pendingRequests

2 years agoAdd reasons for updateRequests to be triggered
Matt Joiner [Fri, 8 Oct 2021 22:14:57 +0000 (09:14 +1100)]
Add reasons for updateRequests to be triggered

2 years agoAdd pprof labels for request updates and remove Client-wide requester
Matt Joiner [Fri, 8 Oct 2021 02:53:36 +0000 (13:53 +1100)]
Add pprof labels for request updates and remove Client-wide requester

2 years agoRemove peer piece inclination and piece request order
Matt Joiner [Fri, 8 Oct 2021 01:41:25 +0000 (12:41 +1100)]
Remove peer piece inclination and piece request order

These are vestigial data structures used with old request strategy implementations.

2 years agoChange peer requesting to spread requests out evenly
Matt Joiner [Thu, 7 Oct 2021 06:31:10 +0000 (17:31 +1100)]
Change peer requesting to spread requests out evenly

2 years agoLog handshook conn run errors with level warning
Matt Joiner [Tue, 19 Oct 2021 02:37:46 +0000 (13:37 +1100)]
Log handshook conn run errors with level warning

(cherry picked from commit 7ca7890e9f30307a1f4942e6d987791a142c3cb3)

2 years agoEnding a conn because we don't like its ID is not an error
Matt Joiner [Tue, 19 Oct 2021 02:37:18 +0000 (13:37 +1100)]
Ending a conn because we don't like its ID is not an error

(cherry picked from commit 2a5ba846cef9b4e035b3b88698f3e7f7d9d79251)

2 years agoFix incomplete Integer.Read
Matt Joiner [Tue, 19 Oct 2021 02:13:23 +0000 (13:13 +1100)]
Fix incomplete Integer.Read

Possibly on read buffer boundaries, this would have caused errant io.ErrUnexpectedEOF.

(cherry picked from commit 22569449dad48e1c5cdeac171625585f8ca976c9)

2 years agoOnly allow chunk size to be set for new Torrents
Matt Joiner [Sun, 10 Oct 2021 00:22:29 +0000 (11:22 +1100)]
Only allow chunk size to be set for new Torrents

(cherry picked from commit 09e73e9fce139cd78da48c442e8610501b6fd26b)

2 years agoUpdate piece priority after processing piece hash result
Matt Joiner [Fri, 8 Oct 2021 23:26:52 +0000 (10:26 +1100)]
Update piece priority after processing piece hash result

Seems to avoid some silly performance issues in triggering request updates due to a piece still pending while being marked.

2 years agoPieces being marked should have priority none
Matt Joiner [Sat, 9 Oct 2021 03:20:54 +0000 (14:20 +1100)]
Pieces being marked should have priority none

Also reorder the checks a bit to favour the cheaper ones.

2 years agoAdd DisableInitialPieceCheck option (#677)
Simon Kirsten [Mon, 11 Oct 2021 00:33:40 +0000 (02:33 +0200)]
Add DisableInitialPieceCheck option (#677)

2 years agoUpdate requests on unchoke
Matt Joiner [Thu, 7 Oct 2021 06:20:04 +0000 (17:20 +1100)]
Update requests on unchoke

2 years agocmd/torrent: Ensure envpprof.Stop runs before exit
Matt Joiner [Thu, 7 Oct 2021 06:00:39 +0000 (17:00 +1100)]
cmd/torrent: Ensure envpprof.Stop runs before exit

2 years agogo mod tidy
Matt Joiner [Thu, 7 Oct 2021 02:56:45 +0000 (13:56 +1100)]
go mod tidy

Fixes broken go doc: https://github.com/golang/go/issues/36811#issuecomment-937401816

2 years agoAdd TorrentStats.PiecesComplete
Matt Joiner [Thu, 7 Oct 2021 02:50:13 +0000 (13:50 +1100)]
Add TorrentStats.PiecesComplete

Fixes https://github.com/anacrolix/torrent/issues/352.

2 years agoAdd to tests for Torrent.Complete v1.33.0
Matt Joiner [Thu, 7 Oct 2021 02:38:29 +0000 (13:38 +1100)]
Add to tests for Torrent.Complete

2 years agoReturn errors from Client.Close
Matt Joiner [Thu, 7 Oct 2021 02:31:08 +0000 (13:31 +1100)]
Return errors from Client.Close

2 years agoFix comment that seems to mess with GoLand's package structure
Matt Joiner [Thu, 7 Oct 2021 01:09:52 +0000 (12:09 +1100)]
Fix comment that seems to mess with GoLand's package structure

2 years agoAdd a flag for Complete
Matt Joiner [Thu, 7 Oct 2021 01:09:20 +0000 (12:09 +1100)]
Add a flag for Complete

2 years agoUpgrade Torrent.GotInfo
Matt Joiner [Thu, 7 Oct 2021 00:50:30 +0000 (11:50 +1100)]
Upgrade Torrent.GotInfo

2 years agoUpdate chansync
Matt Joiner [Thu, 7 Oct 2021 00:45:09 +0000 (11:45 +1100)]
Update chansync

2 years agoApply some lints from GoLand
Matt Joiner [Wed, 6 Oct 2021 07:43:15 +0000 (18:43 +1100)]
Apply some lints from GoLand

2 years agoRemove old-style build tags
Matt Joiner [Wed, 6 Oct 2021 07:36:25 +0000 (18:36 +1100)]
Remove old-style build tags

2 years agoSwitch Peer.PieceAllowedFast and several request strategy inputs to raw roaring.Bitmaps
Matt Joiner [Tue, 5 Oct 2021 09:06:23 +0000 (20:06 +1100)]
Switch Peer.PieceAllowedFast and several request strategy inputs to raw roaring.Bitmaps

This is in preparation to support encoding request strategy run inputs for benchmarking.

2 years agogo fmt ./...
Matt Joiner [Tue, 5 Oct 2021 06:48:52 +0000 (17:48 +1100)]
go fmt ./...

2 years agoChange Peer._peerPieces to use raw roaring Bitmap type
Matt Joiner [Tue, 5 Oct 2021 06:48:34 +0000 (17:48 +1100)]
Change Peer._peerPieces to use raw roaring Bitmap type

The wrapper type was from when roaring didn't support zero-alloc initialization.

2 years agoWrap error with %w in mainReadLoop v1.32.0
Matt Joiner [Thu, 30 Sep 2021 02:01:56 +0000 (12:01 +1000)]
Wrap error with %w in mainReadLoop

2 years agoFix mainReadLoop benchmark
Matt Joiner [Thu, 30 Sep 2021 02:01:34 +0000 (12:01 +1000)]
Fix mainReadLoop benchmark

2 years agoFix scheduling overhead in decoding benchmark
Matt Joiner [Thu, 30 Sep 2021 01:52:50 +0000 (11:52 +1000)]
Fix scheduling overhead in decoding benchmark

2 years agoCombine peer protocol fuzz targets into one file
Matt Joiner [Thu, 30 Sep 2021 01:19:50 +0000 (11:19 +1000)]
Combine peer protocol fuzz targets into one file

2 years agoMore optimizations in peer protocol message decoding
Matt Joiner [Thu, 30 Sep 2021 01:05:01 +0000 (11:05 +1000)]
More optimizations in peer protocol message decoding

2 years agopeer_protocol: Use faster form for Integer.{UnmarshalBinary,Read}
Matt Joiner [Thu, 30 Sep 2021 00:24:03 +0000 (10:24 +1000)]
peer_protocol: Use faster form for Integer.{UnmarshalBinary,Read}

2 years agoRemove an allocation reading message length
Matt Joiner [Wed, 29 Sep 2021 23:59:18 +0000 (09:59 +1000)]
Remove an allocation reading message length

2 years agoTry to avoid allocating buffer
Matt Joiner [Wed, 29 Sep 2021 23:45:50 +0000 (09:45 +1000)]
Try to avoid allocating buffer

2 years agoImprovements to decoder fuzzing
Matt Joiner [Wed, 29 Sep 2021 23:42:54 +0000 (09:42 +1000)]
Improvements to decoder fuzzing

2 years agoAdd some fuzzing in peer_protocol
Matt Joiner [Wed, 29 Sep 2021 23:01:10 +0000 (09:01 +1000)]
Add some fuzzing in peer_protocol

2 years agoLimit decoded bencode string lengths to 32 bits
Matt Joiner [Wed, 29 Sep 2021 06:56:01 +0000 (16:56 +1000)]
Limit decoded bencode string lengths to 32 bits

Found in fuzzing

2 years agoAdd some fuzzing seeds
Matt Joiner [Wed, 29 Sep 2021 02:12:16 +0000 (12:12 +1000)]
Add some fuzzing seeds

2 years agoFix parsing of strings that don't fit in memory
Matt Joiner [Wed, 29 Sep 2021 02:11:58 +0000 (12:11 +1000)]
Fix parsing of strings that don't fit in memory

2 years agoFix fuzz comparison of big.Int
Matt Joiner [Wed, 29 Sep 2021 02:11:17 +0000 (12:11 +1000)]
Fix fuzz comparison of big.Int

2 years agoUpdate fuzz to use new standard library support
Matt Joiner [Wed, 29 Sep 2021 00:13:32 +0000 (10:13 +1000)]
Update fuzz to use new standard library support

2 years agoSimplify bencode.Decoder.parseStringInterface
Matt Joiner [Tue, 28 Sep 2021 23:50:01 +0000 (09:50 +1000)]
Simplify bencode.Decoder.parseStringInterface

2 years agobencode: optimize `(*Decoder).parseStringInterface()` (#659)
YenForYang [Tue, 28 Sep 2021 23:30:35 +0000 (18:30 -0500)]
bencode: optimize `(*Decoder).parseStringInterface()` (#659)

2 years agoInlineable `(*Torrent).BytesMissing()` (#633)
YenForYang [Mon, 27 Sep 2021 02:43:09 +0000 (21:43 -0500)]
Inlineable `(*Torrent).BytesMissing()` (#633)

Honestly a name like `BytesLeft` would have been more suitable, but it's too late for that I guess.

2 years agoInlineable `addrIndex` (#673)
YenForYang [Mon, 27 Sep 2021 02:41:20 +0000 (21:41 -0500)]
Inlineable `addrIndex` (#673)

2 years agoEliminate `expect` dependency (#667)
YenForYang [Thu, 23 Sep 2021 14:10:30 +0000 (09:10 -0500)]
Eliminate `expect` dependency (#667)

2 years agoDefault to peer requesting disabled
Matt Joiner [Tue, 21 Sep 2021 06:07:08 +0000 (16:07 +1000)]
Default to peer requesting disabled

There are some flaws that need to be fixed up before it might be considered the default.

2 years agocmd/torrent: Don't make --stats default to --debug's value
Matt Joiner [Tue, 21 Sep 2021 03:06:06 +0000 (13:06 +1000)]
cmd/torrent: Don't make --stats default to --debug's value

2 years agoRemove unused function
Matt Joiner [Tue, 21 Sep 2021 03:05:16 +0000 (13:05 +1000)]
Remove unused function

2 years agoAdd ClientConfig.AlwaysWantConns
Matt Joiner [Tue, 21 Sep 2021 03:04:51 +0000 (13:04 +1000)]
Add ClientConfig.AlwaysWantConns

2 years agocmd/torrent: Refactor and set progress interval to 3s
Matt Joiner [Mon, 20 Sep 2021 12:01:18 +0000 (22:01 +1000)]
cmd/torrent: Refactor and set progress interval to 3s

2 years agoUse an iterator to skip through dirty chunks
Matt Joiner [Tue, 21 Sep 2021 00:48:15 +0000 (10:48 +1000)]
Use an iterator to skip through dirty chunks

2 years agocmd/torrent: Include download stats on interrupt
Matt Joiner [Mon, 20 Sep 2021 11:33:46 +0000 (21:33 +1000)]
cmd/torrent: Include download stats on interrupt

2 years agoTrack dirty chunks in a single bitmap on Torrent
Matt Joiner [Mon, 20 Sep 2021 08:52:54 +0000 (18:52 +1000)]
Track dirty chunks in a single bitmap on Torrent

2 years agoFix cancellation for global requesting
Matt Joiner [Mon, 20 Sep 2021 05:24:24 +0000 (15:24 +1000)]
Fix cancellation for global requesting

2 years agoMerge branch 'master' into peer-requesting
Matt Joiner [Mon, 20 Sep 2021 05:10:19 +0000 (15:10 +1000)]
Merge branch 'master' into peer-requesting

2 years agoReduce the diff to master and add peerRequesting feature const
Matt Joiner [Mon, 20 Sep 2021 05:09:28 +0000 (15:09 +1000)]
Reduce the diff to master and add peerRequesting feature const

2 years agocmd/torrent: Dump download statistics
Matt Joiner [Mon, 20 Sep 2021 04:17:58 +0000 (14:17 +1000)]
cmd/torrent: Dump download statistics

2 years agoFilter next requests application for peer state changes
Matt Joiner [Mon, 20 Sep 2021 04:16:55 +0000 (14:16 +1000)]
Filter next requests application for peer state changes

2 years agoAdd NOTES
Matt Joiner [Mon, 20 Sep 2021 02:31:44 +0000 (12:31 +1000)]
Add NOTES

2 years agoStore peer requests in a bitmap
Matt Joiner [Sun, 19 Sep 2021 05:16:37 +0000 (15:16 +1000)]
Store peer requests in a bitmap

2 years agoWhen updating requests, only tickle writer if there are no requests pending
Matt Joiner [Sat, 18 Sep 2021 11:01:06 +0000 (21:01 +1000)]
When updating requests, only tickle writer if there are no requests pending

Just a quick attempt to reduce load for now. There'll be a much better way to do this.

2 years agoDo peer requests separately for each peer
Matt Joiner [Sat, 18 Sep 2021 10:34:14 +0000 (20:34 +1000)]
Do peer requests separately for each peer

2 years agoAdd choking/allowed fast checks to Peer.shouldRequest
Matt Joiner [Sat, 18 Sep 2021 09:00:21 +0000 (19:00 +1000)]
Add choking/allowed fast checks to Peer.shouldRequest

2 years agoExport request_strategy.GetRequestablePieces
Matt Joiner [Sat, 18 Sep 2021 08:57:50 +0000 (18:57 +1000)]
Export request_strategy.GetRequestablePieces

2 years agogoimports doing its thing
Matt Joiner [Sat, 18 Sep 2021 08:54:54 +0000 (18:54 +1000)]
goimports doing its thing

2 years agoAdd some tests verifying request map ordering
Matt Joiner [Sat, 18 Sep 2021 03:50:55 +0000 (13:50 +1000)]
Add some tests verifying request map ordering

2 years agoUpdate anacrolix/args for bool Flag fix
Matt Joiner [Sat, 18 Sep 2021 03:23:36 +0000 (13:23 +1000)]
Update anacrolix/args for bool Flag fix

2 years agorequesting: Remove some obsoleted condition checks
Matt Joiner [Thu, 16 Sep 2021 07:10:07 +0000 (17:10 +1000)]
requesting: Remove some obsoleted condition checks

These shouldn't be necessary since peers were pruned for the ability to allocate chunks to the current piece.

2 years agorequest_strategy.Peer.canRequestPiece performance: Reorder conditions
Matt Joiner [Thu, 16 Sep 2021 07:08:59 +0000 (17:08 +1000)]
request_strategy.Peer.canRequestPiece performance: Reorder conditions

2 years agobencode: get type of `big.Int` without creating instance (#651)
YenForYang [Sat, 18 Sep 2021 02:44:47 +0000 (21:44 -0500)]
bencode: get type of `big.Int` without creating instance (#651)

Nope, it doesn't really matter. But anyway, see https://github.com/golang/crypto/blob/c084706c2272f3d44b722e988e70d4a58e60e7f4/cryptobyte/asn1.go#L267

2 years agobencode: simplify getting `marshalerType` and `unmarshalerType` (#652)
YenForYang [Sat, 18 Sep 2021 02:43:53 +0000 (21:43 -0500)]
bencode: simplify getting `marshalerType` and `unmarshalerType` (#652)

2 years agobencode: Simplify `(*Decoder).parseListInterface()` (#656)
YenForYang [Sat, 18 Sep 2021 02:42:20 +0000 (21:42 -0500)]
bencode: Simplify `(*Decoder).parseListInterface()` (#656)

Preserve as much type as possible (it'll be converted to an `interface{}` anyway, but we can return `[]interface{}` instead of `{}interface` here).

2 years agoPrevent allocation when checking interface (#649)
YenForYang [Sat, 18 Sep 2021 02:36:25 +0000 (21:36 -0500)]
Prevent allocation when checking interface (#649)

2 years agometainfo: Remove reflection dependency for `Piece.Hash()` (#661)
YenForYang [Sat, 18 Sep 2021 01:35:21 +0000 (20:35 -0500)]
metainfo: Remove reflection dependency for `Piece.Hash()` (#661)

`copy`  will copy exactly `HashSize` bytes here.

2 years agolockWithDeferreds: inlineable Unlock() (#603)
YenForYang [Fri, 17 Sep 2021 03:08:00 +0000 (22:08 -0500)]
lockWithDeferreds: inlineable Unlock() (#603)

2 years agorequesting: Filter peers that can't be allocated chunks
Matt Joiner [Thu, 16 Sep 2021 00:47:27 +0000 (10:47 +1000)]
requesting: Filter peers that can't be allocated chunks

2 years agoRejig waits between doRequests
Matt Joiner [Wed, 15 Sep 2021 14:12:58 +0000 (00:12 +1000)]
Rejig waits between doRequests

2 years agoPerformance improvements in piece requesting
Matt Joiner [Wed, 15 Sep 2021 14:12:24 +0000 (00:12 +1000)]
Performance improvements in piece requesting

2 years agoChange torrent capacity to not return a pointer
Matt Joiner [Wed, 15 Sep 2021 00:30:37 +0000 (10:30 +1000)]
Change torrent capacity to not return a pointer

It's an unnecessary complication for a storage implementer.

2 years agoSimplify PIece.iterUndirtiedChunks
Matt Joiner [Sat, 11 Sep 2021 11:17:47 +0000 (21:17 +1000)]
Simplify PIece.iterUndirtiedChunks

2 years agoPre-allocate request strategy pieces
Matt Joiner [Sat, 11 Sep 2021 11:17:31 +0000 (21:17 +1000)]
Pre-allocate request strategy pieces

2 years agocmd/torrent: Use anacrolix/args and merge several other cmds in as subcommands
Matt Joiner [Tue, 7 Sep 2021 02:08:31 +0000 (12:08 +1000)]
cmd/torrent: Use anacrolix/args and merge several other cmds in as subcommands

2 years agoInlineable `(*Torrent).Seeding()` (#626)
YenForYang [Wed, 15 Sep 2021 00:28:14 +0000 (19:28 -0500)]
Inlineable `(*Torrent).Seeding()` (#626)

2 years agoInlineable `(*Torrent).SetDisplayName()` (#625)
YenForYang [Wed, 15 Sep 2021 00:27:52 +0000 (19:27 -0500)]
Inlineable `(*Torrent).SetDisplayName()` (#625)

2 years agoInlineable `(*Torrent).PieceState()` (#621)
YenForYang [Wed, 15 Sep 2021 00:14:28 +0000 (19:14 -0500)]
Inlineable `(*Torrent).PieceState()` (#621)

2 years agoInlineable `(*Torrent).AddPeers()` (#622)
YenForYang [Wed, 15 Sep 2021 00:14:07 +0000 (19:14 -0500)]
Inlineable `(*Torrent).AddPeers()` (#622)

2 years agoInlineable `DownloadPieces()` and `CancelPieces()` (#623)
YenForYang [Wed, 15 Sep 2021 00:13:46 +0000 (19:13 -0500)]
Inlineable `DownloadPieces()` and `CancelPieces()` (#623)

2 years agoCreate default constructor for Client (#567)
YenForYang [Tue, 14 Sep 2021 13:01:20 +0000 (08:01 -0500)]
Create default constructor for Client (#567)

Allow for certain values to always be properly initialized on construction -- namely the maps for now. I'm currently working on a change that requires a baseline constructor; this change would make the use of `chansync.BroadcastCond` and `chansync.SetOnce` obsolete -- i.e. one can have channel members without worrying about proper initialization/destruction of a `chan struct{}`.

As for why `makeClient` returns a value instead of a pointer: returning a value gives us more options -- you can always take a pointer from a value later on cheaply, and have things moved to the heap if they weren't already. The same can't be said about getting a value back from a pointer. GC also could potentially have less work to do. Plus I personally find ownership to be an important concept (semi-borrowed from rust) -- use of values make ownership clear.

2 years agoClarify maximum value of "metadata_size" (#609)
YenForYang [Tue, 14 Sep 2021 12:36:19 +0000 (07:36 -0500)]
Clarify maximum value of "metadata_size" (#609)

2 years agoInlineable `(*File).BytesCompleted()` (#612)
YenForYang [Tue, 14 Sep 2021 12:11:35 +0000 (07:11 -0500)]
Inlineable `(*File).BytesCompleted()` (#612)

2 years agoOptimize `LoopbackListenHost()` (#615)
YenForYang [Tue, 14 Sep 2021 11:54:37 +0000 (06:54 -0500)]
Optimize `LoopbackListenHost()` (#615)

2 years agoInlineable (*reader).SetReadAhead and (*reader).Close (#594)
YenForYang [Tue, 14 Sep 2021 04:05:37 +0000 (23:05 -0500)]
Inlineable (*reader).SetReadAhead and (*reader).Close (#594)

Small fixes that prevent inlining of public functions

2 years agoDrop bradfitz/iter dependency (#605)
YenForYang [Tue, 14 Sep 2021 03:46:50 +0000 (22:46 -0500)]
Drop bradfitz/iter dependency (#605)

* Drop bradfitz/iter dependency

`range iter.N` looks nice and doesn't allocate, but unfortunately using a `range` expression blocks a function from being inlined wherever it's used (for now). It's not that we need inlining in all cases, but I do think a C-style for loop looks just as nice and is probably clearer to the majority. There also aren't any clear disadvantages to changing (unless you just happen to dislike the look of C)

* Update misc_test.go

* Update rlreader_test.go

* Update torrent_test.go

* Update bench_test.go

* Update client_test.go

* Update iplist_test.go

* Update mse_test.go

* Update peerconn_test.go

* Update peerconn.go

* Update order_test.go

* Update decoder_test.go

* Update main.go

* Update bench-piece-mark-complete.go

* Update main.go

* Update torrent.go

* Update iplist_test.go

* Update main.go

2 years agoCheaper byte to string conversion (#602)
YenForYang [Tue, 14 Sep 2021 00:41:04 +0000 (19:41 -0500)]
Cheaper byte to string conversion (#602)

Revamped https://github.com/anacrolix/torrent/pull/587

2 years agoInlineable reader.Seek and no-lock return for bad whence (#577)
YenForYang [Mon, 13 Sep 2021 12:52:58 +0000 (07:52 -0500)]
Inlineable reader.Seek and no-lock return for bad whence (#577)

* Inlineable reader.Seek and no-lock return for bad whence

Couldn't find an elegant way to early exit on bad whence. Using a closure could reduce code duplication slightly, but it's overkill for only 3 cases.

Note that returning 0 on an invalid whence is the behavior of `strings.Reader` and `bytes.Reader`.

* Update reader.go

2 years agoFix usages of eachListener() and make inlineable (#599)
YenForYang [Mon, 13 Sep 2021 11:35:51 +0000 (06:35 -0500)]
Fix usages of eachListener() and make inlineable (#599)

2 years agoInlineable methods to access File piecePriority (#601)
YenForYang [Mon, 13 Sep 2021 11:17:54 +0000 (06:17 -0500)]
Inlineable methods to access File piecePriority (#601)

2 years agoSimplify (*Torrent).gotMetainfo (#581)
YenForYang [Mon, 13 Sep 2021 01:41:11 +0000 (20:41 -0500)]
Simplify (*Torrent).gotMetainfo (#581)