]> Sergey Matveev's repositories - btrtrc.git/log
btrtrc.git
2 years agoSimplify request ordering for reversed conditions
Matt Joiner [Tue, 12 Oct 2021 01:31:47 +0000 (12:31 +1100)]
Simplify request ordering for reversed conditions

2 years agoSome fixes to pending request accounting
Matt Joiner [Tue, 12 Oct 2021 00:58:46 +0000 (11:58 +1100)]
Some fixes to pending request accounting

2 years agoHandle allowed fast while choked when requests already exist in the same piece
Matt Joiner [Mon, 11 Oct 2021 11:04:53 +0000 (22:04 +1100)]
Handle allowed fast while choked when requests already exist in the same piece

2 years agoAdd missing request updates on cancels, rejects
Matt Joiner [Mon, 11 Oct 2021 07:22:02 +0000 (18:22 +1100)]
Add missing request updates on cancels, rejects

2 years agoDon't automatically delete requests if we're choked with fast extension
Matt Joiner [Mon, 11 Oct 2021 07:21:24 +0000 (18:21 +1100)]
Don't automatically delete requests if we're choked with fast extension

2 years agoDon't put requests we can't make into the request heap
Matt Joiner [Mon, 11 Oct 2021 05:21:26 +0000 (16:21 +1100)]
Don't put requests we can't make into the request heap

In particular this should save a lot of overhead if we're choked.

2 years agoOptimize Peer.peerHasWantedPieces
Matt Joiner [Mon, 11 Oct 2021 04:46:40 +0000 (15:46 +1100)]
Optimize Peer.peerHasWantedPieces

2 years agoOptimize request updates due to conn and reader state changes
Matt Joiner [Mon, 11 Oct 2021 04:45:55 +0000 (15:45 +1100)]
Optimize request updates due to conn and reader state changes

2 years agoExtend nominal max requests upper bound to 2048
Matt Joiner [Sun, 10 Oct 2021 01:02:29 +0000 (12:02 +1100)]
Extend nominal max requests upper bound to 2048

2 years agoUse a flat slice for pending request counts
Matt Joiner [Sun, 10 Oct 2021 00:54:19 +0000 (11:54 +1100)]
Use a flat slice for pending request counts

Under heavy load, seems to be 2-3x faster.

2 years agoOptimize chunk calculations in request strategy
Matt Joiner [Sun, 10 Oct 2021 00:32:27 +0000 (11:32 +1100)]
Optimize chunk calculations in request strategy

2 years agoIgnore request strategy run gobs and envrcs
Matt Joiner [Sun, 10 Oct 2021 00:19:40 +0000 (11:19 +1100)]
Ignore request strategy run gobs and envrcs

2 years agoImplement pending requests using BSI
Matt Joiner [Sun, 10 Oct 2021 00:19:08 +0000 (11:19 +1100)]
Implement pending requests using BSI

2 years agoExtract pendingRequests
Matt Joiner [Sat, 9 Oct 2021 08:00:58 +0000 (19:00 +1100)]
Extract pendingRequests

2 years agoAdd reasons for updateRequests to be triggered
Matt Joiner [Fri, 8 Oct 2021 22:14:57 +0000 (09:14 +1100)]
Add reasons for updateRequests to be triggered

2 years agoAdd pprof labels for request updates and remove Client-wide requester
Matt Joiner [Fri, 8 Oct 2021 02:53:36 +0000 (13:53 +1100)]
Add pprof labels for request updates and remove Client-wide requester

2 years agoRemove peer piece inclination and piece request order
Matt Joiner [Fri, 8 Oct 2021 01:41:25 +0000 (12:41 +1100)]
Remove peer piece inclination and piece request order

These are vestigial data structures used with old request strategy implementations.

2 years agoChange peer requesting to spread requests out evenly
Matt Joiner [Thu, 7 Oct 2021 06:31:10 +0000 (17:31 +1100)]
Change peer requesting to spread requests out evenly

2 years agoLog handshook conn run errors with level warning
Matt Joiner [Tue, 19 Oct 2021 02:37:46 +0000 (13:37 +1100)]
Log handshook conn run errors with level warning

(cherry picked from commit 7ca7890e9f30307a1f4942e6d987791a142c3cb3)

2 years agoEnding a conn because we don't like its ID is not an error
Matt Joiner [Tue, 19 Oct 2021 02:37:18 +0000 (13:37 +1100)]
Ending a conn because we don't like its ID is not an error

(cherry picked from commit 2a5ba846cef9b4e035b3b88698f3e7f7d9d79251)

2 years agoFix incomplete Integer.Read
Matt Joiner [Tue, 19 Oct 2021 02:13:23 +0000 (13:13 +1100)]
Fix incomplete Integer.Read

Possibly on read buffer boundaries, this would have caused errant io.ErrUnexpectedEOF.

(cherry picked from commit 22569449dad48e1c5cdeac171625585f8ca976c9)

2 years agoOnly allow chunk size to be set for new Torrents
Matt Joiner [Sun, 10 Oct 2021 00:22:29 +0000 (11:22 +1100)]
Only allow chunk size to be set for new Torrents

(cherry picked from commit 09e73e9fce139cd78da48c442e8610501b6fd26b)

2 years agoUpdate piece priority after processing piece hash result
Matt Joiner [Fri, 8 Oct 2021 23:26:52 +0000 (10:26 +1100)]
Update piece priority after processing piece hash result

Seems to avoid some silly performance issues in triggering request updates due to a piece still pending while being marked.

2 years agoPieces being marked should have priority none
Matt Joiner [Sat, 9 Oct 2021 03:20:54 +0000 (14:20 +1100)]
Pieces being marked should have priority none

Also reorder the checks a bit to favour the cheaper ones.

2 years agoAdd DisableInitialPieceCheck option (#677)
Simon Kirsten [Mon, 11 Oct 2021 00:33:40 +0000 (02:33 +0200)]
Add DisableInitialPieceCheck option (#677)

2 years agoUpdate requests on unchoke
Matt Joiner [Thu, 7 Oct 2021 06:20:04 +0000 (17:20 +1100)]
Update requests on unchoke

2 years agocmd/torrent: Ensure envpprof.Stop runs before exit
Matt Joiner [Thu, 7 Oct 2021 06:00:39 +0000 (17:00 +1100)]
cmd/torrent: Ensure envpprof.Stop runs before exit

2 years agogo mod tidy
Matt Joiner [Thu, 7 Oct 2021 02:56:45 +0000 (13:56 +1100)]
go mod tidy

Fixes broken go doc: https://github.com/golang/go/issues/36811#issuecomment-937401816

2 years agoAdd TorrentStats.PiecesComplete
Matt Joiner [Thu, 7 Oct 2021 02:50:13 +0000 (13:50 +1100)]
Add TorrentStats.PiecesComplete

Fixes https://github.com/anacrolix/torrent/issues/352.

2 years agoAdd to tests for Torrent.Complete v1.33.0
Matt Joiner [Thu, 7 Oct 2021 02:38:29 +0000 (13:38 +1100)]
Add to tests for Torrent.Complete

2 years agoReturn errors from Client.Close
Matt Joiner [Thu, 7 Oct 2021 02:31:08 +0000 (13:31 +1100)]
Return errors from Client.Close

2 years agoFix comment that seems to mess with GoLand's package structure
Matt Joiner [Thu, 7 Oct 2021 01:09:52 +0000 (12:09 +1100)]
Fix comment that seems to mess with GoLand's package structure

2 years agoAdd a flag for Complete
Matt Joiner [Thu, 7 Oct 2021 01:09:20 +0000 (12:09 +1100)]
Add a flag for Complete

2 years agoUpgrade Torrent.GotInfo
Matt Joiner [Thu, 7 Oct 2021 00:50:30 +0000 (11:50 +1100)]
Upgrade Torrent.GotInfo

2 years agoUpdate chansync
Matt Joiner [Thu, 7 Oct 2021 00:45:09 +0000 (11:45 +1100)]
Update chansync

2 years agoApply some lints from GoLand
Matt Joiner [Wed, 6 Oct 2021 07:43:15 +0000 (18:43 +1100)]
Apply some lints from GoLand

2 years agoRemove old-style build tags
Matt Joiner [Wed, 6 Oct 2021 07:36:25 +0000 (18:36 +1100)]
Remove old-style build tags

2 years agoSwitch Peer.PieceAllowedFast and several request strategy inputs to raw roaring.Bitmaps
Matt Joiner [Tue, 5 Oct 2021 09:06:23 +0000 (20:06 +1100)]
Switch Peer.PieceAllowedFast and several request strategy inputs to raw roaring.Bitmaps

This is in preparation to support encoding request strategy run inputs for benchmarking.

2 years agogo fmt ./...
Matt Joiner [Tue, 5 Oct 2021 06:48:52 +0000 (17:48 +1100)]
go fmt ./...

2 years agoChange Peer._peerPieces to use raw roaring Bitmap type
Matt Joiner [Tue, 5 Oct 2021 06:48:34 +0000 (17:48 +1100)]
Change Peer._peerPieces to use raw roaring Bitmap type

The wrapper type was from when roaring didn't support zero-alloc initialization.

2 years agoWrap error with %w in mainReadLoop v1.32.0
Matt Joiner [Thu, 30 Sep 2021 02:01:56 +0000 (12:01 +1000)]
Wrap error with %w in mainReadLoop

2 years agoFix mainReadLoop benchmark
Matt Joiner [Thu, 30 Sep 2021 02:01:34 +0000 (12:01 +1000)]
Fix mainReadLoop benchmark

2 years agoFix scheduling overhead in decoding benchmark
Matt Joiner [Thu, 30 Sep 2021 01:52:50 +0000 (11:52 +1000)]
Fix scheduling overhead in decoding benchmark

2 years agoCombine peer protocol fuzz targets into one file
Matt Joiner [Thu, 30 Sep 2021 01:19:50 +0000 (11:19 +1000)]
Combine peer protocol fuzz targets into one file

2 years agoMore optimizations in peer protocol message decoding
Matt Joiner [Thu, 30 Sep 2021 01:05:01 +0000 (11:05 +1000)]
More optimizations in peer protocol message decoding

2 years agopeer_protocol: Use faster form for Integer.{UnmarshalBinary,Read}
Matt Joiner [Thu, 30 Sep 2021 00:24:03 +0000 (10:24 +1000)]
peer_protocol: Use faster form for Integer.{UnmarshalBinary,Read}

2 years agoRemove an allocation reading message length
Matt Joiner [Wed, 29 Sep 2021 23:59:18 +0000 (09:59 +1000)]
Remove an allocation reading message length

2 years agoTry to avoid allocating buffer
Matt Joiner [Wed, 29 Sep 2021 23:45:50 +0000 (09:45 +1000)]
Try to avoid allocating buffer

2 years agoImprovements to decoder fuzzing
Matt Joiner [Wed, 29 Sep 2021 23:42:54 +0000 (09:42 +1000)]
Improvements to decoder fuzzing

2 years agoAdd some fuzzing in peer_protocol
Matt Joiner [Wed, 29 Sep 2021 23:01:10 +0000 (09:01 +1000)]
Add some fuzzing in peer_protocol

2 years agoLimit decoded bencode string lengths to 32 bits
Matt Joiner [Wed, 29 Sep 2021 06:56:01 +0000 (16:56 +1000)]
Limit decoded bencode string lengths to 32 bits

Found in fuzzing

2 years agoAdd some fuzzing seeds
Matt Joiner [Wed, 29 Sep 2021 02:12:16 +0000 (12:12 +1000)]
Add some fuzzing seeds

2 years agoFix parsing of strings that don't fit in memory
Matt Joiner [Wed, 29 Sep 2021 02:11:58 +0000 (12:11 +1000)]
Fix parsing of strings that don't fit in memory

2 years agoFix fuzz comparison of big.Int
Matt Joiner [Wed, 29 Sep 2021 02:11:17 +0000 (12:11 +1000)]
Fix fuzz comparison of big.Int

2 years agoUpdate fuzz to use new standard library support
Matt Joiner [Wed, 29 Sep 2021 00:13:32 +0000 (10:13 +1000)]
Update fuzz to use new standard library support

2 years agoSimplify bencode.Decoder.parseStringInterface
Matt Joiner [Tue, 28 Sep 2021 23:50:01 +0000 (09:50 +1000)]
Simplify bencode.Decoder.parseStringInterface

2 years agobencode: optimize `(*Decoder).parseStringInterface()` (#659)
YenForYang [Tue, 28 Sep 2021 23:30:35 +0000 (18:30 -0500)]
bencode: optimize `(*Decoder).parseStringInterface()` (#659)

2 years agoInlineable `(*Torrent).BytesMissing()` (#633)
YenForYang [Mon, 27 Sep 2021 02:43:09 +0000 (21:43 -0500)]
Inlineable `(*Torrent).BytesMissing()` (#633)

Honestly a name like `BytesLeft` would have been more suitable, but it's too late for that I guess.

2 years agoInlineable `addrIndex` (#673)
YenForYang [Mon, 27 Sep 2021 02:41:20 +0000 (21:41 -0500)]
Inlineable `addrIndex` (#673)

2 years agoEliminate `expect` dependency (#667)
YenForYang [Thu, 23 Sep 2021 14:10:30 +0000 (09:10 -0500)]
Eliminate `expect` dependency (#667)

2 years agoDefault to peer requesting disabled
Matt Joiner [Tue, 21 Sep 2021 06:07:08 +0000 (16:07 +1000)]
Default to peer requesting disabled

There are some flaws that need to be fixed up before it might be considered the default.

2 years agocmd/torrent: Don't make --stats default to --debug's value
Matt Joiner [Tue, 21 Sep 2021 03:06:06 +0000 (13:06 +1000)]
cmd/torrent: Don't make --stats default to --debug's value

2 years agoRemove unused function
Matt Joiner [Tue, 21 Sep 2021 03:05:16 +0000 (13:05 +1000)]
Remove unused function

2 years agoAdd ClientConfig.AlwaysWantConns
Matt Joiner [Tue, 21 Sep 2021 03:04:51 +0000 (13:04 +1000)]
Add ClientConfig.AlwaysWantConns

2 years agocmd/torrent: Refactor and set progress interval to 3s
Matt Joiner [Mon, 20 Sep 2021 12:01:18 +0000 (22:01 +1000)]
cmd/torrent: Refactor and set progress interval to 3s

2 years agoUse an iterator to skip through dirty chunks
Matt Joiner [Tue, 21 Sep 2021 00:48:15 +0000 (10:48 +1000)]
Use an iterator to skip through dirty chunks

2 years agocmd/torrent: Include download stats on interrupt
Matt Joiner [Mon, 20 Sep 2021 11:33:46 +0000 (21:33 +1000)]
cmd/torrent: Include download stats on interrupt

2 years agoTrack dirty chunks in a single bitmap on Torrent
Matt Joiner [Mon, 20 Sep 2021 08:52:54 +0000 (18:52 +1000)]
Track dirty chunks in a single bitmap on Torrent

2 years agoFix cancellation for global requesting
Matt Joiner [Mon, 20 Sep 2021 05:24:24 +0000 (15:24 +1000)]
Fix cancellation for global requesting

2 years agoMerge branch 'master' into peer-requesting
Matt Joiner [Mon, 20 Sep 2021 05:10:19 +0000 (15:10 +1000)]
Merge branch 'master' into peer-requesting

2 years agoReduce the diff to master and add peerRequesting feature const
Matt Joiner [Mon, 20 Sep 2021 05:09:28 +0000 (15:09 +1000)]
Reduce the diff to master and add peerRequesting feature const

2 years agocmd/torrent: Dump download statistics
Matt Joiner [Mon, 20 Sep 2021 04:17:58 +0000 (14:17 +1000)]
cmd/torrent: Dump download statistics

2 years agoFilter next requests application for peer state changes
Matt Joiner [Mon, 20 Sep 2021 04:16:55 +0000 (14:16 +1000)]
Filter next requests application for peer state changes

2 years agoAdd NOTES
Matt Joiner [Mon, 20 Sep 2021 02:31:44 +0000 (12:31 +1000)]
Add NOTES

2 years agoStore peer requests in a bitmap
Matt Joiner [Sun, 19 Sep 2021 05:16:37 +0000 (15:16 +1000)]
Store peer requests in a bitmap

2 years agoWhen updating requests, only tickle writer if there are no requests pending
Matt Joiner [Sat, 18 Sep 2021 11:01:06 +0000 (21:01 +1000)]
When updating requests, only tickle writer if there are no requests pending

Just a quick attempt to reduce load for now. There'll be a much better way to do this.

2 years agoDo peer requests separately for each peer
Matt Joiner [Sat, 18 Sep 2021 10:34:14 +0000 (20:34 +1000)]
Do peer requests separately for each peer

2 years agoAdd choking/allowed fast checks to Peer.shouldRequest
Matt Joiner [Sat, 18 Sep 2021 09:00:21 +0000 (19:00 +1000)]
Add choking/allowed fast checks to Peer.shouldRequest

2 years agoExport request_strategy.GetRequestablePieces
Matt Joiner [Sat, 18 Sep 2021 08:57:50 +0000 (18:57 +1000)]
Export request_strategy.GetRequestablePieces

2 years agogoimports doing its thing
Matt Joiner [Sat, 18 Sep 2021 08:54:54 +0000 (18:54 +1000)]
goimports doing its thing

2 years agoAdd some tests verifying request map ordering
Matt Joiner [Sat, 18 Sep 2021 03:50:55 +0000 (13:50 +1000)]
Add some tests verifying request map ordering

2 years agoUpdate anacrolix/args for bool Flag fix
Matt Joiner [Sat, 18 Sep 2021 03:23:36 +0000 (13:23 +1000)]
Update anacrolix/args for bool Flag fix

2 years agorequesting: Remove some obsoleted condition checks
Matt Joiner [Thu, 16 Sep 2021 07:10:07 +0000 (17:10 +1000)]
requesting: Remove some obsoleted condition checks

These shouldn't be necessary since peers were pruned for the ability to allocate chunks to the current piece.

2 years agorequest_strategy.Peer.canRequestPiece performance: Reorder conditions
Matt Joiner [Thu, 16 Sep 2021 07:08:59 +0000 (17:08 +1000)]
request_strategy.Peer.canRequestPiece performance: Reorder conditions

2 years agobencode: get type of `big.Int` without creating instance (#651)
YenForYang [Sat, 18 Sep 2021 02:44:47 +0000 (21:44 -0500)]
bencode: get type of `big.Int` without creating instance (#651)

Nope, it doesn't really matter. But anyway, see https://github.com/golang/crypto/blob/c084706c2272f3d44b722e988e70d4a58e60e7f4/cryptobyte/asn1.go#L267

2 years agobencode: simplify getting `marshalerType` and `unmarshalerType` (#652)
YenForYang [Sat, 18 Sep 2021 02:43:53 +0000 (21:43 -0500)]
bencode: simplify getting `marshalerType` and `unmarshalerType` (#652)

2 years agobencode: Simplify `(*Decoder).parseListInterface()` (#656)
YenForYang [Sat, 18 Sep 2021 02:42:20 +0000 (21:42 -0500)]
bencode: Simplify `(*Decoder).parseListInterface()` (#656)

Preserve as much type as possible (it'll be converted to an `interface{}` anyway, but we can return `[]interface{}` instead of `{}interface` here).

2 years agoPrevent allocation when checking interface (#649)
YenForYang [Sat, 18 Sep 2021 02:36:25 +0000 (21:36 -0500)]
Prevent allocation when checking interface (#649)

2 years agometainfo: Remove reflection dependency for `Piece.Hash()` (#661)
YenForYang [Sat, 18 Sep 2021 01:35:21 +0000 (20:35 -0500)]
metainfo: Remove reflection dependency for `Piece.Hash()` (#661)

`copy`  will copy exactly `HashSize` bytes here.

2 years agolockWithDeferreds: inlineable Unlock() (#603)
YenForYang [Fri, 17 Sep 2021 03:08:00 +0000 (22:08 -0500)]
lockWithDeferreds: inlineable Unlock() (#603)

2 years agorequesting: Filter peers that can't be allocated chunks
Matt Joiner [Thu, 16 Sep 2021 00:47:27 +0000 (10:47 +1000)]
requesting: Filter peers that can't be allocated chunks

2 years agoRejig waits between doRequests
Matt Joiner [Wed, 15 Sep 2021 14:12:58 +0000 (00:12 +1000)]
Rejig waits between doRequests

2 years agoPerformance improvements in piece requesting
Matt Joiner [Wed, 15 Sep 2021 14:12:24 +0000 (00:12 +1000)]
Performance improvements in piece requesting

2 years agoChange torrent capacity to not return a pointer
Matt Joiner [Wed, 15 Sep 2021 00:30:37 +0000 (10:30 +1000)]
Change torrent capacity to not return a pointer

It's an unnecessary complication for a storage implementer.

2 years agoSimplify PIece.iterUndirtiedChunks
Matt Joiner [Sat, 11 Sep 2021 11:17:47 +0000 (21:17 +1000)]
Simplify PIece.iterUndirtiedChunks

2 years agoPre-allocate request strategy pieces
Matt Joiner [Sat, 11 Sep 2021 11:17:31 +0000 (21:17 +1000)]
Pre-allocate request strategy pieces

2 years agocmd/torrent: Use anacrolix/args and merge several other cmds in as subcommands
Matt Joiner [Tue, 7 Sep 2021 02:08:31 +0000 (12:08 +1000)]
cmd/torrent: Use anacrolix/args and merge several other cmds in as subcommands

2 years agoInlineable `(*Torrent).Seeding()` (#626)
YenForYang [Wed, 15 Sep 2021 00:28:14 +0000 (19:28 -0500)]
Inlineable `(*Torrent).Seeding()` (#626)

2 years agoInlineable `(*Torrent).SetDisplayName()` (#625)
YenForYang [Wed, 15 Sep 2021 00:27:52 +0000 (19:27 -0500)]
Inlineable `(*Torrent).SetDisplayName()` (#625)

2 years agoInlineable `(*Torrent).PieceState()` (#621)
YenForYang [Wed, 15 Sep 2021 00:14:28 +0000 (19:14 -0500)]
Inlineable `(*Torrent).PieceState()` (#621)