]> Sergey Matveev's repositories - public-inbox.git/commit
lei index: new command to index mail w/o git storage
authorEric Wong <e@80x24.org>
Tue, 4 May 2021 09:49:12 +0000 (09:49 +0000)
committerEric Wong <e@80x24.org>
Tue, 4 May 2021 23:08:01 +0000 (23:08 +0000)
commit40f3f2a2c805fc37c7ed35a60948856bd962b493
tree0fe4d70c7a50a0122f92d48b832ada31a3b5a0bf
parent4481b372ba150c669b2fefe2d6ec5dccb5da1d40
lei index: new command to index mail w/o git storage

Since completely purging blobs from git is slow, users may wish
to index messages in Maildirs (and eventually other local
storage) without storing data in git.

Much code from LeiImport and LeiInput is reused, and a new dummy
FakeImport class supplies a non-storing $im->add and minimize
changes to LeiStore.

The tricky part of this command is to support "lei import"
after a message has gone through "lei index".  Relying on
$smsg->{bytes} == 0 (as we do for external-only vmd storage)
does not work here, since it would break searching for "z:"
byte-ranges when not using externals.

This eventually required PublicInbox::Import::add to use a
SharedKV to keep track of imported blobs and prevent
duplication.
12 files changed:
MANIFEST
lib/PublicInbox/FakeImport.pm [new file with mode: 0644]
lib/PublicInbox/Import.pm
lib/PublicInbox/LeiBlob.pm
lib/PublicInbox/LeiImport.pm
lib/PublicInbox/LeiIndex.pm [new file with mode: 0644]
lib/PublicInbox/LeiInput.pm
lib/PublicInbox/LeiMailSync.pm
lib/PublicInbox/LeiStore.pm
lib/PublicInbox/LeiToMail.pm
lib/PublicInbox/OverIdx.pm
t/lei-index.t [new file with mode: 0644]