]> Sergey Matveev's repositories - public-inbox.git/commit
searchidx: disable CoW for SQLite and Xapian under btrfs
authorEric Wong <e@yhbt.net>
Tue, 28 Jul 2020 22:21:58 +0000 (22:21 +0000)
committerEric Wong <e@yhbt.net>
Wed, 29 Jul 2020 11:32:57 +0000 (11:32 +0000)
commitc106504309621b662ce6c7cd914718f7045edca4
treec569ad56cfd8e192c9f087faf9a5e13482dcd27f
parenta3391407c960e4bbd825a34b87d053de6ef3767a
searchidx: disable CoW for SQLite and Xapian under btrfs

SQLite and Xapian files are written randomly, thus they become
fragmented under btrfs with copy-on-write.  This leads to
noticeable performance problems (and probably ENOSPC) as these
files get big.

lore/git (v2, <1GB) indexes around 20% faster with this on an
ancient SSD.  lore/lkml seems to be taking forever and I'll
probably cancel it to save wear on my SSD.

Unfortunately, disabling CoW also means disabling checksumming
(and compression), so we'll be careful to only set the No_COW
attribute on regeneratable data.  We want to keep CoW (and
checksums+compression) on git storage because current ref
storage is neither checksummed nor compressed, and git streams
pack output.
MANIFEST
lib/PublicInbox/NDC_PP.pm [new file with mode: 0644]
lib/PublicInbox/Over.pm
lib/PublicInbox/SearchIdx.pm
lib/PublicInbox/Spawn.pm
t/nodatacow.t [new file with mode: 0644]