]> Sergey Matveev's repositories - public-inbox.git/commit
replace Xapian skeleton with SQLite overview DB
authorEric Wong (Contractor, The Linux Foundation) <e@80x24.org>
Mon, 2 Apr 2018 00:04:52 +0000 (00:04 +0000)
committerEric Wong (Contractor, The Linux Foundation) <e@80x24.org>
Mon, 2 Apr 2018 00:05:39 +0000 (00:05 +0000)
commit35ff6bb106909b1c1232666a9792156dfa398ea8
tree000f656d3daf3a077fbfa02b5853523d66a89329
parent7503aeb540af5afd5cb1b554b3c29f35f5fc918d
replace Xapian skeleton with SQLite overview DB

This ought to provide better performance and scalability
which is less dependent on inbox size.  Xapian does not
seem optimized for some queries used by the WWW homepage,
Atom feeds, XOVER and NEWNEWS NNTP commands.

This can actually make Xapian optional for NNTP usage,
and allow more functionality to work without Xapian
installed.

Indexing performance was extremely bad at first, but
DBI::Profile helped me optimize away problematic queries.
17 files changed:
MANIFEST
lib/PublicInbox/Inbox.pm
lib/PublicInbox/Msgmap.pm
lib/PublicInbox/NNTP.pm
lib/PublicInbox/Over.pm [new file with mode: 0644]
lib/PublicInbox/OverIdx.pm [new file with mode: 0644]
lib/PublicInbox/OverIdxFork.pm [moved from lib/PublicInbox/SearchIdxSkeleton.pm with 56% similarity]
lib/PublicInbox/Search.pm
lib/PublicInbox/SearchIdx.pm
lib/PublicInbox/SearchIdxPart.pm
lib/PublicInbox/SearchMsg.pm
lib/PublicInbox/V2Writable.pm
script/public-inbox-compact
t/over.t [new file with mode: 0644]
t/psgi_search.t
t/search-thr-index.t
t/search.t