3 public-inbox-index - create and update search indices
7 public-inbox-index [OPTIONS] GIT_DIR
11 public-inbox-index creates and updates the search and NNTP
12 article number database used by the read-only public-inbox HTTP
13 and NNTP interfaces. Currently, this requires L<Search::Xapian>
14 and L<DBD::SQlite> and L<DBI> Perl modules.
16 Once the initial indices are created by public-inbox-index,
17 L<public-inbox-mda(1)> and L<public-inbox-watch(1)> will
18 automatically maintain them.
20 Running this manually to update indices is only required if
21 relying on L<git-fetch(1)> to mirror an existing public-inbox;
22 or if upgrading to a new version of public-inbox using
23 the C<--reindex> option.
25 Having a search and article number database is essential to
26 running the NNTP interface, and strongly recommended for the
27 HTTP interface as it provides thread grouping in addition
28 to normal search functionality.
36 Forces a search engine re-index of all messages in the
37 repository. This can be used for in-place upgrades while
38 NNTP/HTTP server processes are utilizing the index. Keep in
39 mind this roughly doubles the size of the already-large
42 This does not touch the NNTP article number database.
48 All public-inbox-specific files are contained within the
49 C<$GIT_DIR/public-inbox/> directory. All files are expected to
50 grow in size as more messages are archived, so using compaction
51 commands (e.g. L<xapian-compact(1)>) is not recommended unless
52 the list is no longer active.
56 =item $GIT_DIR/public-inbox/msgmap.sqlite3
58 The stable NNTP article number to Message-ID mapping is
59 stored in an SQLite3 database.
61 This is required for users of L<public-inbox-nntpd(1)>, but
62 users of the L<PublicInbox::WWW> interface will find it
63 useful for attempting recovery from copy-paste truncations of
64 URLs containing long Message-IDs.
66 Avoid removing this file and regenerating it; it may cause
67 existing NNTP readers to lose sync and miss (or see duplicate)
70 This file is relatively small, and typically less than 5%
71 of the space of the mail stored in a packed git repository.
73 =item $GIT_DIR/public-inbox/xapian*
75 The database used by L<Search::Xapian>. This directory name is
76 followed by a number indicating the index schema version this
77 installation of public-inbox uses.
79 These directories may be safely deleted or removed in full
80 while the NNTP and HTTP interfaces are no longer accessing
83 In addition to providing a search interface for the HTTP
84 interface, the Xapian database is used to group and combine
85 related messages into threads. For NNTP servers, it also
86 provides a cache of metadata and header information often
87 requested by NNTP clients.
89 This directory is large, often two to three times the size of
90 the objects stored in a packed git repository. Using the
91 C<--reindex> option makes it larger, still.
101 Used to override the default "~/.public-inbox/config" value.
107 Feedback welcome via plain-text mail to L<mailto:meta@public-inbox.org>
109 The mail archives are hosted at L<https://public-inbox.org/meta/>
110 and L<http://hjrcffqmbrq6wope.onion/meta/>
114 Copyright 2016-2018 all contributors L<mailto:meta@public-inbox.org>
116 License: AGPL-3.0+ L<https://www.gnu.org/licenses/agpl-3.0.txt>
120 L<Search::Xapian>, L<DBD::SQLite>