1 From: Eric Wong <e@yhbt.net>
2 To: meta@public-inbox.org
3 Subject: [WIP] public-inbox 1.6.0
5 Content-Type: text/plain; charset=utf-8
6 Content-Disposition: inline
8 A big release containing several performance optimizations, a
9 new anonymous IMAP server, and more. It represents an
10 incremental improvement over 1.5 in several areas with more to
13 The read-only httpd and nntpd daemons no longer block the event
14 loop when retrieving blobs from git, making better use of SMP
15 systems while accomodating slow storage.
17 Indexing can be now be tuned to give somewhat usable performance
18 on HDD storage, though we can't defy the laws of physics, either.
22 - ~/.cache/public-inbox/inline-c is automatically used for Inline::C
23 if it exists. PERL_INLINE_DIRECTORY in env remains supported
24 and prioritized to support `nobody'-type users without HOME.
26 - msgmap.sqlite3 uses journal_mode=TRUNCATE, matching over.sqlite3
27 behavior for a minor reduction in VFS traffic
29 - public-inbox-tuning(7) - new manpage containing pointers to
30 various tuning options and tips for certain HW and OS setups.
32 - Copy-on-write is disabled on BTRFS for new indices to avoid
33 fragmentation. See the new public-inbox-tuning(7) manpage.
35 - message/{rfc822,news,global} attachments are decoded recursively
36 and indexed for search. Reindexing (see below) is required
37 to ensure these attachments are indexed in old messages.
39 - inbox.lock (v2) and ssoma.lock (v1) files are written to
40 on message delivery (or spam removal) to wake up read-only
41 daemons via inotify or kqueue.
45 The ordering of these steps is only necessary if you intend to
46 use some new features of this release.
48 0. install (use your OS package manager, or "make install")
50 1. restart public-inbox-watch instances if you have any
52 2. Optional: remove Plack::Middleware::Deflater if you're using
53 a custom .psgi file for PublicInbox::WWW. This only saves
54 some memory and CPU cycles, and you may also skip this step
55 if you expect to roll back to 1.5.0 for any reason.
57 Steps 3a and 3b may happen in any order, 3b is optional
58 and is only required to use new WWW and IMAP features.
60 3a. restart existing read-only daemons if you have them
61 (public-inbox-nntpd, public-inbox-httpd)
63 3b. run "public-inbox-index -c --reindex --rethread --all"
64 to reindex all configured inboxes
66 4. configure and start the new public-inbox-imapd. This
67 requires reindexing in 3b, but there's no obligation to
68 run an IMAP server, either.
72 There are several new options to improve usability on slow,
75 - `--batch-size=BYTES' or publicinbox.indexBatchSize parameter
76 to reduce frequency of random writes on HDDs
78 - `--sequential-shard' or publicInbox.sequentialShard parameter
79 to improve OS page cache utilization on HDDs.
81 - `--no-fsync' when combined with Xapian 1.4+ can be used to
82 speed up indexing on SSDs and small (default) `--batch-size'
84 - `--rethread' option to go with `--reindex' (use sparringly,
87 - parallelize v2 updates by default, `--sequential-shard' and
88 `-j0' is (once again) allowed to disable parallelization
90 - (re-)indexing parallelizes blob reads from git
92 - `--all' may be specified to index all configured inboxes
96 - `rm' supports `--all' to remove from all configured inboxes
100 - new read-only IMAP daemon similar to public-inbox-nntpd
101 `AUTH=ANONYMOUS' is supported, but any username and
102 password for clients without `AUTH=ANONYMOUS' support.
106 - blob reads from git are handled asynchronously
110 - Plack::Middleware::Deflater is no longer loaded by default
111 when no .psgi file is specified; PublicInbox::WWW can rely
112 on gzip for buffering (see below)
116 - use consistent blank line around attachment links
118 - Attachments in message/{rfc822,news,global} messages can be
119 individually downloaded. Downloading the entire message/rfc822
120 file in full remains supported
122 - $INBOX_DIR/description is treated as UTF-8
124 - HTML, Atom, and text/plain responses are gzipped without
125 relying on Plack::Middleware::Deflater
127 - Multi-message endpoints (/t.mbox.gz, /T/, /t/, etc) are ~10% faster
128 when running under public-inbox-httpd with asynchronous blob
131 - mbox search results may now include all messages pertaining to that
132 thread (requires `--reindex' mentioned in `Upgrading').
136 - Linux::Inotify2 or IO::KQueue is used directly,
137 Filesys::Notify::Simple is no longer required
139 Please report bugs via plain-text mail to: meta@public-inbox.org
141 See archives at https://public-inbox.org/meta/ for all history.
142 See https://public-inbox.org/TODO for what the future holds.