1 From: Eric Wong <e@80x24.org>
2 To: meta@public-inbox.org
3 Subject: [ANNOUNCE] public-inbox 1.6.0
4 Date: Wed, 16 Sep 2020 20:03:09 +0000
5 Message-ID: <20200916200309.public-inbox-1.6.0-rele@sed>
7 Content-Type: text/plain; charset=utf-8
8 Content-Disposition: inline
10 A big release containing several performance optimizations, a
11 new anonymous IMAP server, and more. It represents an
12 incremental improvement over 1.5 in several areas with more to
15 The read-only httpd and nntpd daemons no longer block the event
16 loop when retrieving blobs from git, making better use of SMP
17 systems while accomodating slow storage.
19 Indexing can be now be tuned to give somewhat usable performance
20 on HDD storage, though we can't defy the laws of physics, either.
24 - ~/.cache/public-inbox/inline-c is automatically used for Inline::C
25 if it exists. PERL_INLINE_DIRECTORY in env remains supported
26 and prioritized to support `nobody'-type users without HOME.
28 - msgmap.sqlite3 uses journal_mode=TRUNCATE, matching over.sqlite3
29 behavior for a minor reduction in VFS traffic
31 - public-inbox-tuning(7) - new manpage containing pointers to
32 various tuning options and tips for certain HW and OS setups.
34 - Copy-on-write is disabled on BTRFS for new indices to avoid
35 fragmentation. See the new public-inbox-tuning(7) manpage.
37 - message/{rfc822,news,global} attachments are decoded recursively
38 and indexed for search. Reindexing (see below) is required
39 to ensure these attachments are indexed in old messages.
41 - inbox.lock (v2) and ssoma.lock (v1) files are written to
42 on message delivery (or spam removal) to wake up read-only
43 daemons via inotify or kqueue.
45 - `--help' switch supported by command-line tools
47 * Upgrading for new features in 1.6
49 The ordering of these steps is only necessary if you intend to
50 use some new features in 1.6. Future releases may have
51 different instructions (or be entirely transparent).
53 0. install (use your OS package manager, or "make install")
55 1. restart public-inbox-watch instances if you have any
57 2. Optional: remove Plack::Middleware::Deflater if you're using
58 a custom .psgi file for PublicInbox::WWW. This only saves
59 some memory and CPU cycles, and you may also skip this step
60 if you expect to roll back to 1.5.0 for any reason.
62 Steps 3a and 3b may happen in any order, 3b is optional
63 and is only required to use new WWW and IMAP features.
65 3a. restart existing read-only daemons if you have them
66 (public-inbox-nntpd, public-inbox-httpd)
68 3b. run "public-inbox-index -c --reindex --rethread --all"
69 to reindex all configured inboxes
71 4. configure and start the new public-inbox-imapd. This
72 requires reindexing in 3b, but there's no obligation to
73 run an IMAP server, either.
77 There are several new options to improve usability on slow,
80 - `--batch-size=BYTES' or publicinbox.indexBatchSize parameter
81 to reduce frequency of random writes on HDDs
83 - `--sequential-shard' or publicInbox.sequentialShard parameter
84 to improve OS page cache utilization on HDDs.
86 - `--no-fsync' when combined with Xapian 1.4+ can be used to
87 speed up indexing on SSDs and small (default) `--batch-size'
89 - `--rethread' option to go with `--reindex' (use sparringly,
92 - parallelize v2 updates by default, `--sequential-shard' and
93 `-j0' is (once again) allowed to disable parallelization
95 - (re-)indexing parallelizes blob reads from git
97 - `--all' may be specified to index all configured inboxes
101 - `rm' supports `--all' to remove from all configured inboxes
105 - new read-only IMAP daemon similar to public-inbox-nntpd
106 `AUTH=ANONYMOUS' is supported, but any username and
107 password for clients without `AUTH=ANONYMOUS' support.
111 - blob reads from git are handled asynchronously
115 - Plack::Middleware::Deflater is no longer loaded by default
116 when no .psgi file is specified; PublicInbox::WWW can rely
117 on gzip for buffering (see below)
121 - use consistent blank line around attachment links
123 - Attachments in message/{rfc822,news,global} messages can be
124 individually downloaded. Downloading the entire message/rfc822
125 file in full remains supported
127 - $INBOX_DIR/description is treated as UTF-8
129 - HTML, Atom, and text/plain responses are gzipped without
130 relying on Plack::Middleware::Deflater
132 - Multi-message endpoints (/t.mbox.gz, /T/, /t/, etc) are ~10% faster
133 when running under public-inbox-httpd with asynchronous blob
136 - mbox search results may now include all messages pertaining to that
137 thread. Needs `--reindex' mentioned above in
138 `Upgrading for new features in 1.6'.
140 - fix mbox.gz search results downloads for lynx users
142 - small navigation tweaks, more prominent mirroring instructions
146 - Linux::Inotify2 or IO::KQueue is used directly,
147 Filesys::Notify::Simple is no longer required
149 - NNTP groups and IMAP mailboxes may be watched in addition
150 to Maildirs (lightly tested).
152 * Ongoing internal changes
154 - reduce event loop hogging for many-inbox support
156 - use more Perl v5.10-isms, future-proof against Perl 8
158 - more consistent variable and field naming, improve internal
159 documentation and comments
161 - start supporting >=40 char git identifiers for SHA-256
163 - test -httpd-specific code paths via Plack::Test::ExternalServer
164 in addition to generic PSGI paths.
166 Please report bugs via plain-text mail to: meta@public-inbox.org
168 See archives at https://public-inbox.org/meta/ for all history.
169 See https://public-inbox.org/TODO for what the future holds.