1 public-inbox design notes
2 -------------------------
4 Challenges to running normal mailing lists
5 ------------------------------------------
8 2) bounce processing of invalid/bad email addresses
9 3) processing subscribe/unsubscribe requests
11 Issues 2) and 3) are side-stepped entirely by moving reader
12 subscriptions to git repository synchronization and Atom feeds. There's
13 no chance of faked subscription requests and no need to deal with
14 confused users who cannot unsubscribe.
16 Use existing infrastructure
17 ---------------------------
19 * public-inbox can coexist with existing mailing lists, any subscriber
20 to the existing mailing list can begin delivering messages to
23 * public-inbox uses SMTP for posting. Posting a message to a public-inbox
24 instance is no different than sending a message to any _open_ mailing
27 * Existing spam filtering on an SMTP server is also effective on
30 * readers may continue using use their choice of mail clients and
31 mailbox formats, only learning a few commands of the ssoma(1) tool
34 * Atom is a reasonable feed format for casual readers and is supported
35 by a variety of feed readers.
40 * Freedom from proprietary services, tools and APIs. Communicating with
41 developers and users of Free Software should not rely on proprietary
44 * Existing infrastructure, tools, and user familiarity.
45 There is already a large variety of tools, clients, and email providers
46 available. There are also many resources for users to run their own
47 SMTP server on a domain they control.
49 * All public discussion mediums are affected by spam and advertising.
50 There exist several good Free Software anti-spam tools for email.
52 * Privacy is not an issue for public discussion. Public mailing list
53 archives are common and accepted by Free Software communities.
54 There is no need to ask the NSA for backups of your mail archives :)
56 * git, one of the most widely-used version control systems, includes many
57 tools for for email, including: git-format-patch(1), git-send-email(1),
58 git-am(1), git-imap-send(1). Furthermore, the development of git itself
59 is based on the git mailing list: https://public-inbox.org/git/
60 (or http://hjrcffqmbrq6wope.onion/git/ for Tor users)
62 * Email is already the de-facto form of communication in many Free Software
65 * Fallback/transition to private email and other lists, in case the
66 public-inbox host becomes unavailable, users may still directly email
67 each other (or Cc: lists for related/dependent projects).
72 * git is distributed and robust while being both fast and
73 space-efficient with text data. NNTP was considered, but does not
74 support delta-compression and places no guarantees on data/transport
75 integrity. However, a read-only NNTP gateway is implemented.
77 * As of 2016, git is widely used and known to nearly all Free Software
78 developers. For non-developers it is packaged for all major GNU/Linux
79 and *BSD distributions. NNTP is not as widely-used nowadays.
84 * Perl 5 is widely available on modern *nix systems with good a history
85 of backwards and forward compatibility.
87 * git and SpamAssassin both use it, so it should be one less thing for
88 admins to install and waste disk space with.
90 * Distributing compiled binaries has higher costs for storage/cache
91 space is required for each architecture. Having a runnable,
92 source-only distribution means any user already has access to all
98 * Stick to dependencies available in Debian main, this should make it
99 easier for potential users to install, and easier for distro
100 maintainers to pick up.
102 * A list server being turned into an SMTP spam relay and being
103 blacklisted while an admin is asleep is scary.
104 Sidestep that entirely by having clients pull.
106 * Eric has a great Maildir+inotify-based Bayes training setup
107 going back many years. Document, integrate and publicize it for
108 public-inbox usage, encouraging other admins to use it (it works
109 as long as admins read their public-inbox).
111 * Custom, difficult-for-Bayes requires custom anti-spam rules.
112 We may steal rules from the Debian listmasters:
113 svn://anonscm.debian.org/pkg-listmaster
115 * Full archives are easily distributable with git, so somebody else
116 can take over the list if we give up. Anybody may also run an SMTP
117 notifier/delivery service based on the archives.
119 * Avoids bikeshedding about web UI decisions, GUI-lovers can write their
120 own GUI-friendly interfaces (HTML or native) based on public archives.
125 * Getting users to install/run any new tool is difficult.
126 The web views must be easily read/cache/mirror-able.
128 * There may also be a significant number of webmail users without
129 an MUA or feed reader; so a web view is necessary.
131 * Expose Message-ID in web views to encourage replies from drive-by
134 * Raw text endpoint allows users to write client-side endpoints
135 without hosting the data themselves (or on a different server).
137 What sucks about public-inbox
138 -----------------------------
140 * Lack of push notification. On the other hand, feeds seem popular.
142 * some (mostly GUI) mail clients cannot set In-Reply-To headers
143 properly without the original message.
148 Even with shallow clone, storing the history of large/busy mailing lists
149 may place much burden on subscribers and servers. However, having a
150 single (or few) refs representing the entire history of a list is good
151 for small lists since it's easier to look up a message by Message-ID, so
152 we want to avoid splitting refs with independent histories.
154 ssoma will likely grow its own built-in ref rotation system based on
155 message count (not rotating at fixed time intervals). This would
156 split the histories and require O(n) lookup time based on Message-ID,
157 where `n' is the number of history splits.
162 Copyright 2013-2018 all contributors <meta@public-inbox.org>
163 License: AGPL-3.0+ <http://www.gnu.org/licenses/agpl-3.0.txt>