Meaning users can run this without needing a full copy of the
archives in git repositories.
-* HTTP and NNTP proxy support. Allow us to be a frontend for
+* HTTP, IMAP and NNTP proxy support. Allow us to be a frontend for
firewalled off (or Tor-exclusive) instances. The use case is
for offering a publicly accessible IP with a cheap VPS,
yet storing large amounts of data on computers without a
archive locations to avoid SPOF.
* optional Cache::FastMmap support so production deployments won't
- need Varnish (Varnish doesn't protect NNTP, either)
+ need Varnish (Varnish doesn't protect NNTP or IMAP, either)
* dogfood and take advantage of new kernel APIs (while maintaining
portability to older Linux, free BSDs and maybe Hurd).
* Support more of RFC 3977 (NNTP)
Is there anything left for read-only support?
-* Combined "super server" for NNTP/HTTP/POP3 to reduce memory overhead
+* Combined "super server" for NNTP/HTTP/POP3/IMAP to reduce memory,
+ process, and FD overhead
* Configurable linkification for per-inbox shorthands:
"$gmane/123456" could be configured to expand to the
* imperfect scraper importers for obfuscated list archives
(e.g. obfuscated Mailman stuff, Google Groups, etc...)
+* extend public-inbox-watch to support IMAP, NNTP
+
* improve performance and avoid head-of-line blocking on slow storage
+ (done for most git blob retrievals, Xapian needs work)
+
+* HTTP(S) search API (likely JMAP, but GraphQL could be an option)
+ It should support git-specific prefixes (dfpre:, dfpost:, dfn:, etc)
+ as extensions. If JMAP, it should have HTTP(S) analogues to
+ various IMAP extensions.
+
+* search across multiple inboxes, or admin-definable groups of inboxes
+
+ This will require a new detached Xapian index that can be used in
+ parallel with existing per-inbox indices. Using ->add_database
+ with hundreds of shards is unusable in current Xapian as of
+ August 2020 (acknowledged by Xapian upstream).
+
+* scalability to tens/hundreds of thousands of inboxes
+
+ - pagination for WwwListing
+
+ - inotify-based manifest.js.gz updates
+
+ - process/FD reduction (needs to be slow-storage friendly)
+
+ ...
+
+* command-line tool (similar to mairix/notmuch, but solver+git-aware)
+
+* consider removing doc_data from Xapian, redundant with over.sqlite3
+ It's no longer read as of public-inbox 1.6.0, but still written for
+ compatibility.
* share "git cat-file --batch" processes across inboxes to avoid
bumping into /proc/sys/fs/pipe-user-pages-* limits
* linter to check validity of config file
* linter option and WWW endpoint to graph relationships and flows
- between inboxes, addresses maildirs, coderepos, etc...
+ between inboxes, addresses, Maildirs, coderepos, newsgroups,
+ IMAP mailboxes, etc...
* pygments support - via Python script similar to `git cat-file --batch'
to avoid startup penalty. pygments.rb (Ruby) can be inspiration, too.
* highlighting + linkification for "git format-patch --interdiff" output
-* highlighting + linkification for "git format-patch --range-diff" output
- (requires mirroring of git repos)
-
-* parse and allow (semi)automatic-mirroring of "git request-pull" output
- for coderepos
-
-* configurable diff output for solver-generated blobs
-
-* figure out how search for messages with multiple Date: headers
- should work (some wacky examples out there...)
+* highlighting for "git format-patch --range-diff" output
+ (linkification is too expensive, as it requires mirroring)
* support UUCP addresses for legacy archives