X-Git-Url: http://www.git.stargrave.org/?p=public-inbox.git;a=blobdiff_plain;f=TODO;h=7a27fdd2f716e80d1542c94a00662e42c5b90ba3;hp=2d70995201fb10c8d5b1cbf404e9db6707953676;hb=23af251dd607c4e75ab1e68063f2c885c48cc035;hpb=715d371ec61237804ebd988248ee73aa5943b7d9 diff --git a/TODO b/TODO index 2d709952..7a27fdd2 100644 --- a/TODO +++ b/TODO @@ -1,5 +1,158 @@ -* implement RFC 4685 (Atom message threading) -* configurable constants (quote folding, index limits) -* use tags as date-based skiplists for navigating history -* improve + document mlmmj integration, currently only at: - http://bogomips.org/unicorn-public/20140508084301.GA2033%40dcvr.yhbt.net/ +TODO items for public-inbox + +(Not in any particular order, and +performance, ease-of-setup, installation, maintainability, etc +all need to be considered for everything we introduce) + +* general performance improvements, but without relying on + XS or pre-built modules any more than we currently do. + (Optional Inline::C and user-compiled re2c acceptable) + +* mailmap support (same as git) for remapping expired email addresses + +* support remapping of expired URLs similar to mailmap + (coordinate with git.git with this?) + +* POP3 server, since some webmail providers support external POP3: + https://public-inbox.org/meta/20160411034104.GA7817@dcvr.yhbt.net/ + Perhaps make this depend solely the NNTP server and work as a proxy. + Meaning users can run this without needing a full copy of the + archives in git repositories. + +* HTTP, IMAP and NNTP proxy support. Allow us to be a frontend for + firewalled off (or Tor-exclusive) instances. The use case is + for offering a publicly accessible IP with a cheap VPS, + yet storing large amounts of data on computers without a + public IP behind a home Internet connection. + +* support HTTP(S) CONNECT proxying to NNTP for users with + firewall problems + +* DHT (distributed hash table) for mapping Message-IDs to various + archive locations to avoid SPOF. + +* optional Cache::FastMmap support so production deployments won't + need Varnish (Varnish doesn't protect NNTP nor IMAP, either) + +* dogfood and take advantage of new kernel APIs (while maintaining + portability to older Linux, free BSDs and maybe Hurd). + +* dogfood latest Xapian, Perl5, SQLite, git and various modules to + ensure things continue working as they should (or more better) + while retaining compatibility with old versions. + +* Support more of RFC 3977 (NNTP) + Is there anything left for read-only support? + +* Combined "super server" for NNTP/HTTP/POP3/IMAP to reduce memory, + process, and FD overhead + +* Configurable linkification for per-inbox shorthands: + "$gmane/123456" could be configured to expand to the + appropriate link pointing to the gmane.io list archives, + likewise "[Bug #123456]" could be configured to expand to + point to some project's bug tracker at http://example.com/bug/123456 + +* configurable synonym and spelling support in Xapian + +* Support optional "HTTPS Everywhere" for mapping old HTTP to HTTPS + links if (and only if) the user wants to use HTTPS. We may also + be able to configure redirects for expired URLs. + + Note: message bodies rendered as HTML themselves must NOT change, + the links should point to an anchor tag within the same page, + instead; giving the user options. + +* configurable constants (index limits, search results) + +* handle messages with multiple Message-IDs (done for v2, doable for v1) + +* handle broken double-bracketed References properly (maybe) + and totally broken Message-IDs + + cf. https://public-inbox.org/git/20160814012706.GA18784@starla/ + +* improve documentation + +* linkify thread skeletons better + https://public-inbox.org/git/6E3699DEA672430CAEA6DEFEDE6918F4@PhilipOakley/ + +* Further lower mail parser memory usage. We still slurp entire + message bodies into memory and incur 2-3x overhead on + multipart messages. Inline::C (and maybe gmime) could work. + +* use REQUEST_URI properly for CGI / mod_perl2 compatibility + with Message-IDs which include '%' (done?) + +* better test cases, make faster by reusing more setup + code across tests + +* large mbox/Maildir/MH/NNTP spool import (in lei, but not + for public-facing inboxes) + +* MH import support (read-only, at least) + +* Read-only WebDAV interface to the git repo so it can be mounted + via davfs2 or fusedav to avoid full clones. + davfs2 needs Range: request support for this to be feasible: + https://savannah.nongnu.org/bugs/?33259 + https://savannah.nongnu.org/support/?107649 + +* Contribute something like IMAP IDLE for "git fetch". + Inboxes (and any git repos) can be kept up-to-date without + relying on polling. + +* Improve bundle support in git to make it cheaper to host/clone + with dumb HTTP(S) servers. + +* Expose targeted reindexing of individual messages. + Sometimes an indexing bug only affects a handful of messages, + so it's not worth the trouble of doing a full reindex. + +* code repository integration (cgit: done, TODO: gitweb, etc...) + +* migration path to v2 (making it transparent for "git fetch" + may not be possible, but "public-inbox-fetch" will handle it) + +* imperfect scraper importers for obfuscated list archives + (e.g. obfuscated Mailman stuff, Google Groups, etc...) + +* improve performance and avoid head-of-line blocking on slow storage + (done for most git blob retrievals, Xapian needs work) + +* HTTP(S) search API (likely JMAP, but GraphQL could be an option) + It should support git-specific prefixes (dfpre:, dfpost:, dfn:, etc) + as extensions. If JMAP, it should have HTTP(S) analogues to + various IMAP extensions. + +* scalability to tens/hundreds of thousands of inboxes + + - inotify-based manifest.js.gz updates + + ... + +* lei - see %CMD in lib/PublicInbox/LEI.pm + (there's a truckload here..) + +* make "git cat-file --batch" detect unlinked packfiles so we don't + have to restart processes (very long-term) + +* linter to check validity of config file + +* linter option and WWW endpoint to graph relationships and flows + between inboxes, addresses, Maildirs, coderepos, newsgroups, + IMAP mailboxes, etc... + +* pygments support - via Python script similar to `git cat-file --batch' + to avoid startup penalty. pygments.rb (Ruby) can be inspiration, too. + +* highlighting + linkification for "git format-patch --interdiff" output + +* highlighting for "git format-patch --range-diff" output + (linkification is too expensive, as it requires mirroring) + +* support UUCP addresses for legacy archives + +* support pipelining as an IMAP/NNTP client for -watch + lei + +* auto-detect and reload on TLS cert+key changes in daemons