Eric Wong [Fri, 1 Jul 2016 02:09:45 +0000 (02:09 +0000)]
git: allow cloning from the URL root, too
This means we can still show non-git users a somewhat browseable
URL with a link to the README.html file while allowing git users
to type less when cloning.
Eric Wong [Thu, 30 Jun 2016 08:57:49 +0000 (08:57 +0000)]
Merge branch 'thread-view-skel'
* thread-view-skel:
view: show thread size when linking to summary
view: default to flat/hybrid thread display
view: fix up some HTML injection via Message-ID vectors
www: reinstate old thread view as an option
view: show more nearby messages in flat thread view
view: tweak thread/index header slightly
feed: add $INBOX/new.html endpoint
view: merge $state hash with existing $ctx
view: show thread context in the thread-aware flat view
www: use WwwStream for dumping thread and search views
www: implement hybrid flat+thread conversation view
Eric Wong [Thu, 30 Jun 2016 07:46:18 +0000 (07:46 +0000)]
Revert "view: update git-send-email URL"
This reverts commit a391cf5aaf7181b5e5e20eb240c7ce50cbdf8fa2
as kernel.org is updating again. Ref:
http://mid.gmane.org/xmqqpor18a2n.fsf@gitster.mtv.corp.google.com
Eric Wong [Thu, 30 Jun 2016 02:35:19 +0000 (02:35 +0000)]
view: tweak thread/index header slightly
This makes the top permalink/raw as well as the In-Reply-To
show up without search. While we're at it, try to make
the links on the thread index from the "X siblings, Y replies"
more obvious.
Eric Wong [Thu, 30 Jun 2016 02:35:14 +0000 (02:35 +0000)]
www: use WwwStream for dumping thread and search views
This allows us the HTTP server to react to backpressure
from slow clients when writing. As a side effect, this
also makes it easier for us to maintain a consistent
header/footer across our HTML.
This should be more accessible to readers on narrow terminals
(or giant fonts) while providing a chronological view which
is also aware of message threading relationships.
Eric Wong [Sun, 26 Jun 2016 07:58:43 +0000 (07:58 +0000)]
spamcheck/spamc: fix compatibility with Perl 5.14.2
This is necessary for Debian 7.x (wheezy), since
GLOB objects do not seem responsive to the can("fileno")
check (we do a similar check in GitHTTPBackend).
Eric Wong [Sat, 25 Jun 2016 21:29:43 +0000 (21:29 +0000)]
view: safer and optional quoting for --in-reply-to arg
Angle brackets around the --in-reply-to= arg for git send-email
has been optional since git v1.5.3.2, so strip them and make the
command-line argument easier-to-type.
Eric Wong [Sat, 25 Jun 2016 07:19:10 +0000 (07:19 +0000)]
address: beef up the module with name list extaction
We may remove from_name in the future.
...And disallow quotes in email addresses.
Technically I believe they're allowed, but they're definitely
uncommon and unlikely to show up in legitimate mail.
Eric Wong [Fri, 24 Jun 2016 23:40:03 +0000 (23:40 +0000)]
http: cork chunked responses for small savings
This only affects Linux users with MSG_MORE support.
We can avoid extra TCP overhead for sub-optimal chunk sizes
by using MSG_MORE even with chunk trailers under Linux.
This breaks real-time apps which require <= 200ms latency for
streaming small packets (e.g. implementing "tail -F"), but the
public-inbox WWW code does not (and will never) do such things.
Eric Wong [Fri, 24 Jun 2016 20:37:52 +0000 (20:37 +0000)]
watch_maildir: ignore Trash and Drafts, support Dovecot
Trashed messages and drafts are probably not intended for
importing, so do not import them. Dovecot uses extra flags via
lowercase letters, so we must support those (as that's the
server I use).
Eric Wong [Fri, 24 Jun 2016 01:15:12 +0000 (01:15 +0000)]
implement ListMirror SpamAssassin plugin
When running mailing list mirrors, one needs to be careful
spammers do not try to sidestep the list server we want to
mirror from and inject email into our mail directly by setting
the appropriate list headers (e.g. "X-Mailing-List" or
"List-Id"). We trust the top-most Received: header is
the one our own mail server got the mail from.
Bcc:-ing a public mailing list is a very likely indicator of
spam in my experience, so throw in an extra rule mark it.
While public-inbox-mda rejects Bcc: entirely, public-inbox-watch
needs to mirror lists which allow Bcc.
ifplugin PublicInbox::SaPlugin::ListMirror
header LIST_MIRROR_RECEIVED eval:check_list_mirror_received()
describe LIST_MIRROR_RECEIVED Received does not match trusted list server
score LIST_MIRROR_RECEIVED 10
header LIST_MIRROR_BCC eval:check_list_mirror_bcc()
describe LIST_MIRROR_BCC Mailing list was Bcc-ed
score LIST_MIRROR_BCC 1
endif
Eric Wong [Wed, 22 Jun 2016 23:08:40 +0000 (23:08 +0000)]
view: update git-send-email URL
I've setup my own mirror on https://git-htmldocs.bogomips.org/
since kernel.org is no longer updated, and I don't believe
in endorsing commercial hosting sites or interests.
Unfortunately, it still contains JavaScript and CSS, but it's
generated by AsciiDoc. Oh well, it's all Free Software, at
least.
Junio C Hamano writes:
> On Wed, Jun 22, 2016 at 12:00 PM, Eric Wong <e@80x24.org> wrote:
> > Just wondering, who updates
> > https://kernel.org/pub/software/scm/git/docs/
> > and why hasn't it been updated in a while?
> > (currently it says Last updated 2015-06-06 at the bottom)
>
> Nobody. It is too cumbersome to use their upload tool to update many
> files (it is geared towards updating a handful of tarballs at a time).
Eric Wong [Wed, 22 Jun 2016 06:01:07 +0000 (06:01 +0000)]
view: fix topic threading when ghosts are present
This fixes a bug where a message replying to a ghost would
accidentally be added to the wrong topic in the index/topic
view.
Before commit 76d8f68dc273e54809ad69cfe49e141003f790ef ("view:
avoid recursion in topic index"), we would refuse to indent a
topic which started with a ghost which hid the problem. Now we
inform the user the thread started elsewhere.
Eric Wong [Mon, 20 Jun 2016 00:57:16 +0000 (00:57 +0000)]
feed: various object-orientation cleanups
Favor Inbox objects as our primary source of truth to simplify
our code. This increases our coupling with PSGI to make it
easier to write tests in the future.
A lot of this code was originally designed to be usable
standalone without PSGI or CGI at all; but that might increase
development effort.
Eric Wong [Sun, 19 Jun 2016 09:59:31 +0000 (09:59 +0000)]
examples/*@.service: wait one day for graceful shutdown
Because sometimes folks will want to download gigantic mboxes
or make large clones over Tor which are not resume-friendly.
Note: the timeout logic in nntpd is somewhat over-aggressive
and can break some large slrnpulls. This ought to be easily
recoverable on the client-side, though, since it's based on
per-message fetches.
Eric Wong [Sun, 19 Jun 2016 06:55:42 +0000 (06:55 +0000)]
http: constrain getline/close responses by time
This allows us to yield control to other clients gracefully if
getline takes too long to generate a chunk. This is more
expensive but should not cost a syscall on modern 64-bit systems.
Eric Wong [Sun, 19 Jun 2016 04:50:40 +0000 (04:50 +0000)]
mbox: set gzip timestamp to the Unix epoch
This allows consistency between different invocations from
roughly the same period and is no worse for caching any any of
our existing HTML and Atom feeds.
We cannot set the timestamp to the end date since messages
may be added to the repository while we are iterating
(and this streaming mechanism will pick them up).
Eric Wong [Sun, 19 Jun 2016 02:13:52 +0000 (02:13 +0000)]
watch_maildir: tighten up path checks
Only mark seen messages as spam, otherwise it could be
too aggressive and cause problems or over training.
We wouldn't want a wayward FIFO ruining our day, either :)
Eric Wong [Sat, 18 Jun 2016 23:25:20 +0000 (23:25 +0000)]
watch_maildir: spam removal support
We can support spam removal by watching a special "spam"
Maildir, too. We can run public-inbox-learn as a separate
step, and that command will be improved to support
auto-learning, too.
Eric Wong [Sat, 18 Jun 2016 10:53:32 +0000 (10:53 +0000)]
spawn: try to keep signals blocked in spawned child
While we only want to stop our daemons and gracefully destroy
subprocesses, it is common for 'Ctrl-C' from a terminal to kill
the entire pgroup.
Killing an entire pgroup nukes subprocesses like git-upload-pack
breaks graceful shutdown on long clones. Make a best effort to
ensure git-upload-pack processes are not broken when somebody
signals an entire process group.
Followup-to: commit 37bf2db81bbbe114d7fc5a00e30d3d5a6fa74de5
("doc: systemd examples should only kill one process")
Eric Wong [Sat, 18 Jun 2016 08:49:05 +0000 (08:49 +0000)]
view: consolidate per-message newline handling
We don't want to blindly append a trailing newline
if the message ends in quoted text leading to a <span>,
as a newline is already added to a <span>...
Eric Wong [Sat, 18 Jun 2016 00:22:34 +0000 (00:22 +0000)]
view: minor tweaks to reduce long lines
Fold addressee fields to better delimit destinations,
reduce horizontal rule <hr /> to reduce scrolling,
and use spaces to indent "git send-email" example.
Eric Wong [Fri, 17 Jun 2016 21:32:59 +0000 (21:32 +0000)]
view: introduce WwwStream interface
This will allow us to commonalize HTML generation in the future
and is the start of moving existing HTML generation to a "pull"
streaming model (from the existing "push" one).
Using the getline/close pull model is superior to the existing
$fh->write streaming as it allows us to throttle response
generation based on backpressure from slow clients.
Eric Wong [Fri, 17 Jun 2016 01:56:05 +0000 (01:56 +0000)]
import: auto-update index when done
This prevents multiple update processes from stepping over
each other while called under the lock, and also allows the
new -watch process to update the index iff indexing was
desired.