]>
Sergey Matveev's repositories - public-inbox.git/log
Eric Wong [Tue, 6 May 2014 19:30:32 +0000 (19:30 +0000)]
INSTALL: update support status
We probably won't bother with FastCGI until there is interest.
Eric Wong [Mon, 5 May 2014 19:59:56 +0000 (19:59 +0000)]
examples/apache2_perl.conf: set PI_CONFIG
This is easier and lower-impact than setting HOME anywhere.
Eric Wong [Mon, 5 May 2014 19:56:20 +0000 (19:56 +0000)]
www: set charset=UTF-8 in full message view
Oops, this was an oversight.
Eric Wong [Mon, 5 May 2014 08:48:13 +0000 (08:48 +0000)]
learn: set_list_headers on training
This is idempotent, and should actually/really prevent training
loops.
Eric Wong [Mon, 5 May 2014 08:39:14 +0000 (08:39 +0000)]
mda: attempt to prevent training loops
Eric Wong [Mon, 5 May 2014 01:07:54 +0000 (01:07 +0000)]
feed: add atom feed link to the bottom
This should improve navigation as browsers may not make it
obvious there is an Atom feed.
Eric Wong [Mon, 5 May 2014 01:01:13 +0000 (01:01 +0000)]
view: add newline before list-wide footer
Otherwise the reply link gets squished.
Eric Wong [Mon, 5 May 2014 00:57:06 +0000 (00:57 +0000)]
www: multi-line cloneurl maps to multiple lines
URLs may be long, so give each its own line.
Eric Wong [Mon, 5 May 2014 00:47:41 +0000 (00:47 +0000)]
www: add footers to HTML index and message views
Some advertising is necessary.
Eric Wong [Sun, 4 May 2014 01:38:16 +0000 (01:38 +0000)]
add various TODO items
Screen space is precious, and we do not need it in the abbreviated
view.
Eric Wong [Fri, 2 May 2014 23:16:37 +0000 (23:16 +0000)]
view: API cleanup, remove "as_" prefix
These views are not OO, so the "as_" prefix makes little sense
and "as_html" conflicts with Hval, which is OO.
Eric Wong [Fri, 2 May 2014 20:28:43 +0000 (20:28 +0000)]
reduce copies by passing string refs to Email::MIME
This should reduce data copies and memory usage, according
to Email::Simple documentation.
Eric Wong [Fri, 2 May 2014 20:07:59 +0000 (20:07 +0000)]
git cat-file wrapper uses syswrite for writes
The requests we make to git cat-file --batch are guaranteed to be
smaller than the 512 bytes required by PIPE_BUF, so there will be no
partial writes. Bypass Perl IO layers and write directly to the
pipe to avoid needing IO::Handle here.
Eric Wong [Thu, 1 May 2014 18:33:18 +0000 (18:33 +0000)]
thread: limit scope of monkey patch
Future versions of Mail::Thread may be released to fix this bug.
However, since it's been about 8 years since the bug was reported..
Eric Wong [Thu, 1 May 2014 02:30:00 +0000 (02:30 +0000)]
workaround Mail::Thread memory leak
Thanks to Ask for the patch in
https://rt.cpan.org/Public/Bug/Display.html?id=22817
Eric Wong [Thu, 1 May 2014 01:56:34 +0000 (01:56 +0000)]
purge Email::Address cache
This prevents memory bloat in case we're serving many requests
with a large, diverse set of email addresses (potentially from
malicious spammers).
Eric Wong [Wed, 30 Apr 2014 22:58:23 +0000 (22:58 +0000)]
split out WWW package and CGI/PSGI-specific parts
This should allow us to more-easily test with Plack.
Eric Wong [Wed, 30 Apr 2014 22:04:44 +0000 (22:04 +0000)]
examples: make web configs consistent and add README
Hopefully this is enough to get some folks started.
Eric Wong [Wed, 30 Apr 2014 21:39:24 +0000 (21:39 +0000)]
cgi: set charset in responses
This ought to prevent browsers from misguessing/misdetecting
things.
Eric Wong [Wed, 30 Apr 2014 20:24:54 +0000 (20:24 +0000)]
add example configs for Apache2 mod_perl and CGI
These are probably still popular configurations in some
settings, so include sample configurations to get folks
started. These are trickier than an average CGI script
to setup because we rely on pretty URLs from PATH_INFO
and not ugly query parameters.
Eric Wong [Wed, 30 Apr 2014 20:13:54 +0000 (20:13 +0000)]
"git cat-file --batch" wrapper really mod_perl-compatible
IPC::Open* does not work well under mod_perl (probably because
it does funky things with stdout/stderr). So use the POSIX
dup2 functions instead, which work reliably on all POSIX-like
platforms.
Eric Wong [Wed, 30 Apr 2014 19:17:21 +0000 (19:17 +0000)]
cgi: avoid warning after exec
Found with mod_perl2 (as it overrides exit).
Eric Wong [Wed, 30 Apr 2014 01:03:30 +0000 (01:03 +0000)]
cgi: remove dependency on IPC::Run in CGI
We may not have PATH available on some servers (e.g. webrick)
and must rely on the hardcoded system PATH. My installation of
IPC::Run does not seem to work without PATH set in the env,
however normal Perl "open" calls work fine.
Eric Wong [Wed, 30 Apr 2014 01:02:53 +0000 (01:02 +0000)]
config: avoid double-slash in expand_path output
It looks ugly in error messages to have redundant slashes.
Eric Wong [Wed, 30 Apr 2014 00:19:48 +0000 (00:19 +0000)]
cgi: disable warnings for CGI::NOSTICKY
We probably know what we're doing, here...
Eric Wong [Wed, 30 Apr 2014 00:16:55 +0000 (00:16 +0000)]
favor "--git-dir=..." over "--git-dir ..."
This might improve grep-ability slightly and reduce allocations
while we're at it.
Eric Wong [Wed, 30 Apr 2014 00:14:35 +0000 (00:14 +0000)]
feed: unset GIT_DIR and use "git --git-dir=.. log"
This should elimate all of our %ENV dependencies in the WWW portion
of our code.
Eric Wong [Tue, 29 Apr 2014 05:33:45 +0000 (05:33 +0000)]
cgi: mid2blob does not depend on $ENV{GIT_DIR}
ENV changes do not propagate to child processes under mod_perl
Eric Wong [Tue, 29 Apr 2014 05:10:48 +0000 (05:10 +0000)]
implement our own cat-file --batch wrapper
We use --git-dir=... instead of $ENV{GIT_DIR} because ENV changes
do not propagate easily with mod_perl.
Eric Wong [Tue, 29 Apr 2014 03:43:20 +0000 (03:43 +0000)]
config: use "git config --file" to avoid changing %ENV
%ENV changes do not propagate by default under a mod_perl2
environment, and we might as well work towards being thread-safe.
Eric Wong [Mon, 28 Apr 2014 22:09:30 +0000 (22:09 +0000)]
cgi: make CGI optional for Plack users
CGI.pm is on its way out, so do not make it a hard requirement for
Plack users. We still support CGI-only installations because some
sysadmins and distros may be very conservative.
Eric Wong [Mon, 28 Apr 2014 21:46:44 +0000 (21:46 +0000)]
cgi: wrap self_uri calls for Plack::Request
We should be adding automated tests for this, soon.
Eric Wong [Mon, 28 Apr 2014 21:46:04 +0000 (21:46 +0000)]
feed: call URI->as_string explicitly
Implicit string conversion is more confusing.
Eric Wong [Mon, 28 Apr 2014 21:02:41 +0000 (21:02 +0000)]
use relative URLs in HTML index
Browsers know how to follow relative URLs, so we may generate
smaller HTML pages.
Eric Wong [Mon, 28 Apr 2014 20:47:33 +0000 (20:47 +0000)]
cgi: do not decode path_info
We only generate URI-escaped URLs, so we should not be decoding this
into octets.
Eric Wong [Mon, 28 Apr 2014 20:45:42 +0000 (20:45 +0000)]
feed: swap order of "next" and "head" links
This is one less key stroke for somebody paging through the
list history.
Eric Wong [Mon, 28 Apr 2014 20:04:44 +0000 (20:04 +0000)]
feed: avoid redirect on permalink
This saves one round-trip request response so reduces latency over
slow links. The redirect only exists for convenience and isn't 100%
reliable in case the Message-ID terminates with a .(html|txt)
suffix.
Eric Wong [Mon, 28 Apr 2014 10:55:59 +0000 (10:55 +0000)]
feed: dead code elimination around dropped endpoints
public-inbox is for discussion, so top-level posts do not get
special treatment.
Eric Wong [Mon, 28 Apr 2014 10:25:58 +0000 (10:25 +0000)]
mda: support aliased addresses
This mimics functionality found in -learn. Originally the design
allowed for only one address per-list, but when migrating/hijacking
existing mailing lists, having multiple addresses map to the same
inbox is useful.
Eric Wong [Mon, 28 Apr 2014 07:39:41 +0000 (07:39 +0000)]
feed: disable navigation to previous page
This is unfortunately needed for scalability to long histories.
The design of git requires it to traverse full history to walk
forward in time, since commits can only record past history.
Instead, replace "prev" with a "head" link to zip us back to
the most recent page. Users who wish to go backwards can use
browser history, which should always work on our old-fashioned
HTML pages.
Eric Wong [Mon, 28 Apr 2014 03:27:21 +0000 (03:27 +0000)]
feed: pedantically quote HTML attributes
This is more correct, although it costs us 4 bytes.
Eric Wong [Mon, 28 Apr 2014 04:56:47 +0000 (04:56 +0000)]
allow running as a Plack app without CGI emulation
This might be slightly cleaner, though generating the base URL
now has an ugly condition in it.
Eric Wong [Mon, 28 Apr 2014 04:50:17 +0000 (04:50 +0000)]
uri_escape => uri_escape_utf8
We should be able to deal with URIs with non-ASCII characters in
them. I only found this problem when looking at archives with
non-English spam :x
Eric Wong [Mon, 28 Apr 2014 02:15:04 +0000 (02:15 +0000)]
cgi: preliminary Plack compatibility
This needs further testing and refactoring, but seems to work
reasonably well.
Eric Wong [Sun, 27 Apr 2014 06:21:57 +0000 (06:21 +0000)]
feeds use XHTML to avoid tag soup
This should work in most browsers, lets find out!
Eric Wong [Sun, 27 Apr 2014 05:04:02 +0000 (05:04 +0000)]
Feed: <id> element must be a URI, oops :x
For each feed element, we'll just use the link since there's
currently no suitable URN.
Eric Wong [Sun, 27 Apr 2014 01:56:39 +0000 (01:56 +0000)]
view: uri_escape subject
This hopefully improves compatibility with mailers
Eric Wong [Sat, 26 Apr 2014 22:22:35 +0000 (22:22 +0000)]
feed: comment about the %deleted hash
It's strictly not necessary anymore since
commit
fa6168c56bdd1cece178b6b852a9b2cba6ce6ffb
("feed: message must exist in current HEAD to show up")
However it can still save us some unnecessary syscalls and
round-trips to the "git cat-file --batch" process, so it's probably
worth th cost of stuffing it in a hash.
Eric Wong [Sat, 26 Apr 2014 21:32:52 +0000 (21:32 +0000)]
cgi: style: return undef => return
Eric Wong [Sat, 26 Apr 2014 09:07:42 +0000 (09:07 +0000)]
feed: message must exist in current HEAD to show up
We do not want recently-deleted spam showing up when looking
in old histories.
Eric Wong [Sat, 26 Apr 2014 07:42:25 +0000 (07:42 +0000)]
spamassassin rule and config updates
While we're at it, add a script for easy editing of user prefs.
We need some human-maintained rules based on the spam we get.
It's an imperfect world, but I'd _much_ rather deal with the
occassional spam than require signup/registration to post.
Eric Wong [Sat, 26 Apr 2014 02:52:50 +0000 (02:52 +0000)]
view: show References: header, too
Some mail user agents use this header, and Mail::Thread uses
it, too, so show it if possible, but only if it's not redundant
to In-Reply-To.
Eric Wong [Sat, 26 Apr 2014 02:29:22 +0000 (02:29 +0000)]
view: add per-message HTML footer to encourage replies
This may not work with most mail user agents, however.
Eric Wong [Sat, 26 Apr 2014 01:01:10 +0000 (01:01 +0000)]
huge refactor of encoding handling
Hopefully this simplifies and corrects our usage of Perl encoding
APIs.
Eric Wong [Fri, 25 Apr 2014 07:49:14 +0000 (07:49 +0000)]
cgi: eliminate dead/redundant HTML escaping code
Eric Wong [Thu, 24 Apr 2014 00:21:21 +0000 (00:21 +0000)]
html: refactor header value handling to be OO
This helps us keep track of escaping which needs to be done
for various levels.
Eric Wong [Wed, 23 Apr 2014 10:50:08 +0000 (10:50 +0000)]
t/feed: cleanup test and add test for /f/ link
We already depend on IPC::Run, so just use it our tests.
Eric Wong [Wed, 23 Apr 2014 01:05:56 +0000 (01:05 +0000)]
feed: add tests for <id> element setting
We need to ensure this works
This follows commit
bd8fd095067b79a0d2a40bbca2b27b923d02b3f8
("feed: fix address when multiple addresses exist")
Eric Wong [Mon, 21 Apr 2014 20:39:54 +0000 (20:39 +0000)]
t/feed: notify user of missing XML::Feed
One of my dev machines did not have XML::Feed so things were
not tested sufficiently.
Eric Wong [Tue, 22 Apr 2014 09:48:53 +0000 (09:48 +0000)]
view: fix link to raw message from /f/ endpoint
Ugh, at least this has a test...
Eric Wong [Tue, 22 Apr 2014 09:24:45 +0000 (09:24 +0000)]
fix quoted URL generation in feeds
While we're at it, make sure strange characters are escaped properly
in Message-IDs. We'll need tests for all this behavior.
Eric Wong [Tue, 22 Apr 2014 09:22:30 +0000 (09:22 +0000)]
view: do not decode Message-ID
This is ridiculous, nobody (including ssoma) ever does this.
Eric Wong [Mon, 21 Apr 2014 19:29:32 +0000 (19:29 +0000)]
feed: fix address when multiple addresses exist
This needs to be cleaned up
Eric Wong [Mon, 21 Apr 2014 18:33:22 +0000 (18:33 +0000)]
README: add links to try and HTML archives
Eric Wong [Mon, 21 Apr 2014 10:43:59 +0000 (10:43 +0000)]
config: use description file for gitweb
Do not repeat ourselves, just use the same description file
gitweb uses to avoid surprising users.
Eric Wong [Mon, 21 Apr 2014 10:10:14 +0000 (10:10 +0000)]
html/index: fix broken prev/next pagination on short histories
We do not have much history in public-inbox meta, so do
not mislead users with strange navigation elements which
lead nowhere.
Eric Wong [Mon, 21 Apr 2014 10:00:21 +0000 (10:00 +0000)]
feed: there is only one atom feed, with all messages
This is not a blog. All posts, whether replies or not,
carry equal weight.
Eric Wong [Mon, 21 Apr 2014 09:53:53 +0000 (09:53 +0000)]
html/index: do not waste space with non-existent thread roots
Screen real-estate is valuable, and missing roots tend to
be false-positive matches (using Subject, not In-Reply-To
or References).
Eric Wong [Mon, 21 Apr 2014 09:19:13 +0000 (09:19 +0000)]
doc: update design_www and add HACKING file
Document some of the stranger choices I've made.
Eric Wong [Mon, 21 Apr 2014 08:07:53 +0000 (08:07 +0000)]
new scripts for importing slrn spools and maildirs
The old import_gmane_spool script was inflexible,
since we may import from maildir archives as well, so
get everything into maildir, first.
Eric Wong [Mon, 21 Apr 2014 01:45:24 +0000 (01:45 +0000)]
README: fix URL for source code clone
This is an 80x24.org project (more on that at another date).
Eric Wong [Mon, 21 Apr 2014 00:18:33 +0000 (00:18 +0000)]
scripts/dc-dlvr: allow exiting from ~/.dc-dlvr.pre
The ~/.dc-dlvr.pre script for my public-inbox user does this.
Eric Wong [Sun, 20 Apr 2014 23:27:46 +0000 (23:27 +0000)]
use ORIGINAL_RECIPIENT once again
It should be common for a single users to be subscribed to multiple
addresses/lists, so we must use the address before alias expansion.
This partially reverts commit
b949afc9edf89dd494cac6255c78b124d58e11a5
Eric Wong [Sun, 20 Apr 2014 20:40:55 +0000 (20:40 +0000)]
scripts/import_gmane_spool: set git committer date
We normally want committer date to be different so we may
track delivery latencies (which do not differ much).
However, the rules for importing are much different and
tend to screw things up when using time ranges with git-rev-list.
Eric Wong [Sun, 20 Apr 2014 20:35:51 +0000 (20:35 +0000)]
various documentation cleanups
Eric Wong [Sun, 20 Apr 2014 19:31:23 +0000 (19:31 +0000)]
feed: close string ref before returning
Just in case there is an error, this should be more explicit.
Eric Wong [Sun, 20 Apr 2014 11:17:16 +0000 (11:17 +0000)]
cgi: delay some requires
This shaves off nearly 100ms when my Core2Duo is clocked to 800Mhz
when rendering a full HTML index.
Eric Wong [Sun, 20 Apr 2014 11:00:06 +0000 (11:00 +0000)]
feed: speed up blob checks if Git.pm is usable
Git::cat_blob is a handy interface to read multiple emails
without incurring fork + exec overhead. Git.pm is GPLv2+,
not GPLv2-only, so we may link to it.
Eric Wong [Sat, 19 Apr 2014 23:23:10 +0000 (23:23 +0000)]
mda: share commit setup code with -learn
We need -learn to do many of the same things as -mda
when we have a false-positive. We also need -learn to
do HTML filtering in case the training user screws up.
Eric Wong [Sat, 19 Apr 2014 23:11:00 +0000 (23:11 +0000)]
move precheck to MDA namespace
We will be combining common code between -learn and -mda
Eric Wong [Sat, 19 Apr 2014 19:19:06 +0000 (19:19 +0000)]
various documentation updates
We have an HTML homepage, OMG!
Eric Wong [Fri, 18 Apr 2014 22:18:17 +0000 (22:18 +0000)]
mda: rename PI_FAILBOX to PI_EMERGENCY
The emergency destination may be Maildir. A Maildir emergency
destination is better for volatile data which is written to
and deleted-from frequently.
Eric Wong [Sat, 19 Apr 2014 10:37:28 +0000 (10:37 +0000)]
cgi: index pages allow iterating some pagination
This allows WWW readers to slowly page through the entire history
of the mailing list.
Eric Wong [Fri, 18 Apr 2014 22:43:59 +0000 (22:43 +0000)]
view: fix regression in standalone /^>$/ lines
The lack of trailing whitespace in quote prefixes threw us
off and cause t/view to fail.
This failure was caused by
commit
fefea3d7d2484ffbf433aec0dd80026aa7120e07
("ensure per-message short quotes do not get too long")
and not caught before pushing because I failed to run
"make", only "prove" (and not even "prove -l" :x).
Eric Wong [Thu, 17 Apr 2014 22:41:57 +0000 (22:41 +0000)]
ensure per-message short quotes do not get too long
Sometimes a single long word or URL can lengthen the line
too much.
Eric Wong [Thu, 17 Apr 2014 22:23:01 +0000 (22:23 +0000)]
cgi: sort HTML index by most recent date
This is hopefully the most user-friendly method.
Eric Wong [Thu, 17 Apr 2014 22:08:31 +0000 (22:08 +0000)]
view: remove pointless self-linkage
It is pointless to link to ourselves.
Eric Wong [Thu, 17 Apr 2014 22:05:04 +0000 (22:05 +0000)]
view: fix title of HTML views
We need to take care to escape everything properly to avoid
HTML/JS injections.
Eric Wong [Thu, 17 Apr 2014 21:56:12 +0000 (21:56 +0000)]
add example for CGI with Ruby WEBrick
Some people like old-fashioned Ruby and WEBrick is in the Ruby
standard library, so widely available.
Eric Wong [Thu, 17 Apr 2014 21:49:52 +0000 (21:49 +0000)]
view: inline shorter quotes
Breaking up short quote messages at 1 line is too disconcerting,
try 5 lines as proper quotes shouldn't be too long.
Eric Wong [Thu, 17 Apr 2014 21:31:06 +0000 (21:31 +0000)]
cgi: implement suffix-less Message-ID redirects
This may be easier in some cases for copy+paste, but not 100%
reliable in case the .txt and .html suffixes are in the Message-ID
itself.
Eric Wong [Thu, 17 Apr 2014 21:30:21 +0000 (21:30 +0000)]
cgi: include HTTP status in error message body
This makes it slightly easier for out-of-the-box curl users since
curl does not report or show the error by default.
Eric Wong [Thu, 17 Apr 2014 20:10:38 +0000 (20:10 +0000)]
HTML: various encoding fixups
Eric Wong [Thu, 17 Apr 2014 03:19:49 +0000 (03:19 +0000)]
Feed: add bug note on memory cycle
This affects users of long-lived processes (FastCGI/Plack)
Eric Wong [Tue, 15 Apr 2014 07:10:17 +0000 (07:10 +0000)]
Revert "cgi: relax path restriction for top-level"
CGI mounts should probably handle this internally. We're reverting
this since it adds too much potential for abuse with fake/extra
prefixes in the URL. We also need to reorder our redirect handling
as a result.
This reverts commit
c394de9f2c91c2c5ed1f7832a5a7cc0206120b7f .
Eric Wong [Tue, 15 Apr 2014 07:06:45 +0000 (07:06 +0000)]
cgi: correct links to folded quotes
Lightly tested, but this seems to work OK.
Eric Wong [Tue, 15 Apr 2014 06:52:07 +0000 (06:52 +0000)]
view: finish HTML in per-message pages
Eric Wong [Tue, 15 Apr 2014 06:51:41 +0000 (06:51 +0000)]
cgi: support /all.html page with inline threads
Maybe this increases readability for now.
Eric Wong [Tue, 15 Apr 2014 06:35:52 +0000 (06:35 +0000)]
HTML: use shorter URLs in indices
Long URLs are not needed for HTML pages, but may be for feeds since
they're often resyndicated and not consumed by the browser.
Eric Wong [Tue, 15 Apr 2014 06:18:43 +0000 (06:18 +0000)]
HTML: ensure hrefs are quoted properly
We may be breaking some parsers or allowing more breakage
to slip through without quotes. We waste some bytes, though.