From: Eric Wong <e@80x24.org>
Date: Sat, 11 Sep 2021 00:19:17 +0000 (+0000)
Subject: lei: normalize whitespace in remote queries
X-Git-Tag: v1.7.0~408
X-Git-Url: http://www.git.stargrave.org/?p=public-inbox.git;a=commitdiff_plain;h=57fed2e4b78ed394;hp=0c2eaee968cb59ae60c10ce89b2c236fd9369830

lei: normalize whitespace in remote queries

Having redundant "+" in URLs is ugly and can hurt cacheability
of queries.  Even with "quoted phrase searches", Xapian seems
unaffected by redundant spaces, so just normalize the ASCII
white spaces to ' ' (%20) when fed via STDIN or saved-search
config file.

Reported-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Link: https://public-inbox.org/meta/20210910141157.6u5adehpx7wftkor@meerkat.local/
---

diff --git a/lib/PublicInbox/LeiXSearch.pm b/lib/PublicInbox/LeiXSearch.pm
index 709a3b3a..9f7f3885 100644
--- a/lib/PublicInbox/LeiXSearch.pm
+++ b/lib/PublicInbox/LeiXSearch.pm
@@ -297,7 +297,9 @@ sub query_remote_mboxrd {
 	local $SIG{TERM} = sub { exit(0) }; # for DESTROY (File::Temp, $reap)
 	my $lei = $self->{lei};
 	my $opt = $lei->{opt};
-	my @qform = (q => $lei->{mset_opt}->{qstr}, x => 'm');
+	my $qstr = $lei->{mset_opt}->{qstr};
+	$qstr =~ s/[ \n\t]+/ /sg; # make URLs less ugly
+	my @qform = (q => $qstr, x => 'm');
 	push(@qform, t => 1) if $opt->{threads};
 	my $verbose = $opt->{verbose};
 	my ($reap_tail, $reap_curl);