]> Sergey Matveev's repositories - public-inbox.git/commit
lei: non-blocking lei/store->done in lei-daemon
authorEric Wong <e@80x24.org>
Tue, 24 Aug 2021 13:06:39 +0000 (13:06 +0000)
committerEric Wong <e@80x24.org>
Tue, 24 Aug 2021 20:11:39 +0000 (20:11 +0000)
commit5a27f82749d723625848c8d1cf216a0f4de73597
tree65ad9470b8b149b860d8a7863b8c14d3067829b1
parent9d6b13687e0e4403523bff301adb4aae027dfdb1
lei: non-blocking lei/store->done in lei-daemon

This allows client sockets to wait for "done" commits to
lei/store while the daemon reacts asynchronously.  The goal
of this change is to keep the script/lei client alive until
lei/store commits changes to the filesystem, but without
blocking the lei-daemon event loop.  It depends on Perl
refcounting to close the socket.

This change also highlighted our over-use of "done" requests to
lei/store processes, which is now corrected so we only issue it
on collective socket EOF rather than upon reaping every single
worker.

This also fixes "lei forget-mail-sync" when it is the initial
command.

This took several iterations and much debugging to arrive at the
current implementation:

1. The initial iteration of this change utilized socket passing
   from lei-daemon to lei/store, which necessitated switching
   from faster pipes to slower Unix sockets.

2. The second iteration switched to registering notification sockets
   independently of "done" requests, but that could lead to early
   wakeups when "done" was requested by other workers.  This
   appeared to work most of the time, but suffered races under
   high load which were difficult to track down.

Finally, this iteration passes the stringified socket GLOB ref
to lei/store which is echoed back to lei-daemon upon completion
of that particular "done" request.
lib/PublicInbox/LEI.pm
lib/PublicInbox/LeiForgetMailSync.pm
lib/PublicInbox/LeiImportKw.pm
lib/PublicInbox/LeiNoteEvent.pm
lib/PublicInbox/LeiPmdir.pm
lib/PublicInbox/LeiStore.pm
lib/PublicInbox/LeiXSearch.pm
lib/PublicInbox/PktOp.pm