8 @item Get its source code
11 $ git clone git://git.stargrave.org/feeder.git
15 @item Compile @command{feed2mdir} utility
18 $ ( cd cmd/feed2mdir ; go build )
21 @item Create feeds state directories
23 You can create feeds subdirectories under @file{feeds/} manually:
26 $ mkdir -p feeds/my_first_feed/@{cur,new,tmp@}
27 $ echo http://example.com/feed.atom > feeds/my_first_feed/url
30 or convert Newsboat @file{urls} file (containing many lines with URLs)
31 with @command{urls2feeds.zsh} to subdirectories hierarchy:
34 $ ./urls2feeds.zsh < ~/.newsboat/urls
35 $ cat feeds/blog.stargrave.org_russian_feed.atom/url
36 http://blog.stargrave.org/russian/feed.atom
39 @command{urls2feeds.zsh} won't touch already existing directories and will
40 warn if some of them disappeared from @file{urls}.
42 @item Download your feed(s) data
45 $ cmd/download.sh feeds/blog.stargrave.org_russian_feed.atom
46 $ ./feeds-download.zsh # to invoke parallel downloading of everything
49 Probably you want to change its default @env{$PROXY} value. It uses
50 @command{curl}, that is aware of @code{If-Modified-Since} and
51 @code{ETag} headers, compressed content encodings and HTTP redirections.
52 If you want to see verbose output, then set @env{FEEDER_CURL_VERBOSE=1}.
54 @item Parse your feeds
57 $ cmd/parse.sh feeds/blog.stargrave.org_russian_feed.atom
58 $ ./feeds-parse.zsh # to parse all feeds in parallel
61 @item Quick overview of the news:
65 habr.com_ru_rss_interesting: 7
66 habr.com_ru_rss_news: 3
68 naked-science.ru_?yandex_feed=news: 1
69 planet.fsfe.org_atom.xml: 1
70 www.astronews.ru_astronews.xml: 1
71 www.darkside.ru_news_rss: 5
80 That will read all feeds titles and create @file{mutt.rc} sourceable
81 configuration file with predefined helpers and @code{mailboxes}
82 commands. Mutt will be started in mailboxes browser mode (I will skip
86 1 N [ 1|101] 2021-02-17 20:41 Cryptology ePrint Archive/
87 3 [ 0| 8] 2021-12-02 19:28 Thoughts/
88 32 [ 0| 8] 2021-02-17 19:32 apenwarr/
89 101 [ 10| 50] 2021-02-14 13:40 Блог Stargrave на русском comments/
90 102 [ 0| 51] 2021-02-17 19:37 Блог Stargrave на русском/
91 316 [ 0| 44] 2021-02-17 19:33 Eaten By A Grue: Infocom, Text Adventures, and Interactive Fiction/
94 ePrint has new entries since last downloading/parsing. Stargrave's blog
95 comments have nothing new, but still ten unread entries.
97 If we open "Eaten By A Grue" mailbox, then will see its entries:
100 1 [2021-01-30 11:00] Zork Zero: The Revenge of Megaboz (0,8K)
101 2 [2021-06-12 11:01] Journey: The Quest Begins (0,8K)
102 3 [2021-04-28 11:00] Eaten By A Cruise (0,8K)
104 ---Mutt: feeds/monsterfeet.com_grue.rss [Nachr:44 60K]---
107 @item Press @code{q} to return to mailbox browser again
109 This is made for convenience, because you will often switch your
110 mailboxes (feeds), but @code{q} quits Mutt by default.
112 @item Press @code{A} to mark all messages read
114 And again this is made for convenience. It will mark both new
115 (@strong{N}) and old-but-unread (@strong{O}) messages as read. You will
116 see left tag-marks near each message to understand what was touched.
118 @item Press @code{o} to open links and enclosures URLs
120 Do it in pager mode and you message will be piped to
121 @command{cmd/x-urlview.sh}, that will show all @code{X-URL}
122 and @code{X-Enclosure} links.
124 @item Index your messages
130 That will create @file{mu/} and @file{search/} directories and run
131 @command{mu index} indexing, that is safely can be done incrementally
132 after each download/parse cycle.
134 @item Search something
136 Press @code{<F8>} in Mutt's index and enter your mu/Xapian search query.
137 Let's search for articles mentioning
138 @url{https://en.wikipedia.org/wiki/Planetfall, Planetfall} during
139 2019-2021 period: @code{Planetfall date:2019..2021}. @command{mu} will
140 create symbolic links in @file{search/} subdirectory to the message. Press
141 @code{<F9>} to switch that mailbox:
144 1 [2021-12-20 07:08] Missed Classic: Stationfall - When Food Dispensers Attack (The Adventurers Guild)
145 2 [2021-11-20 04:52] Missed Classic 102: Stationfall - Introduction (1987) (The Adventurers Guild)
146 3 [2021-11-19 17:54] Boffo Games (The Digital Antiquarian)
147 4 [2021-10-30 23:05] Missed Classic 100: The Manhole (1988) (The Adventurers Guild)
148 5 [2020-05-17 22:16] Round 04 Reveal (unWinnable State)
149 6 [2020-05-16 22:29] Round 03 Reveal (unWinnable State)
150 7 [2020-04-20 11:00] Planetfall (Eaten By A Grue: Infocom, Text Adventures, and Interactive Fiction)
151 8 [2020-04-09 11:00] Beyond Zork (Eaten By A Grue: Infocom, Text Adventures, and Interactive Fiction)
152 -%-Mutt: =search [Nachr:8 215K]---
155 Pay attention that there is different index format, lacking unnecessary
156 message flags display and adding name of the feed in parenthesis.
158 @item Cleanup excess number of messages
160 By default (@env{$FEEDER_MAX_ITEMS}) only 100 entries are processed.
161 Parser only appends them, but does not remove obsolete ones.
167 will clear everything exceeding the quantity limit. You can set that
168 limit on per-feed basis. For example @code{echo 50 > feed/FEED/max}.
169 0 means no limit and keep all the messages.
171 Pay attention that @file{new/} directory is not touched, so you won't
172 loose completely new and unread messages when you are on vacation and
173 left @command{cron}-ed workers.
175 @item If you want to clean download state
178 $ cmd/download-clean.sh feed/FEED
182 @item Download enclosures
184 Many feeds include links to so-called enclosures, like audio files for
185 podcasts. While you mail is not processed by MUA, its @file{new/}
186 messages still there, you can run enclosure downloading process, that
187 uses @url{https://www.gnu.org/software/wget/, GNU Wget}. Specify the
188 directory where your enclosures should be placed. Each enclosure's
189 filename is more or less filesystem-friendly with the current timestamp
193 $ mkdir path/to/enclosures
194 $ ./feeds-encs.zsh path/to/enclosures
196 traffic.libsyn.com_monsterfeet_grue_018.mp3-20220218-152822
198 $ file path/to/enclosures/traffic.libsyn.com_monsterfeet_grue_018.mp3-20220218-152822
199 path/to/...: Audio file with ID3 version 2.2.0, contains:MPEG ADTS, layer III, v1, 96 kbps, 44.1 kHz, Monaural
202 @command{feeds-encs.zsh} do not parallelize jobs, because enclosure are
203 often heavy enough to satiate your Internet link.