A feed aggregator perfect for data hoarding

IW: 2021-05-28
LU: 2021-05-29

Feed aggregators such as newsboat, liferea or elfeed (for the emacs users) are fantastic tools to keep up with what you care about (without having to jump from one site to another every single day) but also for fighting against the attention-grabbing algorithms that social media deploy. However none of the above are very well suited for data hoarding as they store feeds locally in weird formats, enters sfeed.

Sfeed (here's the repo) is a minimal set of tools with two goals in mind:

This fits quite well with the UNIX philosophy as tab-separated files are very easy to interact with using your standard coreutils (grep, cut and the likes). As a result sfeed is the perfect tool for data hoarding as it is only necessary to

This is particularly useful (and trivial) for hoarding youtube channels among other things.

On the other hand the pretty printing tools that sfeed provides are well suited for command line usage, but (as we will see) they can also be used to generate a nice html interface to your feeds. This effectively solves two problems at once; so how do you setup sfeed?

Once sfeed is installed on your system (you can compile it, but some distro have it in their repos so it's worth to check) create a .sfeed directory in your home.

mkdir -p $HOME/.sfeed/feeds

Now we need a configuration file, sfeeds uses $HOME/.sfeed/feedrc. It's of course possible to write it from scratch, but the default one is not overwhelming. You can find it in the sfeed repo as sfeedrc.example. Now adding you own feeds should be trivial just by looking at the given examples.

Finally to update your feeds just run sfeed_update in your terminal. The downloaded feeds (in tab-separated format) can be found under $HOME/.sfeed/feeds; running sfeed_update again will re-download all feeds and merge any new item found.

To look at a feed in a more human-readable format simply run

sfeed_plain $HOME/.sfeed/feeds/feedname

You can supply multiple arguments or even wildcards so sfeed_plain $HOME/.sfeed/feeds/* prints all your feeds nicely. If that's not to your liking sfeed also provides sfeed_html that outputs html insted of plain text; you can then pipe it into a file and point your browser to it.

Even better is sfeed_frames which creates an html page with a menu (you guessed it, it uses frames) that emulates the look and feel of a GUI aggregator such as Liferea (there's even some default CSS you can use for it (available in the repo), or you can write your own). Finally it is trivial to write a script to do all that (update all feeds, then generate an html file) in a single go.

#!/bin/sh

sfeed_update || exit

cd "$HOME"/.sfeed && sfeed_frames "$HOME"/.sfeed/feeds/*

Just put this on a cronjob and forget about it: you'll always have updated feeds easily checkable by pointing your browser to $HOME/.sfeed/index.html as well as a collection of local feeds easy to manipulate with shell scripts.

Sfeed has many features that go well beyond the basic stuff I've described here (you can read about them from the repo's README). With plain text the limit is the sky!