gemini://://carcosa.net/journal/20210122-maybe-an-aggregator.gmi
#

Maybe I'll write an aggregator

I've been running my own instance of CAPCOM for a while, privately. The URL probably isn't too hard to find if you poke around my capsule, but I haven't advertised it, because I don't want to either throw a lot of load on my server, or feel like I'm somebody's authority for maintaining a list of feeds. I started it a while ago when CAPCOM was fetching all the known feeds, and I wanted one that updated more frequently. Now I'm fetching 99 feeds, while CAPCOM is fetching 100 out of a larger list.

I want to change because

  • I want to curate my feeds more carefully
  • I want to support both atom and gmi feeds

Schapcom, by Sandra Snan (idiomdrottning) supports both atom and gmi feeds, but I kind of feel like writing my own. I don't need the 'curator' feature from Schapcom, and I want to throw in a few extra features.

One thing that I'm interested in is exponential backoff for checking feeds. A lot of the feeds I'm subscribed to are inactive, and don't need to be checked so often. Maybe I should just remove them, but what if they come to life suddenly? I want to run updates every couple of hours or so, but I don't want to repeatedly hit pages that haven't changed.

Another thing I'm thinking about is parallelism. Not multiple connections to a site, of course, but using multiple connections for multiple sites. This could really cut down on how long a run takes.

Don't know if I'll really write it. I have other responsibilities and other things to do, but I need some fun.