JSONFeed River with CouchDB as Backend
I use Pushover to get custom notifications on my phone. I do get all kinds of notifications. I get notifications from my server about their status, Notifications from scrapers like the state of HOPCOMS scraping, Notifications from my weather sensor and AQI etc. Well some of them could have been emails or dashboard1 but I like short messages. I just want to get one liner as to what happened.
As you see not all of them are mission critical. So I really don't need a "push" notification. Even if I get a message after 15 minutes of the event, it's fine. I also wanted something simple and based on open protocol. So I figured if my system just created a feed and pushed it to my feedreader it would be fine. And I am sure this is not a new idea.
Anyway I use CouchDB as my personal DB server. So I thought it would be great to use the same as backend. It has great HTTP API to post data to it. So I don't really need much on the incoming side. On the outgoing side I thought, I just need a script to modify/format CouchDB documents into a valid JSONFeed (with channel and items) that can be easily subscribed on an app like Feeder which supports JSONFeeds along with RSS and Atom.
Well that's all this system is. Its just one db, with two types of documents, defined by an attribute called type. Attribute type is channel for channel definition and item for individual item. Item type of document also has additional attribute called channel which specifically mentions to which channel this item belongs to.
Channel document for listenlater channel
{
"_id": "listenlater",
"_rev": "2-f297bdd5aad677a765d2c42a9284310f",
"version": "https://jsonfeed.org/version/1",
"title": "ListenLater Feed",
"feed_url": "http://scripts.thejeshgn.com/feeds/river?channel=listenlater",
"type": "channel",
"author": {
"name": "Thejesh GN"
}
}
Item Document for listenlater channel with enclosures. Which means I can send audio too.
{
"_id": "2020-09-20T21:28:32+05:30|listenlater|1",
"_rev": "2-cf87a2a700f044066d6238f0b44f285e",
"id": "2020-09-20T21:28:32+05:30|listenlater|1",
"title": "Scripting news",
"summary": "Scripting news",
"content_text": "Jon Udell was tweeting about podcasting, and I had a bunch of things I wanted to say, so I talked for 22 minutes in this podcast. I talk about the Daily podcast and Brian Lehrer, two formats that work. Why has the tech for listening to podcasts evolved so slowly? Who has the ability to innovate? Same with blogging. When was the last time there was a serious innovation?",
"url": "http://scripting.com/2020/09/15.html",
"date_published": "2020-09-20T21:28:32+05:30",
"date_modified": "2020-09-20T21:28:32+05:30",
"author": {
"name": "Dave Winer"
},
"tags": [],
"type": "item",
"channel": "listenlater",
"attachments": [
{
"url": "http://scripting.com/2020/09/15/whyIsPodcastingSoPrimitive.m4a",
"mime_type": "audio/mpeg",
"title": " I talk about the Daily podcast and Brian Lehrer"
}
]
}
On CouchDB I wrote a view get the items of a specific channel.
{
"_id": "_design/items_by_channel",
"_rev": "9-59de6bcceae1a9889eeed008b157df0d",
"views": {
"listenlater": {
"map": "function (doc) {\n if(doc.channel && doc.channel == \"listenlater\"){\n emit(doc._id, 1); \n }\n}"
}
},
"language": "javascript"
}
So now I have a Lamnda which takes the channel name as url parameter. Then it reads the channel document. Then it reads the latest 10 items for that channel, combines them as JSONFeed and sends it back. I have mapped it to a custom domain, so I can subscribe to this feed very easily. URL looks like
https://custom-domain.com/feeds/river?channel=listenlater
Add items to this channel is very easy. You can just do a HTTP post to your CouchDB. Then it will appear in your feed reader
curl --request POST \
--url https://your.couchdb.com/river/ \
--header 'authorization: Basic add_your_auth_code' \
--header 'content-type: application/json' \
--data '{
"_id": "2020-09-22T00:07:47+05:30|listenlater|1",
"id": "2020-09-22T00:07:47+05:30|listenlater|1",
"title": "Scripting news",
"summary": "Scripting news",
"content_text": "Jon Udell was tweeting about podcasting, and I had a bunch of things I wanted to say, so I talked for 22 minutes in this podcast. I talk about the Daily podcast and Brian Lehrer, two formats that work. Why has the tech for listening to podcasts evolved so slowly? Who has the ability to innovate? Same with blogging. When was the last time there was a serious innovation?",
"url": "https://thejeshgn.com/projects/nithya-kannada/",
"date_published": "2020-09-22T00:07:47+05:30",
"date_modified": "2020-09-22T00:07:47+05:30",
"author": {
"name": "Thejesh GN"
},
"tags": [
],
"type": "item",
"channel": "listenlater",
"attachments": [
{
"url": "http://scripting.com/2020/09/15/whyIsPodcastingSoPrimitive.m4a",
"mime_type": "audio/mpeg",
"title": " I talk about the Daily podcast and Brian Lehrer"
}
]
}'
I also have a Lambda on the incoming side. That receives various kinds of Webhooks, transforms them before posting to CouchDB database. At some point I will move this transform function to CouchDB validation function.
In the example I have shown a channel that can be public. To make it private. I have hosted a similar Lambda, connected to another CouchDB database, where the channel is UUID and also it takes api_key as url parameter, for example
https://private.custom-domain.com/feeds/river?channel=<guid>&api_key=<long_passcode>
That's where I get my server alerts now. I am quite happy about how well it works.
- I am looking for a generic dashoard that can work as an app on a phone ↩
3 Responses
[…] available on F-Droid and Play store. I use it everyday. Since all my server updates are JSONFeeds. It’s probably the most viewed app on my […]
[…] is not FOSS but Huffduffer Video is. I am planning to sync the Huffduffer video to my homebrew JSONFeed river. The way I will have my own […]
[…] already know I have a pipeline for generating the JSONFeeds. We store the items in CouchDB. We generate feed by pulling items out of CouchDB using a simple […]