A container for all my views with excerpts from technology, travel, films, books, kannada, friends and other interests. I am Thejesh GN, friends call me Thej.
One of the primary jobs as a Data Archivist at DataMeet is to download and archive the data from the internet. Mostly from government websites. I usually use python scripts to download, scrape and clean the data. But sometimes, I just need to download many files and store them. I could still use python, but its an overkill. So here are some of the methods that I use.
As you would know, I scrape a lot of web pages as a Data Archivist at DataMeet. I usually use BS4 for this, and it’s beautiful, simple, and works. But often don’t want to write a python script to do that, and I need a simple tool to get data out of HTML.
UDISE+ has the most detailed datasets about all aspects of the elementary education system in India. It collects data from 15 Lakh+ schools from across India. It also has aggregated data at Educational Blocks and Educational districts. This post is about these educational blocks and districts. Unified District Information System for Education (UDISE)...
It’s tough to self-host the maps for Indie users. We usually host the wrapper and styles but end up using third-party tile providers. Even though there are good FOSS options for that, it’s still cumbersome if you are Indie. But Protomaps makes it easy.
There are many ways to subscribe to this blog. Email is probably the best.
Thejesh GN (ತೇಜೇಶ್ ಜಿ.ಎನ್) "Thej" is an Independent Technologist, Hacker, Maker, Traveler, Blogger, InfoActivist, Open data and Open internet enthusiast from Bangalore, India. He loves experimenting with all things life and hence some times he is called hacker and other times duct tape. You can read more about him here. Email [ i @ thejeshgn dot com] Twitter: @thej
(C) Thejesh GN.The views expressed on this site are mine (Thejesh GN) alone and do not necessarily reflect the views of my employer. Please read the terms and conditions before reading or commenting on this blog.
