Tagged: Free and Open Source

Building in the times of COVID-19 0

Building in the times of COVID-19

As an engineer, maker, builder, you want to build something during the crisis to help the community. But what can you build as a software engineer that can help the society during this period of crisis? The answer to the question is not easy, As we have not seen something this devastating in...

0

How to be a Good Software Developer

Yesterday I spoke to students of NPTEL. The topic was “How to be a Good Software Developer”. It’s more like my advice to my younger self. The talk went on quite well except for few internet glitches. It was also special we because I was part of NPTEL’s engineering team at Google. I...

1

DataMeet’s COVID-19 India Datasets

DataMeet community has been doing some amazing work in collecting, cleaning and organizing COVID-19 datasets. The focus in on Indian datasets. The data is mainly sourced from different government websites which are freely available to all the Indians. I recorded a video introducing these datasets. In the video we will go through each...

How to Use Docker to run Workflows in Digdag 0

How to Use Docker to run Workflows in Digdag

There is a good chance that you want to allow multiple developers to run their dags/tasks on your Digdag server. It’s painful to maintain the packages/dependencies they need to run their tasks. It specially becomes difficult if they conflict with each other. One way to solve it, is by having some kind of...

3

Seven Useful DigDag Tips and Tricks

Table of Contents1 Setup DigDag Config2 Enable Authentication3 Run a different version of Python4 Use other programming languages or call a binary5 Use Restful APIs6 Use Docker7 Make HTTP Requests DigDag is a pretty simple tool to install and run. There are quite a few tips and tricks that you can use to...