Satyamev Jayate is on Star Plus every sunday @ 11am. Episodes are also on Youtube (Available in other Indian Languages too)

The world Tomorrow is Russia Today and is not available on any Indian Channels. But its on internet :)

Go watch them.

1. Steve Jobs by Walter Isaacson
2. The Art of Deception by Kevin Mitnick
3. Ghost in the Wires: My Adventures as the World’s Most Wanted Hacker by Kevin Mitnick
4. The Selfish Gene by Richard Dawkins
5. Wizard: The Life and Times of Nikola Tesla : Biography of a Genius
6. The Girl with the Dragon Tattoo by Stieg Larsson
7. The Girl Who Played with Fire by Stieg Larsson
8. The Girl Who Kicked the Hornet’s Nest by Stieg Larsson
9. What Do You Care What Other People Think?: Further Adventures of a Curious Character by Richard P. Feynman
10. Burning Chrome by William Gibson (Progress)
11. The Kite Runner by Khaled Hosseini (Progress)

I want to do at least 25 books this year. Suggestions please. You know what I like :)

You should actually read that blog post but the following are the most important points

No chance to defend.
There is no need to inform users before this content is removed. So, even material put up by a political party can be removed based on anyone’s complaint, without telling that party. This was done against a site called *CartoonsAgainstCorruption.com”. This goes against Article 19(1)(a).

Government censorship, not ‘self-regulation’.
The government says these are industry best-practices in existing terms of service agreements. But the Rules require all intermediaries to include the government-prescribed terms in an agreement, no matter what services they provide. It is one thing for a company to choose the terms of its terms of service agreement, and completely another for the government to dictate those terms of service.

Video: why the IT Rules are a threat to your Internet as you know it

Government, police or an angry mob can force your blog (freedom of expression) or your business (web-based) to go offline without notice. Which is kind of insane isn’t it?

Thankfully there are some smart MPs in our parliament who are trying to pass a motion to annul these rules. We can’t just sit and watch. We need to support MPs who are supporting this motion. Call other MPs and ministers to urge them to support this motion.

To call your MP, ministers and to send them an email, use this online form provided by CIS. Its easy doesn’t take more than a minute. If you want to do more use the postal address to send hard-copies of the letter or use the given official phone number to call them.

To support Member of Parliament P. Rajeeve, go change.org and sign the petition. It doesn’t stop there. Share this information with your friends and family. Urge them to do the same.

It is very difficult to stay anonymous online. I believe there is no way you can remain anonymous forever. Finding you depends on how much time, intelligence, computer power and motivation your opponents have. Identification can only be delayed for practically long time depending on the steps you take.

This how-to is not probably complete. My knowledge is quite limited. But if I ever want to remain anonymous, I would follow all of them. In fact I follow most of these steps just to be safe online.

I have tried to make it as simple as possible for non techies. But there are pointers for the geeks to follow into the rabbit hole. They are usually marked with [g] for geek alert.

Trustable Fake Identity

There are two ways to remain anonymous online. Say if you are a blogger, your username could be so obvious that your adversaries will know you are trying to hide or you could use a very realistic fake identity which will confuse them. I would go second way. But I would give a thought about country, city, religion, sex and other identifiable characteristics of my identity before I decide on upon one. It’s very important to create a trustable fake identity. It’s not an easy job. So work on it.

Computer OS – Linux

The most important thing to remain safe online is to keep your computer safe. You need to be able to trust your computer to do the right thing. I use Linux (Debian/Ubuntu). I trust them because its open source and most of the vulnerabilities are available in the public domain. So I know whats going on in the system.

So use Linux as much as possible. If you can’t avoid using Windows or Mac try dual booting or a live CD. Linux like before is not very difficult to use these days. Distros like Ubuntu are very user-friendly. Try it.

Tails is a live CD or live USB that aims at preserving your privacy and anonymity. It helps you to use the Internet anonymously almost anywhere you go and on any computer. All connections to the Internet are forced to go through the Tor network. The OS leaves no trace on the computer you’re using unless you ask it explicitly. It uses state-of-the-art cryptographic tools to encrypt your files, email and instant messaging. It’s an all in one package which is very useful non geeks. Install it on your USB and use it.

For highly paranoid ones there are distros like TinFoilHat [g] available. It even protects you against electromagnetic radiation eavesdropping [g].

Disk Encryption

Have you ever thought what happens to your data if you lose your machine or give it for repair? Remember your data is your identity. Its very important to encrypt your hard disk. On Ubuntu you can enable home directory encryption by default. In case if you find it difficult or you have many portable harddisks or use any other OS then use TrueCrypt. It’s an open-source, transparent, on the fly encryption which is very easy to use. It works on all the platforms. Portable versions are available too.

Firewall and Anti virus

Keeping your computer safe is very important to anonymous online. You really don’t want a trojan horse sending your information to somebody. Ubuntu comes with built-in firewall (it’s not difficult to install one either). You can install Gufw which is a great GUI to support your Ubuntu firewall. Latest version of windows comes with the firewall. Enable it if you haven’t already.

Anti virus is a necessary evil. I use Avast and clamav. Both are available for all three platforms. clamav is an open source project. I prefer using it.

Connecting to Internet – Tor

On internet every machine will have an ip address. It works as a primary identifier online. It’s very important to hide/fake it to remain anonymous. Tor is free software and an open network that helps you defend against a form of network surveillance that threatens personal freedom and privacy, confidential business activities and relationships, and state security known as traffic analysis. Individuals, journalists, whistle blowers from all over the world use Tor. You should use it too.

Tor prevents anyone from knowing your location or browsing. Tor can be used with browsers, instant messaging clients, remote logins, and more. Tor is free and open source for Windows, Mac, Linux/Unix, and Android phones. You should use it.

Tor Browser Bundle[g] lets you use Tor on Windows, Mac OS X, or Linux without needing to install any software. It can run off a USB flash drive, comes with a pre-configured web browser to protect your anonymity, and is self-contained. Very useful when you are traveling and want to remain anonymous.

Tor makes your traffic is channelized anonymously but it doesn’t promise data security between you and the end server. You must always use HTTPS, encrypt your mails, OTR for IM etc.

[g] Active content, such as Java, Javascript, Adobe Flash, Adobe Shockwave, QuickTime, RealAudio, ActiveX controls, and VBScript, are binary applications. These binary applications run as your user account with your permissions in your operating system. They might bypass Tor. I would avoid using them when I want to remain anonymous.

Special case : Open Public WiFi

Public WiFis (or any wifi/lan which you not in control of) are not safe. Don’t fall for free wifi at coffee shops or at any other public spaces if your concern is privacy and safety. Data between your computer and wifi router is plain text and anybody on network can read it. As though you are reading your dairy loud and clear. If you really want to use public wifi, you must use https at the least.
[g] I have a machine at home which runs Tor and acts as VPN server for me. On public Wifi I connect through my home VPN server which goes through Tor. My setup is similar to the one explained in this blog title Tor on SSH. It’s not very difficult to build an SSH tunnel yourself.
[g] SSHing over Tor is very interesting.

Browser – Firefox

I will use Firefox since it’s the best Free and Open Source software available. Keep updating it. Use private browsing mode.

Browser Add Ons

Https Everywhere Many sites on the web offer some limited support for encryption over HTTPS, but make it difficult to use. For instance, they may default to unencrypted HTTP, or fill encrypted pages with links that go back to the unencrypted site. The HTTPS Everywhere extension fixes these problems by rewriting all requests to these sites to HTTPS.
No Script Blocks Javascript. The best security you can get in a web browser!
Allow active content to run only from sites you trust, and protect yourself against XSS and Clickjacking attacks.
Tor Button Torbutton is the component in Tor Browser Bundle that takes care of application-level security and privacy concerns in Firefox. To keep you safe, Torbutton disables many types of active content.
Better Privacy serves to protect against special longterm cookies, a new generation of ‘Super-Cookie’, which silently conquered the Internet. This new cookie generation offers unlimited user tracking to industry and market research. Concerning privacy Flash-cookies are most critical.
This add-on was made to make users aware of those hidden, never expiring objects and to offer an easier way to view and to manage them – since browsers are unable to do that for you.

Special case Browser plug-ins

Active content, such as Java, Javascript, Adobe Flash, Adobe Shockwave, QuickTime, RealAudio, Silverlight, ActiveX controls, VBScript, etc are binary applications. They run inside a sandbox in the browser. They aren’t safe. Disable them if you don’t want.

Email- Encryption – GPG

If you are online you have to use email. Use a provider who gives HTTPS access to email. Keep it in sync with your fake identity. Use it only for the related communications and nothing else. It’s very important to keep it safe. But remember just the HTTPS doesn’t promise security. You need to encrypt it. Get used to GPG encryption if you want complete privacy. It’s not very difficult. You can GPG-Shell which is great UI for GPG or GPG4USB which is a simple and clean UI that can be used by anybody. Try it.

GPG is a must if you want to keep your communications secure. Make sure the keysize is at least 2048 bits.

[g] You can also use GPG with RetroShare is an Open Source cross-platform, private and secure decentralised communication platform. It lets you to securely chat and share files with your friends and family, using a web-of-trust to authenticate peers and OpenSSL to encrypt all communication. RetroShare provides filesharing, chat, messages, forums and channels. I USE IT.

IM – OTR

If you like to use your regualr IM and keep the conversations off the record. This off the record is not same as GChat OTR. Gchat OTR is lame. It’s not actually OTR. You should use OTR Plugin with pidgin and google IM account for this. Installation is quite easy.

[g]Off-the-Record (OTR) Messaging allows you to have private conversations over instant messaging by providing Encryption (No one else can read your instant messages.), Authentication (You are assured the correspondent is who you think it is.) Deniability (The messages you send do not have digital signatures that are checkable by a third-party. Anyone can forge messages after a conversation to make them look like they came from you. However, during a conversation, your correspondent is assured the messages he sees are authentic and unmodified.) Perfect forward secrecy (If you lose control of your private keys, no previous conversation is compromised.)

Blogging

Use the fake identity and email. Avoid using any Javascript or flash embedded content as much as possible. Specially avoid Advt and analytic. Have a very strong password.

[g]if you are paronoid it’s not difficult to sign your blog post. Just to avoid hostile take over your blog account and posting on your blog on your behalf.

Posting pictures or videos

When you are posting picture and videos online remove the metadata. Some cameras and phones are known to leave signatures on them. Be careful about it. Make sure people who appear in the picture/video are okay with it and sworn to confidentiality of your identity. Or you might have to mask them with really good software (lame work can lead to reconstruction of the image, identification of them and then you). Like ObscuraCam for Android camera.

Passwords and reset password settings

Make sure your passwords are strong and long. Specially of your email. Once somebody gets access to your email then he gets access to all your accounts. So it’s very very important. Also make sure reset password(or forgot password) questionnaires of all your accounts are strong.

End Note

I have wanted to share this for a long time. But then its #MakeBlog which got me to write this. Thanks guys.

Obligatory warning: Use it to protect yourself and not to harm anybody. This article is written as per my best knowledge. There could be loopholes. Let me know if you find any. I will buy you a coffee, update the post and give credit.

You can use my GPG public keys to send an encrypted email.

We started at sharp 10 am with me introducing to the concept of BarCamp and the day, followed by panel discussion.

From then on it was a smooth ride. I didn’t have to do other than time management.

I liked all the morning session talks. I could not attend most of the noon sessions. Among the ones I attended Anand’s talk on Pictures through Numbers

and Shekhar’s Open Data & Free Maps are my favorites.

I spend most of my afternoon either tweeting or scheduling or in conversations.I couldn’t do my session, may be next datacamp.

This camp was marginally different from the regular Barcamps. Morning sessions were done in a single hall. And it was curated to keep the audience interested. After noon sessions were in three halls + corridor. This time due to lack of time I had to do the curation of talks all by myself. But I would settle for standard barcamp way next time. Never the less most participants liked my curation. So I am happy.

By six we were done. But the conversation in the corridor and bar continued till eight. I reached home by 10 dead tired. A day well spent.

A big thanks to my co-organizer Nisha and all the volunteers from DataMeet. It would not have been possible without them. Thanks to Meera for all the pictures. They are on Flickr. All videos were shot and edited by our friends at HasGeek. And at last thanks to all the sponsors (Google, MSR, IWP, Gramener, Akshara, CIS and HasGeek) for working with us and trusting us. Now I am waiting for the next DataCamp.

Yesterday, I sent my first invitation mail to datameet group.

We are writing to invite you to the 1st Open Data Camp in India on Saturday, March 24, 2012, at Bangalore.

This event is dedicated to all aspects of open data, from working with data, to getting it, and of course how to use it to create impact. This event is being organized by DataMeet, an online group of data enthusiast who hope to use data to create an impact in the lives of people living and working in India.

This Open Data Camp would bring together all the main development sector actors working with/on open data. Some of them include Nonprofit Organizations like India Water Portal, Akshara Foundation, Accountability Initiative, Azim Premji University and PRS Legislative;
Also, the Indian Government, has just passed the National Data Sharing and Accessibility Policy - essentially the Open Data Policy for India. The policy itself is rotten and nothing to do with ‘open’ and ‘sharing’; but considering that before this policy all data in India was part of the ‘Official Secrets Act’, this is no small gain.

Policy and Advocacy Groups like Centre for Internet and Society, Tactical Tech Collective, and of course the Government of India;

and, the broader interest group that include Technologists, and Technology Companies (from Google and Gramener), designers and Design Schools, Journalists and Media Groups.

I hope you can attend the event. Registration is compulsory to attend the event.   Please register at doattend.com

You can find more about the event @ http://odc.datameet.org    — it’s still under progress. I will keep updating it.

If you have any questions feel free to contact me at any time.

Thanks,
Thej, Nisha and Team.

Please do register.

There are huge systems and frameworks built on the concept of MapReduce. They use distributed filesystem, have fault tolerance and can process petabytes of data. But I wanted something simple. I wanted something that’s minimalistic and does everything that a MapReduce framework should do and is written in Python.

“Map” : The master node takes the input, partitions it up into smaller sub-problems, and distributes them to worker nodes.

“Reduce” : The master node then collects the answers to all the sub-problems and combines them in some way to form the output.

I found MinceMeatPy and Octo.py. Both are single python file MapReduce frameworks. mincemeatpy is actively developed, where as last checkin to octo.py was probably in 2008.

I thought the best way to learn the concept is to write the framework that implements it. But then reinventing the wheel is waste of everybody’s time. So I choose the middle ground and forked Octo.py and called it Octopoda.

I removed lot of code and in turn made it simple and inflexible. Added simple auth, added some examples, created a wiki and road map and how could I forget ASCII art :)

============================================================
        _____                                  _
       / ___ \       _                        | |
      | |   | | ____| |_  ___  ____   ___   _ | | ____
      | |   | |/ ___)  _)/ _ \|  _ \ / _ \ / || |/ _  |
      | |___| ( (___| |_| |_| | | | | |_| ( (_| ( ( | |
       \_____/ \____)\___)___/| ||_/ \___/ \____|\_||_|
                  MapReduce for HumanBeings
          Repo: http://code.thejeshgn.com/octopoda
============================================================

I am now working on channel encryption. I need help. The project is hosted on bitbucket. Go ahead and fork and send me pull request with your changes.

A standard MapReduce example is counting words.

#wordCount.py
source = {1:"Humpty Dumpty sat on a wall",
2:"Humpty Dumpty had a great fall",
3:"All the King's horses and all the King's men",
 4:"Couldn't put Humpty together again" }

def final(key, value):
    print key, value

# client
def mapfn(key, value):
    for w in value.split():
        yield w, 1

def reducefn(key, value):
    result = 0
    for v in value:
        result += v
    return result

On server:
$ python octopoda.py server ./examples/wordCount.py

On client or nodes:
$ python octopoda.py client localhost_or_server_ip

You can start as many clients as you want. Server will handle task distribution and aggregation. I know this is an overly simplistic example. With a little modification the same example can be made to calculate the word count from all the files in a directory. I will write about that in my next post. Until then have fun.

This process is inspired by Heroku git deployment feature. This tutorial works with hg, git and mostly with any other DVCS with minor alteration. It has two major steps
STEP 1: On your *nix Server

1. Install mercurial on your server – it should be easy
2. Setup SSH access to mercurial repository
   Your server should be able to login to code repository and pull the latest code. Its easier to use SSH than passwords.
   On your server machine:
   1. Open terminal
   2. Enter ssh-keygen
   3. Give a name or you can use the default name id_rsa
   4. When it asks “Enter passphrase (empty for no passphrase):” press enter. No Password
   5. Once the key generation is complete. You can verify the same using ls -a ~/.ssh
   6. Add this new identity to SSH agent ssh-add ~/.ssh/id_rsa
   7. Now we need to add this public key to bitbucket or any other provider cat ~/.ssh/id_rsa.pub. Copy the output
   8. For bitbucket, go to account -> SSH keys, add the above output to your ssh keys

   Now your server is set to access your repositories with out the need of password.

3. Now we need to setup the Hg repository inside web accessible directory. For example, your web accessible folder could be
/home/user_home/public_html or /var/www/html
   or in case of phython it can be anywhere /home/user_home/my_project_code
4. To deploy php application tweet4blood, clone the repo inside the directory using ssh url
   
cd /home/user_home/public_html
hg clone ssh://hg@bitbucket.org/thejeshgn/tweet4blood tweet4blood.com

5. Make sure to make the .hg folder (actual repository) inaccessible to the webserver either by .htaccess rule or changing the permissions etc

STEP 2: On your desktop

1. Install fabfile or fabric. On ubuntu search for fabric in Synaptic Package Manager
2. Create your fabfile, you can find the latest version of below example fabfile in my snippets project
3. To call any method in fabfile
   
$ cd /home/thej/my_deployment_scripts/tweet4blood/
$ fab hello


4. Fabric can also chain the calls
   
$ cd /home/thej/my_deployment_scripts/tweet4blood/
$ fab test hello

   Here it calls the test method first which sets the env variables and then calls hello
5. None of the env variables are necessary but providing env.user, env.hosts, env.password will avoid typing them everytime
6. BTW env.user, env.hosts, env.password are that of SERVER machine
7. To deploy the latest version to test
   
$ cd /home/thej/my_deployment_scripts/tweet4blood/
$ fab test deploy:tip

   In this case,
   • test method sets the env variables corresponding to TEST env
   • test method also sets application env specific consumer_key which later we will use to setup config.php, similarly you can use define databas_name, database_user_name etc
   • then as per chain deploy method is called with input variable version whose value now is “tip”
   • inside deploy the first call is cd (change directory) on remote server
   • at this point fab logs into the remote server using env.user, env.hosts, env.password
   • then control goes to the repo directory
   • runs hg pull which gets everything from bitbucket
   • runs hg update -C tip which is clean update to “tip” version
   • then CDs into auth folder
   • uses the Linux sid command to replace the env specific values in config.php
8. To deploy any other version to test. I usually tag the versions, so I will pass the tag name
   
$ cd /home/thej/my_deployment_scripts/tweet4blood/
$ fab test deploy:v.0.1.0


9. In case you want to shutdown the apache before the deployment and restart later, you can chain them too
   $ cd /home/thej/my_deployment_scripts/tweet4blood/
$ fab test apache_stop deploy:tip apache_start

STEP 3: Go deploy
Below I have embedded a rough version of fabfile.py for quick refrence. But as I told you can find the latest version of the same in my snippets.

Questions and suggestions are welcome.

################################################################
# It uses fabfile to build and deploy
# Ref: http://thejeshgn.com/2012/02/01/deployment-php-python-app-using-mercurial-and-fabfile
# usage:
#        cd to the folder
#        $ fab hello
#
#To deploy tagged version to test:
#        cd to the folder which has fab file
#        $fab test deploy:REPO_TAG_NAME
#
#To deploy latest version to test:
#        cd to the folder which has fab file
#        $fab test deploy:tip
#
#To deploy to prod:
#        cd to the folder which has fab file
#        $fab prod deploy:REPO_TAG_NAME
#
#

################################################################
from __future__ import with_statement
from fabric.api import *
from fabric.contrib.console import confirm

def hello():
    print("Welcome to web deployment.")

def test():
    env.user = 'user_name'
    env.hosts = ['100.100.100.100']
    env.password ='password'
    env.consumer_key ="test_xGxRxzXrxoxSxrxYxuxdxnxLxBxzQU00Q"

def prod():
    env.user = 'user_name'
    env.hosts = ['200.200.200.200']
    env.password ='password'
    env.consumer_key ="prod_xGxRxzXrxoxSxrxYxuxdxnxLxBxzQU00Q"

def update(version):
    with cd('/home/user_home/public_html/tweet4blood.com'):
        run('hg pull')
        run('hg update -C '+version)
    with cd('/home/user_home/public_html/tweet4blood.com/html/auth/'):
            consumer_key_defnition = "define('CONSUMER_KEY', '"+ +"');"
            run("sed -i 's/define('CONSUMER_KEY', 'xGxRxzXrxoxSxrxYxuxdxnxLxBxzQU00Q');/"+consumer_key_defnition+"/' config.php")

def apache_restart():
    sudo('/etc/init.d/apache2 restart') 

def apache_stop():
    sudo('/etc/init.d/apache2 stop') 

def apache_start():
    sudo('/etc/init.d/apache2 start')

def apache_status():
    sudo('ps aux | egrep "(PID|apache2)"')

def deploy(version):
    update(version)