Python and Riak DB, a perfect couple for huge scale distributed computing


In this article I am going to show how to use Riak NoSQL database with Python . It been long since I wrote my last post. But this time I am going to introduce a new NoSQL Database which is powerful yet simple,open-source .  ( Riak Database )

What is Riak DB and for god sake why I should learn it ?

Let me tell you a small story . We all know that Amazon is an  E-commerce giant and you some times may be wondering ,how they are handling millions of transactions per second and accuracy is unquestioned . Amazon uses Dynamo DB for all its operations ,some of you have used Amazon S3 service and familiar with concepts like buckets and keys.So it is actually a custom DB developed and used by Amazon . It uses a Dynamo DB database and provides a REST API  access to the users who signs up ,and that setup Amazon call it “Amazon S3 service”.

What Riak to do with Dynamo DB?

Riak DB is created by Basho technologies as an open-source product . It is inspired from Dynamo DB and allow you to enjoy all the features and also additional features than Dynamo DB.If Amazon can do the miracles why can’t you? You can with Riak DB.The structure of Riak DB is almost similar to Dynamo.

In February last year, Facebook acquired the company WhatsApp for $19 billion. WhatsApp serves over 465 million users and there are over 40 billion messages sent in and 19 billion messages sent out every day. Although the company has scaled drastically, they have maintained an uptime of 99%. How have they managed these feats? As WhatsApp engineer, Rick Reed mentioned at Erlang Factory, Erlang is their “secret sauce,” and that has put the language in the spotlight. Riak is built upon Erlang where MongoDB on C++ , Cassandra on Java. That is why Riak is special.

Now what is Riak DB ?

Riak is an open source, distributed database. Riak is architected for:

  • Low-Latency: Riak is designed to store data and serve requests predictably and quickly, even during peak times.
  • Availability: Riak replicates and retrieves data intelligently, making it available for read and write operations even in failure conditions.
  • Fault-Tolerance: Riak is fault-tolerant so you can lose access to nodes due to network partition or hardware failure and never lose data.
  • Operational Simplicity: Riak allows you to add machines to the cluster easily, without a large operational burden.
  • Scalability: Riak automatically distributes data around the cluster and yields a near-linear performance increase as capacity is added.

Riak DB Architecture

Riak uses ring structure for handling keys of multiple nodes


Ok .Leave all the technical things behind construction of DB. I assume you are  a developer who use SQL databases or NoSQL ones like MongoDB,CouchDB. GIve a try to Riak because many big heads like Best Buy  are using Riak comfortably from past few years.

When Riak DB is to be used ?

No database serves all needs. If you are painful while upgrading architecture,and you lose customers or money at down time of server then you should chose Riak because it’s fault-tolerance is supreme. This picture shows where to go.Click on it ,if not visible.



How to install Riak on Linux precisely Ubuntu?

Here we are going to see the installation steps for Riak.You may be guessing it is painful as Cassandra . Nope . It is dead easy to install Riak. Won’t you believe? For other linux installations and Mac OSX visit this link . Linux and Mac Installation Riak

For Ubuntu 12.04+ 64-bit systems follow me now.

First install dependencies for Riak

$ sudo apt-get install libpam0g-dev libssl0.9.8 
$ sudo apt-get install python-dev libffi-dev libssl-dev
 Next use this link to download latest version of Riak.
$ wget
Ok. If your download is too slow,then download from a mirror
then you will get a file called riak_2.0.5-1_amd64.deb in downloaded folder.
Next go to that folder in Linux shell and type the command
$ sudo dpkg -i riak_2.0.5-1_amd64.deb 
 Now Riak will be installed.Start it using command
$ sudo riak start

That’s it a Riak server is running on your system now. See how easy! . For checking whether server is running or not type command

$ sudo riak ping
You will be replied with “pong” message. hurray you installed Riak

How Python can access Riak ?

There is a very good Python client written by Basho for Riak. Using that we can do all CRUD operations on  the database.In Riak,there is a different terminology similar to S3.


  • Bucket  === Table
  • Key-Value === Records


so we are going to store key-value pairs in the buckets. Buckets are name spaces those separate keys. I am going to show you how this can be contrasted with MongoDB documents.

Installing Riak-Python-client

Use pip or easy install command

 $ pip install riak
That’s it . I told you ,it is very simple to setup Riak and jump-start it with Python.

Using Python-client

Let us create buckets,store data in them and next retrieve.

from riak import RiakClient, RiakNode

#All are equivalent , leaving empty means connect to Riak on localhost
RiakClient(protocol='http', host='', http_port=8098)
nodes are the remote systems you are running instances of Riak Servers . When your data size is increasing grow your data horizontally by allocating more nodes to cluster. But for now we are just tweaking single server .So this is required
from riak import RiakClient
client  = RiakClient()
That’s it . We got a client with a connection for Riak. We can create ,delete,modify buckets using that client.Let me create a bucket (table in SQL) called users.

Creating and selecting a bucket

myBucket = myClient.bucket('users')
Remember if bucket already exists it will be selected,else dynamically created instantly.
now myBucket is the bucket for users.I will Insert data into it.

Inserting JSON(or any data) into the bucket

Let us think we got data for insertion from a web form or a product catalog.Normally we store that kind of data MongoDB . I am creating dummy records to insert into Riak.
user1 = {
'name' : 'Naren',
'location' : 'Hyderabad',
'role' : 'Programmer',
'fulname' : ['Naren','Arya','Yellavula']

user2 = {
'name' : 'Saikiran',
'location' : 'Hyderabad',
'role' : 'Dentist',
'fulname' : ['Sai','Kiran','Yellavula']
All python valid data-types  can be inserted into bucket of Riak.
Now we insert those dictionaries into bucket myBucket
obj ='user1',data=user1)
It stores user1 dictionary with key ‘user1’ and user1 information as data. If we wan’t to generate a random key , then use
import uuid
#Keys should be of string type
obj =,data=user1)
or if you skip giving a key ,Riak generates  an arbitrary key and stores value with that key in bucket. Similarly we can add the second user
obj ='user2',data=user2)

Retrieving  data from a bucket

We written data to Riak successfully,now to get it back. get_keys method of bucket returns all keys from that bucket.Using those you can traverse each and every record i.e key=>value pair. use get method  on a bucket to return a record.get takes key of that record.

#Gives <riak.riak_object.RiakObject at 0x7f91607aeb90>
result = myBucket.get('user1')
RiakObject has a property called data where you can see all details of user1. If you need to get keys then use get_keys on bucket.It returns all keys as a list.
#Gives all keys in a bucket
keys = myBucket.get_keys()
Now print roles of all users
for key in myBucket.get_keys():
    print myBucket.get(key).data['role']

#This prints 
I think you got a basic Idea of inserting data into a Riak bucket and also retrieving them.If you are a mongo developer , you got theme by now. There are tons of options for queries and searches. I will discuss them in my next post. First be comfortable with setup and basic CRUD operations as explained above. My next post will be building a REST API with Flask and Riak setup. Don’t miss it, Riak is more powerful than you think.For more details visit these:

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s