Johnny ( Chianing ) Wang : OpenStack - Swift Dev Box - SAIO on Ubuntu 14.04 via VirtualBox

Openstack swift is part of my job and research for these years. I love the concept and feel lucky it's an opensource and allow me to take closer look of it and even trying to add new feature or improve it ( in - process ).

Before we start to build the swift dev box, let me give a introduction for it. We start my favor Q&A way for it.

First of all what's swift ? Swift is part of openstack object storage module and it's a reliably store billion of objects and petabytes of data, distributed across standard H/W ( eg: JBOD: just bunch of disks ). Swift is distributed, reliable and scalable, since software is responsible for ensuring data replication, distribution, and integrity across the cluster.

Except these above three major features, my personal idea regarding what's the majority differential than other storage system is swift breaking the geo boundary than other storage system since swift has RESTful API allows all operation are performed using standard HTTP calls. Everything is over internet ! You can have a storage somewhere else in the world, is it good ! Here is the features of sum.

Distributed
Reliable
Scalable
RESTful API

And what's object storage you might ask ? Object Storage is not a traditional file system, but rather a distributed storage system for static data. For distributed, it's no central or master point of control but provides greater scalability, redundancy, and durability. For reliable, object are written to multiple disks, spread throughout servers in the cluster. When disk fail, data from other active nodes is replicated to new location in the cluster ( this is similar with other distributed file system such as ceph ) and For scalable, swift scale horizontally by simply adding more disk, and existing data in the system is evenly redistributed.

Ok, I guess you might interesting how object storage can provide so many features to make the distributed file/storage happen. Here is the architecture of object storage. It includes three major part 1. object: a file, 2. container: logic grouping of objects, ( such as folder in OS ) and 3. account: owner. Basic logic is Account wrap around multiple containers and container wrap multiple objects. Then you probably can image, the physical / virtual H/W you might need includes Object/Container/Account servers for building up the swift, plus proxy server as entry point for public API and routing requests to appropriate account. Just be aware object is only handle the retrieving/storing/deleting object but container/account server the object/container list are stored in SQLite DB. Here is the archtecture of sum.

Object Server
Container Server
Account Server
Proxy Server

Alright, I hope the explaination above won't confusing you too much. Now, let's try to build a dev box to feel the features and I do believe you will get more clear picture from these hand-on labs. Since I was interest in setting up a Swift environment for testing purpose, then I download the code on my eclipse follow my previous posting and trying to set up test environment for the code. And I found the SAIO ( Swift All In One ) from Swift github. It's easy and has clear instruction to guide me step by step. Thus, I was able to follow it to finish my Swift Dev Box on a VirtuaBox VM like no brain-er. :)

Even it's said that it works only on Ubuntu 12, but I still try to put it on Ubuntu 14.4 Desktop vm. It's working for me and haven't see the major issue happen. Here is the experiment I have, hopefully it helps.

Swift All In One (SAIO)

Before you start, you might want to setup your Ubuntu VM on virtualbox, here has link for my previous post, which show you how to setup Ubuntu VM on VirtualBox.
http://chianingwang.blogspot.com/2014/11/how-to-install-openstack-juno-on.html

SAIO is great tool for you to set up your own Swift test environment, just follow these steps. (http://docs.openstack.org/developer/swift/development_saio.html) and remember to make sure you try 3 hints I shared as below when you are doing the set up. Before we jump into 3 hints, let me list the outline for installation.

Installing Dependencies

apt-update & install

apt-update
apt-install (dependencies)
apt-install (python-relevant binaries)

mount a partition drive for storage

partition (fdisk)
formatting (mkfs.xfs)
edit file system table (/etc/fstab)
create mount point and mount drive
make soft link (ln) - option
grant access authority - option

Get Swift Code

swiftclient: git clone

swiftclient: Build a development installation of python-swiftclient(need to install python-swiftclient's dependencies before the installation of python-swiftclient, due to a bug in an older version of setup tool)

swift: git clone

swift: Build a development installation of swift

install swift's test dependencies: pip install

Setting up Rsync

create configuration file: rsyncd.conf

4x account nodes
4x container nodes
4x object nodes

enable rync: /etc/default/rsync
start rsync daemo: sudo service rsync restart
verify rsync: #rsync rsync://pub@localhost/

Starting memcached

start memcached: #sudo service memcached start
check memcached: #sudo chkconfig memcached on

Configuring each node

Pupulate the /etc/swift directory itself
update user in {USER} references in the Swift config file

Setting up scripts for running Swift

copy SAIO for resetting the environment
Editing resetswift secript: $HOME/bin/resetswift
install sampe config for running test: sample.conf, test.conf
add environment variable for running test
double check "PATH" includes "bin" directory
Construct the initial rings: #remakerings (create in order to test storage policies)
verify unit test: #$HOME/swift/.unittests
stat main Swift daemon: #startmain

Testing

get X-Auth-Token
use token to get account
check python-swiftclient via swift command
verify function test: #$HOME/swift/.functests
verify probe test: #$HOME/swift/.probetests

Hint 1:

When you first set up your virtaul box, remember add one more SATA drive as "sdb", the you should be able to see it under /dev/.

It should be looks like this when you setup the Ubuntu OS, assume our boot drive is sda 80GB and swift drive is sdb 20GB.

Hint 2:

In "Using a partition for storage" - "1. Set up a single partition", when you are setting up your partition, please follow these steps for partition and format for your another SATA drive - "sdb". I just put more detail since the guide on web is not clear for me.

--partition
#sudo fdisk /dev/sdb

#p ( print )
#n ( new )

p ( primary partition )
partition number 1 - xxxx

#p ( print ) < print again to double check >

#w <write table and exist>

then you will have a new sdb1

--format

#sudo mkfs.ext3 /dev/sdb1

Hint 3

When you are in "configure each node" section, at The contents of the configuration files provided by executing the above commands are as follows: remember update each *.conf ( 2 /etc/swift/proxy-server.conf ~ 16 /etc/swift/object-server/4.conf)'s
user = <your-user-name> to eg: user = demoswift.

Then you can try the commands in "Setting up scripts for running Swift" section, such as curl or $HOME/swift/.unittests, functtests or .probetests

Here has something interests for structuring Swift. Look what's I found and realized.

Swift base on xfs

XFS is a high-performance 64-bit journaling file system

Swift replication rely on Rsync

rsync is a widely-used utility to keep copies of a file on two computer systems the same.
PS: Dropbox is pretty much base on RSync.

Swift leverage Mencached as memory caching for performance improvement.

Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
quick example

Cache Results

            function get_foo(foo_id)
                foo = memcached_get("foo:" . foo_id)
                return foo if defined foo

                foo = fetch_foo_from_database(foo_id)
                memcached_set("foo:" . foo_id, foo)
                return foo
            end

Play with telnet

            #telnet localhost 11211
            Trying 127.0.0.1...
            Connected to localhost.
            Escape character is '^]'.
            get foo
            VALUE foo 0 2
            hi
            END
            stats
            STAT pid 8861
            (etc)

Update:

If you would like to update configuration, reset ring and restart the swift, here are the steps you can try.

Add more storage policy (http://docs.openstack.org/developer/swift/policies_saio.html)

Update $HOME/bin/resetswift script for add another mount point

#sudo vim $HOME/bin/resetswift

edit : /dev/sdb1 ... to such as /dev/sdb2 ... etc

#sudo vim /etc/swift/swift.conf

Add the lines as below

[storage-policy:0]
name = gold
default = yes

[storage-policy:1]
name = silver

~/.bashrc:

Make sure test config file and bin path in ~/.bashrc

#echo "export SWIFT_TEST_CONFIG_FILE=/etc/swift/test.conf" >> $HOME/.bashrc

#echo "export PATH=${PATH}:$HOME/bin" >> $HOME/.bashrc

What's bashrc

setting user-specific environmental variables, and usually will want the settings to be avaialbe in both login and non-login shell. This means that the place to define these variables is in ~/.bashrc file

Reconstruct the initial ring

#remakerings

Unit Test

#$HOME/swift/.unnittest

Start the "main" Swift Daemon

#startmain
if it's not working you can run cli as below directly since it's the major command in script

#swift-init main start

Playing Swift Has lot of fun !

Share Movie Quote:

The ultimate happiness is having someone who supports you in whatever it is that you do.

- (Julie & Julia), 2009

Johnny ( Chianing ) Wang

Monday, January 26, 2015

OpenStack - Swift Dev Box - SAIO on Ubuntu 14.04 via VirtualBox