Labs: installation & configuration of GlusterFS as synchronous data storage solution.
By: Pascal Charest, Freesoftware consultant
Date: September, 2008.
Synchronization of files in a cloud environment is a challenge in the path of high-{availability, performance}. From simple load balanced web sites to full-blown applications – some files always need to be in sync. Peoples, for simplicity, rely on asynchronous transfer (ie: rsync ), others deploy bigger solutions (ie: block device replication through DRBD or shared storage through AoE protocol & concurrency management with OCFSv2) or even go for the “lazy” “no-shared-storage” solution through NFS.
To address this problem in the PraizedMedia software stack, I decided to give FUSE based GlusterFS a try. Awesome, really ! The technical knowledge to deploy a basic solution is very very low. The modularity of the program also help to have “something working right now”. This isn’t meant as a direct alternative to DRBD or a good SAN deployment but in my use case, it fit perfectly.
In this lab, I will guide you through the installation of GlusterFS on 2 networked systems. They will be both used as “servers” & “client” for the GlusterFS filesystem. They will be sharing a directory (on both system : /var/production/brick), re-mounted as /var/production/static through GlusterFS. Any write I/O on this directory (of any client server) will be synchronized to the pool. This last feature is called “AFR” (for automatic file replication) and is a module (called a translator) to the GlusterFS file system.
The specificity of my environment is around the file-locking management : I don’t need any. By design, the application will never try to write the same file twice on any of the server.
#Installation of requirement (standard tools)
apt-get install flex bison libfuse-dev linux-headaers-`uname -r` curl
#download of the sources
cd /usr/local/src/
curl -O http://ftp.zresearch.com/pub/gluster/glusterfs/1.3/glusterfs-CURRENT.tar.gz
tar zxf glusterfs-CURRENT.tar.gz
# configure
cd glusterfs-1.3.11
./configure --prefix=/usr/local/glusterfs-1.3.11
make && make install
ln -s /usr/local/glusterfs-1.3.11 /usr/local/glusterfs
So we now have a basic 2 servers GlusterFS systems installed. Lets be honest, that wasn’t really hard! We are still missing configuration files though.
#Editing /usr/local/glusterfs/etc/glusterfs/glusterfs-server.vol
#
# glusterfs-servers definition
# volume definition are on first lvl, other are on second lvl (tabbed)
volume brick
type storage/posix
option directory /mnt/production/brick
end-volume
volume server
type protocol/server
option transport-type tcp/server
option auth.ip.brick.allow *
subvolumes brick
end-volume
#Editing the /usr/local/glusterfs/etc/glusterfs/glusterfs-client.vol
#
# glusterfs-client.vol
# volume definition are on first lvl, other are on second lvl (tabbed)
#
volume remote1
type protocol/client
option transport-type tcp/client
option remote-host 002.praized.com
option remote-subvolume brick
end-volume
volume remote2
type protocol/client
option transport-type tcp/client
option remote-host 001.praized.com
option remote-subvolume brick
end-volume
volume mirror0
type cluster/afr
subvolumes remote1 remote2
end-volume
#Launching services (servers and clients)
mkdir -p /mnt/production/brick
/usr/local/glusterfs-1.3.11/sbin/glusterfsd -f /usr/local/glusterfs-1.3.11/etc/glusterfs/glusterfs-server.vol
mkdir -p /mnt/production/static
/usr/local/glusterfs-1.3.11/sbin/glusterfs -f /usr/local/glusterfs-1.3.11/etc/glusterfs/glusterfs-client.vol /mnt/production/static/
You now possess a synchronized directory between your two systems. Please note that GlusterFS require TCP/6996 port to be open. There is also some improvement that can be done to this setup through adding a locking mechanism & i/o thread – I don’t currently need them, but you might.
Enjoy!
Debugging notes ; after starting the server process you should have a kernel process call glusterfs. All log files are in /usr/local/glusterfs/var/log/glusterfs*. After starting the client, “df -h” should show you your new mount point. Careful with UID/GID (&Permission), there is no such thing as root_squash_fs in GlusterFS yet.
Other notes ; Using Amazon EBS would have been the perfect solution if they did allow multiple servers-volume mount and lets us deal with concurrency / lock problems. But, they don’t.
glusterfs & synchronous data storage
574 Responses to “glusterfs & synchronous data storage”
Trackbacks/Pingbacks:
-
-
May 3, 2011
Taylor Lautner…
By the way you might want to check out this cool site I found……
-
-
May 3, 2011
Embroidery Machine…
[...]the following are some hyperlinks in order to internet sites that any of us link[...]…
What would be your next topic next week on your blog.’~.”:
This post was boring. Life moves on. Appears it is pretty energetic right here
Perfect work you have done, this web site is really cool with wonderful information. -Juanita
I think other web-site proprietors should take this web site as an model, very clean and magnificent user friendly style and design, let alone the content. You are an expert in this topic!
Yups, this is a problem that most people today face.You really hit the nail on the head.
http://mitrprasarn.com/forum/index.php?action=profile;u=80502
Thanks so much for giving everyone a very marvellous possiblity to check tips from this website. It’s usually so awesome and as well , stuffed with a lot of fun for me and my office co-workers to visit your site the equivalent of three times a week to read through the new guidance you have. Not to mention, we’re actually satisfied for the exceptional methods you serve. Selected 1 points in this post are really the most efficient I’ve had.
http://forum.sotovik.ru/member.php?u=398134
Have you tried twitterfeed on your blog, i think it would be cool.:*,-~
I would really love to guest post on your blog.`.;”.
I love blogging and i can say that you also love blogging..”-”"
My spouse and i have been quite joyful that Jordan managed to carry out his researching through the entire ideas he came across in your blog. It is now and again perplexing just to choose to be giving for free points that many the others have been selling. And we also do understand we now have the writer to appreciate because of that. The type of illustrations you have made, the straightforward site menu, the relationships you help to create – it’s got all astounding, and it’s facilitating our son and us reason why the article is cool, and that’s rather essential. Many thanks for all!
Can I just say what a relief to seek out someone who really knows what theyre speaking about on the internet. You definitely know how you can carry a difficulty to light and make it important. Extra individuals have to read this and understand this facet of the story. I cant believe youre no more in style because you definitely have the gift.
Are you aware that industry experts approximate longer than one near two U.Vertisements. high-risk workers occur roughed up instead of trying so much at the the this special wellness are actually dangerous? That may be Fifty four million of us! Really as indicated by a major The year 2007 Zogby Online surveys.
WONDERFUL Post.thanks for share..more wait .. …
http://www.traffictravisdownload.com/
I would really love to guest post on your blog.~.::~
For some cause the picture isn’t loading correctly, is there an issue?
Ugh you might be going say that will mens anti aging products is definitely such as your special goods dude
dntr buggered marbles Thayne refund abco nogata strangest panjabi
I’m really enjoying the theme/design of your web site. Do you ever run into any browser compatibility issues? A few of my blog visitors have complained about my website not operating correctly in Explorer but looks great in Opera. Do you have any ideas to help fix this issue?
Very well said, your blog says it all about that particular topic.:-::~