Pascal Charest - blog d’un consultant en logiciel libre

NOTE: Now on www.mass-storage.org, this blog-post isn’t up-to-date anymore. Please see mass-storage.org for the up-to-date labs note.

Fun stuff with DRBD

Ok, so yesterday, I’ve tried without much success to rebuild my computer lab with Debian/SID and unstable DRBD-8.2.5. Now that I know that the main branch of drbd can contain "unusable version", it will go a bit faster.

Installation of DRBD-8.2.4 took around 60 seconds, most of it being the download from their website and the copy of the source tree between Crystal and Ruby, my two lab systems.

# cd /usr/local/src
# wget http://oss.linbit.com/drbd/8.2/drbd-8.2.4.tar.gz
# tar xvf drbd-8.2.4.tar.gz
# apt-get install linux-headers-`uname -r` build-essential flex docbook-utils
# cd /usr/local/src/drbd-8.2.4
# make all
# make install

Online verification of the sync. status

Now the fun part :

(ruby)# drbdadm verify store

It worked like a charm. I used the "verify-alg md5;" line in my config since the kernel crypto. API already had this algorithm available and loaded. Being able to have an online verify allow me to remove the "data-integrity-alg" function I had in some of my setup - verification once a while does really reduce the cpu processing overhead of DRBD. 

The crypto. API interface speed can be tested with

# openssl speed

and currently available (loaded) functions can be queried with :

# cat /proc/crypto

Adding some security

Another thing I had never tried in the past is activating this security feature :

(/etc/drbd.conf)# cram-hmac-alg "md5" ;
(/etc/drbd.conf)# shared-secret "password";

Once again, worked as supposed. I can now see the HMAC handshake when the peer connect. The module is automatically loaded in the crypto API.

Primary/Primary setup ?

Now, here is the true test I wanted to do.

(/etc/drbd.conf)# uncommenting the "allow-two-primaries" line
(ruby&crystal)# /etc/init.d/drbd stop ; /etc/init.d/drbd start
(ruby&crystal)# drbdadm store primary

I now have a Primary/Primary setup. Fun, yet we need a filesystem with support for concurrent connections. Lets go for OCFS2 (The docs say that GFS is also supported).

(ruby&crystal)# apt-get install ocfs2-tools
(ruby&crystal)# mkdir /etc/ocfs2

The creation of the config file is very straight forward :

(/etc/ocfs2/cluster.conf)

node:
ip_port = 7777
ip_address = 10.0.0.18
number = 0
name = crystal
cluster =lab

node:
ip_port = 7777
ip_address = 10.0.0.19
number = 1
name = ruby
cluster = lab

cluster:
node_count = 2
name = lab

Configuration of the Heartbeat process is also very easy (careful to use the good cluster name).

(ruby&crystal): dpkg-reconfigure ocfs2-tools

Then the magic begin:

(ruby&crystal)# /etc/init.d/o2cb start
(ruby)# mkfs.ocfs2 /dev/drbd0
(ruby&crystal)# mount -t ocfs2 /dev/drbd0 /storage

Et Voila.

Concurrent access to the same filesystem on 2 computers. Some-one said "Cheap load-balancing/hot-fail-over for web-server" ? For the optimization part, can I loudly suggest to go, at the very minimum, with giga speed network interfaced… which bring the point that infiniband isn’t the price it used to be… and performance/latency are really a big step forward…

220-602 is very easy if you already have 640-863 and 642-432 or only 70-297 on your credit. However, going for EX0-100 might be a bit more difficult and doing 70-431 would help tremendously.

While updating my Gnu/Linux lab, I’ve decided to put the latest version of DRBD (stable: 8.2.4, unstable: 8.2.5) on the testing bench. I wanted to try the "online verification" and "primary/primary" state for cluster filesystem (OCFS2, GFS).

The current version available through Debian repository is out-of-date (v8.0.8) and doesn’t have the online verification option, so I’ve had no other choice than to build my own modules & utils. Another problem was the "out-of-date" status of the ./drbd-8.2/INSTALL file. Especially about Debian systems - in fact, most of the debian related stuff seem to be broken.

So here goes the missing "INSTALL.debian" for DRBD-8.2.x. This is hosted on googledocs and will change as I invest time into it.

The whole "normal procedure" for the unstable version of DRBD over a minimal Debian/SID install would be summarized as :

# apt-get install git-core
# cd /usr/local/src
# git-clone git://git.drbd.org/drbd-8.2.git drbd-8.2
# apt-get install linux-headers-`uname -r` build-essential flex docbook-utils
# cd /usr/local/src/drbd-8.2
# make
# make doc
# make install

This will give you a valid DRBD-8.2.5 installation. You’ll need to modify /etc/drbd.conf to match your setup. One cool new feature is the "online verification":

You add the following line inside your syncer section of /etc/drbd.conf and modprobe the kernel module:

// in /etc/drbd.conf, syncer section: verify-alg crc32c;
# modprobe crc32c

# drbdadm verify store

where store is my ressource name. But…. this isn’t the end of my problems… because the command doesn’t work here. This cause my primary system to lose connection with the secondary node. Humfff… i’ll see what I can do about that tomorrow.

NOTE: finall, the problem is easy enough : the unstable is not a working version of DRBD.

For 640-863 or even 642-642 it is important to have some background knowledge of 70-292 and 70-528. If you already have 70-536 to your credit, you may be exempted from SY0-101 as well.

OpenLetter from one computer guy to another. If your from ISF, or DigitalDays, please disregard.

Today’s peoples think they know computer networking when they can configure an IPv4 software firewall on a full layer 2 network. This is the plague of the industry : myriad of self-taught semi-genius with ego the size of the world. I’m one of them… but only with 2/3 of those aspects, chose from the list ;-).

I’ve been doing some research about Zeroconf [Apple's implementation is distributed under the name Bonjour, successor of RendezVous], multicast DNS, DNS Service Discovery… and most of the resources are merely re-hashed version of the same page. When I say “re-hashed”, I’m crudely understating: the wording is the same… mere copies. Yet there seem to be thousand of people waving the flag of “MultiCasting Expert”.

Lets speak of other “expert”. Who really knows what is IGMP [_wp], ASN [_wp] , BGP [_wp] or even a Looking Glass [_wp]? Jeez, it’s pathetic. Or maybe i’ve been interested in WAN technologies and other people were more centered on a local system administrator point-of-view? That must be it, but i’ve been a member of NANOG for so long that their never-ending soul searching isn’t fun anymore and to most of today’s “computer operator”, Internet is all about domain name, http, [maybe ssh], firewall and ipv4.

Forget WAN, lets speak LAN. Who heard of ndb storage engine [_wp] ? federated storage engine [_wp]? Who can, at least, tell me which software they are used in? How is a grey list really working? What state can a network connection be, from iptables point of view? heard of ipchains? What is QOS… not the canned answer, a factual use in your network. Answer that question in front of your boss… I want to know the answer to the “why aren’t we using it?” that will follow.

Ok, you haven’t been playing on technical aspect of IT, lately. What about your view on Digital Right Management? Net Neutrality? [Can you at least explain what the subject is?]

Teaching in the computer industry now seem more of a stochastic process where you are happy if you can fumble on the answer before anyone else. This post is a simple statement of my opinion that we don’t need more “firewall guy”… if you like computers, please, at the very least, learn how it really work.