Solutions include disabling the innodb_file_per_table setting & reducing the buffer_pool size…

Then with games:

Then with cloths:

And… There’s more coming! With Christmas I’ve got a cart of interesting items LabsPhoenix is going to buy for the ‘home’ lab. At the very least, a new mac-mini, a Razer mouse, a eSATA external enclosure (with the HD), 2 SSD drives, 2 Infiniband network card (with associated cables), OEM board (ie: Soekris, Wrap, …) with gigabytes connections and an ubiquity setup (3 antennas + router board). If I’m lucky, I might even buy this 24U enclosure I’ve been watching for a few months. There should also be a couple spending session for the LabsPhoenix own project (3x New Servers, vSphere licences, …) but thats another thing altogether.

photo: Business open as usual, by Pascal Charest

Re-launch of blog.pacharest.com under a new name (cloudweaver.org) & new url. Lots of reasons. Main one? I wanted it to be so – but also :
– because I have contractual engagement that the change of domain name will help to clear up.
– because I want to focus on network infrastructure (mainly dynamic and virtualized ones) from now on.
– because I wanted to integrate twitter somewhere on my blog (now directly in my feed/post page).

But, beside everything that might be happening with my business (work log never been that full, started to get employees), I decided to take 2 minutes for a big fact of life (for sysadmin/management). Watch it, watch it: Nobody (especially YOU) care about backup. You care about successful restore. There! Now, I don’t want to hear about how great your backup are, if you don’t do regular restore (even as test), they’re not worth anything.

Reread those last sentences. Important point in a sysadmin life.

PHP version 5.3.1 was just released. This release contains a patch for a denial of service condition we’ve reported on 27 October 2009. The problem is related with PHP’s handling of RFC 1867 (Form-based File upload in HTML).

Source: http://www.securityfocus.com/archive/1/507982

Exploit already on PacketStorm…

About Hypertec

As a rule, never visit somewhere without background info : Hypertec-BCDR (Business Continuity and Disaster Recovery) (they also use the name Hypertec-AS for the french version) is the hosting, datacenter & high availability services division of the Hypertec Group. The group look like an umbrella corporation which also hold the Hypertec Systems division (kind of a computer retail shop). The exact financial details are private (the group is private / NOT available in the stock market), but from what I’ve heard, the whole group have about 120+ employee and a sales figure of about 20M$/years. Those are very rough numbers, I could be totally off the track, and include all their activities (don’t know for the data center aspect only). There seems to be office in a couple locations (Montreal, Quebec, Ottawa, Toronto… ).

So, its quite strange that I haven’t heard about them… especially since they are located inside the old Nortel building in Saint-Laurent. I’ve also contact friends about them, and they were virtually unknown!

The visit

… and this is why I’m doing a blog post on them: because Jonathan Ahdoot, sales manager, walked me through their data center and I must say, he was able to impress me. The main surface is reserved for tier-4 dedicated cages to which you can add a small quantity of tier-2 rack (about 60) setup. As a reminder, in datacenter higher tier speak of better quality (scale from 1 to 4 – as defined by the uptime institute)(different from Internet peering tier).

The visit make clear quite fast why I hadn’t heard about them : they fish for the big ones and government (which can be considered a big one) contracts. They have rooms for rent that act as office away from office for couples of days, they have a 10 posts technical room, a cafeteria (which can become 24h) and … behold: a lounge. Yes ! a true lounge with satellite TV and couches. How many time would I have given everything (my clients own ;-)) for a nice couch while waiting for a file copy between the SAN and the server I’m restoring @ 2h AM. They also make their conference room available to clients (which is another nice feature, especially for office-less consultant (me!)).

I’m far from being a data center specialist: I build infrastructure and I rack them somewhere – this is mainly what I do. So I cannot go into big details about all the nice features the data center seemed to have or in the small point why it might not be as great as I think. However, there is one thing that did impress me: There is 5 flywheel energy storage system in the main engineering room, all being provided by electricity (Hydro) and hooked on a generator. This was also the first time I’ve heard about flywheel energy storage (FES), but I do find the idea quite neat. There must be a lot of energy lost through friction (even if they are in vaccum), but it does look like a system way more secure than batteries (UPS) for data center. Secure as in : I’ve already been screwed twice by “this was a planned maintenance and the ups didn’t turned on, or the tech turned off the wrong line”.

But the sky is not totally blue: Since they do seem to target tier-4 clients, they lack a bit of the standard facility we require in a tier-2: renting 48U racks rarely leave you the space for screen, mouse, keyboard, screwdriver… you expect them to be readily available on site. From what I’ve saw, they were either lacking or in bad shape (tier-2, again… the tier-4 look awesome). Anyway, if you got a cage (with multiple rack) and you don’t have space for tools, you have others problems.

Anyway, a couples contracts will require me to be in data center for the next few months (migrating 35U, deploying 20U, re-designing 24U…). So I guess I will be posting more reviews as time goes.

The swekey is a small USB key that secures access to any swekey enabled web sites.
Swekey secured web sites won’t let you login without your swekey plugged to your computer.
The swekey can also be used to secure corporate’s intranet, unix servers access, and database administration.
[...]

Swekey device, Photo by Pascal Charest

usb 2-8: new full speed USB device using ohci_hcd and address 3
usb 2-8: configuration #1 chosen from 1 choice
Initializing USB Mass Storage driver…
scsi10 : SCSI emulation for USB Mass Storage devices
usbcore: registered new interface driver usb-storage
USB Mass Storage support registered.
usb-storage: device found at 3
usb-storage: waiting for device to settle before scanning
usb-storage: device scan complete
scsi 10:0:0:0: CD-ROM Musbe Swekey 1.03 PQ: 0 ANSI: 0
sr1: scsi-1 drive
sr 10:0:0:0: Attached scsi CD-ROM sr1
sr 10:0:0:0: Attached scsi generic sg3 type 5
cdrom: This disc doesn’t have any tracks I recognize!
usb 2-8: reset full speed USB device using ohci_hcd and address 3

Then : Nothing. No auto-mount, no dialog box… Kinda of left there. The partition cannot be mounted…

Going to their website, I learn the official working steps: “BUY” (pseudo-done), “PLUG” (done), “REGISTER” (ugh?) and I’m “READY”. The REGISTER (the step I’m at, right ?) section give me an error of ‘missing plug-in’ from Mozilla Firefox 3.0.14. Ok, browsing “Support”/”Download” inform me of missing dependencies (a software must be installed) to access the device. I download the x64 GNU/Linux version and … hum ?

pcharest@hydra:~/Desktop/swekey$ cat README
Swekey client
This package install:
- the swekey-client command line tool
- the swekey HAL module
- the swekey Mozilla plugin

The swekey-client command line tool gives you the list of plugged swekeys
and let you calculate OTPs with them.

type:
swekey-client –help
to get the available options

To install swekey-client just type:
sudo ./install
or
./install
if you are root

To uninstall swekey-client just type:
sudo ./uninstall
or
./uninstall
if you are root

I have no idea what is an OTP but let say I try installing the client:

sudo ./install

and validate the device is detected:

./swekey-client –list

It work and give me a device ID. Good, at least the device is known by the system. I still don’t know how it should work. I guess I should be installing the Mozilla plug-in the readme mentionned, but… I never found it. I guess the client install worked (and it was included) because after a Mozilla reload, the Manage section of their web page give (or might also be one of the random file I clicked on) :

Registration is not mandatory but it will allow you to disable a lost or stolen Swekey.

So… I don’t really need to register the key… lets try it then (which I’ve been trying to do for quite a long time at this point).

I own quite a few Zabbix servers, so, from the list of supported service :

ZABBIX is an enterprise-class open source distributed monitoring solution.
A swekey integration exists, it is still a patch but you can ask for it if you need to test it.

Ok, still want to test the device – So i try with MediaWiki:

And it started to work well : creation of an account (user+password), then I get asked if I want to bind this account to my Swekey. This won’t allow me to auto-login but will require the key to be present in any computer (with the installed software) to access the account.

Summary: As a summary, I’d say that while it give a boosted security (require the Swekey to log) – it does seem to go a bit over the limit of the permanent fight between conviviality and security. Installing the software is complicated and might be very problematic on system without administrator access… Personally, having tried both, I would prefer Paypal key ID to be integrated to more website. There is no need to ‘install’ the software on any computer and it give you the same added security the Swekey does.

apt-get install libc6-xen

echo ‘hwcap 0 nosegneg’ > /etc/ld.so.conf.d/libc6-xen.conf ; ldconfig

mv /lib/tls /lib/tls.disabled

And this is where you will start to despair, because, of the 26 200 Google results for “4gb seg fixup“, 26 000 are either linking to a post asking the same question or answering with one of those two answers and 200 are for 4gb usb key. Neither of which will help the message to go away from your syslog and bring it back NGINX/Ruby to decent speed.

So. At this point, where you are starting to think about wiping everything and starting back from scratch (which won’t help), try this little procedure. The principle is to remove the passenger gem from your system, reinstall it (which will only download the source), modify the makefile, recompile NGINX (which in turn automatically compile the Phusion Passenger module) and take a beer while your system serve ruby pages without (systems) errors.

#> gem uninstall passenger
#> gem install passenger

We have a valid passenger gem source code in /var/lib/gems/1.8/gems/passenger-2.2.4 - version can vary and location is valid for Ubuntu/Debian, but could change on others distro. We will be modifying the optimization flags given to the compiler. Since Phusion Passenger does not accept command line argument and variables declarations, we have no other choices than to modify the rake file pre-compilation.

#> sed ‘s/EXTRA_CXXFLAGS = “-Wall #{OPTIMIZATION_FLAGS}”/EXTRA_CXXFLAGS = “-Wall -mno-tls-direct-seg-refs #{OPTIMIZATION_FLAGS}”/g’ /tmp/rakefile

#> mv /tmp/rakefile /var/lib/gems/1.8/gems/passenger-2.2.4/Rakefile

This being done, we will start an NGINX compilation process which will, in turn, start passenger-2.2.4 compilation. Using the -mno-tls-direct-seg-refs will allows us to work arround the 4gb seg fixup error.

#> CFLAGS=”-mno-tls-direct-seg-refs” CXXFLAGS=”-mno-tls-direct-seg-refs” ./configure –prefix=’/usr/local/nginx-0.7.61′ –add-module=’/var/lib/gems/1.8/gems/passenger-2.2.4/ext/nginx’ –with-http_ssl_module –with-http_stub_status_module

There you go.