Archive - Informatique RSS Feed

Cloudbursting

September 17, 2008 in , , , , , with 129 Comments

Cloudbursting

Technical term referring to the transition to a dynamic state of a network infrastructure as an events mitigation process. It entail to both asynchronous and synchronous expansion toward or inside a computer cloud. Each such burst is followed by a period of quiescence before the next burst occurs.

Event such as high workload and extreme traffic spike can cause a correctly configured environment to cloudburst.

The term seem to have been used first, in July 2008, by William Fellows (Principal Analyst @ 451group) in a report and pushed forward by Jeffrey Barr (Technology evangelist of Amazon AWS) on a blog in august 2008.

This is my personal definition, yet I think it summarize the process. Anything that needs to be added ?

Microsoft & Open Source

April 8, 2008 in , , , with 12 Comments

6. Use the Open Source certification mark to keep things pure

One of the threats we faced was the possibility that the term ‘open source’ would be "embraced and extended" by Microsoft or other large vendors, corrupting it and losing our message.
[...]   
It eventually developed that the U.S. Patent and Trademark office would not issue a trademark for such a descriptive phrase [Open Source]
[...]
The sorts of serious abuse we feared have not (at least, no yet as of November 2000) actually materialized.

The Cathedral & the Bazaar :: Revenge of the Hackers, Eric S. Raymond, page 178

A couple years ago, I almost skipped this part of the Revenge of the Hacker manifesto since it was evident that Microsoft would not try something that obvious. I was so wrong…

It did took them a couple of years to figure it out, but then, they push forward evenement like {Open Source} Heroes happen here where you can [...]

Order you own Hero Hack Pack and get started with Open Source. Each Hack Pack contains a trial copy of Windows Server 2008 and Visual Studio 2008, plus a chance to win a free pass to OSCON 2008!

I am well aware that free software doesn’t need to be free "as in beer", yet It kinda make me sad that people can pull that kind of scheme and still sleep at night. They are clearly piggy backing on a wave they don’t control and have nothing to do with. You can receive a trial version of a software that is closed source, which is non-free, to run a software that is closed source, also non-free to program open-source. Hey, you can’t say no to that offer!

Why don’t they start their own little revolution and try to gain leverage… because, lets be honest here, Microsoft doesn’t even own any open source application. They have the money and the technical talent to, once again, shake the whole computing world with innovation. Their loud "we support open source" is based on money they gave their own concurrent while keeping their standard closed – they really should not play on this level.

Port 25, Microsoft blogging platform for their open source community, is more of a joke than anything else. They have some freak labs, acces to some of the best programmers in the world and … their messages all look, at least, to me, like :

Microsoft is involved in open source since Apache web server run on their server. That’s without saying anything about the price/availability of IIS… And this is no joke, the last few post from port 25 are installation procedures for Apache.

Microsoft is involved in open source since they have a partnership with XEN (this is more present on their website than the port25 blog). They still push forward their own virtualisation system, but with VMWare currently stealing all the "high value target" and Xen taking everything else…. not much left for VirtualPC.

But, there is one thing that does intrigue me : Shared Source at Microsoft. Well designed, this program could have some leverage, but I guess that everything is released under a reference only licence, that a very big entree fee is required and that your soul must be sold to the devil, or something like that.

And when they really want to go open source, the underlying "help us get more money" is so evident that their isn’t even any fun reporting it. The project look cool though : Singularity.

Update: I’ve been pointed toward this image, sad that it hasn’t been kept up-to-date.

Une section de ma vie

April 1, 2008 in , , , , , , , with Comments Off

A partir de hier:

    – Dans le jour : Benchmark d’application Ruby & des nouveaux services d’Amazon (ip statique et zone d’availability).
    – En soire : Mise en production d’un cloud.
    – En fin de soire : Ecriture d’un article pour un magazine europeen.
    – dodo
    – Ce matin ; deploiement de STP.   
  
J’adore ma vie ;-)

-

Edit:
HUMMMFFF… je viens de voir que googleDocs possede maintenant une nouvelle fonction : Dans un docs, faire File -> "New Airplane".  Ce n’est pas MON poisson d’avril, jure!

Whoaaa… quelle semaine!

March 30, 2008 in , , , with 1 Comment

Ouf, je viens de passer à travers une semaine totalement folle.

Normalement, j’ai une soumission acceptée pour deux composées. Depuis deux semaines ? Toutes mes offres sont revenues signées (1). Mais qu’on t’elles toutes de spéciales : un lien avec ce blog. Oui, oui. Ce petit journal perso de mes folleries informatiques. De toute évidence, mes clients mesure mon expertise technique, en privé, avant de me rencontrer ce qui déplace le sujet de l’entrevu sur mon charactère plus que mon expertise. Et comme dirait un de mes clients : "Your a young arrogant jerk" (pour les intéressé, cette phrase était suivi par "but, be there on thursday")

(1). À l’exception d’une seule, qui c’est révélée être un déploiement pour une firme dont un des clients aurait été un client de mon ancien employeur. Selon mon avocat, je suis très loins de violer ma clause de non-maraudage (pas de clause de non-compétition), mais je n’ai vraiment pas de temps à perdre à éduquer qui que ce soit. En réalité, compte tenu du déroulement des opérations, j’ai même une possibilitée de recour légal sur l’aspect procédural du dossier… mais quand les avocats commencent à s’en meler. tout le monde à toujours des recours ;-)…

Sur un autre point: Publicité

Je suis motivé par l’argent, tout le monde le sait. Le boudhisme, ce n’est pas pour moi, j’aime trop les biens matériels.

Donc, j’ai accepté une petite offre pour placer des paragraphes de publicitées (clairement marquées) sur certain vieux post de ce blog. Pour ceux que ca offense, bien, désolé.

Sur un autre autre point : EBay

2e commande à été passée & reçue. Une prise d’alimentation pour un Compaq Presario X1000 : 27$ à la place de 75$ (site Compaq). Cette pièce m’a permis de faire cadeau d’un portable centrino 1.2, 768mb ram, écran 15.4 et un disque dur de 60 gigs à mon père.

Il reste le clavier à changer (2 lettres manquantes), mais, devinez quoi : pour 30$ (encore une fois, à la place de 75), j’en ai trouvé un sur Ebay. Il devrait arriver vers la fin de la semaine (portable est quand meme utilisable entre temps).

Je suis rendu à environ à 90$ de sauvé sur mes achats par EBay. Je ne compte pas vraiment le drop de 600$ à 130$ pour ma lentille Maxxum, parceque je ne suis pas certain que j’aurais investi 600$ pour une focale que je possédais déja – bien que maintenant, je sais qu’une focale fixe vaut fichtrement la peine. Donc, 90$ … je dois m’en rappeler si un jours je me fait "avoir", histoire de garder le compte ;-)

Sur un autre autre autre point : PayPal

Je me suis inscrit à PayPal pour pouvoir payer ce que j’achete sur Ebay. Finalement, le compte risque d’etre beaucoup plus utile que ca! Il est maintenant possible pour mes clients (ayant des comptes paypal – et je regarde pour le "sans compte paypal") de me payer par cartes de crédit. Le point commencait à devenir nécessaire, j’ai des clients situés aux états unis et certain prospect en europe (Je viens de signer un petit contrat à ce propos). Les cheques par la poste, ca commence à etre dépassé ;-)

A walk in the cloud

March 17, 2008 in , , , , , , , with 2 Comments

Executive summary : Give me 10k$, a month, 3 poweredge servers, a gigabytes capable switch and I’ll build you a scalable cloud infrastructure ;-).

And, the post:

Last year dominant meme was "Virtualization". Since you can’t have the same focus for two consecutives years (must be a law about that written somewhere), they (for various definition of "they") had to enhance it. Here come "Cloud Computing".

Cloud computing, as defined here, here, here, here, here and…  is still in condensation phase. Ideas appear and usability should emerge… soon.

While this is concentrated fun for theorician, I would prefer a more technical discussion. I am aware of Montreal based corporations currently studying Cloud/Grid systems. One of the next big player, in Montreal/North-Eastern USA, might be iWeb Technologies – they already have hardware, a customer base and so much to gain on the scalability aspect of cloud computing. Think about dynamically closing unused shared hosting system and relocating instance in relation of their impact on server resources. A lot of other corporation are also present in the field.

But I don’t have access to the same quantity of hardware as they have, so lets see what is available / can be built in my small lab.

SunGrid Engine, as an online service, no hardware needed, have more of a grid heritage than a cloud computing future. Application are launch, run, and a specific output is gathered and sent. The list of application, while impressive, doesn’t have "Apache" – this is a system meant for raw processing power, not offering services. 

IBM’s BlueCloud is still more of a vapor cloud around a press release than anything that has to do with computing. Though, I’m sure it look awesome in their lab. But, again, I’m sure their whole lab look nice.

3TERA’s apps logic does look neat, yet, there is no public price tag. This also look like the kind of system that is built around templates "which should not be modified". I have no idea how the system reliability goes when customization are made. And I won’t know… no price tag is a straight no-go for me. If you are ashamed of your pricing model, there is a problem. If its not the case, there is no reason not to show "figures".

Another online service, Amazon AWS (EC2 & S3), is one of the current market leader. Based on XEN, you can have a remote instance for couples of cents an hour. The main concern with EC2 is the volatile aspect of the storage, which kinda defeat most of services real purpose, dealing with informations.

So ?

While I don’t have much hardware, I still have a labs of 4 dev + 2 prod systems. Lets see what can be done. Lets design a home brewed cloud infrastructure.

Nodes types
ConfigNode :
    role : CNode is a standard Debian sys. It is the DHCP + PXE + tftp server. It hold the HardwareNode kernel. All cloud configuration happen on those systems.
    min : 1 sys.
    normal : 2 sys.. {Primary/Slave}. with software raid + drbd + heartbeat.
    Scalable: no use. 2 systems is more than enough, there isn’t really any CPU/Network load.

StorageNode :
    role: SNode is a network booted GNU/Linux system. It serve AoE devices on the network. All nodee (except ConfigNode) use SNODE as root filesystem.
    min: 1 sys.
    prefered: 2 sys, {Primary/Primary} with software raid + drbd. MD-device Multipathing is required from clients to preserve the P/P coherence and reliability to network failure.
    Scalable : This is a building block. The limit of SNODE is defined by the network fabric speed.

HardwareNode :
    role : HNode is a network booted GNU/Linux/XEN-dom0 system. It use a SNODE array as its root filesystem. This is where INODE will be launched. This node is diskless.
    min: 1 sys.
    prefered: no limit.
    Scalable: This is a building block of the infrastructure. The limit of HNODE is defined by the acceptable speed of the root file system located on a SNODE.

Instance :
    role : an Instance is a network booted GNU/Linux/XEN-domU system. In the presence of VT technologies, it can also be an unmodified guest operating system (hear full-fledge GNU/Linux or Microsoft Windows). It is started on a specific HNODE using SNODE resources.
    min : 1 sys.
    prefered : no limit.
    Scalable : Currently limited to the underlying HNODE ressources.

Summary : Using a specific configuration node we start a StorageNode and an hardware node. Then, once the infrastructure is  "running", Instances can be dynamically started on HardwareNode.

Since Instances are XEN/domU based, running on shared storage, they can be migrated LIVE without downtime between HardwareNode. A ping to the virtual instance would not fail, even in the middle of the live migration.

Since HardwareNode are network booted, adding new server is as simple as adding the MAC addrs in the dhcp configuration and tagging it as HNODE. As long as system are able to PXE boot, it is really a matter of minutes to add new nodes.

Since HardwareNode are network booted with remote root filesystem, they do not need to have hard drive. This remove one of the main failing pieces of current infrastructure. There isn’t much to fail in a server with only a CPU, memory and network interfaces.

The storage aspect is taken care of Storage node where good raid + redundancy + hard-drive snapshot can be used to control the environment. The only limit on the number of storage node is the network… but then, link aggregation is your friend.

Since multipathing is used, with DRBD and AoE, a storage node can be shutdown without impacting running instances.

The creating of new InstanceNode is easy : either copy an instance or debootstrap a new system. Doing something similar from 3Tera would be fairly easy at this point, creating template and preparing configuration interfaces/scripts. 

What now ?

Took me a week-end day. I have a running ConfigNode, StorageNode (using NFS, but AoE /multipathing is next), HardwareNode and an Instance. Much of the time was spent waiting for kernel compilation and deploying distcc on my lan. Had little problems pxe booting a dom0, but found a fix.

I wonder what someone working full time could accomplish in a month…. Someone want to pay me to see ? ;-). Haaa.. and it would cost you (in addition to my salary for a month) a copy of Nicolas Carr’s BigSwitch book (which I haven’t read yet, but plan to, as soon as I can get my hand on a copy). I can even do a little presentation first for some kind of financial retribution (yeah, money drive me ;-)).

Seriously, such setup would be fully scalable and so easy to dynamically configure through scripts/GUI. One of the limiting factor is the CPU/Memory resources limit that instance have since they are linked to a single hardware node but if Xen (as a commercial solution) is able to create a resource pool, I’m sure there is way to go around that limitation.

Jeez, using VT enabled hardware node, you could even start Microsoft Windows instance in your cloud…

Btw, I know that everything i’ve spoke about can be done through VMWare infrastructure with vmotion (and maybe 3Tera’s Apps) but…. then, think about the fact that a 2 CPU licence for VMWARE Infrastructure is a little bits over 6900$USD….

I just don’t understand why there isn’t more cloud out there. This isn’t all that hard to deploy… not even time consuming…

.cloud computing
Page 1 of 2612345»1020...Last »