- Developer Tools
Phil Whelan, February 20, 2014
In this post we will look at common steps IT organizations take when adopting cloud technologies. From starting small, and gaining expertise, to rolling out IaaS, PaaS and continuous integration. We look at one possible route you might take.
1. Start Small, Experiment, Gain Expertise
Large organizations move slowly for good reason. They have immense inertia. Moving your infrastructure to a different platform is going to create a certain amount of churn. Moving to a completely different paradigm is going to be even more challenging. But that does not mean you should not move. You just need to do it with an appropriate amount of caution.
Starting small is your best bet. If you can prove out new technologies on smaller, lower risk projects, you can gain expertise in new technologies. When risk is decreased, confidence increases and so does velocity. There is also room to experiment.
One thing that frustrates me is cloud vendors writing white papers that insinuate that a large Fortune 1000 company has adopted their technology and immediately rolled it out throughout their entire organization overnight. Is that realistic? No.
Start small, experiment, and gain expertise.
2. Go Public
Using the public cloud is low-cost for experimentation, but may not ultimately be suitable for your company. The costs associated with using public ready-to-go infrastructure are very low. If you are starting off, is it easier to build an internal OpenStack cluster or to sign-up for an account with an OpenStack-based provider like HP Cloud or RackSpace and see what you can do with theirs?
Often we see lag in adopting PaaS because a company feels they need to first build, and become an expert in running, an internal IaaS. (IaaS is not a requirement of Stackato, but is recommended.) Even so, PaaS solutions can easily be deployed on public IaaS to begin the exploratory stage of your journey into the cloud.
3. Think Cattle, Not Pets
As you start playing with cloud infrastructure, you will realize that keeping track on your servers becomes harder with the number of machines you can fire up. Instead of a cluster of large custom built servers, that you intimately know by name and dote on with loving care, you will be selecting smaller virtual machines that fit better to specific tasks. These machines are best dedicated to one task. This keeps things simple and allows you to scale up particular aspects of your architecture in isolation.
More machines means dropping your old naming convention and going with something more aligned with the tasks they perform. No longer does "Big Bertha" run half of your backend systems. You now have 5 "message queue" servers, 3 "database" servers, 12 "application" servers. Think cattle, not pets.
Your cattle are ephemeral, but unlike the hamburger variety, when one dies another one is instantly born. Ideally you will not be involved in the birthing process and so naming becomes an automatic incremental process.
4. Config As Code
Adopting the cloud involves accepting the ephemeral nature of servers and building more automation into your infrastructure. You want your machine to be disposable and easily recreatable. To do that we have to carefully define the configuration of those machines.
Machine configuration is a complex thing. Luckily there are many tools available to do this. Puppet, Chef, Ansible and Salt are popular. They provide a scripting language to define the complete setup of a machine and how it is uniquely configured in the cluster.
If you look to software development, it would be near impossible to find a project that was not kept in some form of source repository, such as git or subversion. The configuration of your machines has become just another software project, so why would you ignore this first rule of software development?
Get your server configuration into source control.
5. Embrace DevOps
"Cloud" and "DevOps" are two overused words by marketing. This blog post has probably lost half its audience by me putting "cloud" in the title. But these words define very well the umbrellas that cover a range of technologies and new ways of working with them.
Read posts on DevOps, read The Phoenix Project, go to DevOps Days events, watch the FoodFightShow or HangOps and get involved with the DevOps community. This is where these cloud technologies are best understood. Dig deeper than the generic "cloud" marketing material that is out there. And never, ever read something with a title like "10 Steps To The Cloud".
6. Dive Into CI
Continuous Integration is not new, but cloud technologies make it so much easier, that there is no excuse for not doing it. Once you are thinking about your machines as ephemeral, easily deployable cattle and your configuration is succinctly defined as code, it becomes easy to fire-up test clusters to run tests.
Jenkins is the most common goto tool for running CI jobs. Bamboo is also popular. Both of these run CI jobs that can fire-up machines, run tests and destroy those machines. If your tests fail, then your team can be notified and the configuration changes that causes the failure can be automatically blocked from going into production.
7. Internal IaaS
As I mentioned above, you can start your journey into the cloud by utilizing the public cloud. If you are a large organization and you are serious about moving to cloud-based technologies, then very quickly you will want to be firing up your own internal IaaS service.
OpenStack, CloudStack and Eucalyptus are common solutions for this. OpenStack is probably the one I hear of most often and there is a large community behind it. HP Cloud and Rackspace run this for their public IaaS offering.
To get OpenStack up and running there are many flavors of installer. At ActiveState, we have tried many of them and found Red Hat's RDO to give us the best experience for our needs.
Nebula also provide a full hardware and software appliance to easily deploy IaaS behind your firewall. This provides API compatibility with the OpenStack and Amazon EC2/S3 Web Services.
Once you are running an internal IaaS on your own infrastructure, PaaS is the obvious next step. With IaaS you are bringing automation, control and efficiency to your infrastructure layer. Why not do that with your application layer too?
The goal of IaaS is to quickly provide the machines required for running IT's software solutions. IaaS can create a herd of machines automatically configured for a certain role. Unfortunately, defining those server roles all the way down the developer level is slow and time consuming process. This takes up far too much Ops time.
PaaS fills the gap and gives the developer what the Operations teams get from IaaS. Developers can now more easily dive into CI (step 6) and work with Operations to define CI tests that cover both of their worlds.
In developing Stackato, we have put a strong focus on the communication channels between Dev and Ops, helping organizations embrace a DevOps mentality.
9. Distribution And Redundancy
Just like real clouds, clouds can form from multiple smaller clouds. For instance, one PaaS cluster can span across multiple IaaSes, given the appropriate latency and network configuration.
The way cloud-enabled applications tend to be architected enables them to be easily adaptable for redundancy and low-latency distribution. High-latency distribution (east coast to west coast) is a little harder and out the scope of these 10 steps.
Legacy applications, that are not already designed for running in a distributed redundant manner, often require being broken apart to be more modular. How this is done is very much dependant on the specific legacy application.
The idea here is that IaaS can be defined in such a way that it utilizes both private infrastructure, such as an internal OpenStack cluster, and public infrastructure, such as Amazon Web Services. You use internal infrastructure for the base-line resources that you consistently use and use public resources for the spikes. Bursting out to the public cloud gives you an unlimited ability to scale up, even it is for only a few hours.
As discussed in the recent Cloud Foundry advisor board meeting, IBM have contributed code for "DEA Zones", which allows fine-grain control over where an application runs and how it is distributed. This can utilize a hybrid infrastructure by controlling whether it runs on the private infrastructure, public infrastructure, or across the two. Possibly, some components of a larger application could run in the private infrastructure and some in the public infrastructure, while still being part of the same VPN/VPC.
Where are you in your journey to the clouds? Where are the pain-points? What is working?
Remember to start small, experiment and gain expertise. Utilize the public cloud for quick experimentation and get involved with the DevOps community to ensure you are aware of all the new ideas and tools in the space.
Image courtesy of georgio@flickr
If you are looking for more information about PaaS and maximizing your cloud investment,view the recording of ActiveState's recent webinar.
Firefighters are the ones who are often up at 3am frantically trying to keep systems online.
Sometimes they are too busy putting out fires to step back and look for preventative measures. Fire fighting is a noble job, a job for the brave, it saves companies from outage, from loss in revenue, from loss due to bad publicity. Yes, it is a job for heroes. And this is where the problem lies.
It is inefficient for Operations teams to manage the individual needs of developers and applications. Developers should be able to select and provision their own needs.
We look at the evolution of self-service in grocery stores and how it might related to DevOps.
There is a lot of confusion about "PaaS" and what it means. The definitions are vast, vague and varying. The market and the media are understandably confused.
Subscribe to ActiveState Blogs by Email
Share this post:
Tags: ansible, chef, ci, continuous integration, devops, devops days, foodfightshow, hangops, hybrid, IaaS, nebula, openstack, PaaS, public cloud, puppet, redhat rdo, salt