Next month:
May 2010

April 2010

All Roads Lead to the Cloud - Cloud Automation

Imported from published Apr 25 2010


There has been and continues to be a huge amount of material generated regarding the 'Cloud'. There are many definitions ( and I am not going to repeat them here. As part of EMC Consulting, we see many different approaches from clients on how to reach the characteristic functions of the cloud. There tends to be a lot of focus currently on the theme of 'control' within the cloud.


As I mentioned in my last blog, the folks that have been certified in virtualization products (in the mainstream VMware, Citrix or Microsoft certifications....and there are others of course) tend to start using the main management consoles provided with the virtualization solution or a 3rd party product that has integrated API functionality used for management of all/part of the computing real estate. This is a bottom-up approach and works reasonably well. There are other more programmatic approaches, in the sense of an overlay to the computing resources being created. A portal-style application is used to capture service instructions and process them. This is a middle up-down approach focused on the existing IT landscape.


However, what strikes me as being of significance is that there are many roads leading to the cloud, and that it is perhaps necessary to understand that many of those journeys are mandated by the current contingencies operating within different organizations. The unique mix of market needs, skills and the configuration within an organization tend to lend shape to the transformation approach leading to the cloud. Interestingly, these initial forays into this domain provide significant learning experiences for organizations, ultimately allowing them to determine which cloud configuration will best support their business ambitions. This is also true of organizations operating within the same market - they all take a different internal approach to building out their clouds.


However, when an organization has been tasked with creating the necessary capabilities allowing cloud transformations to take place on very large scales, then some upfront thought is definitely going to pay off. In the race for product sets providing panaceas to cloud control, some of the good 'ole fashioned computing management lessons learned over the last 30-40 years tend to be pushed out of scope, although they may well still be relevant.


One of the areas that I was reminded of the other day in discussion with some vendors and clients was how to control the various activities within the cloud - once it is actually thereWink I was struck by the incredible complexity and simplicity of this statement. Back in the early 80s I used to work on IBM mainframes, and many of the characteristics of the cloud that we see had some of their early, and arguably from a GUI perspectiveSurprise, primitive beginnings. I recall that job scheduling was a big thing at that time! There were literally thousands of activities taking place in the background that nobody was aware of, and they kept the business running.


In a cloud, once the infrastructure levels are instantiated, and the virtual compute resources have been apportioned to specific guest operating systems within a virtual machine container (yes - I know there are other ways of giving resources in the cloud - just taking this one as an example as most organizations are familiar with this) - the fun really starts. So let's take this further. We have suddenly 10,000 virtual machines running server operating systems, and another, say, 100,000 virtual desktops running in our cloud. Great stuff - well done folks!Cool


Well, as most administrators and IT shops know, the work is just starting. There are all the activities regarding data backup, replication of data, servicing restores, rolling out anti-virus updates, controlling the flow of agents within each of those machines (e.g. update programs running on desktops offering to update the Adobe Acrobats of this world, and indeed the operating system itself all directed at a limited number of source machines), patching and the list goes on and on.


There are many ways to deal with these types of activities, but ultimately they come back to some form of console where these unique events are scheduled. For example, typically backups are grouped, scheduled and hopefully executed. Reporting on an exception basis focuses the administrator on potentially re-running some of the failed backups. This could be partially automated using semi-automatic event-driven intelligence - where specific alerts generate specific actions - that are then triggered and managed - much like a scheduled job.


As you can see, some of the typical stuff that IT shops have been doing over the years are still relevant. Don't get me wrong here; there are other ways of doing things. Indeed the paradigm of data protection through backup has seen substantial revision in the last years with the widespread use of disk media technologies. However, the reality at IT shops is still to have control and accountability of the backup process. Control is a very important part of IT Service Delivery displines in the sense of reporting to your business service clients (internally or externally) that you are doing what they are perhaps purchasing as a service, and that the service is running 'just fine!'Big Smile


The point here is that the need for massively scalable job scheduling in the cloud providing event/schedule driven activity intelligence is definitely still there. IT operations would have a very difficult job of actually being able to control the potentially millions of operational activities that need to take place daily. Ensuring for example that all virtual machines are backed up, and providing the reporting data to management with a breakdown per business unit, utilizing cost and performance dimensions is potentially a 'job' that would need to be run at a certain time. This stuff does not just happen on its own automagically!


I was speaking about this theme with a particular vendor UC4 (you can find these folks at and there are others in the market of course - but the beer was very good in Belgium of course - thanks Lennaert De JongWink and we were discussing the backup 'job' when there are potentially hundreds of thousands/millions of clients. Never mind that the technological way of realizing this would probably differ vastly from the traditional backup program approach of streaming to storage medium. The task itself was still there. In such a large cloud environment, I realized that the all the tricks of the datacenters, ICT shops and service providers still apply - with some significant modifications needed.


However, the sheer scale under discussion requires the effective means of control - this is absolutely essential. Think about it - patching a million virtual machines in the cloud that require a critical patch may not allow the luxury of rolling out the patch (hopefully regression tested first please) in small groups of machines, verifying if that is ok, and then rolling out to larger and larger groups.


The patch in question may be against a particularly virulent viral infection. There may well be twists and turns in the logic such as' patch-if ok reboot- if not ok bring back a previous image of the machine - patch again - if still failing power off virtual machine and call your nearest IT Virus Buster through an alerting mechanism'. The poor IT administrator may potentially get thousands or millions of alerts in this way. Basically, the IT operation could be swampedAngry.


It definitely pays dividends for organizations embarking on the cloud transformation to ensure their IT house has been brought in order to handle massive numbers of parallel events. Even simple activities that currently take place in organization such as file transfers can on this scale become a seriously complex issue when things start to go wrong.


So scale as well as preparing for things going wrong and mapping these to some of those traditional ICT management skills will certainly help to move further on the cloud journey. Go on, don't be afraid to dust off some of that 'old' knowledge and get it working again for the cloudSmile





The opinions expressed here are my personal opinions. Content published here is not read or approved in advance by EMC and does not necessarily reflect the views and opinions of EMC.

The Significance of the VMware VCP4 Certification

Imported from published Apr 21 2010


I have recently taken the VCP4 examination, and luckily, passed. Being with EMC for only 6 months, I wanted to get some insight on how other people had prepared for this momentous event. Talking with some very experienced colleagues in VMware vSphere ESX4 environments, I had realized that approaches ranged from brain cramming, deep hands-on experience all the way to 'understanding the concepts'.


Personally, I took a hybrid approach to learning, supplementing concept understanding with a 'how would one do the following?' questioning approach to make the material a little more interestingSmile This combined with reading a huge list of VMware whitepapers Wink Seemed to have worked for my style of learning.


I actually have many exams under my belt mainly in the academic and Microsoft fields of study. It was interesting for me personally to compare the various approaches used in Microsoft exams - because they have been around and evolving for a long time. Now, don't get me wrong here, this is comparing apples to pears, but the basic rationale behind doing the examination and how that gets one ready for the job was the key focus.


The Microsoft exams I used as reference were the Windows 2008 MCITP:EA/EMS series (Microsoft Certified IT Professional: Enterprise Administrator, and Enterprise Messaging Administrator with Exchange 2007). These exams used a combination of scenarios (somewhat long winded in many cases - but suprisingly accurate in the field), technical focused questioning, and my favorite the simulation of the actual product screens together with a scenario.


I am actually an infrastructure architect for Microsoft Windows 2008 environments and Citrix/VMware/Microsoft Virtual Infrastructures within large scale datacenter environments ranging from 50,000+ users. It is essential therefore that one knows the boundaries for designing for each environment together with a holistic approach taking into account processes and people aspects.


After simply passing the VCP4 exam, I realized that this exam is really only an introduction to the topic of vSphere and virtualization. Perhaps I had set my expectations a little too high in terms of what one would learn, with a design/architect background in mind. Now what was interesting was that the VCP4 is suprisingly 'fit-for-purpose'. However, that purpose, as I see it, needed to be put into a better mental frame.


The VCP4 gives you the nuts and bolts understanding to be able to run a reasonable sized ESX Cluster environment using vCenter. There is some minor discussion about networking, storage, hardware and administrative processes, but this is vastly different to the indepth grilling one gets doing the Microsoft exams (there are focused exams on each of the major topics e.g. An exam for networking and one for Active Directory Services.)


Looking at this with my consultant hat on, I would say the VCP4 does exactly what it aims to do. Whether that is enough to 'get you fit-for-the-job' is another question. Meeting many different clients on a regular basis gives a great insight into how the knowlegde gleaned from this exam is used. I would go further and say that this is probably the first step in realising what virtualization can do for an organisation and get energised around those capabilities. Further, this may well be the first step on the journey to the cloud, in that the first internal discussions about where virtualization is heading are initiated.


This plays out at many different levels in organisations. Administrators have a newfound confidence in their VMware vSphere activities. They are able to make those critical suggestions allowing an organisation to gather more latent value from a virtual infrastructure and even technically 'spar' with consultantsCool However, they tend to fall short when a consultant asks them to abstract their knowledge to larger scale operations.


Design authorities in organisations tend to be more focused on the 'limits+parameters' of the vSphere environment. This helps guide decisions on what can, and should not be done when bringing in new applications/virtual machines into service. There is certainly more awareness of the IT ecosystem at this level, but still difficulty in structuring this knowledge to scale to large service environments.


What interests me is that looking at the service managers, there tends to be a good awareness that things that were difficult to do before, can now be done rather quickly and easily. This is the level where one starts to hear about the ability to manage at scale, support line of business on an operational basis and indeed questions related to the IT Services Value Chain. In other words, some of the key value promises of the cloud.


Working further up the chain of command, major themes such as business IT strategy alignment, overall security of services, compliance, governance and investment value extraction strategies start to be elicited. These are the true values of being able to leverage cloud-based ICT services supporting the basic rationale and business strategy of an organisation.


All the traditional themes of service management design, service oriented architectures and full-scale virtualization at any and every level tends to drive through value at all layers. In some cases, the virtual solution has intrinsic value that is necessary to extract and highlight for service consumers. For example, a virtual desktop, whilst being of high interest to IT shops, requires some explanation for end users such that the additional value and features can be highlighted and appreciated.


To that end, it appears that there is a rather large gap between the nuts-and-bolts information of the VCP4 and the VCXD Designer oriented examinations. A series of smaller more topic-focused examinations in between would ensure that the level of awareness to be 'fit-for-the-job' are addressed. For example, having a VCP4 does not mean that you are well versed in the dark arts of networking. A network engineer is better able to answer those types of questions, but does not necessarily know anything about a vSphere Virtual Distrubuted Switch environment. The same can also be said for vCenter as a powerful console to the virtual estate and operations management disciplines in general.


These specific levels of topic-focused examinations also help to stimulate creative discussions around some of the practical issues of scaling cloud infrastructures, particularly if one is also a cloud provider for other organisations.


There is a lot to be said for experience of course, however the need to manage and structure knowledge of virtual service operations will allow organisations to be able to extract far more value and be more nimble in service delivery.


I understand from colleagues and some of the recent blogs here that VMware has indeed released certifications partially addressing the gap between administrators and designers, and that is certainly welcome news for all!


I would be interested to get some feedback on some of the areas that other fellow VCP4'ers feel would warrant a specific examination/certification.


Jas Dhalliwal


The opinions expressed here are my personal opinions. Content published here is not read or approved in advance by EMC and does not necessarily reflect the views and opinions of EMC.