Virtualization as ‘Continuous Workload Consolidation’ Paradigm
The Journey to the Cloud –The Physical Construct (IT’s the Small Things in Life that Really Matter)

Planetary Cloud: The Rise of the Billion-Node Machine

Imported from published May 22 2010


In a previous blog (Virtualization as ‘Continuous Workload Consolidation’ Paradigm ), I described a paradigm change whilst being on the journey to the private cloud in how one examines the very substance of virtualization. Not simply a one-off activity but a fundamental restructuring of IT operations toward a continuous improvement activity facilitated by every increasing capabilities of component infrastructure elements e.g. the processor, network and so on.

The processor is of course one of those big facilitators. I was speaking to some of my ‘virtualization and Cloud‘ colleagues the other day in Berlin (nice place to visit folksSmile) regarding just how big an influence processors are. Some of these guys are born philosophers and well, we wanted to explore the outer boundaries of where the current multi-core processors would/could take us.

We know already about some of the gigantic leaps in the abilities of processors (see blog link above) from Intel / AMD and many others in the market – but where can this go? EMC Consulting, as trusted advisors for organizations large and small, maintains a technology watch on this, and we discuss the cumulative effects of technology on the core business of these firms in the form of envisioning workshops. Indeed, we do this as a continuous activity for EMC itself.

Whilst in discussion over some good cold German beers (well we do need to keep the thoughts lubricated as well) I mentioned something I had read about some time ago regarding RAM Clouds (The Case for RAMClouds: Scalable High-Performance Storage Entirely in DRAM). Well this got us going regarding scale, and as one colleague quoted (Richard Branson of Virgin fame) ‘maybe size matters after all’.

We are reaching a cross roads in the industry as more processing cores are made available in smaller formats. Indeed Intel already dubs its 48-core wonder as the ‘single-chip cloud computer’. Well let’s take this further. As core count increases, we start to reach rack loads of traditional computers being represented on a single chip. The 48 core chip is effectively the equivalent of 24 dual processor servers (2 server racks’ worth) or 12 four processor servers (also more than a single rack’s worth of servers) in the traditional uni-core (not unicorn!) world.

The next step up may well be the ‘datacenter on a chip’. That is the entire CPU processing power of a datacenter represented on a single chip. The close proximity of such cores presents several magnitudes of advantage over the equivalent physical server constructs of the past, and indeed today. Having a 1000-core machine available is definitely a competitive advantage in many industries!

Putting a couple of these machines online, with all the automation and management abilities needed (see All Roads Lead to the Cloud - Cloud Automation), and a dash of virtualization source, and we have a veritable cloud. Indeed this private cloud could be described, due to its density and several thousand core processing ability within say a single square meter, as the first Planetary-IT-on-a-Chip.

Folks this is tremendous amounts of power. This would be like putting in the entire complex of Google datacenters around the world in literally a single server rack. That begs the question of whether such datacenters should be built as they are - I think that is a blog for another timeWink.

Imagine all those datacenters represented within this construct. The RAM Cloud takes shape around this, with several Petabytes worth of RAM available instantly. Whole chunks of the Internet, perhaps all of it, stored in memory! This is instant-on IT!

Planetary IT on this scale would dwarf the constructs we have today. This would make the so-called mega datacenters of today look like a ‘very large desktop PC’ in comparison. EMC Consulting as Information Consultants, need to think about the long-range effects of such massive amounts of power and the information ocean that is held there. I mean cooling such a construct is no longer a discussion of putting in some chilling units. We are talking about all of this immersed in liquid just to stop it from frying!

The effects on businesses can be described as disruptive at least. Imagine all those DBAs perpetually raising concerns about never having servers large enough. What are they going to talk about in the future when we have 1000 cores available and Petabytes of RAM all provisioned in seconds[:'(] (before you ask - yes I have been a DBA - and I said this too)

The infrastructure and information management technologies of today need to keep these challenges in mind. Not simply multi-tenancy ability, but also the guardian and stewardship roles associated with, dare I say, the information cultural heritage of mankind.

EMC (The EMC Heritage Trust Project), IBM (Vatican Library Automates for the 21st Century), Gates Foundation (Bill & Melinda Gates Foundation) and many other organizations are investing in that future. Planetary IT may well be the tool they needed to perform this on a scale previously undreamt of.

Indeed this may be the means of IT boldly reaching parts of the world where no IT has gone before (thanks Gene Roddenberry - Where no man has gone before – some truly astonishing articles on Wikipedia)

Planetary IT, not just Public Clouds, may well be the next big thing. The ability to get the IT resources of the planet working on problems such as genome sequencing, climate challenges, simulation for nuclear fusion, spaceship design amongst others with a billion-node system composed each of 1000-cores is like getting a brain the size of the entire Earth working for a fraction of a second to find potential solutions to our most challenging concerns of today. This affects everyone. This is the Rise of the Billion-Node Machine – the billion-node IT nexus serving the planet.

Anyway, coming back down to reality, the challenges of administering, metering usage, maintaining and operations of such a Planetary Cloud are similarly challenging. The tools, services and products that EMC & others are introducing into the marketplace take these huge scales as a backdrop for development (see The Diverse and Exploding Digital Universe ).

Efforts such as the SETI@Home screen saver that harness the spare processing power of millions of volunteers dispersed around the globe concurrently to look for sentient life in the universe, Folding@home doing something similar for protein folding to solve the challenges of diseases are paving the way for what it means to make millions of machines work on a problem.

Pioneers and frontrunners are necessary in society. Indeed, the push towards Cloud thinking is a means of pushing the frontiers of IT as we know it today.

On this blog and others of course, the ripple effects of this infrastructure and paradigm play are being felt. The challenges of programming and presenting ever increasing volumes of information are discussed here daily. How to get teams organized, which methodology to be able to use, discussions of agility, outsourcing, working with colleagues in far-flung parts of the world concurrently (open source Linux kernel development) are attempts through dialogue to get the maximum out of these changes.

This know-how is filtering through into society through the consulting efforts engaged with organizations around the world. It is helping to directly address the varied challenges organizations bring to our attention every day. Before you ask, we don’t always have the answer, but the greatest ideas EMC has are stimulated through robust debates with these very same organizations.

The robust ‘Star Wars / Star Trek’ like discussions are important to stimulate the thoughts of tomorrow. To provoke these thoughts and stimulate our greatest minds, to get us all thinking about a world where processing limitations are a thing of the past and where that can lead us – will drive our young minds in Academia, Business and Society at large to develop the planetary IT frameworks needed to harness this power.

The Journey to the Cloud, or in this case Planetary scale IT is important as a concept whilst the practical realities and challenges unfold. It may well be the step that stimulates the IT Renaissance and leads us to the stars!


The opinions expressed here are my personal opinions. Content published here is not read or approved in advance by EMC and does not necessarily reflect the views and opinions of EMC.


Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.