IDEAS Insights

IDEAS Predictions for IT Industry in 2010

As in previous years, IDEAS analysts recently conferred to identify the key industry trends that are poised to play out in 2010. Following are IDEAS top 10 predictions for next year (see the December issue of Tech Trends Monthly, the IDEAS newsletter, for much more detail on each topic – or sign up for a free subscription here):

1. More mergers, acquisitions and alliances will occur. IDEAS expects to see some surprising moves, many of which will be opportunistic, but others due to strategic shifts in response to competitive realignments in the industry. Acquisitions could include hardware companies buying large system software companies, and more high-profile software companies buying hardware companies.

2. Demand for server systems will rebound. Server deployment will pick up in 2010 due to pent-up demand, and as customers take advantage of new technology to streamline their infrastructure. Companies that do not make their IT more efficient will find themselves at a competitive disadvantage. As the economy recovers, companies that optimize their business processes to maximize use of IT will gain an advantage over those that have concentrated only on cutting costs.

3. Green marketing hype will subside as the issue matures. Corporate executives will remain highly focused on environmental goals for the overall corporation, but for departmental-level IT managers, "efficient IT" will gradually replace "green computing" as a priority.

4. x86 servers will continue their drive into the datacenter. The sharp increase in performance of Intel's Nehalem processor will accelerate the use of x86 servers for hosting enterprise-level workloads, but x86 servers will not take on the same characteristics as the "big-iron" systems that traditionally made up the heart of datacenters. Instead, servers and storage systems will employ various forms of clustering software on inexpensive hardware to achieve scalability and reliability.

5. Server form factors will progress from towers and racks to integrated appliance-like systems. Integrated solutions that combine multiple layers of IT infrastructure, including servers, storage, networking, and software, will become increasingly appealing to organizations that have the need for next-generation datacenter capabilities, but lack the depth of personnel to manage complex new technology such as virtual infrastructure.

6. Server virtualization will converge with storage and network virtualization. I/O infrastructure virtualization, which applies virtualization functions to storage and networking infrastructure, will continue to evolve very rapidly throughout 2010, with high stakes for vendors to chart out positions in next-generation datacenter infrastructures.

7. System software will fragment, and heterogeneous virtualization management will become a requirement. As virtualization takes hold across the industry, the operating system will begin to lose its grip on IT as the center of the software universe. Users will apply virtualization to workloads that span multiple departments or business units, and as a result, virtualization management tools will have to support multiple virtualization platforms.

8. Organizations will build their own "secure" clouds. Despite continued fascination with the prospect of tapping into third-party computing infrastructures, the most pressing concern for the majority of users will be to virtualize as much as possible of their internal infrastructure into "secure" or "private" clouds.

9. Public clouds will draw startups, and some experimental use by larger organizations. Startups will use cloud-based services to minimize costs and accelerate time to market, and some small companies will move routine workloads to Software as a Service (SaaS) for cost savings. Mid-sized and large enterprise companies will experiment with third-party cloud services by temporarily deploying some production workloads on them.

10. IT administrator roles will take on a broader scope. With the rise of virtual infrastructure in next-generation datacenters, skill sets that traditionally fell into separate silos for servers, storage, software, and networking, will have to converge. Few people have such converged skills today, and those that do have them will become more valuable.

IT is about to get very complex as a result of consolidation and virtualization, and the very definition of a server and its components will undergo significant changes. Vendors who control parts of the IT stack today may not be the ones that control tomorrow's IT stack, and a huge battle will erupt for supplying the critical functions in new datacenters being rolled out. At the same time, IT managers will increasingly evaluate SaaS and outsourcing options for production workloads, ultimately promoting cloud computing to become one of their central business issues. 2010 will be the turning point in which organizations begin to confront some fundamental questions of where and how IT should be implemented.

We would like to hear your feedback on these projections. Please leave comments below, or to discuss any specific issue in more detail, we invite you to visit our Facebook discussion page, where there are separate threads for each of these topics.

Empirically Tracking Interest in IT Trends

IDEAS analysts are currently putting together their predictions for IT industry developments in 2010. One way to make predictions is to extrapolate existing trends using time-series analysis, and then inject educated guess work and insights along the way. Two useful tools that can help with this process are Google Insights and Google Trends, which reveal the frequency of search terms that users have requested over time. To illustrate how search behavior can be driven by real-world developments, see this example of the relationship between searches for the term “fuel efficient” and actual gas prices (collected by the Department of Energy):

Fuel Efficient v. Gas Price

 

Clearly, the two items are correlated, with interest in "fuel efficient" rising and falling roughly in sync with the price of fuel. We decided to apply this analysis to several IT terms, starting with virtualization and cloud computing.

The data would seem to imply that the term “virtualization” is showing no signs of slowing down as a key business term (see Figure 1). When constraining the data to only the searches relating to Computers and Electronics (as predefined by Google), the normalized search volume for virtualization has two features that stand out: the overall growth in requests for the term, and the significant stalactite-like dips during the month of December. These crevices, highlighted by the red arrows, indicate that people use the word in business settings, and then refer to it less while on Christmas break. This does not happen, or is less pronounced, in previous years. 

Virtualization

 

Figure 1: Google search volume for term "virtualization"

The empirical evidence indicates that cloud computing has also increasingly become part of day-to-day office language as managers consider the marketing claims about its promised benefits. Some businesses may have already started to adopt cloud computing to save on IT cost, and hence are already convinced of the pricing benefits. In the percent growth of searches for all-things-cloud (see Figure 2), there appears one salient feature in the second quarter of 2009. Albeit delayed, this bump corresponds to the NYSE crash, while the large growth during the end of 2009 points to increased interest during the recession. This is in contrast to the negative growth overall of searches for terms in the Computer and Electronic category. From 2006 until now, there has been over a 350% increase in this class of searches while the forecast for 2010 is for at least a 25% increase over the current year. Because these spikes in volume correspond to periods of economic decline, the increases are at least partly due to managers looking for ways to reduce budgets (meaning that many are already convinced of the savings). Software currently takes a larger share of this volume (50-75%), but the top non-company specific search is "server virtualization."

Cloud Computing

   

Figure 2: Google search volume for term "cloud"

If companies are truly convinced in the cost savings of cloud computing, one would expect increased scrutiny over other issues. This is exactly what happened in 2009, when many companies started focusing on issues other than price that come with adoption of the cloud. Figure 3 gives the corresponding volume for "Cloud Security" since July 2008. Again, the spike after the stock market crash and rapid growth thereafter, is apparent. The percent growth in this short time period is over 600%. This could imply that businesses are attempting to find security options that can address anxieties relating to the protection of their data after migration to either a private or public cloud.

Cloud Security

Figure 3: Google search volume for term "cloud security"

In early 2009, many analyst thought that cloud computing would explode in the coming year. This prediction has been shown to be true, but not for the reasons many were quoting. The boost came from a struggling economy, and the real growth did not occur until the latter part of the year. There is no reason to expect that this most recent exponential growth of the last few months will be subdued, and so the interest in cloud computing in 2010 should greatly exceed that of the current year. The next blog will focus on our empirically observed trends for Green IT.

After HP’s Acquisition of 3Com, What’s Next for Storage and Network Virtualization?

Following HP's announcement last week that it would acquire 3Com, speculation has arisen that more acquisitions related to storage and networking could happen soon, possibly including HP's absorption of 3PAR or Pillar. Another leading network company, Brocade, issued a denial that it was for sale, which nonetheless raises questions about whether its products and technology would fit well into a larger company.

As the industry moves into the next stage of applying virtualization, going beyond basic use cases such as consolidation, users are focusing increasingly on the broader impact that virtualization will have across storage and network infrastructures. Especially in large organizations, system managers now seek to achieve some of the same benefits from virtualizing storage and networking resources that they gained with server virtualization. In particular, they would like to reconfigure storage and network infrastructures with the same flexibility as virtual machines. For example, just as virtual machines can be moved around a network with functions such as VMware vMotion or Hyper-V Live Migration, virtual storage can be transparently relocated from one physical storage device to another with tools such VMware Storage vMotion. Similarly, administrators expect to be able to dynamically reconfigure virtual networks independently of the physical network and storage connections, which are generally changed as little as possible to avoid burdening network and storage management personnel.

Some of the requirements for these capabilities must be handled at the level of server hardware. Indeed, I/O virtualization technologies such as HP Virtual Connect, IBM BladeCenter Open Fabric Manager, and Egenera PAN Manager enable flexible connections between servers and the physical topology of networks and storage. However, to fully integrate virtual machines into virtual networks spanning datacenters or enterprises, specific functions must be implemented within virtual machine hypervisors. For this reason, the integration of Cisco's Nexus 1000V virtual switch with VMware vSphere 4.0 represents an especially crucial link. The commonality between Cisco and VMware's virtual networking technology enables VMware-based virtual networks to be managed with the same tools used for managing Cisco-based physical networks, in a way that allows virtual networks to be flexibly reconfigured. A key strength of the Virtual Computing Environment (VCE) coalition between EMC, Cisco, and VMware is their ability to further optimize the hypervisor, virtualized storage, and virtualized networking as a single integrated stack. Coupled with Cisco's entry into the server market with its Unified Computing System (UCS), VCE's thrust deep into the heart of virtual infrastructure steps up the challenge for other major systems vendors to respond with a credible alternative for end-to-end virtual infrastructures spanning servers, storage, and networking.

HP's acquisition of 3Com, following its renewed focus on the HP ProCurve line of networking gear in the past year, clearly represents a dramatic first step towards adapting to the new realities of the IT systems business, in which Cisco has become a strategic competitor. HP has also quietly started promoting an industry standard for virtualized networking called Virtual Ethernet Port Aggregator (VEPA), which it hopes will be more palatable for adoption by multiple server vendors than the VN-Link technology underlying Cisco's Nexus 1000V switch and the vNetwork distributed switch in VMware vSphere 4.0. As the strategic importance of virtual switching has become clearer, some other alternatives have sprung up in the industry as well, including Open vSwitch, which is based on open source technology developed in part by Nicira Networks and is being promoted by Citrix for its XenServer hypervisor.

How would additional acquisitions then help HP to compete with Cisco and VCE? First, acquiring 3PAR does not appear to be necessary. HP already has a broad range of storage solutions, including Modular Smart Array (MSA), Enterprise Virtual Array (EVA), XP, X9000, ExDS9900, AiO (Proliant Storage Server), RISS, and Lefthand. It is not clear how 3PAR would fit into this lineup, unless some existing product family were phased out. Indeed, most of the big storage companies have now made their clustered storage technology acquisitions, leaving 3PAR to compete independently. At this stage, it is unlikely that any of the major vendors would want to acquire the company, except for an irresistible fire sale price. One possibility would be for 3PAR to be acquired by Rackable (now SGI), should it still have any money left after its acquisition of SGI. More far-fetched, 3PAR could be acquired by someone out of left field, such as a cloud service provider like Amazon, Salesforce.com, Google, or even Microsoft. Any of these companies could take advantage of 3PAR's technology to build out a cloud service infrastructure that requires vast amounts of high-performance storage.

Acquiring Brocade could be more strategically significant, either for HP or another vendor such as Cisco. From the perspective of storage customers, Brocade is far more relevant than 3Com, which does not make Storage Area Network (SAN) switches. Brocade is one of only two independent suppliers of enterprise-grade Fibre Channel (FC) switches, the other being Cisco, which entered the FC market relatively late. While Brocade insists it is not for sale, the company now faces a long-term strategic issue as its core market of FC solutions nears the end of its natural lifespan. FC technology is now 30 years old, and while it has been the SAN interface of choice for the last 10 years, it will soon enter the sunset phase when FCoE and/or Infiniband start taking off as expected. As a result, Brocade needs to break out with some major new innovation or be acquired, otherwise it will perish.

One key advantage for Brocade is its DCX Backbones technology, which enables true Storage Network Virtualization, whereby SAN switches themselves are transformed into hosting virtual networks. Cisco is targeting converged storage networking with Fibre Channel over Ethernet (FCoE), but its technology is optimized more for the virtualized Ethernet layer of the stack, and it does not have quite the same degree of storage virtualization functionality as DCX Backbones. Whichever company acquires Brocade and its DCX Backbones technology could gain a crucial edge in the pursuit of Storage Network Virtualization.

Integrating DCX Backbones with Nexus 1000V technology would put Cisco, and the VCE coalition, in an even stronger position than before to drive the agenda of virtual infrastructure and Storage Network Virtualization, since most of the major systems vendors now OEM Brocade's technology for their existing SAN products. Conversely, should HP proceed to acquire Brocade, it would significantly strengthen its defenses against Cisco's drive into the server market. At minimum, having another switch supplier in its fold would help HP establish VEPA as a standard for virtual switching. Further, integrating Brocade's technology with other elements of HP's virtual infrastructure such Virtual Connect would go far in enabling HP to propose an alternative to VCE for virtualizing everything outside of the hypervisor (regardless of which hypervisor is installed, i.e. VMware, Hyper-V, or XenServer). While there is no evidence at present that either Cisco or HP is about to make a move on Brocade, the control of that company's assets remains a critical variable for determining how virtual infrastructure will be adopted across the industry.

Building Large SMP Systems with Virtualization

As a former designer of big iron, scale-up computing remains dear to my heart. Some specialized applications still perform best on large, shared memory, tightly-coupled SMP servers. Even for workloads that need only fractions of a large server, the opportunity to share "headroom" allows multiple applications to reside on the same large SMP, typically via virtualization partitioning. These systems can run at a much higher utilization percentage by pooling the resources that are "reserved for peaks" than if each workload were deployed on separate, individual servers. That flexible pool of hardware resources is often just as valuable as the ability to host a large, monolithic application. 

Unfortunately, the sophisticated scale-up designs created by innovative engineers must amortize their development expense across a relatively small volume of servers, compared to the ubiquitous industry-standard x86 platforms shipping as near-commodities. While I remain convinced that large, scale-up, shared memory servers are crucial for many computing environments, I acknowledge that x86 implementations offer compelling price/performance.

Is there a way to combine the best of both attributes? That is, can large shared memory servers be created from commodity-priced, industry-standard building blocks? Can some "glue," be it hardware or software, allow a collection of small, inexpensive servers to appear to perform as if a large, shared memory system? This week, two firms, 3Leaf Systems and ScaleMP, have introduced alternatives they each believe will address significant portions of the market for economical scale-up systems by coupling standard x86 servers.

Both firms unveiled their solutions just before SC09, November's annual USA venue showcasing innovative high performance computing solutions. Why SC09 and not a trade show targeting the market for broader commercial computing? Attendees of the supercomputing conference usually have very demanding compute requirements, often needing very large servers, but they are also typically budget-constrained. The promise of a large, shared memory, server at commodity price points understandably excites these users.   

Of course, a virtual "appears like a very large shared memory SMP" server will not perform identically to an actual large hardware-optimized SMP server. Therein lies the difference between 3Leaf and ScaleMP solutions. The 3Leaf platform employs a 3Leaf-designed ASIC hardware chip that melds together the memory space of multiple x86 motherboards (currently AMD Opteron, but planning Xeon support with the next iteration of Intel QuickPath) to deliver low-latency shared memory reminiscent of SMPs. ScaleMP, on the other hand, forgoes the expense of unique hardware, and accomplishes coherent shared memory via software. Admittedly, 3Leaf's hardware memory coherence is faster than ScaleMP's software mechanism, but ScaleMP claims its pre-fetch and local caching algorithms deliver the desired low latency appearance without the expense of specialized hardware.

High Performance Computing (HPC) users are acutely aware that the road promising affordable high performance is littered with firms who dreamed of delivering champagne performance at beer budgets. NUMA, non uniform memory access, is at the heart of most "affordable" scale-up designs. Flat, uniform memory access, designs prove to be costly as systems grow larger and larger. The key is to have NUMA's apparent non-uniformity seem as small as possible. KSR in the early 90s, Convex, SGI's Distributed Shared Memory, and other Scalable Coherent Interface (SCI) users (such as Sequent and Data General) all promised the performance of sophisticated, optimized SMPs at commodity server prices. Fairly recently, Virtual Iron attempted a hypervisor that could join multiple smaller servers into a virtual larger SMP, but it eventually abandoned that effort to focus only on applying virtualization for partitioning (the company has since been absorbed into Oracle.)

Clearly, a virtualized NUMA SMP will not adequately substitute for a real SMP for all environments. The key question is which workloads will be satisfied by ScaleMP's software virtualization, which will benefit from 3Leaf's hardware memory coherence, and which applications demand the more uniform memory access of true large SMP implementations.  All current operating systems have evolved to understand how to efficiently keep processing and associated memory physically close to each other on NUMA platforms. However, some workloads have less sensitivity to NUMA than others. Targeting an announcement at the supercomputing conference recognizes that many HPC applications depend less on the shared memory updating that plagued early NUMA attempts to address general purpose commercial computing. The inherent parallelization of HPC applications designed to exploit large infrastructures makes them better prepared to run well on 3Leaf and ScaleMP platforms. Optimized large SMP hardware designs will continue to offer the best performance for applications that exploit multiple compute engines and large shared memory. However, other application environments may benefit from hardware and/or software mechanisms that lash together lower cost commodity servers to create the virtual appearance of a large system, yet with very attractive pricing.

We believe there will be growing customer demand for virtualized SMP solutions like 3Leaf and ScaleMP, which offer lower cost, even with the tradeoff of potentially lower performance.  3Leaf Systems' hardware ASIC is impressive in its use of InfiniBand switching as a substitute for custom SMP backplanes. The low latencies of today's InfiniBand implementations are likely to provide an adequate SMP-appearance for many applications. ScaleMP's software-only solution may not offer the low latencies of 3Leaf, but it is likely to attract a number of customers who desire large memory accessibility at a very economical cost, albeit with somewhat slower latencies. We predict that 3Leaf and ScaleMP will see first adoption among HPC users, but each will eventually carve out a niche among commercial application users. The optimized design of monolithic large SMPs will still be needed by many customers, but virtualized SMPs, aggregated from low cost commodity servers, will offer an attractive price/performance alternative for a variety of workloads.

End User Interest High at Virtualization Forum

Yesterday I attended VMware’s Virtualization Forum 2009 held at the Sydney Convention & Exhibition Centre, Darling Harbour. Virtualization in its various forms has been a hot topic for years and looking at the growth of this event in terms of the number of attendees, is probably an independent verification of the interest in virtualization in the market with real users. Even in these tough economic times.

In his keynote address, Paul Harapin, Vice President VMware Australia and New Zealand, commented that this was the fourth annual event of the Virtualization Forum, which began with just 350 attendees a few years ago. This year’s event attracted 2,500 attendees. This may not be a big number by international standards, but Mr. Harapin noted that by VMware’s measurements, virtualization penetration in Australia is higher than any other country. Other high-adopters are Holland and Israel. This doesn’t come as a great surprise because Australia has had a history of adopting new technologies. In the 80s for example, Australia had a high market penetration of what was then a hot new technology, the fax machine. In fact, probably because of this predisposition for all things new, Australia has a reputation as a test market for many companies where they will trial new products and/or packaging, before rolling them out around the world.

Carl Eschenbach, EVP of Worldwide Field Operations was the second keynote speaker and he took the audience through VMware’s product portfolios and directions for the company.  VMware claims around 150,000 VMware customers globally and although initially, the motivation for adopting virtualization is often to save costs, according to Mr. Eschenbach he is being told by CFOs that greater business agility is the key motivator, especially at the big end of town.

It wouldn’t have been an event on Virtualization without “Cloud” being mentioned and so the audience wasn’t disappointed. In fact, Cloud Bingo was alive and well on the Twitter feeds. (#vForum #vForumAust & vForum). Having said that I don’t think VMware overplayed its hand with Cloud for the event, given the hype out there, it has to be addressed in such a forum.

VMware’s definition of Cloud is very broad, being simply “Deliver IT as a Service”. In a subsequent discussion with Dr. Stephen Herrod, CTO at VMware, he pointed to a VMware survey that found 15% or respondents had adopted Cloud in some form and 26% were evaluating. Although, it wasn’t clear what ‘adopting Cloud in some form’, means exactly.

Beyond Cloud a clear message from the event was that VMware is getting involved up and down the stack in middleware and now applications, with its recent acquisition of SpringSource.  In fact, VMware appears to have interests everywhere except for the hardware and operating system layers. Mr. Herrod was quite clear however, that it is VMware’s intention to stay away from the operating system and for VMware to add value virtualizing elsewhere.

The event saw a strong push to message Desktop Virtualization and Mr. Eschenbach’s believes that 2010 is the tipping point for desktop virtualization. This is because in his opinion many factors are coming together, such as the increased interest in virtualization generally and the expected Windows 7 inspired desktop refresh. I would add to that the 3G telecommunications rollouts around the world means that mobile devices and applications give viable alternatives for users to still be productive when not at their virtualized PC and fixed line broadband connection.

The business justification is certainly there. If you think servers have low utilization, desktops generally have an even lower utilization. Because of this low relatively lower utilization, potential consolidation ratios for PCs are even higher than desktops. So, where you might be able to consolidate say 4 to 8 physical servers onto a virtual platform, the desktop consolidation ratio can be as high as 10 or 20 to 1. Desktop virtualization has always had what seemed good business cases, but for one reason or another it has never taken off in a big way.
 
I would be interested in your views on the future of Desktop Virtualization. Is this something you are now considering and why now? Or, do you still see impediments to the take up of this technology?

Beyond the keynotes, the Forum had an exhibition hall with around 60 vendors marketing their offerings including many well known brands and not so well known organizations. Feedback was generally positive and response in terms of the number of delegates shows that even in these difficult financial times, whether they are committed VMware users or novices interested in understanding more about moving into the world of virtualization, there is still interest from users where they perceive there is value to be had.

Did you attend the event? If so, what did you think of it? Let us know by posting a response to this blog.

Gary Burgess
Senior Vice President, Research & Operations
 Follow Gary on Twitter