IDEAS Insights

Building Large SMP Systems with Virtualization

As a former designer of big iron, scale-up computing remains dear to my heart. Some specialized applications still perform best on large, shared memory, tightly-coupled SMP servers. Even for workloads that need only fractions of a large server, the opportunity to share "headroom" allows multiple applications to reside on the same large SMP, typically via virtualization partitioning. These systems can run at a much higher utilization percentage by pooling the resources that are "reserved for peaks" than if each workload were deployed on separate, individual servers. That flexible pool of hardware resources is often just as valuable as the ability to host a large, monolithic application. 

Unfortunately, the sophisticated scale-up designs created by innovative engineers must amortize their development expense across a relatively small volume of servers, compared to the ubiquitous industry-standard x86 platforms shipping as near-commodities. While I remain convinced that large, scale-up, shared memory servers are crucial for many computing environments, I acknowledge that x86 implementations offer compelling price/performance.

Is there a way to combine the best of both attributes? That is, can large shared memory servers be created from commodity-priced, industry-standard building blocks? Can some "glue," be it hardware or software, allow a collection of small, inexpensive servers to appear to perform as if a large, shared memory system? This week, two firms, 3Leaf Systems and ScaleMP, have introduced alternatives they each believe will address significant portions of the market for economical scale-up systems by coupling standard x86 servers.

Both firms unveiled their solutions just before SC09, November's annual USA venue showcasing innovative high performance computing solutions. Why SC09 and not a trade show targeting the market for broader commercial computing? Attendees of the supercomputing conference usually have very demanding compute requirements, often needing very large servers, but they are also typically budget-constrained. The promise of a large, shared memory, server at commodity price points understandably excites these users.   

Of course, a virtual "appears like a very large shared memory SMP" server will not perform identically to an actual large hardware-optimized SMP server. Therein lies the difference between 3Leaf and ScaleMP solutions. The 3Leaf platform employs a 3Leaf-designed ASIC hardware chip that melds together the memory space of multiple x86 motherboards (currently AMD Opteron, but planning Xeon support with the next iteration of Intel QuickPath) to deliver low-latency shared memory reminiscent of SMPs. ScaleMP, on the other hand, forgoes the expense of unique hardware, and accomplishes coherent shared memory via software. Admittedly, 3Leaf's hardware memory coherence is faster than ScaleMP's software mechanism, but ScaleMP claims its pre-fetch and local caching algorithms deliver the desired low latency appearance without the expense of specialized hardware.

High Performance Computing (HPC) users are acutely aware that the road promising affordable high performance is littered with firms who dreamed of delivering champagne performance at beer budgets. NUMA, non uniform memory access, is at the heart of most "affordable" scale-up designs. Flat, uniform memory access, designs prove to be costly as systems grow larger and larger. The key is to have NUMA's apparent non-uniformity seem as small as possible. KSR in the early 90s, Convex, SGI's Distributed Shared Memory, and other Scalable Coherent Interface (SCI) users (such as Sequent and Data General) all promised the performance of sophisticated, optimized SMPs at commodity server prices. Fairly recently, Virtual Iron attempted a hypervisor that could join multiple smaller servers into a virtual larger SMP, but it eventually abandoned that effort to focus only on applying virtualization for partitioning (the company has since been absorbed into Oracle.)

Clearly, a virtualized NUMA SMP will not adequately substitute for a real SMP for all environments. The key question is which workloads will be satisfied by ScaleMP's software virtualization, which will benefit from 3Leaf's hardware memory coherence, and which applications demand the more uniform memory access of true large SMP implementations.  All current operating systems have evolved to understand how to efficiently keep processing and associated memory physically close to each other on NUMA platforms. However, some workloads have less sensitivity to NUMA than others. Targeting an announcement at the supercomputing conference recognizes that many HPC applications depend less on the shared memory updating that plagued early NUMA attempts to address general purpose commercial computing. The inherent parallelization of HPC applications designed to exploit large infrastructures makes them better prepared to run well on 3Leaf and ScaleMP platforms. Optimized large SMP hardware designs will continue to offer the best performance for applications that exploit multiple compute engines and large shared memory. However, other application environments may benefit from hardware and/or software mechanisms that lash together lower cost commodity servers to create the virtual appearance of a large system, yet with very attractive pricing.

We believe there will be growing customer demand for virtualized SMP solutions like 3Leaf and ScaleMP, which offer lower cost, even with the tradeoff of potentially lower performance.  3Leaf Systems' hardware ASIC is impressive in its use of InfiniBand switching as a substitute for custom SMP backplanes. The low latencies of today's InfiniBand implementations are likely to provide an adequate SMP-appearance for many applications. ScaleMP's software-only solution may not offer the low latencies of 3Leaf, but it is likely to attract a number of customers who desire large memory accessibility at a very economical cost, albeit with somewhat slower latencies. We predict that 3Leaf and ScaleMP will see first adoption among HPC users, but each will eventually carve out a niche among commercial application users. The optimized design of monolithic large SMPs will still be needed by many customers, but virtualized SMPs, aggregated from low cost commodity servers, will offer an attractive price/performance alternative for a variety of workloads.

End User Interest High at Virtualization Forum

Yesterday I attended VMware’s Virtualization Forum 2009 held at the Sydney Convention & Exhibition Centre, Darling Harbour. Virtualization in its various forms has been a hot topic for years and looking at the growth of this event in terms of the number of attendees, is probably an independent verification of the interest in virtualization in the market with real users. Even in these tough economic times.

In his keynote address, Paul Harapin, Vice President VMware Australia and New Zealand, commented that this was the fourth annual event of the Virtualization Forum, which began with just 350 attendees a few years ago. This year’s event attracted 2,500 attendees. This may not be a big number by international standards, but Mr. Harapin noted that by VMware’s measurements, virtualization penetration in Australia is higher than any other country. Other high-adopters are Holland and Israel. This doesn’t come as a great surprise because Australia has had a history of adopting new technologies. In the 80s for example, Australia had a high market penetration of what was then a hot new technology, the fax machine. In fact, probably because of this predisposition for all things new, Australia has a reputation as a test market for many companies where they will trial new products and/or packaging, before rolling them out around the world.

Carl Eschenbach, EVP of Worldwide Field Operations was the second keynote speaker and he took the audience through VMware’s product portfolios and directions for the company.  VMware claims around 150,000 VMware customers globally and although initially, the motivation for adopting virtualization is often to save costs, according to Mr. Eschenbach he is being told by CFOs that greater business agility is the key motivator, especially at the big end of town.

It wouldn’t have been an event on Virtualization without “Cloud” being mentioned and so the audience wasn’t disappointed. In fact, Cloud Bingo was alive and well on the Twitter feeds. (#vForum #vForumAust & vForum). Having said that I don’t think VMware overplayed its hand with Cloud for the event, given the hype out there, it has to be addressed in such a forum.

VMware’s definition of Cloud is very broad, being simply “Deliver IT as a Service”. In a subsequent discussion with Dr. Stephen Herrod, CTO at VMware, he pointed to a VMware survey that found 15% or respondents had adopted Cloud in some form and 26% were evaluating. Although, it wasn’t clear what ‘adopting Cloud in some form’, means exactly.

Beyond Cloud a clear message from the event was that VMware is getting involved up and down the stack in middleware and now applications, with its recent acquisition of SpringSource.  In fact, VMware appears to have interests everywhere except for the hardware and operating system layers. Mr. Herrod was quite clear however, that it is VMware’s intention to stay away from the operating system and for VMware to add value virtualizing elsewhere.

The event saw a strong push to message Desktop Virtualization and Mr. Eschenbach’s believes that 2010 is the tipping point for desktop virtualization. This is because in his opinion many factors are coming together, such as the increased interest in virtualization generally and the expected Windows 7 inspired desktop refresh. I would add to that the 3G telecommunications rollouts around the world means that mobile devices and applications give viable alternatives for users to still be productive when not at their virtualized PC and fixed line broadband connection.

The business justification is certainly there. If you think servers have low utilization, desktops generally have an even lower utilization. Because of this low relatively lower utilization, potential consolidation ratios for PCs are even higher than desktops. So, where you might be able to consolidate say 4 to 8 physical servers onto a virtual platform, the desktop consolidation ratio can be as high as 10 or 20 to 1. Desktop virtualization has always had what seemed good business cases, but for one reason or another it has never taken off in a big way.
 
I would be interested in your views on the future of Desktop Virtualization. Is this something you are now considering and why now? Or, do you still see impediments to the take up of this technology?

Beyond the keynotes, the Forum had an exhibition hall with around 60 vendors marketing their offerings including many well known brands and not so well known organizations. Feedback was generally positive and response in terms of the number of delegates shows that even in these difficult financial times, whether they are committed VMware users or novices interested in understanding more about moving into the world of virtualization, there is still interest from users where they perceive there is value to be had.

Did you attend the event? If so, what did you think of it? Let us know by posting a response to this blog.

Gary Burgess
Senior Vice President, Research & Operations
 Follow Gary on Twitter

IBM’s Overlapping Design Teams Ensure Flow of New Servers

IBM's recent earnings release - lower revenues but higher earnings - brings to mind an old saying "You can't save your way to prosperity."  Perhaps some cutbacks may turn out to be short-sighted, but those that challenge/change business fundamentals may yield long lasting benefit.

Locales suffering IBM's "stealth layoffs," under the guise of "resource rebalancing," may bemoan the shift of employment offshore.  But, the merging of "z," "p," and "i" design engineers allows more frequent product refreshes across all server platforms.  

Designing a complex system does take many years.  Mainframes used to see significant updates every 5+ years, but now that mainframes leverage POWER building blocks, IBM has indicated that its next mainframe update is targeted for 2010, just a couple years after its 2008 introduction of z10.

Back in the pre-90s mainframe heyday, IBM had multiple overlapped design teams – while one was wrapping up a design, another was in the midst of design and yet another was exploring alternatives for the next, future design.  As one design finished, engineers were shifted onto a future design.  The overlapped teams meant that a new system could be released twice as often than if there were only a single team.  Over the years, cost cutting reduced mainframe engineers from 2.5 teams to something like 1.5 teams, which lengthened the introduction of new architecture.

That overlapping team approach is still used for complex chip designs like IBM's POWER.  The POWER5/5+ team had a different crew of engineers than the POWER6/6+ team.  The POWER7 team picked up some of the POWER5 guys while the POWER6/6+ guys were still hard at work.  POWER8 is now the skeleton exploration team.  Sure, there is a cadre of gurus who oversee all, but the grunt work involves overlapping teams of engineers.  Like clockwork, the overlapped POWER teams have introduced new designs every 3 years (POWER4=2001, POWER5=2004, POWER6=2007, POWER7 promised for 2010) with speed-bump midlife-kickers in between.

The mainframe continues to support an instruction set architecture (ISA) that has evolved over the past 40+ years while steadily maintaining backwards compatibility.  It is clearly not the same ISA as POWER, with its RISC roots.  But, ISA aside, there are a lot of common functions – cache hierarchies, memory interfaces, even execution units (IEEE floating point, Decimal floating point, etc.) – that can be leveraged between POWER RISC and mainframe CISC.  That's just what happened with z10 – a bit of unique mainframe ISA design coupled with some fundamental cache/memory building blocks. 

In 2010, the same design leverage is expected.  POWER7 is promised for staggered 2010 rollout across Power Systems UNIX servers.  And, IBM's recent earnings call indicated that the next mainframe launch will also be in 2010.  Downsizing the number of design teams is truly heartbreaking for now unemployed system designers.  But, combining design teams across products and leveraging fundamental building blocks has allowed IBM to reduce its costs and still deliver timely product refreshes.

Do Economy-Size Servers Make Business Sense?

IT managers constantly seek the optimal size of servers that will most efficiently handle their computing workloads. Clearly, if a particular application needs a large memory and can use a lot of processing cores, then a large server seems like the right choice. However, if individual applications do not need such capacity, then a collection of smaller, less costly servers may seem attractive. On the other side of the coin, the cost to power and manage server sprawl has increased dramatically in recent years, making it unacceptable in many cases. The question ultimately comes down to how much to consolidate into each server. IT managers must carefully balance the higher hardware costs and the reduced number of software images to purchase, manage and patch in a large server, against the lower hardware cost, higher software cost, and higher management costs of a number of smaller servers. Does an “economy size” larger server provide the execution efficiency economies of scale to justify the typically higher cost of the higher-end design?

Here at Ideas International, we deal with server pricing, energy efficiency, and performance, so we thought we would provide some basic calculations to illustrate some of the costs involved in determining the optimal size of servers. Note that we are not attempting a comprehensive TCO analysis, which is beyond the scope of a blog. For this exercise, we selected the HP ProLiant DL785 G6 and four DL385 G6 servers. We could have selected IBM servers because they have an industry-standard server that can scale to eight sockets, but we felt that HP was a better example because the DL785 G6 can scale within the same physical box. Next, we loaded each configuration to its maximum number of six-core “Istanbul” Opteron processors, populated a total of 32GB of memory, added an equal number of 300GB SAS drives, selected the redundant power options, and loaded Windows 2008 Server (Enterprise Edition). Essentially, the stack of four DL385 G6 rack servers had an identical configuration to the single DL785 G6. And these servers where not stripped configurations; they were configured to host real application loads.

When we looked at the overall hardware cost of our configurations, the four DL385 G6 servers cost a total of $39,796 before any discounts and allowances, and the DL785 G6 cost $55,819. That 29% price differential was to be expected because designing and building eight sockets into a single server requires  more technology and engineering than two socket servers. What did surprise us was how small the difference in cost actually was when selecting real-life configurations rather than the low cost entry configurations showcased on HPs’ website. One point worth noting is that the four DL385 G6 servers require four Windows Enterprise licenses at a cost of $2,710 each versus just one license for the DL785 G6. Those license costs have already been factored into the total costs above.

Finally, as a rough gauge of energy costs, we looked at the cost to power the power supplies rather than the more detailed summation of the power consumption of all of the individual server components. In this exercise we ignored the redundant power supplies and concentrated only on the base power options, assuming they were running at 100%. It is important to note that both ProLiants support the optional HP Insight Power Manager to help monitor and regulate power usage. The DL385 G6 has a choice of three power supplies, 460W, 750W and 1200W, and the DL785 G6 only supports three 1200W power supplies. In our case, we selected the 750W power supply for our DL385 G6 configuration due to the two six-core processors, the amount of memory and the two SAS drives. If we had installed more memory and other options, the 1200W would have been a more appropriate choice. Calculations were based on three years of ownership and a commercial power cost of 15.32 cents per kWh (New Hampshire, USA). Power costs were kept constant for all three years, although that is probably an unrealistic assumption.

HP ProLiant DL385 G6 Power Costs

Power Supplies (qty of 4)	36 month power cost (USD)
460W	$6,800
750W *	11,100
1200W	17,800

*Used in our configuration of four DL385 G6 servers

HP ProLiant DL785 G6 Power Costs

Power Supplies (qty of 3)	36 month power cost (USD)
1200W **	$13,300

**Used in our configuration of the DL785 G6

The results above show that there really is not much difference in energy cost between the two stacks. The DL785 G6 power costs at $13,300 are roughly the same as the $11,100 for the four DL385 G6 servers.

We were surprised by the relatively small difference in hardware and energy cost between the DL385 and DL785 alternatives. Once the IT Manager has considered all associated costs, including the management of multiple servers, there may not be much cost difference at all between the two options. What that means is if a large server like the DL785 G6 will handle the application stack more efficiently, then the 29% difference in hardware and software cost may not be significant in our opinion. Other driving factors for selecting the DL785 G6 include its ability to run Windows Datacenter and the fact that there is a single entity to manage versus four servers with the DL385 G6. There is also the issue of virtualization efficiency on a large server versus numerous smaller servers. With a common pool of shared compute “headroom,” the larger, eight-socket server may be able to run at higher overall utilization than the smaller, two socket servers.

As IT managers confront the question of choosing the optimal hardware footprint for x86 workloads, they still have a tough decision to make. This pricing exercise indicates that the hardware and energy costs may not be the deciding factor for selecting a larger x86 server instead of multiple smaller servers.  Rather, it will more likely be the “soft” costs related to management that will determine whether eight-socket x86 servers make economic sense. While these costs are notoriously challenging to quantify with a generalized approach, the relatively modest pricing premium of the DL785 G6 sets a low hurdle for achieving tangible returns on its investment.

Low Latency and Clouds Pervade at High Performance Computing On Wall Street Conference

At the 6th Annual High Performance Computing (HPC) on Wall Street conference, the key words of the day were low latency and cloud computing. The two topics were often combined in displays and talks that interwove both concepts. Despite the collapse of some of Wall Street’s biggest investment banks in 2008, the drive for faster low latency computing not only continues, but is coveted, as microsecond trading continues to reap profits for the ones who survived. 

Most of the session speakers focused on the question of what cloud computing means for the financial sector, while vendors displayed their top-of-the line equipment in the exhibit area. IBM’s booth focused on its Computing on Demand offering. Unlike some cloud competitors, who offer cycles distributed across a myriad of servers, IBM believes that HPC users need to have dedicated servers and the knowledge of exactly what they are getting (IBM’s offering includes some POWER servers, and a good deal of x86 servers). IBM also emphasized its security credentials. Beyond the typical standard offerings related to security (i.e. firewalls), IBM offers separate memory and storage for consumers’ virtual machines.

High performance computing remains a key priority for Wall Street, but so does low-latency networking. As reported in the New York Times, the NASDAQ exchange allows users, for a fee, to obtain stock data 30 milliseconds before it is made public. Although early trading may yield only miniscule profit per share, when performed many times in aggregate, a hefty profit can be made with such "high-frequency" or "flash" trading (note that the practice is now being scrutinized by the Securities and Exchange Comission). In order to beat the competition in this type and other forms of quantitative trading, low network latency is crucial. Fujitsu, Sun, Arista and other vendors all displayed high-end 10 Gigabit Ethernet data center switches to help improve networking performance. Arista showcased hardware that can reduce latency to as little as 600 nanoseconds. For the users who were concerned about how software and hardware components would function together, the Securities Technology Analysis Center (STAC) has been devising performance benchmarks that focus exclusively on evaluating systems for Wall Street requirements. STAC highlights that, unlike other “standard” benchmarks, the end users are the ones who set the specifications. This comes at a price though. It cost $15,000 to have a say in the specifications, while access to the published data costs $5,000.

Even after the collapse of the financial sector, and its continually slow recovery, algorithmic trading is here to stay. This is good news for the vendors as they improve their speed, capabilities, and find more end users in the emerging market of cloud computing. It's also good for the organizers of this event, for they will surely have more conferences in the future, as users continue to seek out new ways to turn computational cycles into wealth.