IDEAS Insights

HP File System Technology to Help Drive Linux into Open Storage

HP announced that it is releasing the source code for its Tru64 UNIX Advanced File System (AdvFS) under a license that will allow it to be integrated deeply into the Linux kernel. AdvFS is one of the most proven 64-bit system software technologies in the industry. It offers a number of powerful storage features, including the ability to configure flexible storage pools that can be shared by multiple file systems without needing to add a volume manager; the ability to expand, shrink or reconfigure file systems without taking them offline; flexible transaction log management, including the ability to log both metadata and data asynchronously or synchronously; availability features such as snapshots for consistent backups and support for recovering deleted files; and performance features such as online rebalancing of files and free space across storage pools. Moreover, these advanced features have proven in production for 16 years as a key component of Tru64 UNIX, which itself was one of the most advanced UNIX implementations on the market, before becoming retired in the fallout of HP's acquisition of Compaq.

HP emphasizes that its release of the AdvFS code into open source represents a technology contribution, and is not simply a port that will result in another competing file system for Linux, alongside ReiserFS or Global File System (GFS). The AdvFS code is being released under the same GPL Version 2 license used for the Linux kernel, which means that developers will be able to drive its features as deeply into the kernel as necessary to make them work effectively. HP is actually providing two generations of the AdvFS source code: the existing code taken from Tru64 UNIX, and work-in-progress code that HP produced during its abandoned effort to port AdvFS to its HP-UX operating system. HP is also providing an extensive set of design documentation, test suites and engineering resource related to the code. Linux programmers will be able to refer to this broad base of knowledge as they work on the next-generation Linux file systems currently under development, such as ext4 or BTRFS.

Indeed, it will take several years before the results of HP's AdvFS offering show up in mainstream Linux distributions. Aside from sheer altruism on HP's part, though, the contribution has several implications, both for HP and its customers, as well as the industry at large. First, the availability of proven AdvFS technology on a Linux foundation will make that OS more attractive to any remaining Tru64 UNIX users who may still be resisting HP's recommended migration to HP-UX (even if HP decided to offer innovative migration technology such as Transitive QuickTransit). If these users move to Linux, HP could still retain some as customers, which they could not if the users migrate to a competing UNIX system.

Further, by strengthening Linux storage management functions at the kernel level, HP will help to qualify Linux as a foundation for Open Storage, allowing it to serve as the core software for scalable and reliable network storage solutions that are based on standard hardware. The Open Storage concept has been promoted heavily by Sun with its OpenSolaris initiative, which allows users to couple the advanced storage functions in Solaris, i.e. the ZFS file system, with inexpensive, industry-standard hardware components. As the storage management function in standard operating systems become more powerful, such solutions could start to disrupt traditional enterprise storage systems, most of which are based on proprietary hardware. It is not possible to add ZFS technology into Linux, because it is licensed under Sun's Common Development and Distribution License (CDDL), which is incompatible with GPL v2. AdvFS matches several key ZFS features, and as Linux developers embrace AdvFS technology, it will become more feasible to offer Open Storage solutions based on standard Linux distributions, whereby solution providers add value with various data services (copy, replication, de-duplication, thin provisioning) on top of standard interfaces in the Linux kernel. By becoming fully engaged with efforts to enable Linux for open storage at the earliest stages, HP can start to position itself in a leadership role should open storage solutions start to generate serious momentum in the market.

Sun's Two-Tier OS Support Targets Next Wave of IT Infrastructure

Much of the discussion around Sun's OpenSolaris operating system has focused on comparing it with Linux in terms of open source development communities and processes. Indeed, the relationship between OpenSolaris and Solaris somewhat parallels the segmentation of leading Linux distributions into "development" and "production" releases, i.e. Red Hat Fedora and Red Hat Enterprise Linux, and Novell’s OpenSUSE and SUSE Linux Enterprise Server. But the key development of this announcement is Sun’s introduction of robust support offerings for its dynamic, rapidly evolving operating environment. Neither Red Hat nor Novell offer formal support for Fedora or OpenSUSE (although Novell does provide free installation support for OpenSUSE). Thus, Sun has broken new ground in the open source OS business model by arriving at a solution that envisions the long-term needs of web-centric enterprises.

It is clear that much of the growth in the market is increasingly being driven by a new class of customer with a set of needs that diverge considerably from those of traditional commercial server environments. Users in this class consider massive levels of scale-out computing to be the normal way to grow capacity, and they rely heavily on proprietary software that is developed internally, which represents the “secret sauce” of their operations. Currently, the leading-edge proponents of this approach are predominantly in high-performance computing (HPC) environments; entertainment and media companies delivering content over the web; firms with web 2.0 business models; and some financial services organizations. Over time, though, their style of computing could impact a variety of organizations, as the lines between software and services start to blur, and end-users become more comfortable relying on remote computing resources that are accessed over the web.

As this segment of the market matures and gears up for operations, its customers are introducing new requirements for how systems should be designed in order to best meet their needs, some of which could send vendors back to the drawing board (as shown by IBM’s introduction of its iDataPlex architecture). Among other demands, these customers are creating a new set of rules for operating systems that will play a central role in their infrastructure. To be relevant in the rising wave of web-based computing, an OS should have the following characteristics:

• Support for industry-standard hardware based on x86/x64 processors, which are taking on ever more demanding workloads as a result of their continuously growing performance;
• Integrated support for virtualization, which is becoming a standard part of IT infrastructures both in hardware and software;
• Support for cloud/grid/utility computing, i.e. the ability for organizations to achieve optimal utilization of computing resources, regardless of their source or physical location;
• Real-time computing capabilities, in which services are performed within a guaranteed time span;
• Open source business models, whereby the OS supplier offers value based on service and support, rather than licensing;
• Choice of release streams, including a relatively fluid version with frequent updates and rapid functional improvements, as well as a more stable version suitable for critical production workloads.

Sun has addressed this last requirement with the first commercial release of OpenSolaris. While OpenSolaris has been available for some time, Sun announced that it will now provide support for the binary distribution of the OpenSolaris code. As a result, Sun now effectively offers its customers a choice of two supported operating systems: Solaris and OpenSolaris. Sun positions Solaris for “high speed developers and development teams”, as well as users who treat the operating system as a source of competitive advantage for applications such as high performance computing and social networking. By contrast, the traditional version of Solaris is intended for IT departments that value stability over rapid innovation.

At the front line of the largest web-based applications, resiliency is typically built into the user's software, so that few reliability features are needed at the level of individual servers. These applications are usually designed to continue processing in the event that servers fail due to outages in hardware or OS software. Thus, Sun’s offering of support for OpenSolaris may not be relevant for many of the cutting-edge web applications that might be deployed on it today. However, today’s web 2.0 workloads are tomorrow’s enterprise workloads representing the corporate backbone. As individual nodes in the cloud are used to host ever more critical workloads that are sensitive to critical points of failure at the level of individual systems, and thus require higher levels of uptime in the OS, OpenSolaris users may start to take Sun’s support offerings more seriously.

Sun offers two levels of support for OpenSolaris: "Essential" and "Production". While the pricing of these offerings has not yet been announced, based on their description, the Production level of support for OpenSolaris appears to be nearly identical to the "Premium" level of support for Solaris (except OpenSolaris Production promises one hour response time for Priority 1 calls, as opposed to immediate transfer for Solaris Premium). Therefore, the pricing of OpenSolaris Production compared to Solaris Premium will determine whether it makes more sense to deploy Solaris (which is free to deploy as well) or OpenSolaris. Compatibility between OpenSolaris and Solaris will also be a critical factor in determining which path customers take as they upgrade, thus controlling the flow of users from OpenSolaris to classic Solaris.

The ability for Sun to drive Solaris into IT infrastructures that are central to customers’ businesses remains a key milestone on its path to long-term success. Although Sun is aggressively optimizing its systems for web-based workloads, software serves as a much "stickier" bond with customers than most hardware, which can be swapped out relatively easy. As Scott McNealy was fond of saying, “users date their hardware, but marry their operating systems”. In the bigger picture, as more and more new businesses are seeded with infrastructures that are based on the web and scaling out, a requirement is emerging for new class of OS with a specific set of attributes, only one of which is an open source development model. Solaris and the leading Linux distributions are all well positioned to deliver these attributes, but Sun’s innovation in the business of operating systems could allow it to leap ahead of others in defining how this critical software component becomes integrated to customers’ organizations.

Dell Embraces Egenera's PAN Manager to Create a Virtual Datacenter

On Tuesday, Dell and Egenera will be announcing a strategic OEM agreement where Egenera’s PAN Manager will be available on Dell PowerEdge servers. PAN Manager is a unique management tool that is at the heart of Egenera’s enterprise BladeFrame servers, providing the ability to virtualize and manage not only virtual server environments, but also storage and I/O. Essentially, PAN Manager allows IT managers to create an entire virtual datacenter where nothing is tied to physical hardware. Compute, storage, and network resources can be dynamically allocated when needed and where needed. PAN Manager includes application clustering that allows virtual servers to failover automatically, and also full server failover to remote locations for disaster tolerance. All of these capabilities are tied together with a strong Egenera management framework that implements the high level of security demanded by many of Egenera’s financial services and government customers.

How will Dell benefit from this relationship? Dell has traditionally been firmly positioned in the industry-standard server space, providing a wide selection of servers at a great value. For virtualization, Dell PowerEdge servers support a host of third-party software, including VMware. Dell servers can be managed using the Dell OpenManage framework. In today’s market, Dell can compete very effectively with other vendors on simple server virtualization and SANs. But what it lacks is a management tool that can pull everything together into an entirely virtualized datacenter. That is where PAN Manager comes into play. With PAN Manager, Dell leaps over many of its competitors with the ability to create the virtualized datacenter of the future today using inexpensive industry-standard components.

Will this OEM relationship ultimately be extended to Sun, HP, and IBM, allowing the creation of heterogeneous virtualized datacenters? IDEAS feels that it is unlikely that HP and IBM will be offered the opportunity to partner with Egenera due to the intense competition Egenera experiences from these two vendors on a daily basis. But Sun may be a real possibility in the future. Sun already has a long-term OEM relationship with Fujitsu and Fujitsu Siemens has an OEM relationship with Egenera for PAN Manager. However, neither Sun nor Egenera has indicated that a deal is in the works.

We at IDEAS feel the OEM relationship is a win-win for Dell and Egenera, as well as the customers of both companies. This deal actually began when Egenera’s customers, who also own Dell servers, asked both companies to extend the benefits of Egenera’s virtual datacenter to Dell’s commodity hardware. The Egenera BladeFrame with PAN Manager is an outstanding product, but its cost precludes many IT departments from deploying it in general computing environments. However, the low price of the PowerEdge servers allows Egenera’s virtual datacenter to be brought into the mainstream server market, giving Dell an advantage that IBM, HP, and Sun cannot match. This relationship could really become very interesting should Dell decide to deploy PAN Manager on its new PowerEdge M1000e blade servers, or link PAN Manager into the OpenManage framework. Neither of these projects has been formally announced, but they both make sense as the relationship grows and matures. If you currently deploy Dell servers, you owe it to yourself to learn a little more about this new Dell/Egenera relationship.

Tackling Virtualization Management with Open Source

As basic virtualization functions become a standard part of infrastructure both in hardware and software, many vendors have concluded that virtualization management will offer their greatest opportunities for adding value, and they are rushing to stake out positions with a variety of offerings in this space. Virtualization management solutions are undergoing rapid innovation, and users face a growing set of product choices as a result.

At the head of the pack is VMware with its VirtualCenter platform, which plays by default as a result of VMware's leadership in the x86 virtual machine market. Microsoft, which now claims to be the fastest growing major systems management supplier overall, is in pursuit with its System Center Virtual Machine Manager (VMM). Novell also continues to build out its ZENworks management platform for managing virtual infrastructure, as shown by the recent acquisition of PlateSpin. Systems providers like IBM, HP, and Dell are extending their hardware management frameworks for virtualization, but are treading more carefully so as not to antagonize their key software partners. Finally, a profusion of startups have emerged peddling point products targeting various aspects of virtualization management, and established systems management platforms suppliers such as CA and BMC are gradually evolving to embrace the mass virtualization era as well.

Users typically have to acquire these virtualization management offerings in the manner of traditional software, licensed in some controlled fashion based on hardware footprint or some other metric. But as with any software category, virtualization management is potentially an opportunity for disruption by open source software. Until now, the most successful open source efforts have targeted relatively mature areas like operating systems and databases, and the emergence of open source-based virtualization management solutions will be an interesting opportunity to observe how open source can potentially take hold from the ground up in a major new growth area involving critical infrastructure. Two such efforts to deliver open source virtualization management have recently materialized.

Sun announced that it has released a production-ready version of xVM Ops Center, its new datacenter automation tool that is designed to simplify the management of heterogeneous IT environments. xVM Ops Center is based on a combination of Sun’s N1 System Manager infrastructure management framework with the Aduva technology that Sun acquired in 2006, which is optimized for patch and dependency management. The xVM Ops Center console is designed to automate routine system administration tasks, covering such tasks as discovery and inventory management; firmware and bare-metal server provisioning; patch management & updating; managing and monitoring system operations; and compliance reporting. xVM Ops Center supports Solaris, Red Hat Linux, or SUSE Linux operating systems, and while the initial release only works with physical servers, Sun describes the platform as a critical milestone in its xVM virtualization platform. xVM Ops Center is intended to work equally well with two of Sun’s virtualization technologies: a Solaris-based implementation of the Xen hypervisor which is about to be released, and Logical Domains (LDOMs) in Sun's CoolThreads servers. Sun is releasing the Ops Center code under the GPLv3 license, and it is available at http://www.openxvm.org/.

Red Hat recently revealed an open source virtualization project call oVirt, which includes a new web-based console to manage virtual machine platforms. The goals for oVirt are to provide a secure way for owners of virtual machines to manage their environment without granting them access to the host hardware itself. oVirt is also intended to automate virtual machine clustering, load balancing, and Service Level Agreement (SLA) maintenance; and generally simplify the management of large numbers of machines. oVirt uses a Kerberos/LDAP server for authentication and authorization, and it is tightly integrated with the freeIPA project, so that administrators will be able to authenticate, authorize, and audit their virtual resources across the enterprise. oVirt is also designed to work across platforms and architectures. The current release of oVirt is based on Red Hat’s Fedora 8 Linux distribution and the Kernel Virtual Machine (KVM) virtualization technology, but it uses the libvirt API to control virtualization functions, so it could be adapted to work with any libvirt-compatible platform, including the Xen implementation in Red Hat Enterprise Linux (note that Sun is also embracing libvirt in its virtualization platform). oVirt is currently at an earlier stage of development than Sun’s Ops Center. However, Red Hat has promised to aggressively pursue virtualization opportunities as part of its ambitious expansion plans, so expect to hear more about oVirt in the future.

As with other open source offerings, users will have free access to the code itself for these projects, and they will have the option of paying for subscriptions that provide support and a feed of updates. Consequently, these systems will potentially have lower barriers to entry than traditional software packages, since users will be able to "kick the tires" as long as they want before opting to establish a formal commercial relationship. Neither of these solutions is yet functionally competitive with heavyweight products like VMware VirtualCenter for Microsoft VMM but over time, their open source basis could help them mature rapidly, depending on the momentum that their respective development communities build.

There are many possible motivations for adopting open source software, including cost, practical benefits, and/or ideology. In some cases, users may wish to avoid entrusting their most critical IT functions to a proprietary solution. Indeed, some of the decisions that organizations make now as they start to deploy virtual infrastructure will have a long-term impact. As virtualization management policies are defined, they will become the junction at which virtualization is woven directly into business processes. While service and support are essential for such a critical system component, users may be wary of committing to a long-term dependency on a single vendor to obtain their virtualization management solution. There are few better ways to minimize the risks of such dependency than using open source software. Sun and Red Hat appeal to this reasoning, and they are betting that the open source nature of their offerings will give them a significant advantage over competitors who are using a traditional software commercialization approach.

Microsoft Kicks Off Home Server Era

Microsoft recently held a reviewer's workshop for its upcoming Windows Server 2008 operating system, due for introduction in February. The conference was attended by about 100 press, bloggers, and a handful of analysts, who were largely subdued for much of the three-day conference as the new system's many functional improvements were itemized in detail. The audience noticeably perked up during the very last presentation, though, which reviewed Microsoft’s just-released Windows Home Server operating system. That session generated a swell of questions, revealing considerable interest in the potential for a server platform targeting the needs of consumers.

Windows Home Server is designed for homes with two or more PCs/laptops, and it will typically be delivered in OEM hardware like HP’s new MediaSmart server. The home server is supposed to be turned on all the time and connects directly – i.e. via cable – to the home router (which is usually inside the home's wireless base station). As an appliance, it runs headlessly, so it doesn't have a monitor, mouse, or keyboard - all interaction with it occurs remotely over the network. Microsoft showed prototypes of devices that were very unobtrusive, some fanless and not much larger than a  car battery, designed to be bolted inside closets or under cabinets.

One of the main functions of Windows Home Server is to automatically back up the home’s PCs and laptops (possibly even if they are turned off, since many clients can be remotely powered up on demand using Intel’s Wake-on-LAN protocol). The Windows Home Server backup engine uses a very space-efficient method that duplicates redundant blocks to minimize disk consumption. To protect files from disk failure, data is mirrored over available drives using an adaptive algorithm, so it is not necessary to have strict RAID configurations, which also makes it easy to add or upgrade drives. The server is designed to share media like videos and music between different clients around the home, which will be increasingly valuable to users with media collections swelling to hundreds of GBs (more than currently fits on most Apple iPods), which they may want to consume on multiple devices in addition to portable players, i.e. home audio-visual systems. Any content on the server can be shared over the web, allowing it to be accessed from anywhere, and authorized remote users such as family and friends can deposit content on the server over the web.

The difference in positioning of Windows Home Server compared to Microsoft’s previous “home appliance"  efforts, i.e. Windows XP Media Center Edition, could be critical to its success. As an entertainment device, the Windows Media PC had to compete for living room shelf space that was already cluttered with a variety of other gadgets (and their remote controls), including video games, Tivo boxes, audio/video receivers, VHS/DVD players, and even the TV itself. By contrast, the Windows Home Server device is oriented towards the home network, which allows it to be placed far less obtrusively, like in the basement, attic etc. and promises to give the platform significant leverage as the “gatekeeper” between the home and the web.

The home server concept is still controversial, and skeptics point out that consumers may have little interest in becoming server administrators. However, it appears that Microsoft has done a lot of work to make Windows Home Server as simple as possible to set up, manage, and use. If Windows Home Server devices are successful at becoming as unobtrusive and maintenance-free as other home appliances, while supporting a variety of functions based the growing use of digital resources in day-to-day life, why wouldn’t every future home have one of these built in? If so, Windows Home Server promises to create opportunities for a huge range of applications. ISVs are already starting to line up with add-ons, offering applications for home control & security monitoring etc. Centralized backup solutions targeting consumers in an appliance form factor and based on other operating systems have been on the market for some time. However, it is Microsoft's vast and diverse ecosystem of developers, OEMs and partners that will give Windows Home Server the best chance to reach its full potential as a true server platform meeting the needs of a broad new set of customers.