Primary Storage

Originally posted by: Tom_Clark

In my previous blog post, I discussed the role of the storage administrator in virtual server environments and whether they would continue to manage storage or largely cede their current role to the virtual server administrator. A related topic is where advanced storage function itself resides in a virtual server environment. This is interesting both for virtual server and storage management as well as compute clouds, with virtualization of servers and storage as a key onramp to the cloud.

The types of function I’m referring to are the typical advanced storage functions we’ve grown accustomed to: volume management and striping of data across volumes, snapshots, copy services, thin provisioning, compression, etc. There are two primary options for location of advanced storage function – on the host in the hypervisor or in the storage network. The question of where to locate advanced storage function is not a new one. We’ve had logical volume managers (and IBM’s DFSMS for the mainframe) provide this function on servers for many years and have seen storage controllers dominate advanced storage function for more than a decade.

In order to determine where it’s best to place advanced storage function in a virtual server environment, it’s helpful to briefly review why this function largely migrated to the storage network in the first place. The rationale was very simple: shift the CPU, memory, network, and bus impacts off the host to purpose-built storage systems. This shift in storage resource consumption allowed the host to focus on efficient application usage of resources without so much storage processing interference. The concentration of function in storage-focused systems allowed for creation of optimal performance and efficiency in storage. This continues to be the case in spite of more off the shelf hardware components and less special purpose hardware in modern controllers. The function is also available on a variety of host platforms by simply porting the storage device driver rather than the full set of storage function. Over time, additional advanced function has been added to storage systems, including sophisticated copy services, data compression, load balancing and striping across device types, etc. As storage systems have increased in capacity and density, the advanced functions had to be designed to scale and perform for increasingly demanding workloads. These characteristics and advantages hold for both block storage systems and NAS filers. The scale and complexity of storage networks and controllers called for sophisticated tools and focused storage administration.

Now back to the question of where storage function should reside in a virtual server environment. Server hypervisors are taking a two-pronged approach to storage management. On the one hand, they are adding advanced storage functions to their hypervisors. On the other hand, they are creating APIs to integrated with storage controllers to request and coordinate with advanced storage function. Virtual server environments tend to be much more dynamic and require a level of flexibility (workloads coming and going, workload movement across systems, etc.) than non-virtual environments. They require the storage environment to have the same level of dynamism and flexibility as the virtual server. One way to obtain this required storage capability is to depend on the hypervisor to provide this function. There are a few issues, however, with this approach: storage operations will take resources from the applications, scalability of the storage operations on the host is limited, and the hypervisor is trying to catch up to a decade of innovation in advanced function in storage systems. For these reasons, use of storage function in the hypervisor should be limited to lower scale environments and ephemeral storage (where many advanced storage functions are unnecessary).

The ideal storage solution will have similar characteristics to the server hypervisor, a “storage hypervisor” if you will. The combination of IBM’s System Storage SAN Volume Controller (SVC) for storage virtualization and Tivoli Storage Productivity Center (TPC) for storage management is an example of such a storage hypervisor. Analysts have begun writing about the need for such a storage solution in virtual server environments (see Gartner and ESG (requires free registration)). In addition, my colleague Ron Riffe has written a series of blog entries about the IBM storage hypervisor. I encourage you to read those entries as well as you consider how to evolve your storage environment to meet the needs of virtual server environments and ultimately compute clouds.