By Tony Lock and Bob McDowall Network Appliance
Thursday, 21 July 2005 11:07 EST

Compliance, Regulation and Statutory Legislation have become increasing burdens for the long-term storage of data. Legislation operates in tandem with a fierce economic climate that is keeping a tight grip on budgets and is dictating that IT operates with fewer skilled resources, whilst demands for service intensify. It is clear that a way must be found to satisfy these demands cost effectively whilst minimising corporate risk and exposure.

The importance of Compliance and regulations covering records retention at national, European and International levels are imposing severe demands on the long-term storage of data. These, in turn, are placing supplementary burdens on the associated storage infrastructure and its management. This is notable particularly within financial services organisations, where there are many challenges to ensure that data storage systems can handle the new obligations and demands. These include Basel II, the International Auditing Standards (IAS), Sarbanes-Oxley and FSA guidelines specific to UK operations. The numerous regulations imposed upon the Banking sector requires careful assessment of data management policies:-

• What information is stored
• How the information should be classified
• Which storage platforms should hold the data
• To whom access should be granted
• How long the data should be retained
• How rapidly it needs to be accessed
• How it should be archived and/or deleted at the end of its lifecycle

In order to meet the requirements placed upon Financial services organisations, which require them to retain relevant data for long (sometimes unspecified) periods of time, it is essential that they actively manage the information throughout its lifetime. The value of data held varies enormously over its lifetime and the speed with which it needs to be recoverable also fluctuates. The concept of Information Lifecycle Management, frequently referred to as ILM enables management of data to ensure efficient recovery within specific regulatory timeframes. It is the opinion of Bloor Research that effective data storage, data management and ILM lie at the heart of all solutions to regulatory and compliance challenges.

Storage Strategy for Regulatory and Compliance Data

The increasing demand for organisations to store these ever growing volumes of information for extended periods of time makes it unlikely that many of them will wish to store all such data on their most performant (and usually therefore most expensive) disk sub-systems for the entire lifetime of the data. It is clear that the service levels demanded by the business for access to the data are likely to vary considerably over its lifetime. Within the majority of organisations, the value of data usually decreases over time. For example, several recent studies have indicated that around 90 percent of data held in disk storage systems is seldom, if ever, accessed more than ninety days after its creation.

Storage/Data Management software now makes it possible to migrate data between various storage platforms each of which possesses differing performance and cost characteristics. This makes it possible to move data between differing storage platforms in accordance with varying business requirements for access to the data. This approach has become known as Information Lifecycle Management, or ILM. In essence, ILM is an extension to the entire IT infrastructure of the hierarchical storage management principles long practised in mainframe environments.

For example, when data is created by one of the core business financial applications it might be held on a primary storage system that is very responsive with sophisticated data mirroring and rolling-backup systems in place to ensure that the information may be retrieved rapidly and is robustly protected. As the data ages, the business requirement to have rapid access to it may diminish. Consequently, after a period of time that meets all business usage requirements, the data could be migrated to a less expensive, ‘capacity’ disk system, thereby freeing up space on the primary storage platform to store newly created data whilst still providing online access to the information.

After a longer period of time defined by business requirements, the data might then be moved to a Nearline storage platform and eventually to an archive system.

ILM - the building blocks for managing compliance data

The first step in implementing an ILM strategy to manage compliance and regulatory data requires that organisations identify all data that has defined data retention periods. Once the information is identified and the necessary data retention demands assessed, the data should be classified according to its retention characteristics.

Information Lifecycle Management has the potential to play a pivotal role in helping organisations meet data retention and compliance obligations.

• The first step that must be taken is to ensure that all of the compliance and regulations that apply to the business are understood and classify the impact that these have on the ongoing retention and handling of data.
• The next step is to catalogue and classify the data held. The storage management software should ensure that the data is assigned to the most appropriate storage platform as cost effectively as possible whilst ensuring that all service and regulatory obligations are met.
• The third step would then see the underlying storage platforms classified into a number of storage containers each with understood performance and protection. It then becomes possible to manage data through its lifecycle using centrally administered policies that moves data around the storage infrastructure simply on the basis of its classification and the classification of the storage platform required.

ILM and Storage Architectures

For any ILM solution to operate it is essential that the underlying storage infrastructure be flexible and secure, yet capable of delivering differing levels of service at a variety of price points. The requirement for flexibility and tiered performance characteristics dictates that the storage platforms be simple to manage and built utilising open standards.

A flexible, open storage infrastructure coupled with an understanding of the data held by the organisation, its value and its classification in terms of its retention and deletion policies will make it possible to implement an effective ILM solution to handle all regulatory and compliance requirements.

Getting Started with ILM

• Implement an Open, Flexible Simple to Manage Storage Infrastructure
• Identify all data held that is subject to retention/deletion requirements
• Set appropriate Compliance Policy Classes for the required retention
• Classify Data into applicable Policy Classes
• Archive to Write Once Read Many (WORM) Compliant Media (Disk/Optical etc.)
• Set compliance policies for the supervision of content

Summary

Without the adoption of an ILM approach, it is likely that the cost to organisations of ensuring that they remain compliant with all legislative, regulatory and corporate governance drivers will prove to be extremely expensive. ILM demands that not only the storage infrastructure is suitable to meet the needs of the organisation but that each and every person working in the entity understands the importance of working in accordance with appropriate procedures.

Many of the latest compliance initiatives, including Sarbanes-Oxley, Basel II, IAS, EU Data Protection legislation and various industry-specific regulations are at an early stage in their development. As is always the case with any form of regulation it will take time for many of them to evolve and mature. It is in the very nature of compliance solutions that they are always subject to refinement and, especially in Europe, to local legal interpretation.

It is the opinion of Bloor Research that the deployment of open, flexible storage
infrastructures will play a crucial role in determining the ability of organisations to meet their obligations in respect of compliance and regulation. Without adequate ILM it is likely that organisations will be unable to meet their obligations without imposing a massive burden on personnel and/or incurring potentially excessive costs. A simplified, open storage infrastructure, good storage management software and well-defined and refined data classification and management policies will prove to be invaluable.

It is clear that many organisations have already decided to adopt an approach that stores everything rather than risk the consequences of not having access to data when it is requested or, more likely, demanded by a regulatory body. This methodology has the benefit of simplicity, speed and, when coupled with a suitable ILM strategy, could prove to be financially attractive.

Network Appliance are exhibiting at Storage Expo the UK's largest and most important event dedicated to data storage, now in its 5th year, the show features a comprehensive FREE education programme, and over 90 exhibitors at the National Hall, Olympia, London from 12 - 13 October 2005 www.storage-expo.com