Leverage the object storage to store your compliance data
Anbazhagan Mani, Cloud Storage Architect & Master Inventor, IBM Systems Group ,
Nils Haustein, Executive IT Specialist, Master Inventor, IBM EMEA Storage Competence Center
Note: This blog reflects the understanding of the authors in regard to archiving & WORM solutions with IBM hardware and software. This document is presented “As-Is” and IBM does not assume responsibility for the statements expressed herein. It reflects the opinions of the authors.
Background - WORM Storage
Long-term records retention is mandated by regulations and compliance rules, demanding to keep certain types of records (files, object, data) in an immutable manner for seven and more years. For example, in the financial services industry, SEC Rule 17a-4(f) specifies that “electronic records must be preserved exclusively in a non-rewriteable and non-erasable format” for the retention period. Like the SEC 17a in the U.S, there are other regulations in other countries and industries that mandates long term data retention.
WORM (Write Once and Read Many) is a key technology concept behind implementing data storage archive solution to comply with regulations such as SEC 17a. WORM storage can be implemented via optical media hardware solutions as well as software-controlled mechanisms (such as file systems). Recently, Object Storage has also evolved into a viable storage medium for WORM data.
Use Case
This blog post presents a WORM storage solution that combines the tiering capability available in the WORM compliant file system (IBM Spectrum Scale) and locked vault capability available on the object storage (IBM Cloud Object Storage). This solution can be useful for clients who wish to leverage object storage to store and manage compliance data for long term retention.
Cloud Object Storage – Locked Vaults
IBM Cloud Object Storage (previously Cleversafe) enables administrators to create vaults, which are under the exclusive control of a given external application (by using a secure private key infrastructure). This allows the application to have full control over the vault, but will not allow a user or administrator to bypass the application and access the vault directly. Administrators are allowed to create WORM-style vaults that will enforce read or write restrictions on the objects in the vault, which they will not be able to bypass.
More details on Locked Vaults can be found in “Locked Vault Guide’ from the Help section of IBM Cloud Object
Storage documentation.
Spectrum Scale Immutability
Immutability means to associate a file with a retention time and prevent any changes or deletion of the file data during the retention time. Immutable files are write-once-read-many protected (WORM) for a given period of time which can also be unlimited. After the retention time has expired the file can be deleted but not changed. Append-only mode allows data to be appended to the files, but no over-write of existing data or deletion of file is allowed.
With this immutability function Spectrum Scale can be used for archiving data subject to regulatory compliance by preventing changes and deletion of files during the life cycle.
More details on Spectrum Scale immutability can be found in the document https://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102620
The Spectrum Scale immutability function in Version 4.2 has been assessed for compliance by a globally recognized auditor in accordance to US regulations (SEC17a-4f) as well as German and Swiss tax and trade laws. The detailed assessment report can be found at http://www.kpmg.de/bescheinigungen/RequestReport.aspx?41742
Spectrum Scale Transparent Cloud Tiering
The tiered storage function in Spectrum Scale allows for optimal placement of files on the most appropriate storage technology during the entire lifecycle of the data. Tiering is important for archiving as large volumes of data that has to be retained for long period of time can be archived on lower cost storage tier such as cloud storage.
Transparent Cloud Tiering on Spectrum Scale allows policy based or manual migration of files from the file system to the cloud storage. After the migration, the file will be present in the file system as stub. Accessing the stub(s) will transparently recall the file(s) from the cloud storage. Transparent Cloud Tiering supports many Cloud Storage Providers including IBM Cloud Object Storage. The WORM solution support is offered only with IBM Cloud Object Storage On-Premise deployments that supports Locked Vaults. More details of Transparent Cloud Tiering can be found on the following links.
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/General%20Parallel%20File%20System%20%28GPFS%29/page/Transparent%20Cloud%20Tiering
https://www.ibm.com/support/knowledgecenter/STXKQY_4.2.3/com.ibm.spectrum.scale.v4r23.doc/bl1ins_mcstore_intro.htm
WORM Solution Support
With Spectrum Scale 4.2.3, we now support WORM solution that leverages immutable file sets in Spectrum Scale in combination with transparent cloud tiering and Cloud Object Storage Locked Vaults.
The key characteristics of the WORM solution are
- Immutable files cannot be deleted or modified from the file system (until the retention period has expired).
- Additionally, IBM Spectrum Scale supports a POSIX interface based method of setting immutability (what NetApp refers to as SnapLock® ) . This POSIX interface can be leveraged by archiving applications to set immutability on the files.
- Immutable files on the file system can be tiered on to the Cloud Object Storage system by using transparent cloud tiering. Thereby the control of file immutability is within the Spectrum Scale immutable fileset. All the data is encrypted at client(file system) side before it is migrated to the object storage. Also HTTPS is used to secure the communication.
- Object on the Locked Vaults of Cloud Object Storage cannot be deleted by administrators or other users.
Combining Spectrum Scale Immutability and Cloud Object Storage Locked Vaults allows to place the archived data on the most appropriate storage medium over the lifecycle of data while retaining immutability of files. This solution is based on IBM software defined storage with no requirement for hardware appliances.
For more information on the above solution, please contact your IBM representative or relationship
manager.
-------------------------------------------------------------------------------------------------------------------------------------------------------
SnapLock® is a registered trade mark of NetApp Inc. in the United States and other countries.