Friday 11 January 2013

NETAPP_SNAPLOCK


NETAPP_SNAPLOCK

One day I was giving an technical interview, in which interviewer asked me a question, it was related to NetApp storage technology, and the question was that what is the “Compliance volume”, I got stuck and was not able to give any reply, because I was not aware of compliance volume, so after the interview got finished, I star searching for the compliance volume in NetApp and then I found that compliance volume is also know or called as snap lock volume.

I knew about snap lock volume in NetApp, but I did not know that the snap lock volume is also known as compliance volume.

After getting rejected in that interview, I thought of writing something about the snap lock volume, as the snap lock is not much used in most of the companies, so most of the NetApp administrator is not aware of this feature in NetApp and some who knows about this feature have not used it.

SnapLock is NetApp WORM (Write Once Read Many)

Some regulated environment require business records be archived on WORM media to maintain a non-erasable and nonrewritable electronic audit trail that can be used for discovery or investigation.

DataONTAP SnapLock software complies with the regulations of those markets. With SnapLock, the media is rewritable but the integrated hardware and software controlling access to the media prevents modification or deletion.

SnapLock is available in tow version: SnapLock compliance for strict regulatory environment, and SnapLock Enterprise, for more flexible environments.

SnapLock can integrates with the snap Mirror and snap vault, and snap mirror allows the SnapLock volumes to be replicated to another storage system and Lock vault backs up SnapLock volumes to a secondary storage system to ensure that if the original data is destroyed than the data can be restored or accessed from another location.

Once the data is created in the SnapLock volume they comes under the retention period and these files get treated as WORM, so that nobody can delete or modify the data until and unless it reach to its retention period, the SnapLock volumes cannot be deleted by the user, administrator nor by the application, the retention date on a WORM file is set when the file is committed to WORM state, but it can be extended at any time. The retention period can never be shortened for any WORM file.

SnapLock Compliance (SLC)

SnapLock Compliance is used in strictly regulated environment, where data is retained for longer period of time and these data are accessed frequently only for readable purpose.

SnapLock Compliance even does not allow the storage administrator’s to perform any operations that might modify the file, it uses the feature called “ComplianceClock” to enforce the retention periods. SnapLock Compliance requires the SnapLock license to enable the SnapLock features and to restrict the administration access to the file.

SnapLock Enterprise (SLE)

SnapLock Enterprise allows the administrator to destroy the SnapLock volume before all the file on the volume reach their expiration date. However no one else can delete or modify the files.

It requires the SnapLock _enterprise license

NOTE:

1.       From dataontap 7.1 and later, SnapLock compliance and SnapLock Enterprise can be installed on the same storage system.

2.       Uninstalling the SnapLock license does not undo the SnapLock volume or the Worm file properties. When uninstalling the license, existing SnapLock volumes are still accessible for general reading and writing, but won’t allow new files to be committed to WORM state on the SnapLock volume, or the creation of any new SnapLock volumes.

SnapLock Configuration Steps

1.       Verify that the storage systems time zone time and date are correct.

2.       License SnapLock.

3.       Initialize  the compliance clock

4.       Create SnapLock aggregate and volume.

5.       Set retention periods

6.       Create files in the SnapLock volume

7.       Commit files to a WORM stat.

Use the “date –c initialize” command to initialize the compliance clock.

After compliance clock is initialized, it cannot be altered and WORM files, SnapLock compliance volumes, or aggregates cannot be destroyed. As compliance volume get in sync mode with the system clock, so when the volumes is taken offline then the compliance volume will get out of sync with the system clock, but when the volumes is brought online back the data ontap will start to make up the time difference, the rate at which the compliance clock catches up depends on the version of data ontap your system is running.

Creating the snaplock volumes

Snaplock type, compliance or Enterprise, is determined by the installed license.

 

Creating the snap lock traditional volumes

“vol create –L vol_name <n_disk>”

 

For ex :

System> vol create viptrd –L 14

 

For creating the SnapLock flexible volumes, you need to create the snaplock aggregate first

 

“aggr create –L aggr_name <n_disks>”

“vol create vol_name aggr_name <size>”

 

For ex:

System> aggr create vipaggr –L 3

System> vol create vipflex  vipaggr 100m

System> vol status vipflex (for checking the status of the vipflex volume)

 

Volume Retention Periods.

The retention periods are the volume’s attributes. If you do not explicitly specify an expiration date for a WORM file, the file inherits the retention periods set at the volume level.

 

There are three retention periods per volume: a minimum, a maximum and a default.

1.       Until you explicitly reconfigure it, the maximum retention period for all WORM files in a snap Lock Compliance volume is 30 years.

2.       If you change the maximum retention period on a SnapLock Compliance volume, you also change the default retention period.

3.       If you change the minimum retention period on a SnapLock Enterprise volume you also change the default retention period. The minimum retention period is 0 years.

How to set the Retention Periods.

Use the vol options command to set the retention period values.

For ex:

System> vol options vol_name snaplock_minimum_period [{period} |min |max | infinite <count>d|m|y]

System>vol options vipvol snaplock_minimum_period 1d

System> vol options vipvol snaplock_default_period min

Min is the retention period specified by the snaplock_minimum_period option, max is the retention period specified by the snaplock_maximum_period option, infinite specifies that files committed to WORM state are retained forever.

For storage system using dataOntap 7.1 and later , the maximum retention period can be extended up to 70 years.

You can also extend the retention date for a WORM file before or after the file retention date has expired by updating its last accessed timestamp. This will re_enable WORM protection on the file as if it were being committed for the first time. Once the retention date expires, you can re_enable the write permission on the file and then delete it. However the file contents still cannot be modified.

Note: the SnapLock volume maximum retention period restrictions are not applied when extending the retention date of a WORM file.

Committing Data to WORM

After placing a file on a snaplock volume, the file attributes must be explicitly changed to read-only before it becomes a WORM file. These operations can be done interactively or programmatically, the exact command or program required depends on the file access protocol (CIFS, NFS, and so on) and client operating system being used.

Below command could be used to commit the document .txt file to a WORM state, with a retention date of November 21,2020 ,using a unix shell.

touch –a –t 202011210600 document.txt

chmod –w document.txt

For a windows client, to make a file read-only, right-click the file, select properties, checks the read only checkbox and then clicks OK.

The last accessed timestamp of the file at the time it is committed to WORM state becomes its retention date unless it is limited by the minimum or maximum retention period of the SnapLock volume. If the date was never set, the default retention period of the SnapLock volume is used.

If a file is initially copied read-only, it is not automatically committed to WORM state. The only way to guarantee that a file is committed to WORM state is explicitly assign a read-only attribute and optionally modify the last access time to set a retention date while the file is in the SnapLock volume.

Auto-Committing Files.

A time-delay commit to WORM with an adjustable timer is available in Data Ontap 7.2 or later. If the file does not change during the delay period, the file is committed to WORM at the end of the delay period. Auto-commit does not take place instantly when the delay period ends. Auto-commit are performed using a scanner and can take some time. The retention date on the committed file will be determined by the volume’s default retention period.

To specify a time delay, set the global option snaplock.autocommit_period to a value consisting of an integer count followed by an indicator of the time period: “h” for hours, “d” for days. “m” for months, or “y” for years.

System> options snaplock.autocommit_period none | [count (h | d | y)]

The default value is none and the minimum delay that can be specified is two hours.

The following example sets the auto-commit period to 24 days.

System> options snaplock.autocommit_period 24d

Appending WORM File

A WORM file can be appended by simple creating an empty Snaplock file, appending data to it, and then locking the file in place again, data is committed to WORM state in 256 Kb chunks; the previous 256 kb segment automatically becomes immutable. This procedure can be done from ontap7.1 onward.

For ex:

1.       Inside a WORM volume create a zero-length file with the desired retention date.

touch  -a  -t  202012150600 file

 

2.       Update the file access time to indicate the file’s desired expiration.

 

3.       Make the file read-only.

chmod 444 file

 

4.       Make the file writable again.

chmod 755 file

 

5.       Start writing data to the file.

echo test data > file

 

at this stage you have a WORM appendable file. Data is committed to WORM in 256 kb chunks. Once data is written to byte n*256K+1 of the file, the previous 256Kb segment is in a WORM state and cannot be rewritten.

 

6.       Make the file read-only again. The entire file is now in the WORM state and cannot be overwritten or erased.

chmod 444 file

 

Deleting WORM volumes and Aggregates

 

You cannot delete SnapLock compliance volumes that contain unexpired WORM data. You can delete the volume only when all the WORM files have passed their retention dates.

 

Note: If a compliance volume cannot be destroyed, it remains offline. It should be immediately brought online so that retention period problem can be fixed and to keep the complianceclock from falling behind.

 

You can destroy the compliance aggregates if they don’t contain volumes. The volume contained by a snaplock compliance aggregate must be destroyed first.

 

You can destroy snaplock Enterprise volumes at any time, provided you have the appropriate authority.

 

………..if you like this blog please comment……..Thanks…….

 

 

 

 

 

 

 

Wednesday 2 January 2013

Quantum DXi6700 series vs. EMC DataDomain


Quantum DXi6700 series vs. EMC DataDomain
As we know that the quantum DXi series appliance is a disk based backup solution and their solution are quite similar to the data domain solutions, one thing they claim in market that the DD uses their patent variable size segment technology in their DD box. So the quantum is giving quite a tough fight to the EMC data domain, but quantum is new to the Indian market, it will take time for quantum to make their footprint in Indian market.

Quantum has launched their Dxi Accent software with their new quantum Dxi6700 series boxes which is Dxi6701 and Dxi6702

Dxi Accent software distributes the deduplication work between the backup software and the Dxi appliance. This software is similar to the EMC DDBoost software.

Quantum have one big plus point is that they have the direct tape out capability in their quantum Dxi storage box and they know very well that the tape is not going to get replaced because they are still the cheaper options for the long term retention of data. And quantum produces their own tape devices , so by this direct tape out capability the quantum Dxi boxes can get attached to the quantum tape libraries and send data directly to the tape, and in data domain there is no such type of features, which again creates a performance bottle neck when we need to send the data from the data domain boxes to the tape which will again use the separate or existing network, and as we know that EMC does not manufacture any tape device , so if some issue get occurred while sending the data from the datadomain to the tape via network then we can get stuck in getting support from two different vendors, one of EMC datadomain and one of the tape vendor, while in case of quantum if you are using the quantum Dxi appliance and their tape in your environment , if some issue happens then you can get the complete support for both the Dxi appliance and the tape device, which is again a plus point for quantum in support and services.

Quantum vmpro software is a specially designed software to take the vmware backup as, we know that in today’s world as fast the things are getting virtualized the more the data getting created and tougher the process is getting to take backup of these virtual machines, how to take a proper virtual machines backup, the quantum vmpro is the solution for taking the dedup backup of the virtual machine properly and speedily to the Dxi appliance.

Quantum Dxi 6700 series is compared by the EMC DD 670, and now I need to show you all, some of the performance details published by the Evaluator group regarding the quantum Dxi6702 performance in their report.

The Dxi6702 was the highest performing data deduplication system evaluated by evaluator group to date. Perhaps more important than the raw performance level, the price performance of the Dxi6700 systems outpace all currently competing system in their category.

Performance met or exceeded claimed values for the Dxi6700 with multiple protocols (NFS, VTL and OST)

1.   OST performance was 5.9TB/hr. for Dxi6702 (vs. 5.4TB/hr. for DataDomain DD670 with DD Boost per published specifications).

2.   VTL performance was 5.9 TB/hr. for Dxi6701 (vs. 3.6 TB/hr. for DataDomain DD670 per published specifications).

3.   NFS performance was 4.4 TB/hr. for Dxi6701 (vs. no specification published by the datdomain DD670).

As per the Evaluator group, Evaluation report.

1.   Quantum’s deduplication delivered the best result in the midrange category across multiple protocols, with a minimal disk configuration.

2.   Industry leading midrange performance for CIFS, NFS, VTL, OST with the distributed deduplication.

3.   Performance with or without distributed deduplication (Dxi Accent) met or exceeded that of midrange industry leaders.

4.   The throughput wasn’t limited with number of drives- maximum performance was available with the minimum configuration of 8TB in a raid 6 configuration.

Additionally the following results were observed.

1.   Hybrid backup modes using DXi Accent and traditional backup are supported simultaneously.

2.   DXi Accent is able to improve the backup performance when network bottleneck exist and significantly accelerate WAN based backup.

3.   The performance scaled rapidly, surpassing the stated performance levels with 35 streams.

4.   Unique path to tape feature can enhance the performance when moving a copy of data to tape, providing a practical method for creating data for long term retention.

As we know that the deduplication technology is so resource utilization technology, so proper care should be taken for indexing the data files and because the DXi with solid state disk not only increases the performance by storing the index files in the solid state but also tune the indexing operations.
As datadomin also do its indexing operation on the cache, but it does not store it in cache, so loading the index file from the sata to the cache again decrease some of the performance.

Some of the Plus point features of Quantum.

Virtual backup Enhancement: the new Quantum VmPRO virtual backup appliance combines a DXi system with software to enhance backing up VMware virtual machines. This software runs as a virtual appliance and discover all virtual machines in the network. Each VM’s data store is then mapped as individual file system, with complete access to internal file system. This enables the backup application to treat and protect VMware virtual machines as a collection of files.

As we know that DataDomain does not have some of specific software feature to take care of the virtual machines backup, so mostly we need to depend on the backup software for backing up of the virtual machine, which again can be bottle neck when we face issue on backing up of virtual machines.

DXi6700 Direct Tape Creation: I would like to say that this is one of the best features that will not only reduce the EMC footprints in backup world but also help many customers to send the data back to the tape for the long term retention by using the direct-tape-path features of quantum, as we know that keeping data for long term on disk can be very costly, so long term retention data has to be moved to the tape, so again sending data to the tape from the disk based backup solution becomes again a big issue , because of not having the direct tape out features, so we need to use either the production lan or create a spate network to send the data from the disk based backup device to the tape device.

Quantum has integrated the use of D2D appliance with tape by including direct tape creation in their Dxi6700 D2D and VTL products. Quantum path to tape support application directed movement of data to tape with either VTL or OST protocols. This capability offloads production SAN and LAN networks and backup server when making the secondary copies to the tape.

With direct connectivity between the Dxi system and tape via dedicated Fiber Channel ports, data may be sent directly to tape drives, offloading backup servers, primary storage system and application servers. Data is sent  under the direction of data protection software, allowing  backup application to maintain awareness of where all copies of data resides, and data copied to tape is expanded back to its native format and then compressed using industry-standards algorithms. By storing data in non-deduplication native format on tape, the long-term retention of data is enhanced while minimizing requirements for data restoration from tape. This provides the LAN and SAN free method of moving data to tape for off-site storage or long-term retention.
Well EMC data domain does not have direct –tape –path features, so sending the data from the datadomain device to tape can again be a headache for customer or backup administrator.

Ease of Management

Well at the end the management  tools comes into picture after buying the compliance if it’s GUI is difficult to understand or not much user friendly then the survival change of that particular product is very less in market.
So the Management of that particular device from its GUI should be very user-friendly and should also provide the useful monitoring data, to keep records for future reference, proper graph like charts and maximum details should be gathered or viewed by that tool.

I have not used any management tool of quantum device, so I cannot explain my experience of that tool, but below given is the data which Evaluation group has provided of Quantum Dxi GUI features and its ease of usability.

Quantum DXi GUI:

Evaluator Group found Quantum’s DXi GUI and other management tools score well in comparison to other deduplication system evaluated by EGI for ease of use. This is based on the following factors:

1.   Bundled pricing, without capacity-based licensing- results in lower and predictable cost for users(significant benefit vs many competitors)

2.   Both GUI and CLI can be used to perform the entire task.

3.   The GUI was intuitive and as easy to use as any competing system.

4.   The embedded GUI is accessible via web browser, requiring no external software installation.

5.   The additional management tools were well integrated and provided significant value.

6.   The DXi  Advanced reporting tool provided  significant  performance analysis benefits such as

The ability to view six year of historical data for logged parameters exceeds the leading competitor

DXi advance reporting can be run simultaneously for multiple appliances through quantum vision

Enables administrator to find and resolve bottlenecks and generate historical trend report.

7.   Quantum vision now provides tighter integration and ability to monitor, troubleshoot and analyze performance in enterprise setting.

8.   New ability for remote access to the vision server via an iPhone, iPad or iPod Touch, even over a VPN.

9.   Integrated path–to-tape provides capability to move data to tape without impact of the production storage or servers, thus offloading data copies to tape for archive or compliance.

Quantum with good management tool and good throughput and direct-tape-out and with other features is really going to rock in Indian marker by giving tough fight to the EMC datadomain.