SnapProtect Guide on NetApp Storage System

SnapProtect Guide Objective and Overview

The SnapProtect technology of the Calypso software is a modernized approach for data protection, allowing hardware storage systems snapshots to merge directly into the backup process. By automatically integrating application intelligence with array snapshot abilities, Calypso is able to reach through the application and file systems into the storage array to accomplish the following:

• discover volume/disk configurations for the snapshot operations
• coordinate snapshot operations with proper application quiesce
• minimize administrative configuration and eliminating scripting requirements

When a scheduled backup job for a defined application runs, the source system quiesces the selected applications and automatically creates a set of persistent snapshots within the production storage system. With a confirmation of the successful snapshot creation by the host, the workload in the protection job shifts to a secondary proxy server to offload backup operations. This shift releases the production host, which returns to full production side operations. This allows the creation of a consistent data image in minutes with Recovery Point Objectives (RPOs) aligning with the frequency of schedule.

The Proxy completes the second half of the protection job by reaching into the SAN, mounting the snapshot and automatically indexing and cataloging the file-level contents of the snapshot(s). Unlike the hardware-based snapshot approach, Calypso blends the speed and efficiency of array snapshots directly into the backup and restore process, offering full system recovery or single file restore. Once the content-aware indexing completes, the snapshot is retained in the array as a persistent recovery copy to provide a quick recovery option to revert or restore the data volumes.

The Proxy serves a secondary role after the snapshot executes. The same snapshots will mount and copy the relevant file contents from the snapshot and apply deduplication to the data during transport to the backup copy destination. The backup copy employs a separate retention than the snapshot allowing aggressive snapshot retention to preserve tier1 space to meet the RTO/RPO needs. As data moves into the backup copy, the original indices are preserved and stored along with the data to ensure access from any location. Data encryption is also another critical feature to apply to data to keep it highly secured from unwanted eyes.

SnapProtect supports the leading SAN/NAS storage solutions from Dell, EMC, NetApp, LSI, HP, SUN, IBM and HDS.

In addition, SnapProtect can use SnapVault and SnapMirror to perform disk to disk backups between NetApp arrays by moving only the changed blocks from one NetApp Array to another, based on the relationships that are defined on the storage. When Snapshots replicate from one NetApp Array to another, they maintain all capabilities for off-host proxy backups and index all snapshot copies, etc. Therefore, you can manage large data sets with one data management platform to do all of the following:

• Create, mount, dismount, retain, recover, and delete snapshots
• Create thin provision replication policies
• Provision storage on the secondary NetApp Array

NetApp Storage System Overview

NetApp provides customers with a storage platform that meets all storage needs, from NAS-based storage to block-based storage. NetApp File-Attached Storage (FAS) devices provide a unified storage platform for mixed data types and access protocols via the ONTAP operating system. NetApp’s Data ONTAP implements snapshots by tracking changes to disk-blocks on the WAFL file system.

While very similar in function, NetApp's SnapVault and SnapMirror are two different technologies. SnapMirror relationships exist between a source volume on a primary array and a destination volume where the copy resides on the secondary array. SnapVault relationships exist between a source volume/qtree on a primary array and a destination volume/qtree where the copy baseline and snapshot-vaulted versions reside on the secondary array.

In SnapProtect all of these operations are orchestrated by a set of APIs built into DFM (Data Fabric Manager). DFM registers each of the arrays, and creates policies and relationships that define the replication topology, schedule, and retention. Required DFM components are:

• Operations Manager
• Protection Manager
• Provisioning Manager

This guide is a brief overview of NetApp FAS arrays and the necessary steps to configure Calypso with NetApp FAS systems.

For further NetApp FAS support, documentation, and training material, visit now.netapp.com.

Integration Requirements for SnapProtect

Array and Solutions Enabler Software Licenses

The Calypso software requires enabling certain array components and licenses on the NetApp storage system to achieve a successful integration with the software. Refer to the SnapProtect Backup - NetApp and SnapProtect Backup - NetApp SnapVault/SnapMirror documentation for more information.

Supported Applications and Operating Environments

Refer to the SnapProtect Backup - Support document for a complete list of supported applications and platforms.

Calypso Software

SnapProtect solutions require the appropriate agents as defined by the customer configuration. See the following terminology for reference in this document:

A CommServe, a necessary storage capacity, and MediaAgents must exist to enable a completely functional solution. See the SnapProtect Getting Started Guide for step-by-step instructions on:

• CommServe, MediaAgent and File System installation
• Storage Device and Storage Policy Configuration

On top of this basic infrastructure, you can configure the environments described below.

Basic File System Environments

The SnapProtect base configuration requires the following agents on the Production Host:

• Windows/Unix File System iDataAgent
• MediaAgent
• VSS Software Provider (Windows Only)

For a configuration where snapshots mount off host to a Proxy server, implement the following agents on the Proxy server:

• File System iDataAgent (Must be similar to Production Host Operating System)
• MediaAgent

Refer to Getting Started - Setup Clients to select Windows/Unix iDataAgent and perform the required deployment and configuration steps.

Application Environments

Protecting application databases and log volumes through an Array snapshot provide fast access for recovery and many flexible options for backups. SnapProtect integrates application awareness together with the Array and Calypso to deliver all of the benefits of traditional streaming backups with all of the performance and proxy capabilities of a snapshot. This application awareness allows backups with appropriate log management operations based on the snapshot data. When implementing for Application Environments, add the appropriate Application iDataAgent to the SnapProtect base configuration, as follows, on the Production Host: Windows/Unix File System iDataAgent MediaAgent VSS Software Provider (Windows Only) Application iDataAgent for selected Application (e.g., Exchange, SQL, Oracle) For a configuration where snapshots mount off host to a Proxy server, implement the following agents on the Proxy server.   With SnapVault and SnapMirror capabilities, you can also choose a Proxy Server from the Secondary Array. To do so, the same software components must be available in the Proxy to Tape Server. File System iDataAgent (Must be similar to Production Host Operating System) MediaAgent Application specific iDataAgent (Must be the same to Production Host) to enable Proxy. This is required for RMAN integrated backups. Application API (e.g., Exchange Management Pack, Oracle for RMAN integration) Refer to Getting Started - Setup Clients to select your application iDataAgent and perform the required deployment and configuration steps.

NAS Environments

For Network Attached Storage environments (NetApp Only) the configuration is slightly different. Since a NAS iDA is a logical agent pointing to an assigned IP address to the NetApp FAS, nothing must be installed for the Production host. Simply configure the NAS iDA per Books Online and check the “enable SnapProtect” box under the client advanced properties to enable local ONTAP snaps to drive the protection operation. There is no notion of a Proxy in this configuration due to the local nature of the ONTAP snapshot. NDMP operations will simply function from the primary FAS. In the scenario where the customer wants to execute “crash-consistent” snapshots of VM, database, or application environments, the best way to achieve this is through the NAS iDA. It allows the customer to define the Volume, qtree, or LUN as the contents of a NAS iDA subclient and execute a crash consistent snap at the schedule defined on the subclient. Refer to Getting Started - NAS for configuration steps.

VMWare Environments

SnapProtect enables fast protection of large or volatile VMware environments without placing load on the production Virtualization Farm. SnapProtect technology integration with the Virtual Server Agent enables the Array to perform backups in minutes even with large numbers of virtual machines and sizable data stores. A dedicated ESX server for proxy data movement removes any utilization on the ESX farm providing file and folder recovery from the secondary tier of storage. To enable SnapProtect for the VMWare install the following on the physical server(s) or virtual hot-add guest(s): Windows/Unix File System iDataAgent MediaAgent Virtual Server Agent (VSA) iDataAgent Refer to Getting Started - VMware for deployment and configuration steps.

Microsoft Hyper-V Environments

SnapProtect enables fast protection of large or volatile Hyper-V environments placing minimal load on the production Virtualization Farm. SnapProtect technology integration with the Virtual Server Agent (VSA) enables the Array to perform backups in minutes even with large numbers of virtual machines and sizable data stores. Granular access provides individual file and folder recovery from the secondary tier of storage along with the full guest .vhd files. Prior to configuring the virtualization environment, deploy the proper agents requiring snapshot integration with the Array. Microsoft Hyper-V is very similar to an application environment for the components necessary to execute SnapProtect operations. Microsoft Hyper-V virtualization environments require the following agents: To enable SnapProtect for the Virtual Environment install the following on the physical server(s) or virtual hot-add guest(s): Windows/Unix File System iDataAgent MediaAgent Virtual Server Agent (VSA) iDataAgent VSS Software Provider Refer to Getting Started - Microsoft Hyper-V for deployment and configuration steps.

SnapVault and SnapMirror Environments

SnapProtect now extends enterprise management for backup and recovery in the data center for local Snapshot copies on NetApp primary storage and replication to secondary and tertiary storage, as well as tape creation. Any of the above environments discussed (application data, file data for NAS, file data in LUNs, or data in virtualized environments) with SnapVault and SnapMirror enables management, storage provisioning, cataloging, and the granular recoverability for seamless operation off of any copy. Refer to SnapProtect Backup - NetApp SnapVault/SnapMirror for configuration steps.

SnapProtect Array Configuration Details with NetApp Fabric-Attached Storage (FAS)

Provisioning storage as a NFS/CIFS target or a LUN requires physical disks assignment in the Array to an Aggregate for storage usage. Volumes creation occurs within the Aggregate grouping of disks internal to the Array. Generation of Shares and Qtrees for NFS/CIFS access occurs within these volumes. Additionally LUNS live internal to the volumes for block-based access to hosts. NetApp’s snapshot mechanism is a pointer-based approach so blocks of the active file system blocks co-mingle with snapshot blocks internal to the LUN. Ensure sufficient space is available for snapshot generation and retention of the point in time copies. The required space directly correlates to the change rate of the data stored on the LUN and the number of snapshots retained. NetApp’s default setting is 20% for snapshot capacity. In the graphic below, note that the total volume size is greater than the total capacity. The extra space designates the reserved area for snapshots.

Refer to the SnapProtect Backup - NetApp documentation for array configuration steps.

SnapVault and SnapMirror Configuration - DataFabric Manager (DFM)

To Integrate SnapVault and SnapMirror it required to install Data Fabric Manager 4.0.2 and register all pertaining NetApp arrays with the Management Console. DFM may be installed on any Windows server in the environment and will require licensing keys for Operations Manager, Provisioning Manager and Protection Manager. Multiple DFM instances can be leveraged for distributed environments and redundancy control.

Once installed it is required to define “Resource Pools” for the SnapVault secondary and SnapMirror secondary targets. SnapProtect will automatically provision the proper Qtree, volume, and/or LUN configuration for the desired source contents of the defined resource pools. Resource Pools are allocated by adding defined aggregates in the NetApp FAS. These can be created from within the NetApp Management Console.

Refer to the SnapProtect Backup - NetApp SnapVault/SnapMirror documentation for array configuration steps.

The SnapProtect Process

The following sections describe the different SnapProtect operations.

SnapProtect Backup Operation

SnapProtect backups consist of the following operations:

1. The backup job (on-demand or scheduled) starts from the CommCell Console.
2. The file system, associated applications, or Virtual Machines properly acquiesce (VSS calls Windows or through application interfaces such as RMAN the database goes in a Hot Backup mode). In VMware configurations, vStorage APIs are called to create software snapshots and enable delta file creation for each of the guests targeted as contents of the snapshot.
3. Array API is called to:
   • Verify the backup job contents (validating the underlying disk structure for file systems, databases, etc).
   • Create a snapshot or clone.
   • Mount the snapshot on the source or the selected proxy computer for post-snapshot operations (e.g., Scan and Catalog for File System). For VMware and RMAN proxy configurations, the Virtual Machines and database files are registered by the proxy application software.
4. Unmount the snapshot and retain based on retention rules.

This snapshot now provides availability for backup copy operations and high speed restore, mount, and revert operations.

Backup Copy Operation

During the Backup Copy operation:

• The snapshots are copied to media in a sequential order.
• The snapshots are mounted to the source or proxy computer. The mounted snapshot receives commands to scan, and backups like a normal file system reading the required contents.
• The file system backup is performed in the Primary Copy of the storage policy of all defined files. The data is indexed and linked back to the original production host.
• When the backup copy job is finished, the snapshot is unmounted and retained based upon Snapshot Copy retention settings.

With SnapVault and SnapMirror operations, you can perform backup operations with the SnapVault or SnapMirror copy. To do so, you must define the SnapCopy for Backup Copy operation in the Storage Policy properties. You can set the Subclient Option, Separate Proxy for Snap to Tape to activate servers that are local to the secondary and tertiary copies to mount snapshots and drive the data protection operations to disk or tape.

Proxy Configuration

For a configuration where snapshots mount off the host to a proxy server, deploy the following agents on the proxy server:

• File System iDataAgent (Must be similar to Production Host Operating System)
• MediaAgent
• Application specific binaries for proxy (e.g., Exchange Management Pack, Oracle for RMAN, etc)

Verify Configuration using SnapTest Tool

Security and Storage Policy Best Practices

Security Roles

Rather than risking the business with scripts or standalone tool sets, the embedded role based security system that Calypso provides allows you to entrust critical actions to the right users at the right time. In most medium to large environments, application, backup, and audit responsibilities may be distributed functions that need to be coordinated into a single policy.

For Example, a customer may have three specific roles within an operations environment:

• Backup Administrators. They are the day to day operators with access to perform standard backup and recoveries, manage media, issue reports, etc.
• Application owners (or Database Administrators). They manage the applications running the business such as application level recoveries and array-based operations.
• Audit & Business Compliance leads. They provide security and process proof of who can perform what and how.

Specific roles should be defined for the SnapProtect and Application iDataAgents within the CommCell. The following table is an example of the Calypso Security Roles basic structure:

Security Roles (for application Clients or Groups)	Backup	Application	Audit Team
Administrative Management
Agent Management
Agent Scheduling
Alert Management
Browse
Browse and In-Place Recover
Browse and Out of Place Recover
Compliance Search
Data Protection
Data Protection Management
End User Search
Job Management
Library Management
Library Administration
License Management
MediaAgent Management
Report Management
Storage Policy Management
User Management
VaultTracker Operations

Storage Policies

• increase the amount of tier 1 storage that is holding recovery points.
• cause the snapshots to fall short of fully meeting SLA requirements for the business.

Calypso storage policies are broken down into copies for managing retention on the proper tier of storage. In a typical storage policy for SnapProtect, three copies will be available:

• Primary snapshot copy
• Primary backup copy (movement to media)
• Offsite disk/tape copy

Storage policy configuration varies from environment to environment. For example, SLAs for Sub 24 hours RPO/RTO drastically lower the returns on leveraging snapshot technology on copies beyond 48 hours. Based on this example, you may set the retention in the following way:

• Primary snapshot copy – 2 days & 0 Cycles
• Primary backup copy – 28 days & 1 Cycle
• Offsite Disk/Tape copy – 60 days & 1 Cycle

This configuration allows snapshot retention on a 48 hour rotation providing multiple high-speed recovery points available on the array to meet the SLA requirement. It also requires storage space allocation to maintain two persistent days of change for the associated clients. By setting “cycles” to 0, the removal of old snapshots occurs regardless of success, so proper alerting and monitoring is required. Improperly setting retention and effects of days and cycles can adversely affect the available recovery scenarios for the business applications.

Manipulating Snapshots

Appendix

VADP-SPE User Permissions

A dedicated account can be used for performing VSA with SnapProtect. This account requires additional permissions for the VADP User role. This account is only required in the event that you want to create a restricted account.

Back to Top