Disk Libraries

Topics | Configure | How To | Troubleshoot | Related Topics


Disk Libraries

Overview of Disk Libraries

Best Practices for Disk Libraries

Shared Disk Libraries

Overview of Shared Disk Libraries

Disk Libraries on Replicated Disks

Overview of Disk Libraries on Replicated Disks

Common

Administering Disk Libraries

Administering Mount Paths

Audit Trail

License Requirements

Other Considerations

Related Alerts


Overview of Disk Libraries

A disk library is a virtual library associated with one or more mount paths. The disk library does not represent a specific hardware entity; it is a software entity that contains a list of mount paths through which data can be sent to a disk media.

Disk media, whether individual hard disks or RAID arrays, is logically divided into partitions. A partition can include some or all of the total disk storage space available. Each partition is associated with a file system path called a mount path, through which data is written to and read from.

A disk library can be configured by assigning one or more mount paths to it. The storage capacity of a disk library is determined by the total storage space in its mount paths.

Note that the maximum size of a volume that can be used as a mount path can be a maximum of 4,096 Terabytes.

Clustered File System Storage

Disk libraries are supported on clustered file systems such as Global File Systems (GFS), Cluster File Systems (CFS), IBM General Parallel File System (GPFS), and Polyserve File System.

Automatically Create Mount Path when adding new Disk Storage

Automated disk library is a process to automatically add new mount paths when new LUN is mounted under given folder as Mount point. It is supported for all Windows MediaAgents.

This is useful when:

See Configuring Automatic Mount Paths for New Disk Storage for step-by-step instructions.


Best Practices for Disk Libraries

The following list provides a set of guidelines to help you ensure that the MediaAgent always accesses the mount paths allocated to disk libraries:

If you have defined multiple mount paths for a disk library, the system determines the specific path to which data secured by a data protection operation is written.

Writers and Storage Policy Data Streams

The maximum number of writers for a disk library is used to determine the maximum number of simultaneous data protection operations. Hence this value is linked to the maximum number of data streams that can be established for a storage policy accessing the disk library.

Storage Policy data streams are logical channels that connect client data to the media in which data secured by data protection operations are stored. Multiple data streams can be used to parallelize an operation, in order to improve the rate at which data can be written to the media.

The practical limits for establishing data streams in a storage policy accessing a disk library is determined by the minimum value established in either the disk library or the sum of mount paths in the disk library.

The values for writers can be modified from the Disk Library Properties and Mount Path Properties dialog box available in the CommCell Console. The values for the maximum number of data streams can be modified from the Storage Policy Properties dialog box also available in the CommCell Console.

Managed Disk Space

For new deduplicated storage policy copies, Enable Managed Disk Space for disk data option on Copy Properties dialog box (Retention tab) is disabled by default. If an existing deduplication copy (for CommServe with Service Pack 8B) has this option enabled, then while updating and saving the deduplication copy, you will be asked to manually disable this option. This option will not be changed automatically for any existing deduplication copy. Disabling this option helps in faster pruning of the aged data in order to reclaim space on the disk.

For step-by-step instructions to disable Enable Managed Disk Space for Disk Library check box, see Enable Managed Disk Space for Disk Data.


Overview of Shared Disk Libraries

Shared disk libraries provide the following benefits:

A shared disk library supports all the operations supported (except disk migration) by a (non-shared) disk library.

Shared Disk libraries with Static Mount Paths

The concept of shared disk libraries is expanded further to include volumes that are statically mounted or accessible in a network, such as PolyServe File System, Sistina GFS (Global File systems), NFS mounted volumes or network shares. Such volumes need not be explicitly mounted and can be accessed by multiple MediaAgents at the same time.

See Configure Shared Disk Libraries With Static Mount Paths for step-by-step instructions on configuring disk libraries with static mount paths.

See Configure Multiple MediaAgents for a Static Shared Disk Device for step-by-step instructions on configuring multiple MediaAgents (of same operating system type) with static shared disk devices.

Configuring PolyServe Disks as Static Mount Paths

Disks with PolyServe File System can be configured in a Storage Area Network (SAN) as a shared disk library with static mount paths, as illustrated in the diagram.

Note that when configuring such disks, use the local PolyServe exposed volume instead of using UNC paths.

See Also: MediaAgents - Supported Features, Agents and Devices for information on MediaAgents that support shared disk libraries with static mount paths


Best Practices for Shared Disk Libraries

The following sections provide a set of guidelines for the successful implementation of shared disk libraries.

Sharing the Same Disk Storage across MediaAgents

LAN Based Backup

LAN Free Backup

Modifying the Access Information for a Shared Disk

It may be necessary to change the access information, such as access credentials and/or path on a shared disk. For example if the access credentials is changed in the operating system you may want to set the new credentials from the CommCell Console for the disk library.

For Windows devices, you can modify the access information from the device level, which will change access information for all the MediaAgents sharing the device.

For Unix MediaAgents this must be performed from each folder level which will change the access credentials for each of the specific MediaAgents accessing the device.

See Modify the Access Information for a Shared Disk for step-by-step instructions.


Overview of Disk Libraries on Replicated Disks

For disks that are replicated to another disk in real-time using software/hardware replication tools, the mount paths from both the primary and replicated disks can be configured as a disk library. In such a configuration, the data written by the primary MediaAgent will have read-write access while the MediaAgent on the replica will have read-only access. Note that the primary and replica mount paths can be configured on any MediaAgent type (Windows, Solaris, Linux, etc) or with any network or cluster file systems. (CIFS,  NTFS, etc.)

The following example illustrates a disk library configuration on a replicated disk:

In this example MediaAgent 1 is configured to access a disk library on the primary computer, to backup a mission-critical production server, such as a server with a large database. (This MediaAgent will have read-write access to the disk library on the primary.)

The data is replicated to a replica that contains MediaAgent 2. (This MediaAgent will have read access to the disk library on the replica.)

Note that when the system performs a data protection operation using the MediaAgent on the primary disk, two copies of the data are available as soon as the disk is replicated. Thus MediaAgent 2 can be used to perform auxiliary copy operations to a tape library, thus limiting the load on the processing capabilities in the production server. Also note that all data protection operations (backup and auxiliary copy) are performed LAN free.

Data can be restored from either the primary or replicated disk or from a copy using the tape library.

See Configure Disk Libraries on Replicated Disks for step-by-step- instructions on configuring such libraries.

See Disk Library Replication solution to seamlessly replicate data from remote offices to centralized data centers using disk libraries on replicated disks.


Administering Disk Libraries

There are several configurable parameters available for Disk libraries. The following sections describe each of these parameters.

Modify the Library Name and Description

The system initially assigns a default name for the disk library. If necessary, you can modify the name. You can also add relevant information about the library as a description.

This operation is supported by all disk libraries.

View the MediaAgent and the Number of Mount Paths

You can view the MediaAgent name and the number of mount paths associated with each Disk library.

This operation is supported by all disk libraries.

View the Status

The status of the disk library indicates whether the library is online or offline, and if offline, the reason for the offline status.

You can use the enable/disable indicator to logically enable or disable the disk library.

This operation is supported by all disk libraries.

See Enable or Disable the Disk Library and View the Disk Library Offline Reason for step-by-step instructions.

Modify the Low Watermark

You can establish a low watermark for each disk library. The low watermark is the minimum amount of free space at which the low watermark warning should be generated. If the amount of free space, for all the combined mount paths, reaches or falls below the low watermark, the system logs a message in the Event Viewer and generates the Insufficient Storage Alert, if configured. See Establish the Low Watermark for a Disk Library for step-by-step instructions.

This operation is supported by all disk libraries.

Configure a Low Disk Space Alert

By default, low disk space alert is based on the low watermark level. However, if you set the Number of days in advance to trigger alert for low disk space for disk library option to a value greater than 0, you receive a low disk space alert when the disk library is about to run out of space (measured in days). The warning message that you receive includes a failure reason. For example:

Failure Reason: At the current usage rate of 0.35GB per day, library <library name> with freespace 0.36GB will run out of space in 2 days.

To calculate the amount of space that remains, the system averages the amount of data that you backed up to the library over the last 30 days. In addition, the system subtracts the reserve space from the library’s free space so that the prediction is based only on the disk space that is available for jobs. Therefore, the calculation is: (disk space remaining – reserve space)/(average amount of data backed up over last 30 days) = # days when the library will be full.

For instructions, see Configuring a Low Disk Space Alert.

For more information about alerts, see Alerts and Monitoring.

Configure Disk Volume Size

Disk volumes are created based on the volume size. When the size of the volume reaches the maximum size, then a new volume is created. The maximum size of a disk volume is set to 100 GB by default, and this value can be modified. See Configure the Size of Disk Volumes for step-by-step instructions.

This operation is supported by all disk libraries.

Configure Archive Files as Read-Only

You can configure the library to create archive files as read-only files on filers. This feature can be enabled using the Mark Archive Files as Read-Only option in the Library Properties dialog box.

When this option is enabled, it engages the corresponding read-only lock mechanism on the destination filers and saves the archive/backup files as read-only files. The expiration date for the read-only lock is set to match the data retention time established in the storage policy copies. These archive files cannot be modified or deleted by any user or application until the specified retention date. Once the retention expires, the system deletes the archives as a part of Data Aging.

This feature is supported on the following filers:

This option will only affect files that are subsequently created by data protection and auxiliary copy operations to this library. Permission to files that are already available in this library will not be affected.

See Create Read-Only Archive Files on filers for step-by-step instructions.

Configure Timeouts

The timeout parameter determines how long a job waits for a status response after an operation is requested. If the job does not receive a success or failure status within the timeout period, the job is terminated and a failure message is displayed. You can set the following timeout periods:

See Modify the Mount and Unmount Timeouts for Disk Libraries for step-by-step instructions.

This operation is supported by all disk libraries.

Enable Support for Single Instancing of Data (Content Address Storage)

For appliances that support hardware single instancing of data (e.g., Centera) you can enable the option to write the data in a single instancing format.

See Enable Single Instancing of Data on a Disk Library for step-by-step instructions.

This feature requires a Feature License to be available in the CommServe® Server.

Review general license requirements included in License Administration. Also, View All Licenses provides step-by-step instructions on how to view the license information.

Support Information

The following Agents support this feature:

Agent

What is Deduplicated

Windows File System iDataAgent Files (Meta-data associated with files are not deduplicated.)*
Unix File System iDataAgents Files
Macintosh File System iDataAgent Files
NetWare File System iDataAgent Files
Exchange Mailbox iDataAgent Attachments
Exchange Mailbox Archiver Attachments
Exchange Compliance Archiver Attachments
SharePoint Document iDataAgent Documents (Meta-data associated with documents are not deduplicated.)*
File Archiver for Windows Files (Meta-data associated with files are not deduplicated.)*
File Archiver for Unix Files

*Metadata includes Access Control Lists (ACLs) and any additional streams associated with the files.

The following operations are not supported:

See Enable Single Instancing of Data on a Disk Library for step-by-step instructions.

 

Establish the Parameters for Mount Path Usage

These parameters may be used to specify how the system must perform write operations on disk libraries with multiple mount paths. Two parameters for mount path usage can be configured. They are:

To illustrate the usage of the above mentioned options, consider the following example:

Assume that a disk library that allows multiple writes is configured with two mount paths. Start two jobs - the jobs may either be run concurrently, or individually, one after another. If the Fill and Spill mount paths option is selected, both jobs will use the first mount path. If the Spill and Fill mount paths option is selected, the first job will use the first mount path and the second will use the second mount path.

Note that data protection operations will continue with an available mount path if a required mount path is not available, thereby taking precedence over the option specified for mount path usage. For example, assume that that there are two mount paths, with the Spill and Fill mount path option selected. If one of the mount paths is not available, all data protection operations will use the available mount path.

See Establish the Mount Path Usage for a Disk Library for step-by-step instructions.

This operation is supported by all disk libraries.

Thresholds for Managed Disk Space

Disks can be used to spool data prior to being permanently stored on tape for long term/offsite storage. Defining managed disk space thresholds allows you to retain your disk data longer by pruning data according to disk capacity and existing retention criteria.

You can define two-disk capacitcy thresholds for managed disk space:

When disk capacity reaches a high threshold (e.g., 85%), any data exceeding its retention criteria is qualified for aging. When disk capacity reaches the low threshold (e.g., 70%),the aging process automatically stops.

Once the  managed disk is set up the process runs automatically without user intervention to manage disk capacity. Data protection operations are retained on disk longer than usual providing the benefits of disk storage without having to spend manual efforts to manage the disk capacity.

The Enable Managed Disk Space for disk data option is available in the Retention tab of the Copy Properties dialog box. If a storage policy is created with a valid retention criteria other than infinite retention, then this option is automatically enabled in the copies.

The pre-defined thresholds for disk capacity for a disk library can be defined in the Mount Paths tab of the Library Properties (associated with a disk library) dialog box.

Data Aging determines the pruning based on jobs -  jobs with older data (based on the creation time of its first archive file) is pruned first. Once the Data Aging operation determines the jobs to be pruned, data will be deleted based on the established threshold. The frequency for checking the disk space and deleting data is determined by the frequency established in the Interval (Minutes) between disk space updates option established in the Service Configuration tab of the Media Management Configuration dialog box in the Control Panel.

See Enable Managed Disk Space for Disk Data for step-by-step instructions.

See Data Aging for comprehensive information on data aging.

This operation is supported by all disk libraries.

Establish the Mount Path Allocation Policy for the Library

The maximum number of writers for a disk library is used to determine the maximum number of simultaneous data protection operations. Hence this value is linked to the maximum number of data streams that can be established for a storage policy accessing the disk library.

Storage Policy data streams are logical channels that connect client data to the media in which data secured by data protection operations are stored. Multiple data streams can be used to parallelize an operation, in order to improve the rate at which data can be written to the media.

The practical limits for establishing data streams in a storage policy accessing a disk library is determined by the minimum value established in either the disk library or the sum of mount paths in the disk library.

The values for writers can be modified from the Disk Library Properties and Mount Path Properties dialog box available in the CommCell Console. The values for the maximum number of data streams can be modified from the Storage Policy Properties dialog box also available in the CommCell Console. See Establish the Mount Path Allocation Policy for a Disk Library for step-by-step instructions.

See Also:

See Establish the Mount Path Allocation Policy for a Disk Library for step-by-step instructions.

This operation is supported by all disk libraries.

View the storage policy and storage policy copies accessing the disk library

You can view the list of Storage Policies and the Storage Policy Copies accessing the disk library. See View a List of Storage Policies Accessing the Disk Library for step-step instructions.

This operation is supported by all disk libraries.

Configure Security

Security allows you to associate the Disk library with one or more CommCell user groups. See User Administration and Security for a detailed explanation on user security.

This operation is supported by all disk libraries.

Migrate Disk Libraries

You can migrate a disk library to another MediaAgent within the CommCell. Consider the following while performing this operation:

See Migrate a Disk Library for step-by-step instructions.

Fragmentation Analysis

Fragmentation levels in the disk mount paths can be analyzed using the Fragmentation Analysis administrative job. When executed, the files in the selected mount paths are analyzed and the fragmentation statistics are reported in the Library and Drive Report. Based on the file fragmentation statistics reported, appropriate tools can be used to defragment the disk library.

  Bull Calypso recommends defragmenting disk library volumes when needed to ensure performance. Bull Calypso does not support calls on any Third-party defragmentation tools. However, Diskeeper has certified its 2011 (and higher) release version for online volume defragmentation.

Before using this feature, set the desired fragmentation margin using the following two parameters in the Media Management Configuration (Service Configuration) tab:

See Perform Fragmentation Analysis for step-by-step instructions. The results of the job can be viewed from the Disk library Maintenance Job Summary Report.

This feature is supported on Windows MediaAgents. Note that this feature is not supported on Centera Clusters.

Fragmentation levels in the mount path can be reduced by using configuration options. See Reduce Fragmentation of Data on a Mount Path for more information.

How is disk fragmentation calculated for disk libraries?

For all files in a mount path, Bull Calypso utilizes the following formula to determine the fragmentation percentage:

(Total number of excess fragmentation/Total number of allowed fragmentation) * 100

The Per File Fragmentation Threshold configuration parameter available in Media Management Configuration (Service Configuration) tab sets the number of allowed fragments per file. The default value is 10. This means within the 2GB chunks of a mount path, if the number of fragments is above 10 (suppose n, where n>10), then the file is categorized as a fragment and the excess fragment (n-10) is counted.

The following is an example of a mount path with five chunks (assuming a block size of 4096K as block content, below a value of 4096 will have no fragments).

Per-file fragmentation threshold = 10

Name (files) Size Number of Fragments Number of Excess Fragments
MEDIA_LABEL 64K 0 0
CHUNK_1 2GB 53 43
CHUNK_2 2GB 9 0
CHUNK_3 2GB 14 4
CHUNK_4 2GB 26 16
CHUNK_5 2GB 5 0
CMT_1 64K 0 0
CMT_2 64K 0 0
CMT_3 64K 0 0
CMT_4 64K 0 0
CMT_5 64K 0 0

The mount path in the above table shows excess fragmentation = 63 (43 + 4 + 16)

Total number of files = 11

Allowed fragmentation = (total number of files * per-file fragmentation threshold) = 11 * 10 = 110

Total fragmentation percentage = (Total excess fragmentation / Total allowed fragmentation) * 100

                                              = (63 * 110) * 100 = 63%

  You can increase/decrease per-file fragmentation threshold to control the fragmentation percentage watermark.

Disk Volume Reconciliation

Disk volume reconciliation allows you to reconcile the discrepancies in the disk volume associated with a storage policy copy data. This feature compares the physical data located in the disk mount path with the metadata records in the CommServe database and provides the list of orphaned jobs and orphaned media space. Orphaned jobs are jobs without corresponding data on the media, and orphaned media is the list of volumes without corresponding jobs in the CommServe. This feature is applicable for disk storage media only.

The orphaned media is displayed with associated details like the Storage Policy, Storage Policy Copy, Library, volume path, No. of Chunks, Size of the data etc., and the orphaned jobs are displayed with details such as Job ID, Client Name, Application Type, Library, Storage Policy, Storage Policy Copy, etc. You can review the information and delete the orphaned jobs and/or orphaned media. When an orphaned media space is deleted, the space is marked for deletion and pruned based on the retention rules set for the copy. When an orphaned job is deleted, the job is removed from the CommServe database. Jobs deleted using this feature are recorded in the Audit Trial. See Perform a Disk Volume Reconciliation Job and Reconcile the Differences for step-by-step instructions.

Once a disk volume reconciliation job is executed, you can use the results of the analysis to reconcile the differences at a later time. This allows you to reconcile the differences without executing the job again. See View the Results of a Disk Volume Reconciliation Job and Reconcile the Differences for step-by-step instructions.

  This feature is supported only for Non-Deduplicated Storage Policy copy.

You can use the Media Information Report to review the results of  this job. The details of the orphaned jobs, orphaned media, and the volumes deleted using the job are available in the report.


Administering Mount Paths

From the CommCell Browser, you can perform the following operations on a mount path:

View or modify the properties of the mount path

You can view the mount path location and information on the free space available on the mount path and the total amount of valid data. If necessary you can record pertinent information about the mount path as a description in the mount path properties.

This operation is supported by all disk libraries.

Use Unbuffered I/O

To increase the speed of operations accessing the mount path, you can enable the MediaAgent to bypass the Microsoft Windows file system buffering. Note that this option is only applicable for Windows MediaAgents and disks that are directly mounted (not UNC paths. (See Enable Unbuffered I/O on a Mount Path or step-by step instructions on enabling this option.)

Note that this is a licensed feature and requires a feature license to be available on the CommServe.

See Enable Unbuffered I/O on a Mount Path for step-by-step instructions.

Enable or disable a mount path

The mount path can be enabled or disabled, if required, to control the access to the mount path. If the mount path is disabled, it will be displayed as Offline in the General tab of the Mount Path Properties dialog box. If the mount path is disabled by the system due to inaccessibility, the offline reason can also viewed from the General tab of the Mount Path Properties dialog box. Note that when a MediaAgent itself is offline, the individual mount paths are not displayed as offline.

See Enable or Disable a Mount Path for step-by-step instructions.

This operation is supported by all disk libraries.

Establish the Mount Path Allocation Policy

The mount path allocation policy allows you to establish the maximum number of concurrent writers or the maximum number of simultaneous data protection operations on the mount path. See Establish the Mount Path Allocation Policy for the Library for a detailed description.

If necessary you can also disable the mount path for write operation. This is useful in situations where you wish to retire or phase-out a mount path. See Retire a Mount Path for step-by-step instructions.

This operation is supported by all disk libraries.

Establish Space Allocation on the Mount Path

Space allocation allows you to establish the maximum amount of space that must be used by the mount path. This can be done by specifying the reserve space and selecting either the Use until the mount path reaches the reserved space or Do not consume more than n GB options.

  The Do not consume more than n GB option considers the backup size stored in the mount path. That is, the data size after compression and before deduplication.

See Establish Maximum Space Allocation on a Mount Path for step-by-step instructions.

This operation is supported by all disk libraries.

Reduce Fragmentation of Data on a Mount Path

You can reduce the fragmentation of data on a mount path by enabling the Reduce fragmentation by growing the backup chunk by n MB option. This will pre-allocate the space on the mount path for a write operations resulting in reduced fragmentation which in turn will speed-up read operations (auxiliary copy, restore operations) from the disk. This option is supported only for Windows MediaAgents. See Reduce Data Fragmentation on a Mount Path for step-by-step instructions.

Note that fragmentation levels in the mount path can be analyzed using the Fragmentation Analysis feature.

View the Contents of a Mount Path

You can view the contents of a specific mount path. This feature can be used to view a list of data protection operations residing in the mount path. All the details associated with the data protection operation(s) available in the media are displayed. This includes the following:

See View the Contents of a Mount Path for step-by-step instructions.

This operation is supported by all disk libraries.

Delete the Contents of a Mount Path

The delete contents option can be used to logically delete the contents of a mount path.

This operation deletes the data from the CommServe database. Note that this operation does not free-up the disk space in the mount path. A Data Aging operation must be run subsequently to free-up the disk space.

This option can be used to make media available to complete an important data protection job when there is no free-space available in the library.

CAUTION

Extreme caution should be exercised while using this option as once deleted, the contents of the mount path will not be available for data recovery operations.

The Delete Contents operation is recorded in the Audit Trail.

See Delete the Contents of a Mount Path for step-by-step instructions.

This operation is supported by all disk libraries.

Prevent Accidentally Deleting the Contents of a Mount Path

You can protect your mount path contents from being accidentally deleted outside of the CommCell Console. Contents on mount paths created when this feature is enabled are protected from accidental deletion. See Prevent Accidentally Deleting Mount Path Contents for step-by-step instructions to enable/disable this feature. This feature is enabled by default.

This feature applies to mount paths created on NTFS volumes. This feature is not supported on libraries attached to BlueArc NAS NDMP file servers, Hitachi HNAS storage array and Data Domain storage systems.

Moving Mount Paths

There are two ways to move the mount paths to another location:

You can also attach the disk library to another MediaAgent by migrating a disk library. See Migrate a Disk Library for step-by-step instructions.

View or Modify the Properties of Shared Disk Devices for Static Mount Paths

You can view or modify the device alias name and can enable or disable the device. See View or Modify the Device Properties for Disk Libraries with Static Mount Paths for step-by step instructions.

View or Modify the Device Paths Associated with Shared Disk Devices

You can also view and modify the following properties associated with device paths associated with shared disk devices from a mount path:

See View or Modify the Properties of Disk Devices in a Shared Mount Path for step-by-step instructions.

This operation is supported by all shared disk libraries, which includes disk libraries on static shares and replicated disks.

Modify the Read/Write Access to a Mount Path in Replicated Disks

See View or Modify the Properties of Disk Devices in a Shared Mount Path for step-by-step instructions.


Audit Trail

Operations performed with this feature are recorded in the Audit Trail. See Audit Trail for more information.


License Requirements

This feature requires a Feature License to be available in the CommServe® Server.

Review general license requirements included in License Administration. Also, View All Licenses provides step-by-step instructions on how to view the license information.


Other Considerations


Related Alerts

The following Media Management Library Management alerts can be configured from the Alerts Wizard:

For more information, see:


Back to Top