Topics | How To | Troubleshoot | Related Topics
Calculating the Storage Space Required for Index Cache
The index cache directory is the directory in which index data resides. Each MediaAgent maintains an index cache for the data protection operations performed using that MediaAgent. The index data maintained in the index cache is accessed by the system during data protection, browse, and data recovery operations.
To ensure that other files do not use up disk space that is needed for index data, you can create a partition specifically for the index cache directory. The partition must be large enough to accommodate four percent of the estimated amount of data managed by the MediaAgent.
During the MediaAgent installation, the install program prompts for an index cache location for the specific MediaAgent. This information can be viewed or modified from the Catalog tab of the MediaAgent Properties dialog box.
|
Indexes are generated and maintained at the job level in the index cache on a MediaAgent. When a full or synthetic full backup is run, a new index file is generated. Indexing information gets appended to the index files in the existing index cache with each subsequent incremental backup (from the last job in the cycle). Later the existing index folder gets pruned.
The Index Cache Directory requires following suggested iOPs specifications. This can be done using IO meter tool which measures the IOPs (Input Output Operations per second). For more information on how to use IoMeter, see IOPs and Capacity Planning Guidelines for Indexing.
MediaAgent Class |
Estimated Data Backed Up Per Week (File and E-mail Data) |
Estimated Index Cache |
Recommended IOPs |
| Large | 40 - 60 TB | 1 TB | 400 |
| Medium | 20 - 40 TB | 500 GB | 300 |
| Small | Up to 20 TB | 200 GB | 250 |
| For best performance and scalability, it is not recommended to host the Index Cache on the CommServe unless your CommServe is your only MediaAgent. |
Index Cache Retention is the mechanism for retaining and cleaning up index data stored in the Index Cache Directory based on certain rules. This is achieved by specifying appropriate retention rules using the MediaAgent Properties (Catalog) tab. Since index cache can grow over a period of time, it is important to specify appropriate retention criteria based on which the index folders are retained.
On a regular basis, the index files older than the value set for Index retention time in days are cleaned up once in 24 hours irrespective of disk free percentage. The default value of Index Retention Criteria is 35 days.
You can specify the number of days to retain the index cache using the Index retention time in days value in MediaAgent Properties (Catalog) tab. The MediaAgent takes care of removing all index files that have not been accessed within the specified number of days.
The amount of index data that can be cached is based on the following parameters:
Specifies the number of days for which the index would be retained in the MediaAgent for a specific subclient. Once the number of retention days for a specific index is met, it gets aged once every 24 hours.
It is the disk percentage until which the index cache must be removed after the minimum free space percentage threshold is met. Whenever a backup is initiated, the MediaAgent checks for the available disk space in the disk housing the index cache. If the amount of used disk space exceeds the Minimum Free Space threshold, the MediaAgent removes index files in the index cache on a least recently used basis until the used disk space becomes equal to the Index Cleanup Percent specified here. However, keep in mind that the index retention time will not be taken into consideration during this process.
Specifies the total amount of free space that must be available at all times in the index cache. If the Index Cleanup Percent is not met, MediaAgent is brought offline and most recent indices are cleaned until Minimum Free Space amount is reached.
If the amount of free space falls below the specified amount in the volume in which the Index Cache is stored, the MediaAgent generates an event message and generates the MediaAgents (Disk Space Low) alert, if configured. It also starts cleaning up the index based on Least Recently Used (LRU) basis until the free space that specified in Index cleanup Percent. See Available Alerts and Alert Descriptions and Space Check Thresholds for the Software Installation and System Directories for detailed information on setting up the alert.
| The above parameters are not applicable for NetWare MediaAgents. Ensure that you have sufficient disk space on the NetWare volume that hosts the index cache before performing data protection operations. |
A local index cache is maintained in a local path on the MediaAgent computer and can be accessed by that MediaAgent only. The location of the local index cache is configured during the installation of the MediaAgent software.
Local index cache is supported on Windows, Unix, and Netware MediaAgents.
MediaAgents create the index cache by default, and the location of the index cache is supplied when installing the MediaAgent. Ensure that you have enough space to accommodate the index cache. You might want to estimate the space required using the recommendation specified at Calculating the Storage Space Required for the Index Cache Directory.
You can change the location of the index cache and designate a new directory at any time from the Catalog tab of the MediaAgent Properties dialog box. When you change the directory for index cache, the existing cache contents are moved and all new cache entries are directed to the new location. See Move an Index Cache for step-by-step instructions.
Since index cache can grow over a period of time, you might have to specify the criteria based on which the index cache is retained. See Index Retention Criteria for the available index cache retention settings. See Set the Index Retention Criteria for step-by-step instructions to specify index cache retention.
The index cache of a MediaAgent can be shared with other MediaAgents. You might want to share the index cache if you wish to define alternate data paths (GridStor) for Storage Policies. When a storage policy uses multiple data paths, then the index cache of all the MediaAgents associated with those data paths must be shared for accessibility reasons. Note that MediaAgents that share the index cache must be of the same release version.
Index cache can be shared using a Index Cache Server or a Network Share.
To create Alternate Data Paths in the CommCell, you must share the index cache so that it can be accessed by all the MediaAgents that are configured with alternate data paths. (See Alternate Data Paths (GridStor) for a detailed description of the topic.)
Follow the guidelines listed below while creating a Shared Index Cache Directory for alternate data paths:
Index Cache Server is an index cache sharing mechanism that saves an additional copy of the index cache for sharing purposes. See Index Cache Server for more information on this feature.
A Network Share is a designated location on the network where one or more MediaAgents store their index cache. The index cache stored in a network share can be accessed all participating MediaAgents. You might use a network share if you have a dedicated partition created exclusively for index cache and you wish to use this partition for index cache sharing.
Ensure that you have enough space to accommodate the index cache from All participating MediaAgents. You might want to estimate the space required using the recommendation specified at Calculating the Storage Space Required for the Index Cache Directory.
| WARNING: Note that when using a network share, the local index and the shared index is one and the same. A network disruption might damage the index and jobs might have to be restarted due to index cache failure. So it is recommended that you use Index Cache Server for index cache sharing instead. It is highly recommended that the option Enable Intermediate Index Cache Directory be used when configuring Index Cache on a network share. With this option turned on the index is written to the local disk first and at commit points uploaded to the Network share. This will avoid failures due to network disruptions/failures writing to the index on the network share. |
Network Share is supported on Windows and Unix MediaAgents.
| When using Network Share with alternate data paths, the create index phase of a data protection job, will round-robin between the MediaAgents that share the index. The appropriate MediaAgent's name will be displayed in the Job Controller during the create index phase of the job. |
Use the following procedures to create a shared index cache using Network Share:
You can change the location of the Network Share. When you change the location, the existing cache contents are moved and all new cache entries are directed to the new location. See Move a Shared Index Cache for step-by-step instructions. Note that you will have to change the location on all MediaAgents pointing to the network share.
Retention rules for the index cache entries in a Network Share are specified in the Network Share profile. See Edit a Shared Catalog Server Profile for step-by-step instructions to modify the retention settings.
See Index Retention Criteria for information on index cache retention settings.
You need to allocate space for index cache on the local disk of a MediaAgent. The necessary space is typically between 100 GB to 500 GB depending on the amount of total data protected using the MediaAgent.
It is recommended to allot a minimum disk space of 100 GB for intermediate index cache. To calculate the disk space, divide the index cache size by the number of cycles retained in the cache. For example, if the index cache allotted size is 500 GB and retention is 14 days (2 cycles), then you will need 500/2 (= 250) GB for intermediate index cache.
The following are recommended as best practices for maintaining index cache:
When clients are added to a MediaAgent.
When the backup cycle is modified.
Factors that affect the composition of backups, such as the data size, file names, etc.
Index Check Pointing option, which provided job restartability in failover scenarios, is a deprecated feature in this release. On upgraded clients, if you have been using the index check pointing feature in the previous release, then you might continue to see the option in the Advanced Backup Options - Data tab. We strongly recommend that you disable this option and use Catalog Server with transaction logging feature instead. For more information on using this feature, see Using Index Cache Server for Job Restarts in Failover Scenarios
MediaAgents participating in index cache sharing must operate in the same version. When the Index Cache of a MediaAgent is shared, then all MediaAgents participating in the share must be upgraded together.
The CommCell Configuration Report provides the following information on index cache disk usage for each MediaAgent in the CommCell. This information is useful to identify index cache growth and manage the disk space accordingly.
See CommCell Configuration Report for information on how to use the report.
Operations performed with this feature are recorded in the Audit Trail. See Audit Trail for more information.