Data Aging

Data Aging is the removal of data from media that has aged. Data Aging first ages data based on the retention rules of a storage policy copy, and then removes that data based on the media recycling rules of the associated media. This will ensure data recoverability going back n number of days, which is specified in the user-defined retention rules. Only data that is classified as aged is removed when data aging operations are run.

Storage policy copies have configurable parameters called retention rules. These rules are called Basic Retention Rules and Extended Retention Rules (These are explained in the following sections.) In order to age data, both the Basic Retention Rules of retention days and retention cycles, and the Extended Retention Rules based on extended intervals of days must be exceeded for all jobs in a cycle. Most agents follow these rules. However, some have their own unique rules, as described in Agents With Unique Rules.

Data must be copied from a source copy (either the primary or a specified source copy) to a secondary copy during an auxiliary copy operation before data can be aged from that source copy.

After data is qualified to be aged, this data will be removed from media based on the following rules:

Data on tape media will be removed when media needs to be reused by data protection operations.
Data on magnetic media will be removed based on the Managed Disk Space rule. All magnetic data on a copy will be removed by data aging if this option is disabled. Data will be removed based on the threshold for Managed Disk Space defined in the library properties.

When all the data on a specific media is aged, that media is automatically recycled back to a scratch pool to be used again for future data protection operations. If you wish to perform data recovery operations from data that has been aged or save the media containing the data for future use, see Accessing Aged Data.

The Retention tab of a Job Details dialog box provides the retention information for the data protection job's storage policy. The associated storage policy copies will be listed with their defined retention rules. From here, you can quickly identify whether the storage policy copies are defined with basic or extended retention rules, and the date(s) until which the data will be retained for each storage policy copy. For more information, see View the Details of a Job.

Basic Retention Rules

Basic retention rules are defined by retention time and retention cycles. These parameters determine how much data is retained and for how long. For a cycle to be eligible for data aging, both of its retention time and retention cycles must be exceeded.

Most agents follow basic retention rules. See Support Information - Data Aging for more information.

Days

Retention time is defined as the amount of time that a cycle needs to be available for a single subclient. Retention time is calculated in terms of 24-hour periods (days) from the start time of the last data protection job in a cycle until the start time of the data aging job.

Cycles

A retention cycle is defined as a group of data protection operations, starting with a full backup, including all subsequent data protection operations up to, but not including, the next full backup that are to be retained for a given subclient. A full backup without any subsequent incremental or differential backups is still counted as a cycle.

Each subclient has its own cycles. Each subclient is associated with a storage policy, and multiple subclients can share the same storage policy. Therefore, a single storage policy can contain cycles for multiple subclients.

The backup data is retained in terms of the following:

Days
Cycles

The default settings for time and cycle parameters are set to infinite, but can be changed to better suit the retention requirement of the data being secured.

The retention period implicitly suggests a relation between the amount of time that you want the data to remain restorable and the number of full backup cycles that you expect to complete during that time period. For example, a retention period of 14 days, 2 full cycles suggests that 2 full backup cycles are completed in a time period of 14 days, one full cycle a week. Consequently, for this subclient, it would be appropriate to schedule full backups on a weekly basis.

As another example, a retention period of 28 days, 2 full cycles suggests that 2 full backup cycles are completed in a time period of 28 days, one full cycle every two weeks. It would therefore be appropriate to schedule full backups on a biweekly basis.

As demonstrated by these examples, the full backup cycle (i.e., the length of time between full backups) for subclients, should be established by the ratio of the retention period parameters, specifically:

Length of time/Number of full cycles = Full backup cycle

Data Aging Based on Basic Retention Rules

For data to be aged from a primary or source copy, the following basic retention rules apply:

Retention days must be exceeded for all jobs in a cycle.
Retention cycles must be exceeded.
Data protection operations that are candidates to be copied to secondary copies must be copied before they are eligible for data aging.

For data to be aged from a secondary copy (that is not a source copy), the following basic retention rules apply:

Retention days must be exceeded for all jobs in a cycle.
Retention cycles must be exceeded.

The following Storage Policy Copy Properties settings would result in data not being retained in the storage policy copy. Thus, these settings are only permitted for primary copies configured with at least one synchronous copy association:

Spool Copy (No retention)
Retain for n number of cycles - set to zero (0).

Basic retention rules can be established from the Basic Retention Rules pane of the Retention tab of the Copy Properties dialog box, then the aging of data on that copy is based on time and cycles.

When a user changes the storage policy association of a subclient, a subclient is deleted, or a client or an agent is deleted, only the retention days must be exceeded for data to be aged. In these cases, retention cycles are set to zero (0). However, when a client or an agent is deconfigured, the associated data will be aged according to the associated storage policy copy’s defined retention time and cycle rules. In this case, retention cycles are honored. If necessary, you can enable the Ignore Cycles Retention on De-Configured Clients option from the control panel’s Media Management Configuration (Service Configuration) dialog box so that the defined retention cycle rules are ignored for the data associated with deconfigured clients.
With this, users can temporarily suspend the activity of a client to age data without uninstalling the client software and without meeting the cycle retention requirement, thereby freeing up media faster for new data. For more information, see Suspend Use of a Client Computer Temporarily.
If a Storage Policy was changed without converting the next backup operation to a full backup, data aging operations will not age the data after the last full backup until:

a new full backup operation is run on the subclient for which the storage policy changed
it has met its retention criteria.

Extended Retention Rules

Extended retention rules are defined in terms of weeks, months, and years. This enables you to retain data for a much longer period of time than with Basic Retention Rules. Each rule starts on the current day and counts backward in time according to each rule. Hence, if there are multiple extended retention rules, then there may be multiple reasons why a full backup is retained and not aged. Note that only full backup jobs can be retained by extended retention rules.

For each extended retention rule you must specify whether the first or last full backup within the time period of the rule is to be kept. If you select last full backup, only the last full backup within the time period will be picked if there are no remaining full data protection schedules for that subclient during that period. For additional information, see Grandfather-Father-Son (GFS) Tape Rotation.

Extended retention rules, which can be defined from the Retention tab of the Copy Properties dialog box, are based on the following:

All Fulls	All Full Backups
Weekly Full	The first or last full backup of every calendar week.
Monthly Full	The first or last full backup of every standard or custom calendar month.
Quarterly Full	The first or last full backup of every standard or custom calendar quarter year.
Half-yearly Full	The first or last full backup of each standard or custom calendar half year.
Yearly Full	The first or last full backup of each standard or custom calendar year.

You can use a standard calendar or define and use a specific custom calendar. If there are no custom months defined for the current time period, data aging operations will use the default standard calendar month definitions. See Custom Calendar for more information.

Full backup jobs for some agents are not self contained because they require data from subsequent jobs in order to be successfully restored, and therefore, cannot be retained by extended retention rules. Since Oracle online and SQL File/File Group (FFG) full backup jobs are dependant upon corresponding transaction logs for restorability, their data will not be retained by extended retention rules.
Data backed up through file/file group subclients for the Microsoft SQL Server iDataAgents cannot be pruned through extended retention rules.
For the Oracle iDataAgent, extended retention rules are supported for offline and selective online full backups only.
For the SAP for Oracle and MAXDB iDataAgents, extended retention rules are supported for selective online full backups only.
Extended retention rules are not supported on Storage Policy Copies configured for Silo Storage.

Extended Retention and Job Priority

Your environment may require that you increase the Job Priority of those jobs that qualify for extended retention, which would ensure that the jobs would have enough resources for completion. You can set the priority of a job individually, see Change the Job Priority of a Scheduled Job, or you can set the priority for these jobs globally, by configuring the JMExtnRtnCombinedPriority database key (a Global Parameter of the CommServe software). Once set, this key can increase the job priority of all those jobs that qualify for extended retention.

Extended Retention and Job Retry

If jobs are interrupted, for various reasons, they can be restarted. You can set the Job Retry (number of job restarts) of those jobs that qualify for extended retention individually, see How to Configure Job Restarts, or you can set the number of job restarts for these jobs globally, by configuring the JMRestartsNoForExtnRtnJob database key (a Global Parameter of the CommServe software). Once set, this key can change the maximum job restart number for all those jobs that qualify for extended retention.

These database registry keys can be created by utilizing the following QScript: SetKeyIntoGlobalParam.sql. For more information, see Command Line Interface: qoperation execscript.

Extended Retention for SnapVault Snapshots for Quick Recovery

The Quick Recovery agent provides the capability to maintain SnapVault destination snapshots (OSSV and ONTAP) and ONTAP SnapShots for extended periods. This capability is provided using registry keys.

This feature enables you to specify a retention period for the daily, weekly, or monthly snapshots. By default, the last snapshot added to the database in the period (daily, weekly, monthly) is selected for extended retention. For example, if the QR policy is set to keep snaps for 10 days and the SaveDailySnaps registry key is set to 20 days, then for snaps older than 10 days, the pruning will remove all but the last one on a day.

The extended retention can be either specified to apply to all QR policies or to a specific QR policy.

For extended retention to apply to all the QR policies, use the SaveDailySnaps, SaveWeeklySnaps, or SaveMonthlySnaps registry keys.
For extended retention to apply for a specific QR policy, use the <QRPolicyName>_daily, <QRPolicyName>_weekly, or <QRPolicyName>_monthly registry keys.

The extended retention specified for a specific QR policy overrides the extended retention applied to all QR policies.

If you change the name of a QR policy then it will be required to update the registry keys to use the updated name for extended retention of snapshots to work.

Transaction/Archive/Logical Log Retention Rules

Transaction/archive/logical log (log) backups are not considered part of the backup cycle. Therefore, storage policy cycle retention parameters do not apply to them. However, log backups may be linked to data backup operations, which can affect their retention: refer to the following:

Log backups are linked to a full backup if they are running at the same time, regardless of whether or not the full backup included data only, or data and log backups.
If a full backup job includes data only, then the next log backup job will be linked to this full backup job.
If a full backup job includes data and log phases, then the next log backup will not be linked to this full backup job.

These linked or chained log backups are not aged until the linked data protection job is aged (according to the associated storage policy copy's defined retention rules). All other log backups, which are not linked to any data backups, will follow the unique data aging rules (listed below) for log backups, unless a user enables the Prune All Database Agent Logs Only By Days Retention Rule option from the Control Panel’s Media Management Configuration (Service Configuration) dialog; with this, the log backups will be aged according to the defined days retention rule.

The following are the unique data aging rules for log backups:

To-be-copied logs will not be aged (on primary or non-primary source copy).
Logs that exist on only one copy will be aged when they are older than the oldest non-removable data.
Logs that exist on multiple copies (except those logs with the longest retention days) will be aged according to copy retention days. Logs that exist on multiple copies with the longest retention days will be aged when they are older than the oldest non-removable data.
Oracle online full logs will not be aged if their data on the same copy is non-removable data.
Partial, disabled logs will be aged when they are older than the oldest non-removable data.

These log rules pertain to the following agents:

DB2
DB2 DPF
Informix
Sybase
Lotus Notes Database
Microsoft SQL Server
Oracle
Oracle RAC
SAP for Oracle
SAP for MAXDB
MySQL

For these database agents, retention cycles are not used for copies involved in operations from the command line. For such operations, data is aged according to the associated retention time.
For MySQL iDataAgents, ensure that the MySQL server does not prune the logs automatically. To do this, you need to set the expire_log_days parameter to 0 on the MySQL Server.

Exceptions

Not all agents support standard data aging rules. The following agents have unique data aging rules due to the nature of their data and operations. See Support Information - Data Aging for more information.

Oracle and Oracle RAC

Data Aging rules are unique for the Oracle and Oracle RAC iDataAgents. The following sections describe these rules.

Data Aging of the Oracle Recovery Catalog Database

When a Data Aging job is run, the BackupPieceName UNAVAILABLE command is automatically issued to RMAN to disable specific backup pieces in the Oracle Recovery Catalog database that were pruned from the Media Manager CommServe tables. Any backup pieces that were aged from the system's database that have exceeded their retention criteria will be marked as unavailable in the Oracle Recovery Catalog database through this methodology. You can delete these specific backup pieces by creating and enabling the OracleDeleteAgedBackupPiece registry key.

Timeout for Oracle Crosscheck Per Instance During Data Aging

By default the timeout for Oracle CROSSCHECK per instance is 600 seconds during data aging in mixed mode environments. You can modify this value (or disable the option) by using the OraCrossCheckTimeOut registry key.

For the Oracle and Oracle RAC iDataAgents, after uninstalling the iDataAgent software, CROSSCHECK will no longer be performed by the system to synchronize entries in the CommServe Database with the RMAN catalog. If either of these iDataAgents is later re-installed, then the next data aging job will synchronize the RMAN catalog with the CommServe Database unless the data on tape has been deleted (such as the case where the tape/volume was used for other backups and has been pruned).

When data aging is running, if the Oracle services go down, the data aging operation will complete successfully. However, you need to manually execute Oracle CROSSCHECK to synchronize the Oracle Recovery Catalog database with that of the CommServe database.

Data Aging Rules for Oracle Archive Logs

Oracle archive logs get deleted for those clients/instances where the Oracle CROSSCHECK has been completed successfully. However, if the timeout for the Oracle CROSSCHECK is small (between 1 - 300) and if there are many archive logs, then the crosscheck will fail with a timeout error (or any other error). In such cases, the archive logs will get deleted from the CommServe database during the next data aging operation.

For more information on archive log retention rules, see Transaction/Archive/Logical Log Retention Rules.

Data Aging Rules for Oracle Archive Index

Oracle archive index is deleted when the associated backup data is deleted. This applies to Table Level Backup.

For more information, see Backup - Oracle.

Age Data Using the qoperation agedata Command

The qoperation agedata command can age data and logs simultaneously based on the Job ID, and it is especially useful for aging each of these items separately.

Disable Oracle RMAN Cross Checks During Data Aging

By default, during a data aging operation, an Oracle CROSSCHECK is performed by the system to synchronize the entries in the CommServe database with the RMAN catalog. If required you can disable this CROSSCHECK operation using the Disable RMAN Cross Check option in the Instance Properties (Details) tab for the specific Oracle or Oracle RAC instance. For step-by-step instructions, see Disable RMAN Cross Check.

Data Aging Rules for Selective Online Full Backups

A selective online full operation that consists of archive logs and oracle data can also be linked to the logs of a separate job, which was initiated within the time frame of the selective online full operation. These logs and the selective online full are then considered as one entity within the software, regardless of whether or not separate jobs have the same job ID. Therefore, they are copied to synchronous and selective copies together during auxiliary copy operations and are aged together. If any part of the selective online full is missing from a copy, the full will not be considered as a valid full and will not be counted as a cycle during data aging. Consider the following:

Data from selective online full backups are considered the same as regular full and offline full backups for each Oracle subclient in terms of basic retention rules of cycles and days. However, if any logs on a primary copy have not been fully copied to a secondary copy, the selective online full cannot be aged.
Data from selective online full backups are considered the same as offline full backups for each Oracle subclient in terms of extended retention rules of days. Selective online full backups and all logs linked with it must be retained together on the same storage policy copy.
Those Logs that are linked with a selective online full (and the logs of the selective online full) can be aged only if they are older than the oldest data that can be aged and the corresponding data of the selective online full that can be or have been aged.
Selective online full backup jobs that are completed with errors will not be retained by extended retention rules during data aging operation.

Data Aging Rules for Third-Party Command Line Backups

Data from third-party command line (i.e., RMAN) backups for Oracle and Oracle RAC is aged differently than data from backups initiated through the CommCell Console. Retention cycles are not used for copies involved in operations from the third-party command line. For such operations, data is aged according to the associated retention time.
To manually retain Oracle third party command line job running on multiple streams, you need to manually retain the job IDs of all the streams of a single rman session.

Data Aging Rules for Oracle On Demand Backups

Data Aging for Oracle On Demand backup jobs uses days/time, and ignores cycles, as the determining factor for pruning the data. Therefore, once the retention time criteria has been met, all data (for both data and logs) is pruned that was backed up using the storage policy specified in the RMAN script that was run through the Command Line Interface.

Consider the following when developing a storage policy strategy for Oracle On Demand backups:

The same storage policy should not be used for regular Oracle backups and Oracle On Demand backups.
The storage policy copy containing logs of Oracle On Demand backups should have a much longer retention time than other storage policies used by regular Oracle backups for the same instance. This is to prevent the logs of Oracle On Demand backups from being pruned before the data of regular Oracle backups, and allow the database to be fully restored and recovered using the data of old regular Oracle backups and logs afterwards.

See Running RMAN Scripts using the Command Line Interface for more information.

Oracle RMAN Retention Policy

An Oracle RMAN retention policy can be configured for each database. When RMAN retention rules are in effect, RMAN considers the backup jobs comprising data files and control files as obsolete, that is, no longer needed for recovery, according to criteria that you specify in the CONFIGURE RETENTION POLICY command. When you run DELETE OBSOLETE or CROSS CHECK operations, RMAN ages data by freeing disk and tape space used by backups that are no longer needed.

Do not configure RMAN retention policy if you want to retain data using the data aging feature provided in the CommCell console. To disable the RMAN retention policy, use the following command: CONFIGURE RETENTION POLICY TO NONE. This ensures that data will only be aged according to the retention rules specified in the associated storage policy copy.

SAP for Oracle and SAP for MAXDB

Similar to Oracle, data aging rules are unique for the SAP for Oracle and MAXDB iDataAgents. The following sections describe these rules.

Data Aging Rules for Selective Online Full Backups

A selective online full operation that consists of archive logs and SAP for Oracle or MAXDB data can also be linked to the logs of a separate job, which was initiated within the time frame of the selective online full operation. These logs and the selective online full are then considered as one entity within the software, regardless of whether or not separate jobs have the same job ID. Therefore, they are copied to synchronous and selective copies together during auxiliary copy operations and are aged together. If any part of the selective online full is missing from a copy, the full will not be considered as a valid full and will not be counted as a cycle during data aging. Consider the following:

Data from selective online full backups are considered the same as regular full and offline full backups for each SAP subclient in terms of basic retention rules of cycles and days. However, if any logs on a primary copy have not been fully copied to a secondary copy, the selective online full cannot be aged.
Data from selective online full backups are considered the same as offline full backups for each SAP subclient in terms of extended retention rules of days. Selective online full backups and all logs linked with it must be retained together on the same storage policy copy.
Those Logs that are linked with a selective online full (and the logs of the selective online full) can be aged only if they are older than the oldest data that can be aged and the corresponding data of the selective online full that can be or have been aged.

Data Aging Rules for Third-Party Command Line Backups

Data from third-party command line (i.e., RMAN) backups for SAP for Oracle or MAXDB is aged differently than data from backups initiated through the CommCell Console. Retention cycles are not used for copies involved in operations from the third-party command line. For such operations, data is aged according to the associated retention time.

Microsoft SQL Server

Data Aging for the SQL Server iDataAgents performs the following stored procedures that you may have been manually running on Enterprise Manager. When Data Aging is run, the system ages these histories from the CommServe database and the SQL Server.

sp_delete_backuphistory
sp_delete_database_backuphistory
sp_delete_backup_and_restore_history

SQL Back in Time Restores and Data Aging Rules

When you perform a back in time restore (i.e., restoring to a backup cycle earlier than the current backup cycle), all differential and transaction log backups which were run after the full backup from which the restored data was obtained will not be able to be aged until a new full backup is run. Running a full backup after performing a back in time restore releases the older backups and subsequent log backups for data aging.

Prune SQL Logs Only By Days Retention Rule

This property is located in the Control Panel, Media Management, Service Configuration tab and specifies that when set to 1, the SQL Chain Check and SQL Log Rule will be skipped and the SQL transaction logs will be pruned for each copy using the days retention rule only. Default is 0, which means this property has no effect. Typically used when Skip Full Backup option is selected.

Migration Archiver Agents and Compliance Archiver Agents

All data and jobs in a migration archiving or compliance archiving operation, must meet the basic or extended retention rules in days (as specified in a storage policy copy) in order to be aged. The Migration Archiver and Compliance Archiver Agents do not support retention cycles.

To promote consistent data availability of all of the data archived by the Migration Archiver and Compliance Archiver agents, it is recommended that all subclients be associated with storage policies whose primary copies have the same retention period.

Quick Recovery

The Quick Recovery Agent can age, or delete, QR Volumes that are older than the retention period specified in the associated QR Policy. The Quick Recovery Agent will also attempt to free any destination volumes that were marked as allocated, or locked, but are not currently referenced by any QR Volume. For example, this condition may arise when a Create/Update QR Volume job is terminated unexpectedly. QR Volume data aging takes place in the context of the data aging job that runs periodically on the CommServe.

Special Data Aging Rules for Quick Recovery Data Aging

QR Volumes are not aged if any Quick Recovery Agent data protection or recovery operations are running.
QR Volumes with incremental updates scheduled will not be aged; these volumes must be deleted manually.

Controls for Quick Recovery Data Aging

QR Volumes that have the In Use flag selected will not be deleted. Select this flag in the QR Volume Details screen of a QR Volume that you wish to exclude from being aged.
For VSS shadows and Generic Enabler snapshots, you can override the special data aging rules by creating the ReplicationMgr registry key and setting the value to 1 or 2. Overriding the special rules with this key means that VSS shadows and Generic Enabler snapshots can be pruned even when Quick Recovery jobs are running or when QR Volumes have incremental updates scheduled. Following are the functions for each of the three possible values:
- A value of 1 will prune only VSS hardware snapshots and Generic Enabler snapshots.
- A value of 2 will prune VSS hardware snapshots, VSS software snapshots, and Generic Enabler snapshots.
- A value of 0 turns the key off.

The value for the ReplicationMgr key does not override the In Use flag value.
The ReplicationMgr key should only be activated when Quick Recovery is used to create VSS hardware shadows and Generic Enabler snapshots, not when VSS software shadows are used to create normal QR Volumes. Otherwise, the source VSS snapshots may get pruned.

Recovery Director

The QR Agent creates snapshots or QR Volumes for use with Recovery Director. These snapshots and QR Volumes are aged as described in the Quick Recovery section above.

Recovery Director has the following exception to the conditions described in the Quick Recovery section: When a snapshot or QR Volume is part of a work flow, and jobs from that work flow are running, the snapshot or QR Volume cannot be aged until all of the jobs in the work flow are complete. This is true even in cases where the snapshot or QR Volume has passed its retention period, and Data Aging is run on the CommServe.

For a workflow that includes VSS for the Quick Recovery Agent, setting the ReplicationMgr registry key to 1 or 2 could cause a VSS shadow to be pruned before the workflow is completed. Refer to Controls for Quick Recovery Data Aging.

Silo Storage

When an active Silo store has been sealed and moved to storage, all the backup jobs that went to that store must meet the retention rules (defined in their associated storage policy copy) for the store to become aged (prunable). Once all of the jobs have met their retention criteria, the entire store is considered prunable, and the Silo (tape) backup jobs are then aged. The tape designated for the Silo storage is then refreshed and available for re-use.

For more information, see Silo Storage.

Data Aging of Other Scenarios

The sections that follow include the data aging of other scenarios.

Data Aging of a Primary Copy Without Secondary Copies

If data aging is performed on a primary copy and there are no secondary copies defined, the data on the primary copy can be aged provided the data has exceeded its specified retention criteria.

Data Aging of a Secondary Copy

The data aging of a secondary copy is dependent on the selected retention criteria set for that copy.

Data Aging of a Primary Copy with Synchronous and Selective Copies

Data aging can be performed on a storage policy that has synchronous and/or selective copies defined. Data can be aged according to its retention rule from the primary copy only when all data that is eligible to be aged has been copied to all active copies during an auxiliary copy operation.

Data Aging from a Source Copy that is not a Primary Copy

The source copy feature allows you to select a copy other than the primary copy to be used as the source copy from which data is copied during an auxiliary copy operation. A source copy can be selected from the Copy Policy tab of the Copy Properties dialog box.

The rules for data aging on source copies are as follows:

Data can be aged from the primary copy when there are To Be Copied jobs and the primary copy is not the source copy. See Jobs on a Storage Policy Copy for an overview of To Be Copied jobs.
Data can be aged from a secondary copy that is a source copy once all of its data is copied to the secondary copy.

The following examples illustrate how data is aged from a storage

Primary_01

Secondary_01

Secondary_02

Secondary_01

Primary_01 = 15 days and 2 cycles
Secondary_01 = 1 day and 1 cycle
Secondary_02 = 30 days and 2 cycles

Data Aging of an Incremental Storage Policy

If data aging is performed on a storage policy that has an incremental storage policy enabled, the data aging operation counts backup cycles across both full and incremental storage policies. Data on a full storage policy is aged based on the retention of the full storage policy, and data on the incremental policy is aged based on the retention rules of the incremental policy.

If the incremental storage policy is also being used as a regular storage policy (and has full backups), the full backups will be also aged according to any basic and extended retention rules that are set.

It is recommended that the retention rule for the full storage policy is greater than the incremental storage policy. Data on incremental policy will be aged earlier if it has shorter retention than the full storage policy. If the incremental storage policy has longer retention than a full storage policy, this may result in dangling incremental jobs.

For more information on incremental storage policies, see Incremental Storage Policy.

Effect of Disabled Jobs on Data Aging

If data aging is performed on a storage policy copy that has disabled jobs, these jobs are aged differently. If the disabled job is a full backup job, the entire cycle is marked as disabled. In this case, data aging does not count the disabled full backup as a valid cycle. If the disabled job is an incremental or differential backup and the full backup job is not disabled, the cycle is counted as a valid cycle. For more information on disabled jobs, see Storage Policy Copy Operations.

Data Aging and Thresholds for Managed Disk Space on Magnetic Libraries

Magnetic disks offer the best choice of media type for fast data protection and recovery operations. Magnetic disks are often used to spool the data before it is archived to tape for long term and/or offsite storage. Using Managed Disk Space data on magnetic disks can be retained as long as possible without running out of disk capacity and affecting future data protection operations. Managed Disk Space provides a way to prune data according to disk capacity in addition to the existing retention criteria which is usually defined by number of days as well as full cycles of data. This adds another layer of retention parameter in addition to specified days/cycles.

Two disk capacity thresholds for managed disk space can be defined. They are:

A threshold (in percentage) for starting the data aging operation (upper limit)
A threshold (in percentage) for stopping the data aging (lower limit)

When disk capacity reaches a high threshold, e.g., 85%, older data automatically qualify for removal. They are removed from the disk if they meet their retention criteria and have been copied to appropriate secondary copies. The aging process automatically stops when the disk capacity reaches a low threshold, e.g., 70%.

Once the managed disk is set up the process runs automatically without user intervention to manage disk capacity. Data protection operations are retained on disk longer than usual providing the benefits of magnetic storage without having to spend manual efforts to manage the disk capacity.

The Enable Managed Disk Space for magnetic data option is available in the Retention tab of the Copy Properties dialog box. If a storage policy is created with a valid retention criteria other than infinite retention, then this option is automatically enabled in the copies.

The pre-defined thresholds for disk capacity for a magnetic library can be defined in the Mount Paths tab of the Library Properties (associated with a magnetic library) dialog box.

Data Aging determines the pruning based on jobs - jobs with older data (based on the creation time of its first archive file) is pruned first. Once the Data Aging operation determines the jobs to be pruned, data will be deleted based on the established threshold. The frequency for checking the disk space and deleting data is determined by the frequency established in the Interval (Minutes) between magnetic space updates option established in the Service Configuration tab of the Media Management Configuration dialog box in the Control Panel.

See Enable Managed Disk Space for Magnetic Data for step-by-step instructions.

Data Aging from Spool Copies

Spool Copy data are aged upon the successful completion of the auxiliary copy job. For more information, see Spool Copy.

Data Aging of Other Types of Data

The following other types of data can be removed during a data aging operation:

Job History Data

The following table describes when job history is removed based on the type of job history and the status of the job:

Job Type	Job Status	When it is Aged
Data Protection Job History/Disaster Recovery Backup Job History	Successful	With its associated data, which is aged based on the associated storage policy copy's defined retention rules.
	Failed/Killed	90 Days
Data Recovery Job History (including CDR Recovery operations)	Any	90 Days
Administration Job History	Any	90 Days

Note that job history data is also removed during the deletion of a storage policy or storage policy copy.

The number of days job history will be retained before it is aged can be changed from the default of 90 days by editing the Days to keep the backup job histories field in the Control Panel's Media Management Configuration (Service Configuration) dialog box. Additionally, the number of days that DataArchiver recovery job history is kept can be changed using the Days to keep the archiver restore job histories field. For more information, see Change Media Management Service Configurations.

Erase Backup/Archived Data

Data from Erase Backup/Archive operations can be aged as follows:

If the Erase Backup/Archive operation is older than all the backup data of the agent that the Erase backup/archive job was run for.
If there are already three Erase Backup/Archive operations that were run for the agent.
Archive files are aged upon completion of auxiliary copy operations.

Audit Trail Data

When the specified retention rule of Audit Trail data is exceeded, Audit Trail operations are aged when a data aging operation is run. See Audit Trail for more information.

Data Aging Operations

Operation

Description

Start or Schedule

Data aging operations can either be run on demand or started immediately. By default, data aging is run every day at 12:00 p.m.

The Data Aging Options dialog box allows you to either start a data aging operation immediately from the Run Immediately option, or schedule it using the Schedule option. You can also run a data aging operation from the Command Line Interface. For more information, see Command Line Interface.

Data Aging for On Demand backup jobs only honors the days/time specified, and ignores cycles, as the determining factor for pruning the data. Therefore, once the retention time criteria has been met, all data is pruned that was backed up using the storage policy specified.

Enable or Disable

Data Aging can be enabled or disabled on a storage policy copy. Disabling Data Aging prevents a Data Aging operation from aging data from a copy when the operation is run, even after the retention rules for that copy have been met.

Suspend and Resume

Data aging operations can be suspended and resumed. When resumed, the operation will continue from the point where it was suspended. For more information, see Suspend a Job and Resume a Job.

Granular Data Aging

Granular data aging is the aging of a specific client and/or storage policy copy data, once all retention rules criteria have been met. Users have the option of specifying the items for removal in the Data Aging Options (Aging Options) dialog box. Entities that can be selected for granular aging options are:

Copies
- Storage Policy Copies: Data associated with the storage policy copies selected will be aged once all defined retention rules criteria have been met.
- Libraries: Data associated with the storage policy copies found on the specified library will be aged from all the libraries to which the storage policy copies are associated once all defined retention rules criteria have been met.
Clients
- All Clients: Data associated with all clients will be aged once all retention rules criteria from their associated storage policy copies have been met.
- Selected Clients: Data associated with selected clients will be aged once all defined retention rules criteria from their associated storage policy copies have been met.

For more information, see Configure Granular Data Aging Options.

Save As Script does not support Granular Data Aging operations. If selected during scheduling of Granular Data Aging operations, users will be prompted with a message indicating that selections will default to standard data aging rules where all data based on the retention rules of a storage policy copy are aged first, and then data based on the media recycling rules of the associated media is aged.

Alert Configuration

Alerts allow you to send notifications related to data aging events. Alerts can be configured during the initiation of a data aging operation. For a detailed explanation of Alerts, see Alerts and Monitoring.

Job Restarts and Job Running Time

Click the Job Retry tab in the Data Aging Options dialog box to access the Job Running Time options, when you either Start Data Aging or Schedule Data Aging.

You can also specify the maximum number of allowed restart attempts and the interval between restart attempts for all data aging jobs. For procedures, see Specify Job Restartability for the CommCell. For more information on these subjects, see Restarting Jobs and Job Running Time.

Manually Retain Jobs

A job can be manually retained from the Jobs for Storage Policy Copy window. From here, users can specify an expiration date for the specific job. If defined, a job's data will not be pruned until after the specified date and all retention rules criteria are met. If defined, and the retention rules criteria are met prior to the defined date, the job's data will not be aged until after the expiration date specified. For more information, see Retain A Job.

Manually Delete Jobs

A job (or multiple jobs) can be manually deleted from the Jobs for Storage Policy Copy. Note that if a job is pruned from a storage policy copy, it cannot be restored. Additionally, if you delete a job of the last cycle that has occurred, the next job will be automatically converted to a full data protection operation, which will ensure a consistent cycle that includes all available data. For more information, see Delete a Job.

Delete Contents

The contents of a media can be deleted and moved to a specified scratch pool so that media is available to complete a data protection job when there are no spare media available in the library. For more information, see Delete the Contents of a Media.

Audit Trail

The following operations are recorded in the Audit Trail, if Audit Trail is enabled:

Job based pruning

Operations performed with this feature are recorded in the Audit Trail. See Audit Trail for more information.

Data Aging and Media Recycling

If data stored on tape media exceeds its retention rule and the data aging operation is run, the data is logically deleted. If all of the data on a media is aged, the media is recycled; that is, it is returned to the scratch pool that is currently associated with the storage policy copy that writes to the media. If you wish to save the data on the media for future recovery, see Accessing Aged Data. Once the tape media is re-used, the data that was originally written to it cannot be restored.

Media that has an active status is not recycled back to the scratch pool until the media has a non-active status.

Data Aging and Jobs that have Completed with Errors

The Control Panel's Media Management parameter, Consider partial successful Full job as valid Full job, specifies whether a partially successful job will be considered as a valid full job. If enabled (default), these specific jobs will be considered as a valid full job in calculating retention cycles, and therefore, be retained until all retention rules have been met.

For example, if several jobs completed with errors (CWE) as displayed below, and retention was set at 2 days, 2 cycles, the following would occur with the data aging operation:

F₁ I₁ I₂ F₂ I₃ I₄ F₃(CWE) F₄(CWE) F₅(CWE) F₆ I₅ I₆

When Consider partial successful Full job as valid Full job is:

Enabled (Set to 1): Jobs F₅(CWE)F₆ I₅ I₆ will be retained, and the remaining jobs will be prunable.
Disabled (Set to 0): Jobs F₂ I₃ I₄ F₆ I₅ I₆ will be retained, and the remaining jobs will be prunable.

For more information on Completed With One or More Errors, see Job Status Levels.

Special Considerations

Keep in mind these considerations when performing a data aging operation:

For data to be aged from a spool copy, the following rule must apply:
- The data must be copied to an active synchronous copy. For more information about spool copies, see Spool Copy.
  - The way in which data is aged is also contingent upon the following:
    - Whether the data protection operation was successful, killed, or failed.
    - Whether the storage policy has secondary copies and whether they are active.
    - Whether jobs are disabled on the storage policy copy.
    - Whether jobs still need to be copied to secondary copies.
- System may override the user defined retention rules under the following cases:
  - Data to-be-copied jobs on source copy won't be pruned regardless of its retention defined by user.
  - When a user changes the storage policy association of a subclient, a subclient is deleted, or a client or an agent is deleted, only the retention days must be exceeded for data to be aged. In these cases, retention cycles are set to zero (0). However, when a client or an agent is deconfigured, the associated data will be aged according to the associated storage policy copy’s defined retention time and cycle rules. In this case, retention cycles are honored. If necessary, you can enable the Ignore Cycles Retention on De-Configured Clients option from the control panel’s Media Management Configuration (Service Configuration) dialog box so that the defined retention cycle rules are ignored for the data associated with deconfigured clients.
    
    If a user changes the storage policy association of a subclient:
    
    All subclient data that was backed up through the previous storage policy will be aged based on its storage policy copy retention time (days) rule only. If you select to run a full backup after changing the storage policy, all subclient data on the new storage policy will be aged according to its retention time and cycle rules. If you select to run a non-full backup as the next backup operation, it is recommended that you run a full backup as soon as possible. All non-full backups run before a full backup will be retained as a partial cycle according to the new storage policy copy's retention cycle rule (even though not a full cycle). The non-full backups (partial cycle) will be aged when the new storage policy copy's retention time and cycle rules are met.
    
    For more information, see Associate a Subclient to a Storage Policy.
By default, the retention period for Disaster Recovery Backup data is set to be retained for 60 days and 60 cycles. If you want to change the retention time for Disaster Recovery Backup data, it is recommended that you keep the default setting as the minimum with predefined extended retention rules defined as: weekly = 180 days, and monthly = infinite.
Multiplexed data is aged when all jobs (multiplexed) on a single chunk have met the defined retention rules of their associated storage policy copy. For more information, see Data Multiplexing.

Related Reports

Data Retention Forecast and Compliance Report

The Data Retention Forecast and Compliance Report provides real-time and historical information of data protection jobs, associated media, data aging forecast and recyclable media based on the selected filter criteria.

Related Alerts

Job Management Data Aging Alert

The Job Management Data Aging alert can be configured to notify users with critical information pertaining to jobs that are active, have failed, succeeded, or even if they have been skipped.