Best Practices - Deduplication to Disk

Review the following best practices before using deduplication.

Agents Specific Guidelines To Improve Deduplication Performance

Use the following settings on agents to get better deduplication.

Agents (Data Type)	Deduplication Block Size (Recommended)	Compression	Application Recommendation	Comments
File System iDataAgent (For File Systems, Messages and Documents)	128 KB	ON
Oracle iDataAgent (Database)	128 KB	Turn off deduplication compression from subclient, instance and storage policy copy level. See Setting Up Data Compression for step-by-step instructions.	Turn off Oracle compression. Increase Oracle block size to 512 KB. See Running RMAN Scripts from Third Party Command Line for modifying the block size. Depending upon your environment, modify Data Files per BFS value to 4 or 8. See Enhancing Backup Performance for step-by-step instructions. Make sure RMAN backup optimization is OFF.	Use Non-deduplicated Storage Policy with Compression ON settings to back up the following: Oracle Archive Logs Flash Recovery Area (FRA) Oracle Export dump files using File System Agent

Space Requirements for Deduplication Store

The following calculations can be used to approximately determine the amount of space required for the deduplication store partition:

(0.2 * Number of blocks above the deduplication size threshold to be backed up on the mount path)

For example:

If a client has 100 files to be backed up
With a schedule of every day full backups
For 30 days

Then the size of deduplication store should be:

(0.2 * 100 * 30) = 600 KB

Note that this size is the maximum space required and the deduplication store size could be much lesser if there is a good level of deduplication possible.

Hence in the above example, if the 100 files do not change at all, then the amount will be (0.1*100*30) KB = 300 KB.

Age of the Primary Block

The period of time that a deduplicated data is used as the primary reference for any new secondary object can be defined in the Do not Deduplicate against objects older than parameter. This will ensure that very old blocks are not allowed as the 'Origin' data for newer data protection jobs that are deduplicated. See Setting the Age of the Primary Block for step-by-step instructions to set the age of the primary object/block.

Free Space Warning for Deduplication Store Using Alerts

If the amount of free space falls below the specified amount in the volume in which the deduplication store is stored, the MediaAgent generates an event message and generates the MediaAgents (Disk Space Low) alert, if configured. If the space is less than 10% the job will not continue. See Available Alerts and Alert Descriptions and Space Check Thresholds for the Software Installation and System Directories for detailed information on setting up the alert.

If the free space on the volume in which Deduplication Store is configured goes below 10% of the total space in the volume, deduplicated jobs will fail.

CommServe Database Restore

After performing the CommServe database restore using CommServe Disaster Recovery Tool, the current Deduplication store must be sealed, and backup jobs must be performed with a new deduplication store to ensure the consistency between the deduplication database and CommServe database.

See Perform Post-Recovery Operations for step-by-step procedure on the post recovery operations performed after CommServe database restore.

Installing Software Updates

When installing updates or patches in a deduplication-enabled setup, ensure all deduplication-enabled jobs are either suspended or stopped prior to installing the updates or patches. This will prevent accidental sealing of deduplication stores due to services being stopped when data protection operations are in progress.

Deleting a MediaAgent Store

Never delete the Deduplication Store manually. The Deduplication Store facilitates the deduplication of backup jobs and data aging jobs. If deleted, new deduplicated backup jobs cannot be performed and the existing data in the disk mount paths will never be pruned.

Deduplication Tuning for Reduction

To get higher deduplication reductions for smaller deduplication block size, disable the data compression options.
This causes first full backup to increase in size due to no compression, but subsequent backups reduce as expected.
This is applicable for data sets that compress > 50%, this occurs for databases agents and NDMP.

Data Aging

Ensure that every week, all data aging operations are run in a specific window. Also, make sure that all data aging jobs are successful.

Graceful Shutdown of MediaAgent Hosting the Deduplication Database

When a MediaAgent hosting the deduplication database (DDB) attempts to reboot or power off, by default the system doesn't halt and the operating system will shut down regardless of any processes that are running.

However, SIDB process has a built-in capability to receive the shutdown notification and to bring down the deduplication database gracefully if there is enough amount of time between the shutdown notification and the actual machine shutdown. In case where the graceful stopping of the DDB takes more time than the OS allows, it may still damage the DDB. In order to prevent the shutdown while the SIDB process is still running, the following method is suggested, which will prevent the shutdown in most of the cases.

To allow system to shut down gracefully when SIDB process is running, See Graceful Shutdown of MediaAgent Hosting the Deduplication Database for detail information.

Managed Disk Space

For new deduplicated storage policy copies, Enable Managed Disk Space for disk data option on Copy Properties dialog box (Retention tab) is disabled by default. If an existing deduplication copy (for CommServe with Service Pack 8B) has this option enabled, then while updating and saving the deduplication copy, you will be asked to manually disable this option. This option will not be changed automatically for any existing deduplication copy. Disabling this option helps in faster pruning of the aged data in order to reclaim space on the disk.

For step-by-step instructions to disable Enable Managed Disk Space for Disk Library check box, see Enable Managed Disk Space for Disk Data.