OnePass Agent for Unix File System Advanced Configuration

Table Of Contents

Archiving Rules

File System Size

File Access Time

File Modified Time

File Size

Configuring Stub Rules

Pruning Archived Data for Space Reclamation

Example 1

Example 2

Filtering Files From Archiving

Manually Filtering Files from Archiving

Manually Including Files for Archiving

Filtering Files Using Registry Key

Configuring Archiving for IBRIX File System

Setting Up Stub Cache

Archiving Rules

Archiving rules specify the criteria for the type of data to be archived. You can configure these rules on a requirement basis.

 File System Size

The backed up files can be stubbed based on the available free space on the volume. During a backup job if the free space on the volume is less than this value, files matching the remaining rules will get stubbed.

  1. From the CommCell Console, navigate to Client Computers | File System. 
  2. Right-click the Subclient and click Properties.
  3. Select Rules tab.
  4. Enter the low watermark value for the volume in Begin scan for file selection when free volume is less than n%.
  5. Enter the high watermark value for the volume in End scan if volume is freed to n%.
  6. Click OK.

File Access Time

This configuration will stub files based on the last accessed date of each file within the volume or folder/directory. In case it is older than the number of days specified, the files will get stubbed.

  1. From the CommCell Console, navigate to Client Computers | File System.
  2. Right-click the Subclient and click Properties.
  3. Select Rules tab.
  4. Enter the number of days elapsed since the file was last accessed Archive file access time older than n Days.
  5. Select Any one of the file selection rules are satisfied.
  6. Click OK.

File Modified Time

This configuration will stub files based on the last modified date of each file within the volume or folder/directory.

  1. From the CommCell Console, navigate to Client Computers | File System.
  2. Right-click the Subclient and click Properties.
  3. Select Rules tab.
  4. Enter the number of days elapsed since the file was last modified Archive file modified time older than n Days.
  5. Select Any one of the file selection rules are satisfied.
  6. Click OK.

File Size

This configuration will stub files based on the file size. If a file size within a scanned volume or folder/directory is in between the two limit values set, it will get stubbed.

  1. From the CommCell Console, navigate to Client Computers | File System.
  2. Right-click the Subclient and click Properties.
  3. Select Rules tab.
  4. Enter the minimum file size in Archive file if size is greater than n KB.
  5. Enter the maximum file size in Maximum file size.
These options can be used individually or in combination to specify a range.
  1. Select All of the file selection rules are satisfied.
  2. Click OK.

For Example:

To archive files more than 5KB but less than 20KB:

  • Enter 5 KB in Archive file if size is greater than n KB.
  • Enter 20 KB in Maximum file size.

Configuring Stub Rules

By default, the stubs are deleted 730 days from the day of stubbing when you enable the Prune archived items option. However, you can set the number of days after which the stubs will be pruned using the following steps:

  1. From the CommCell Console, navigate to Client Computers | File System.
  2. In the right pane right-click the Subclient and click Properties.
  3. Click the Rules tab.
  4. Select the Stub Rule tab and make sure that the Prune archived items checkbox is enabled.
  5. You can enter the number of days after which the stubs will be pruned.
  6. Click OK.

Pruning Archived data for space reclamation

Archived data is pruned and secondary storage space is reclaimed based on the stub retention setting and the storage policy cycle - based retention criteria. Running a synthetic full backup can reduce space on the secondary storage for deleted stubs that meet the expiration criteria. Running data-aging can reclaim space on the secondary storage by pruning the expired backup jobs. The following section describes two scenarios for space reclamation:

Example 1

By default stub retention is set to 730 days from the Stub Rules tab and the Storage Policy retention is set for 2 cycles and 15 days. The following procedure demonstrates the reduction in the secondary storage usage:

1.
  • Run a few backup jobs. This will lead to stub generation of the data meeting the defined archiving rules. The subsequent incremental backup will backup these stubs.
  • Delete a few stubs.
  • Perform an incremental backup operation. The time-interval between this incremental backup and the subsequent synthetic full will be calculated as the number of days specified in the retention criteria. The deleted stubs meeting the retention criteria will get expired . For example, if a user has set 20 days for pruning archived items, the deleted stubs will expire after 20 days.
  • Run synthetic full backup with Check for Deleted Stub option selected in the advanced tab. The expired stubs and the corresponding data would not get backed up in this operation.
  • From the backup job history, verify the backup file list. Ensure that the deleted stubs and their corresponding data are no longer included in the synthetic full backup image.
  • Note that space on secondary storage has been freed up by pruning of expired stubs and their corresponding data by running a synthetic full backup.
2. The following procedure demonstrates the reclamation of secondary storage space:
  • Run backup jobs to generate 2 backup cycles.
  • Wait for the last job of the first cycle to expire (based on the Storage Policy retention, which is 15 days by default).
  • Run a Synthetic full backup.
  • Running the synthetic full job will start the third backup cycle and will render the first cycle prune-able. 
  • Run a data aging job. Notice that the secondary storage space occupied by the first backup cycle has been reclaimed. 
 

Example 2

In order to reclaim space on secondary storage by completely removing the archived data and their stubs, follow the steps given below:

Pre-requisites

1. Change the stub retention to a minimum of 1 day from Subclient properties Stub Rule tab..
2. Change the storage policy retention setting to 2 cycles and 15 days.
3. Delete the unwanted stubs.
4 Perform incremental backup twice to generate two backup cycles.
5 Wait for the deleted stubs pertaining to last backup job in the first backup cycle to expire.
6 Run synthetic full backup with Check for Deleted Stub option selected in the advanced tab.

Note that the synthetic full backup will be considered as the third backup cycle and it will render the stubs in the first cycle prunable.
7 Running the synthetic full backup will start a new backup cycle. Due to the change in storage policy retention all the previous backup have expired.
8 Run a data aging job. All jobs from the first backup cycle will be pruned.

Note that space on secondary storage has been freed up as all the unwanted stubs and their corresponding data have been pruned.

Filtering Files from Archiving

Prior to archiving the files, you can set filters to prevent specific type of files from being archived. Filters can also be used to exclude certain system-related files and database application files which consistently fail during a archive operation and are not integral to the operation of the system or application.

The platform specific files that are automatically filtered from archiving are:

Manually Filtering Files from Archiving

You can manually set filters on selected file types, folders or files to be excluded from archiving. For example if you specify *.txt in the filters then all the text files will be excluded from being archived. 

  1. From the CommCell Console, navigate to Client Computers | File System.
  2. In the right pane right-click the Subclient and click Properties.
  3. Click the Filters tab.
  4. Click Add from the Exclude these files/folders/patterns group.
  5. Enter the complete path of the file, folder, or directory that you want to exclude from the archiving operations

    or

    Click the Browse and expand the file system of the client computer. Click the file, folder, or directory that you want to exclude from the archiving operations and then click Add.

  6. Click OK to save your changes.

Manually Including Files for Archiving

When excluding files from archiving, you can also provide exceptions to the data that you are excluding. For example, if you want to exclude from your migration archive operation all the files in the /Ops directory except the Maintenance file, you can add a subclient filter to exclude the /Ops directory from being archived and also add a corresponding subclient filter to allow the Maintenance file to be archived.

  1. From the CommCell Console, navigate to Client Computers | File System.
  2. In the right pane right-click the Subclient and click Properties.
  3. Click the Filters tab.
  4. Click Add from the Include these files/folder/patters: group.
  5. Enter the complete path of the file, folder, or directory that you want to be included in the archiving operation.

    or

    Click the Browse and expand the file system of the client computer. Click the file, folder, or directory that you want to be included in the archiving operation. and then click Add.

  6. Click OK.

Filtering Files Using Registry Key

The following registry keys can be used to set additional filters for an archive operation:

sSTUB_PRUNE_PATTERNS Use this registry key to specify additional filters.

Ex: sSTUB_PRUNE_PATTERNS *.so *.so.* *.a *.lib *.sh *.ksh *.csh /usr/** /bin/** /lib/** /etc/** /var/**
sSTUB_PRUNE_EXCEPTIONS Use this registry key to set any exceptions to the filter list defined in sSTUB_PRUNE_PATTERNS.

Ex: sSTUB_PRUNE_EXCEPTIONS /var/log/abcd/**

Configuring Archiving for IBRIX File System

For X9000 IBRIX , by default the stubs are recalled during the data tiering process, which moves the files and the data stubs from one storage tier to another based on pre determined parameters. However you can configure the sTranProcName registry key to seamlessly migrate data stubs from one storage tier to another without triggering a stub recall. The steps to configure the key are as follows:

  1. From the CommCell Browser, navigate to Client Computers.
  2. Right-click the <Client> and then click Properties.
  3. Click the Registry Key Settings tab.
  4. Click Add.
  5. In the Name field, type sTranProcName
  6. In the Location list, type or select UnixFsDataMigrator.
  7. In the Type list, select Value.
  8. In the Value field, type the process name used migrate stubs from one storage tier to another.

    Example migrator_daemon

    On specifying this value the stubs will not be recalled during the data tiering process.

  9. Click OK.

 

Setting Up Stub Cache

After recalling the files, you can re-archive the recalled files by preserving them in a stub cache. The recalled file will be re-archived from the cache if the file meets the archiving rules and has not been modified after the recall. The following registry keys allow you to set the stub cache and re-stub the recalled files from the cache memory.

nUSE_STUB_CACHE This key allows a file to be re-stubbed after recall by preserving it in cache memory.
dSTUB_CACHE_ROOT By default the stubs are cached under , /opt/calypso/stubcache folder. Use this key to specify a different location of the root folder for the File Archiver stub cache.

Back to Top