Recover Troubleshooting - Celerra File Archiver

Archive Recover  

The following section provides information on some of the troubleshooting scenarios related to Windows file archiving:

Improper Data Displayed during recovery Operations

Problem Cause

The space quota for an object such as volume, directory or user is set lower than the amount already in use. Doing so may cause improper results for data protection or recovery operations.

Solution

Do not set a quota for an object such as a volume, directory, or user that is lower than the amount that has already been used.

Previewing the archived files is triggering a recall

In case the archived files are MAC files and their metadata has been archived then previewing them will cause the entire file and its metadata to be recalled.

Celerra Proxy Stub Folder has no memory space left

When the volume where the proxy stub folder resides runs out of space, you can move the proxy stub folder to a new location in another volume where more space is available.
Follow the procedure given below to move the Celerra proxy stub folder to a new location:

  1. Create a new proxy folder on the new drive.
  2. Remove the share from the original proxy folder.
  3. Give the new proxy folder the original share name.
  4. Change the sub client properties to point to the new proxy folder.
  5. Restart DataArchiver services.
  6. Run archive job on all associated sub clients whose proxy folder has been changed.

Archiving is completing without stubbing

If archiving is completing without stubbing, follow the steps given below to troubleshoot:

  1. Ensure that the file server and the proxy computer connection is established.
  2. Ensure that the proxy IP address is listed among the target hosts.
  3. Run an archive.
  4. If archiving is completes without stubbing, remove all IP address from the list of target hosts.
  5. Run an archive.
  6. If archiving completes with stubbing determine the IP address that the proxy server is using.
  7. Add the IP address to the target hosts.

Recall fails With Access Denied Error

Problem Cause

When you double click a stub to perform a recall, the recall operation fails with the message Access Denied. You will receive an error in the CommCell Console which says Recall abort due to exported media. Afile-Data=[number].

This happens because the Archiver Services perform a pre-check to see if the required media is available in the library.

Solution

Import the required media into the library. We recommend you to have a latest backup cycle available always so that the stubs can be recalled from that cycle. Backup operations performed on disk libraries enable faster recalls as they are not bound by the read and seek rules binding to the tape drives.

Unable To Do A Stub Recall If The Proxy Computer Is Switched

Pre-Deployment Steps for CIFS access

  1. Make sure that the maximum time is set for Proxy File Expiration and Proxy Prune Interval in the Modify Celerra property dialog box. Restart Data Archiver Services on that computer.
  2. On your new proxy computer, install Windows File System iDataAgent and make sure that Install Agents for Restore Only option is selected during the install.
  3. Make sure that the RestoreOnlyOfflineFiles registry key is enabled in your proxy computer. This will restore the stubs to the new proxy computer and makes sure that the stubs in the old proxy cache are not recalled.
  4. Perform an archiving job with the old proxy computer so that the new stubs that are created and the proxy cache are consolidated.
  5. Specify the name of the new proxy computer Destination Client box. You can change the drive or volume to which the data/stubs need to be restored by entering the new path in the Specify destination path box.
  6. Uninstall the Windows File System iDataAgent installed with Install Agents for Restore Only option selected.

Steps to Replace Proxy Computer

  1. From the CommCell Console, right click <proxy_client> and point to All Tasks | Release License for Client. Make sure that you do not hard delete the <proxy_client> from the CommCell Console.
  2. You must give the new proxy computer the same Domain Name Server name as that of the old proxy computer. Make sure that the old proxy computer is shutdown completely to avoid a naming conflict.
  3. Register the new proxy computer using the following command:

    [root@celerra_cs nasadmin]# server_http server_4 -append dhsm -hosts 172.19.97.208

    In this example:

    server_4 refers to the server name and 172.19.27.208 is the IP of the new proxy computer to be registered.

  4. On the file server control machine, use DNS name to ping the new proxy computer. If you cannot ping the new proxy computer, add the IP address of the new proxy computer manually into the hosts file located in etc folder.
  5. Install the required packages in the new Proxy computer and point to the original CommServe where the archive operations are performed.
  6. The new proxy computer is automatically added to the CommCell Console. If it is not, configure the proxy computer manually with the required agents in the CommCell Console.
  7. Log in to the CommServe/MediaAgent computer using the new DNS name. If you cannot ping the new proxy computer, add the IP address of the proxy computer manually under the hosts files located at C:\Windows\System32\drivers\etc.

Follow the steps given below if you are using a proxy on http:

  1. Configure IIS on the new proxy computer. For more information, see Setup Proxy Client
  2. Log in to the file server control machine and verify if the original registration is still valid.

  3. Use the following command to make sure that the registration information is correct.

    [root@celerra_cs_admin]# fs_dhsm -connection FS6 -info

    In this example:

    FS6 refers to file system volume.

  4. Perform an archive operation on all existing subclients. This operation synchronizes all existing stubs with the new proxy computer.

    The stub recall operations should now complete successfully.

If you want to change the location of the proxy stub folder, enter the new location in the Proxy Stub Path box in the Subclient Properties dialog box.

stubs recalled automatically

If you have PGP Disk Encryption tool used by SAP installed on your computer, you might find that the stubs (archived data) are recalled automatically without triggering a recall operation.

This is because certain permissions might be enabled in the PGP Options dialog box.

To resolve this issue, make sure that the Overlay PGP icon on secured files and folders and Protect individual files options are disabled in the PGP Options (Netware tab) dialog box.

Error opening Terminal server <hostname>. Error [1722]:The RPC server is unavailable

Error opening Terminal server <hostname>. Error [5]:Access is denied

Problem

During recalls the File Archiver recall popup message is not being sent even though the Enable Recall Popup Message has been selected from the system tray settings of the notification enabler.

Solution

Recovering Data Associated with Deleted Clients and Storage Policies

The following procedure describes the steps involved in recovering data associated with the following entities:

Before you Begin

This procedure can be performed when the following are available:

Recovering Deleted Data

  1. Locate the latest Disaster Recovery Backup which contains the information on the entity (Storage Policy, Client, Agent, Backup Set or Instance) that you are trying to restore.
  2. On a standby computer, install the CommServe software. For more information on installing the CommServe, see CommServe Deployment.
  3. Restore the CommServe database using the CommServe Disaster Recovery Tool from the Disaster Recovery Backup described in Step 1. (See Restore a Disaster Recovery Backup for step-by-step instructions.)
  4. Verify and ensure that the Bull Calypso Client Event Manager Bull Calypso Communications Service (EvMgrS) is running.
  5. If you did not have a CommCell Migration license available in the CommServe when the disaster recovery backup was performed, apply the IP Address Change license and the CommCell Migration license on the standby CommServe. See Activate Licenses for step-by-step instructions.
  6. Export the data associated with the affected clients from the standby CommServe as described in Export Data from the Source CommCell.
    When you start the Command Line Interface to capture data, use the name of the standby CommServe in the -commcell argument.
  7. Import the exported data to the main CommServe as described in Import Data on the Destination CommCell.

    This will bring back the entity in the CommServe database and the entity will now be visible in the CommCell Browser. (Press F5 to refresh the CommCell Browser if the entity is not displayed after a successful merge.)

  8. If you have additional data that was backed up after the disaster recovery backup and before the deletion of the entity, use the procedure described in Import Metadata from a Tape or Optical Media to obtain the necessary information.
  9. You can now browse and restore the data from the appropriate entity.

    As a precaution, mark media (tape and optical media) associated with the source CommCell as READ ONLY before performing a data recovery operation in the destination CommCell.

Recovering Data from a Bad Stub

Problem

If you terminate a stub recall job while it is still running, the stub is rendered useless and that data can never be recalled from the stub.

Solution

In case of such scenario, when data cannot be recovered from the stub, it is recommended to browse and restore the data.

Registry Keys for Recovery Operations

Use the following registry keys to modify the default behavior of the Migration Archiver Agent:

Topic Registry Key(s) Description
Access and Modified Times GXHSMSERVICEKEEPTIMES

GXHSMSTUBKEEPTIMES

 Migration archiving and stub recovery operations change the access and modified time on files. An enhancement that allows the recovery administrator to preserve the original file access time and modified time during migration archiving and stub recovery. See Registry Keys (GXHSMSERVICEKEEPTIMES and GXHSMSTUBKEEPTIMES) for more information on enabling this feature for all instance types except Celerra.
GXHSMSERVICEKEEPTIMES

GXHSMSTUBKEEPTIMES

 Stub pruning looks at the modified time of the file to determine eligibility for pruning stubs. If you use the GXHSMSERVICEKEEPTIMES and GXHSMSTUBKEEPTIMES registry keys, the modified time will be preserved. This means that when you run a migration with these registry keys enabled, the preserved modified time will always be older than the time that the file was stubbed. As such, if you use the Prune stub only after n days option, you should ensure that the modified time is not being preserved. Therefore, be sure to disable the GXHSMSERVICEKEEPTIMES and GXHSMSTUBKEEPTIMES registry keys (which are enabled by default) when you use this option.
Stub Recalls GXHSMSERVICEMINRECALLFREESPACE If you would like to ensure that there is sufficient free space on a volume before initiating stub recalls, you can use the GXHSMSERVICEMINRECALLFREESPACE registry key to enable this feature.
nRETRY_RECALL_TIMEOUT

nRETRY_RECALL_DELAY

 Two registry keys are available for administering the timing of recalls. The nRETRY_RECALL_TIMEOUT registry key allows you to set the time of duration for a recall attempt. Once this time expires, the recall attempt will stop. The nRETRY_RECALL_DELAY key indicates the delay time for a recall attempt. Once this time expires, the recall attempt will resume. These keys work hand-in-hand with each other, and they are especially useful if the CommServe goes down.
EVMGRC_REMOTE_PORT_OVERRIDE

EVMGRC_REMOTE_PORT

 For File Archiver for Windows, you can override the stub file port number by using the EVMGRC_REMOTE_PORT_OVERRIDE registry key.  Use this key when the port number changes. Keep in mind that using this key defeats the benefit of being able to recall data using different port numbers for multiple Agents. When the key does not exist, the File Share Archiver client uses the stub content to determine port number. For 7.0 stubs, refer to the EVMGRC_REMOTE_PORT registry key.
GXHSM_DISABLE_ARCHIVE_FILE_RECALL_AND_RESTORE This registry key disables stub recalls at the driver level and allows service to send custom popup messages. This key should be used with a service level key GXHSM_DISABLE_ARCHIVE_FILE_RECALL_AND_RESTORE_MESSAGE.

It is created in computers which has File Share Archiver Client installed.
DMReadMaxMountRetryVal By default no failure message is displayed when a recall job waits for the media to be imported. This key allows for setting a finite number of mount retries during restore.
Stub Recall History nDMRSendFileStatus

The stub recall history is viewable at the client and agent levels in the Commcell Console and is associated with the first instance, which was created in the Migration Archiver agent. Stub recall history can be turned on by creating nDMRSendFileStatus key and setting the value to 1.

You can use the nDMRSendFileStatus registry key to reduce the frequency at which the stub recovery job statistics are sent to Job Manager for updating the Recovery Job History views and reports. This is useful for increasing the efficiency of system resources in cases where there are frequent stub recalls and there is no need to update the Job History immediately after each stub recall.
GXHSM_ENABLE_INPLACE_RESTORE By default the recalls occur on a staged path, and then the recalled files are copied to the target path. This is a user defined key which allows the user to direct the recalls in place rather than the staging path.
GXHSM_ALTERNATE_GXHSMCACHE_PATH By default at the time of stub recall the stubs are stored in a stub cache path, which is generally in the root volume path. The Maximum supported path length for it is 1000. Any path longer than this specified length will get proportionately reduced if this key is specified. This user defined key allows the user to specify the path for all recalls.
Persistent Pipelines nDMRSIDLETIMEOUT

To change the default value of how long the persistent pipeline of this Archiver Agent will stay open during idle periods of a recovery operation, you can use the nDMRSIDLETIMEOUT registry key to accomplish this task. The pipeline stays persistent for 24 hours by default.

To ensure the registry key settings take effect, it is recommended that you restart the CommVault Client Event Manager Service (EvMgrc).
nDMRMaxRestoreThreads If you would like to spawn multiple persistent pipelines when stub recall requests are from different archive file IDs, the nDMRMaxRestoreThreads registry key can be used to accomplish this task.
Recall Messages and Pop-Ups GXHSM_MAGNETIC_MSG_NAME You can use the GXHSM_MAGNETIC_MSG_NAME key to replace the default "MAGNETIC" key word for a recall message with a message of your choice.
GXHSMNOPOPUP To enable the display of a pop-up message for stub recoveries indicating that the recovery operation has completed and showing the location of the file that was recovered, change the GXHSMNOPOPUP registry key value to N. To disable the pop-up, create the GXHSMNOPOPUP registry key and set the value to Y.
It is recommended to use UNC based stub recalls on computers with Vista or Windows 2008.

This is enabled by default on the local client for the File Archiver for Windows Agent when configured for NAS archiving.

The GXHSMNOPOPUP registry key is not supported in Celerra proxy.
GXHSM_DISABLE_ARCHIVE_FILE_RECALL_AND_RESTORE_MESSAGE This is a service level key used in conjunction with the driver level key GXHSM_DISABLE_ARCHIVE_FILE_RECALL_AND_RESTORE on computers where File Share Archiver Cleint is installed. The key allows for customizing the popup message when GXHSM_DISABLE_ARCHIVE_FILE_RECALL_AND_RESTORE is used to disallow the recall.

The key can be used with the following to get desirable popup messages:

  1. "%s" can be used in the registry key value to represent the file name.
  2. The popup message can be customized into top and bottom fields. Anything before <gxhd> would be message in the bottom field. Anything after <gxhd> will go in the top field.
    File %s is being recalled. <gxhd>Please wait....
GXHSM_MAX_DISPLAY_TIME_FOR_DISABLERECALLRESTORE_MESSAGE The key is used in conjunction with GXHSM_DISABLE_ARCHIVE_FILE_RECALL_AND_RESTORE_MESSAGE as it allows the user to set a desired popup message display time.
Log Files GXHSMSTUBLOGFAILFILESTRUNCATE For File Archiver for Windows, you can determine whether the existing log file associated with your job should be truncated or appended to. See GXHSMSTUBLOGFAILFILESTRUNCATE in Registry Keys for more information.
Sparse Files GXHSMSTUBCREATESPARSE If you have enabled the archive option to create stub files that are not sparse files (using the GXHSMSTUBCREATESPARSE registry key), you must use WordPad — not Notepad — to open the resulting text files.
GXHSM_NAS_REMOVESPARSE_ON_RECALL In case of archiving FPolicy and Network File Share data, use GXHSM_NAS_REMOVESPARSE_ON_RECALL to remove the sparse attribute upon recall if the stub has been created as a sparse file.