Two-Way Sync

Configure Source To Destination Replication

Overview

You can use this feature to synchronize data between the source and destination computers (two-way sync). Two-way sync ensures that the data on the source and destination computers is the same and in-sync with each other. This feature is useful if there are changes made to the data on the destination computer that need to be synced up with the source computer.

For e.g., if the data is being replicated between the head office (source) and the remote office (destination), the two-way sync will enable synchronization of data modifications from the head office to the remote office using CDR and from remote office to head office using the file system backup and restore.

How It Works

Two-way sync works as follows:

Data from the source client is replicated continuously to the destination using ContinuousDataReplicator.
Data on the destination client is backed up using File System iDataAgent. The data modified on the destination computer is included in the scheduled backups.
Modified data on the destination computer is restored to the source computer. The data is only restored if the data on the source client is older than the data on the destination client.

Applies To

Environments where the data on the source and destination computers does not change simultaneously.

Limitations

The data modified at the destination will be overwritten:
- If the data is changed at the source before the sync.
- If a Full ReSync or Smart Sync operation is started.
Deleted file(s) at the destination will not be synced up at the source.
Application data will not be consistent.

Install Requirements

Install ContinuousDataReplicator and File System iDataAgent on the source and destination computers.

Configure Source To Destination Replication

Follow the procedure given below to configure two-way sync between the source and the destination computer.

Create Replication Set and Replication pair on the Source Computer

In the CommCell Browser, right-click the ContinuousDataReplicator icon and select All Tasks | Add Replication Set.
In the General tab, specify a Name for the Replication Set.
Enter a Destination Host that has CDR installed.
In the Orphan Files tab, ensure that the Log orphan file names and Automatically delete orphan files options are cleared.
Clearing these options, may prevent orphan files from being cleaned-up on the destination computer but will be synced-up on the source through the sync process.
Click OK to save your changes.
Right-click the Replication Set and select All Tasks | Manage Pairs.
Click Add. In the Add Replication Pair dialog box, click Browse.
Select a source volume or directory for the Source Path and Destination Path.
Click OK.
Click Apply and Close.

Configure Destination To Source Sync

Use the following sections to configure the data sync from the destination to source computer.

Create a subclient to backup data on the destination computer

From the CommCell Browser, navigate to Client Computers | <destination client> | File System | defaultBackupSet.
Right-click the defaultBackupSet, point to All Tasks and then click New Subclient.
In the Subclient Name box, type a name.
In the Storage Policy list, click a storage policy name.
Click the Content tab. Click Browse. Select the destination path specified in the Replication Pair. Click Add.
Click the Filters tab. Under Exclude these files/folders/patterns, click Add.
Add all the temporary file extensions that you wish to filter from the backup job, so that they are not restored back to the source.
Click Close. Click OK.

Filter Job Results Directory

Add the following registry keys on the destination computer to filter Job Results directory from the backup.

For Unix, add the sSkipSubclientFilesBackup registry key.
For Windows, create the nSkipSubclientFilesBackup registry key.

Create a Map File For Restore

Create a map file to be used for the restore operation. Map file is a text file in the CSV (Command Separated Value) format and can be created as follows:

Create a text file on the computer where you wish to restore the files i.e. the source computer in this case.
Add the mapping information in the file in the following format:
"<source path>","<destination path>"

The specified paths for both the source and destination should be absolute paths within the same client computer. The following information can be provided in the map file.

Information	Example
Information	Unix	Windows
File to File Mapping	"/dir1/file1.txt","/dir2/file1.txt"	"C:\dir1\file1.txt","E:\dir2\file1.txt"
Folder to folder mappings	"/dir2/info/","/dir4/info/"	"C:\dir2","E:\dir2"
Drive to drive mappings	"/"(root),"/mnt","/usr"	"C:\","E:\"
Rename files/folder during restore	"/dir4/file2.txt","/dir5/file3.txt"	"C:\dir4\file2.txt","F:\dir5\file3.txt"
Filter certain files	"/dir5",""	"C:\dir5",""

Sample content of a map file.

Unix

"/swrlv1/export/Data","/UsbBigMag/export/Data"

"/swrlv1/export/home","/UsbBigMag/export/home"

"/swrlv1/export/project","/UsbBigMag/export/project"

"/swrlv1/export/pub","/UsbBigMag/export/pub"

Windows

"C:\dir1\file1.txt","E:\dir2\file1.txt"

Record the Username And Password

For performing the two-way sync operation, user account information is required to run the jobs.

Use the following steps to obtain the user account information:

From the CommCell Browser, navigate to navigate to Client Computers | <destination client> | File System | deafultBackupSet.
Right click the subclient and then click Backup.
Click the Save As Script button.
From Save As Script dialog box, specify the following:
- From the Client list, select a client to save the script.
- In the Path box, type or Browse to a path and name for the script file to be saved e.g., C:\user_account.cmd.
- Click OK to save the operation as a script file.
Using Notepad or any other text editor, open user_account.cmd.

Navigate to the qlogin command section.

Note down the user name available with the option -u and encrypted password available with the option -ps.

This username and password is required during the batch file configuration.

Generate Scripts for Data Sync

Right-click the Download Sample XML File link and click Save Link As... or Save Target As... and save the restore_by_job.xml file on the CommServe.

Open restore_by_job.xml using an XML editor and provide a value for the following parameters:

<clientName> - Name of the Destination Computer
<destClient> - Name of the Source Computer
<destPath> - Path for Restoring the Files
<mapFilePath> - Path of the Map File

The screenshot of the parameters to be modified is displayed on the right

This XML file will be used as an input file for the batch file which picks up the last successful backup job and triggers a restore by job operation for that job.

Download Sample XML File

The parameters that need to be changed in the provided XML file are highlighted in the screenshot below:

Add the following content to a file and save the file as restore_last_backup.bat on the CommServe. Make sure that the file is saved as a batch file (.bat extension).
@echo off setlocal ENABLEDELAYEDEXPANSION if not defined GALAXY_BASE set GALAXY_BASE=<SoftwareInstallationDirectory>\Base set PATH=%PATH%;%GALAXY_BASE% set response="RBJ_13328124.tmp" set BackupJobID =0 qlogin -u "<Username>" -ps "<Password>" -cs "<CommServeName>" if %errorlevel% NEQ 0 ( echo Login failed. goto endOfScript ) :The following command fetches the last successful backup job id. FOR /F "skip=2" %%JID IN ('qlist lastjob -c <Destination_computer> -a Q_FILESYSTEM -b "defaultBackupSet" -s "<SubClientName>" -js Completed') do ( set BackupJobID= %%JID ) :The following command triggers the restore by job using Command Line. qoperation execute -cs <CommServeName> -af <Path_of_the_xml_file> -jobids %BackupJobID% -sourceItem "2:%BackupJobID%" qlogout
Provide the following inputs in the saved batch file:
- <SoftwareInstallationDirectory> - Software Installation Directory
- <Username>, <Password> - User details saved in the Record the Username and Password section above.
- <CommServeName> - Name of the CommServe
- <Destination_computer> - Name of the Destination Computer
- <SubClientName> - Name of the Subclient
- <Path_of_the_xml_file> - Path of the restore_by_job.xml file.

Schedule Destination to Source Sync

From the CommCell Browser, navigate to <destination client> | File System | defaultBackupSet.
Right-click the subclient and click Backup.
Select Incremental as the backup type and click Schedule.

Click the Configure button. Select the required scheduling options. Click OK.

Schedule the sync in such a way that the synchronization does not happen too frequently that a backup is started every time a file is modified; or too infrequently that it is overwritten by a Smart Sync or Full ReSync operation.

Click Advanced.
In the Advanced Backup Options dialog box click the Alert tab.
Click Add Alert...
Select Job Succeeded and Job Succeeded with Error options on the Threshold and Notification Criteria in the Add Alert Wizard. Ensure that all other options are cleared and click Next.
Select the Select [E-mail] for notification checkbox in the Notification Type Selection.
Click the Run Command tab. Specify the location of the batch file (restore_last_backup.bat) in the Command Script Location box and click Next.
Select the CommCell Users and/or CommCell User Groups to receive the alert and click Next.
Review the options you selected in Summary. Click Finish.

The backups will now run as per the schedule and once the backup is complete, a restore by job operation will be triggered which will sync the data on the source computer.