Monitoring Data Replication

Overview

All details of on-going replication activity are shown in the Data Replication Monitor in the CommCell Console, since this activity is continuous, and not job-based. All other activity, such as Recovery Point creation, is reflected in the Job Controller.

In the Data Replication Monitor, right-click any Replication Pair to view additional details, view any failed or orphan files, or select/create a filter to specify which clients' activity is displayed in the Data Replication Monitor.

Job Phases

CDR utilizes phases to perform three types of operations -- initial data transfer or baselining, smart synchronization, and continuous data replication. The sequence of these phases is listed below along with details of CDR activities during each phase, and the consequence of an interruption, such as a temporary loss of connectivity:

JOB PHASE	ASSOCIATED ACTIVITY	COMMENTS
Baseline Scan	For Windows only, start NTFS journaling on the source to track any file operations that occur during the entire Baseline phase. Scan source path to obtain the number of files and bytes to transfer. Generate Collect File.	The Replication Pair will show a Job State of "Preparing for Replication" in the Data Replication Monitor. If this phase is interrupted: for Windows, it can resume again at the same point for UNIX, it will start over A Full Re-Sync will start at this phase.
Baseline	Data is transferred from the Replication Pair source path to the destination path.	If this phase is interrupted, it can resume again at the same point.
SmartSync Scan	Create a non-persistent snapshot; for Windows, compare it to the change journal. Scan snapshot and generate a new Collect File for any files or directories that were added or data that was modified since the beginning of the Baseline Scan phase.	For Windows: During the SmartSync Scan phase, CDR requires a short period of no disk activity on the source to begin monitoring the source path, and if there is significant I/O on the source, this can fail, although CDR will continue making successive attempts to find a short period of inactivity. When you have multiple, active Replication Pairs on the same source computer, CDR requires this short period of no disk activity for all of them, even for Replication Pairs that use different drives as their source. For example, if you have a Replication Pair on the Client which is already replicating, using F:\ as its source, and you create a new Replication Pair on the same Client using G:\ as its source, too much I/O on F:\ can cause CDR to fail to begin monitoring G:\. For UNIX: To create the non-persistent snapshot on the source, CDR requires a short period of time, during which no files are deleted, created, or renamed. (File writes will not affect the snapshot.) If one of these three operations occurs while the snapshot is being taken, the process of creating the snapshot will begin again. Note that while new files and directories will be copied in their entirety, modified files do not need to be; thus, for larger files, only the modified portion is re-copied, while smaller files with substantial change may be copied in their entirety. Modified files below a certain size threshold are copied again as complete files, while files above that size are broken into blocks with just the changed blocks copied to the destination computer. Files smaller than 256KB will be copied in their entirety whether they match the destination or not. For files above 256KB in size, only the changed blocks will be transferred; the default block size for hashing is 64KB. The default values of the minimum file size and the block size for hashing, can be configured using the Advanced tab of the Replication Set Properties dialog box. If this phase is interrupted: for Windows, it can resume again at the same point for UNIX, it will start over A Smart Re-Sync will start at this phase.
Processing Orphan Files	Compare the Collect File to the Destination to identify orphan files, and apply orphan file settings.	Any data that was deleted on the replication source during the Baselining phases are treated according to your settings for Orphan Files. If this phase is interrupted: for Windows, it will resume again from the beginning of this phase; however, if the snapshot is no longer available, it will return to the SmartSync Scan phase. for UNIX, it will return to the SmartSync Scan phase.
Checksum Calculation (CDR on Windows only)	Calculate checksums on the source and destination to identify files that have changed since Baseline Scan.	If this phase is interrupted, it will resume again from the beginning of this phase; however, if the snapshot is no longer available, it will return to the SmartSync Scan phase.
SmartSync	Transfer all changed files to destination from the new Collect File.	If this phase is interrupted: for Windows, it will resume again from the beginning of this phase; however, if the snapshot is no longer available, it will return to the SmartSync Scan phase. for UNIX, it will return to the SmartSync Scan phase.
Updating Smart Sync (CDR on Windows only)	Compare time stamps on source and destination and update. Temporary snapshot is deleted.	If this phase is interrupted, it will resume again from the beginning of this phase.
Replication	Data is continuously replicated from the source to destination.	Log Transfer & Log Replay activity is on-going. For more information, refer to Replication Logs. The Replication Pair will show a Job State of "Replicating" in the Data Replication Monitor. If the Replication phase is interrupted, when restarted, if it is possible, replication will begin again from the last log replayed on the destination; if this is not possible, the Replication Pair will return to the Baseline Scan phase (Full Re-Sync) or to the SmartSync Scan phase (a Smart Re-Sync) depending on the nature and duration of the interruption. Note that if a user manually restarts Replication by choosing "Start Full Resync", the Replication Pair will return to the Baseline Scan phase.

Job States

The Data Replication Monitor shows the state of each Replication Pair. These states are briefly described:

New Pair - The Replication Pair has been created, but no activity has taken place yet.
Preparing for Replication - CDR is scanning the source paths, preparing for initial transfer or Full Re-Sync.
Baseline - For detailed information, see Baseline.
Initial Sync - For detailed information, see Baseline Scan.
SmartSync Scan - For detailed information, see SmartSync Scan.
SmartSync - For detailed information, see SmartSync.
Processing - For detailed information, see Processing Orphan Files.
Replicating - Data is being continuously replicated.
Replicating (Not verifiable) - The most recent communication between the CommServe and CDR Client indicated the job was in the Replicating state, but this cannot be verified because communication has been interrupted.
Suspended - Replication activity has been temporarily halted, either by a user, or because communication between the source and destination has been interrupted. Logs continue to be written on the source.
Pending - There has been a temporary interruption and CDR is attempting to reconnect and resume operations.
Failed - Phase failed to complete, or log transfer has stopped, perhaps for connectivity issues; logs continue to be written on the source.
Paused - CDR is trying to resume replication activity.
Stopped - Replication activity has been halted by one of the following:
- the Replication Log is inaccessible to be read from on the destination computer
- the system, because communication between the source and destination has been interrupted and cannot be successfully resumed by the system
- the system, because the replication destination has run out of space
- a source or destination throttling condition has not been cleared; if the condition is on a source computer, the job will be System Aborted at first, and if on a destination computer, the job will be Paused at first
System Aborted - For CDR on Windows only, a Replication Pair will be in this state for 3 minutes if the source disk hosting replication logs runs out of space, after which the system will attempt to restart.

To see more information about a particular Replication Pair, see View details of data replication activities.

You can change the state of a Replication Pair, or several at the same time. See Start/Suspend/Resume/Abort Data Replication Activity.

Considerations

The status of all Replication Pairs is not immediately updated when one Replication Pair is resumed. For instance, when all Replication Pairs had been placed in the Paused state, and you Resume one of them, a prompt will ask if you want all Pairs to be resumed. If you choose to do so, all the Replication Pairs that were placed in the Paused state will Resume, and be placed back in the same state they were in previously. However, the CommCell Console will not immediately reflect the status of all the other Replication Pairs that were resumed, and they may still be shown in the Paused state. The CommCell Console will properly synchronize and display the correct state of the Replication Pairs within a few minutes.
During SmartSync of application data, Data Replication Monitor may display more than the actual number of files transferred.

Job Details

The following information is available in the Data Replication Monitor:

Active - When the symbol is green, it indicates recent activity for the Replication Pair; an orange symbol indicates no recent activity.
Phase - the current phase of the job; for more detailed information see Job Phases.

General

Job ID - A unique number allocated by the Job Manager for the operation.
State - The current state of the Replication Pair; for more detailed information see Job States.
Last Update Time - The date and time of the CommServe when the Job Manager last updated the Data Replication Monitor.
Pair Abort Reason - For a Replication Pair that was aborted, the reason is listed.
Last Error - The most recent error message for this Replication Pair.

Initial Sync Information

Start Time - The date and time of the CommServe when data replication activity began for the Replication Pair.

Number of Files To Be Transferred - The files remaining to be transferred for the Replication Log file currently being replayed on the destination.
Number of Files Already Transferred - The files transferred for the Replication Log file currently being replayed on the destination.

Data To Be Transferred during Initial Sync On Source - The aggregate size of all files to be transferred between the source and destination for the Replication Pair. The actual data transferred may differ slightly from this number, based on whether a given file actually gets transferred in full or in part.
Data Transferred during Initial Sync On Destination - The sum of all data already transferred between the source and destination for the Replication Pair.
Throughput Unit - The rate of data transfer during Baseline phase, in GB/hour.

Progress - The percentage of files transferred for the Replication Log file currently being replayed on the destination.

Replicating State Information

Last Log Played Time - The date and time of the CommServe when the most recent Replication Log was played on the destination computer.
Replicated Data - The sum of all data transferred between the source and destination machines since the Start Time.
Attempts - The number of attempts at replication the system has made for the Replication Pair.

Latest Source Log - The number of the most recent Replication Log that was created on the source computer.
Latest Destination Log - The number of the most recent Replication Log that was replayed on the destination computer. If this number is lower than the Latest Source Log number, it indicates that the destination computer has not yet replayed all of the Replication Logs that have been created on the source computer.

Configuration

Pair ID - A unique number allocated by the Job Manager that identifies the Replication Pair.
Source Path - The path on the source computer for the Replication Pair.
Destination Path - The path on the destination computer for the Replication Pair.
Replication Set - The name of the Replication Set.
Replication Type - The type of replication configured for the Replication Set. (See Data Replication Type.)
Client - The CDR Client that is the source computer for the Replication Pair.
Destination Host - The CDR Client that is the destination computer for the Replication Pair.

Attempts

The following information is available in the Attempts window:

Phase - The phase that the Replication Pair was in at the time of the attempted activity.
State - Current state of the Replication Pair.
Start Time - The date and time of the CommServe when the attempted activity began for the Replication Pair.
End Time - The date and time of the CommServe when the attempted activity ended for the Replication Pair.
Elapsed Time - The amount of time that elapsed while the activity was being attempted for the Replication Pair.
Files to Transfer - Files to be transferred to the destination computer for the Replication Pair, based on the initial scan.
Files Transferred - Files already transferred to the destination computer for the Replication Pair.
Data Transferred - The sum of all data already transferred between the source and destination during the attempted activity.
Data to Transfer - The aggregate size of all files to be transferred between the source and destination for the Replication Pair. The actual data transferred may differ slightly from this number, based on whether a given file actually gets transferred in full or in part.