QR Disaster Recovery Solution for Building a UNIX Standby Oracle Server


Overview

Configuration

Bring the Standby Server online

Appendices


Overview

In this scenario, the user has at least two servers, a Production Server, which runs the Oracle Database and a Standby Server.

The Quick Recovery Agent takes a snapshot of the running instance. All tablespaces of the database are put in backup mode before taking the snapshot (Point-In-Time-Image) of the data partitions; as soon as the snapshot is taken, all the tablespaces are put back in open mode, then, after archiving the online redo logs, a snapshot of the archive log partition(s) is taken. This ensures that any transactions happening during the backup mode of the tablespaces are archived. This archived redo log can be used for later Quick Recovery.

The Production Server contains the source (Primary) volumes, which are copied and updated incrementally via the LAN Copy Manager to QR Volumes on the Standby Server. The data is therefore physically ready to use (no lengthy recovery from tape or other media is necessary) on the Standby Server. The Standby Server should have the same version of the Operating system and the same version of Oracle installed on the same path as the Production Server.

In cases where the Production Server suffers a failure, or requires downtime, the system administrator can remove the Production Server from the network and then add the Standby Server to the network (with the same name and configuration as the disconnected Production Server).

The key to a successful QR Agent recovery is properly configuring the Production and Standby Servers. This ensures that the database(s) can quickly and easily be run from the QR volumes on the Standby Server.

This procedure is written for bringing up one instance on the Standby Server. If multiple instances are present on one client we strongly recommend creating a separate subclient for each instance and following the same procedure for each instance individually. This procedure has been tested on Oracle 9.2 database with Sun Solaris 8 and the Sparc platform. The procedure for other versions of Oracle and the OS may vary slightly.


Configuration

The following sections discuss preparing the Production and Standby Servers as well as configuring the QR Agent. The basic workflow is described below. For detailed instructions on installation and configuration options, see Quick Recovery Agent.

Prepare the Production Server

The Quick Recovery Agent supports Oracle 9.2. The database must be in “archive log mode” for QR Volume Creation. We recommend that data files, archive logs, control files, and online redo logs each reside on different partitions (different mount points), and that none of these four types of files reside on a partition with any of the others.

Following is a sample configuration of an Oracle 9.2 installation on a Solaris 2.8 machine, which has an Oracle instance, “CV”, running. Oracle requires that the Standby database be on the same OS level as that on the Production Server.

Mount Point Description
/oracle901 Contains the Oracle 9.2 installed binaries
/oracle901 Is the ORACLE_HOME of instance CV
/cv_data Contains all the data files of instance CV
/cv_archlog Contains all the archived logs of instance CV
/cv_control1 Contains current control file 1 of instance CV
/cv_control02 Contains current control file 2 of instance CV
/cv_redologs Contains online redo logs of instance CV

The redo logs and control files are not backed up since they are constantly changing. In order to take a consistent control file backup, Oracle provides specific commands to back up online control files. It is strongly recommended to have each database's data and archived logs on different volumes exclusively for each instance. (Example: /cv_data or /cv_archlog is exclusively for instance CV. It should not have data or archive logs of any other instance running on that machine).

  1. If the Production Server is not already installed with Oracle 9.2, install the desired version of Oracle and apply any required patches.
  2. Create the required Oracle database.
    NOTE: When creating the instances or when installing Oracle, record your Oracle database configuration and storage locations so that Oracle can be installed identically on the Standby Server.
  3. Install the QR Agent. If the installation detects that you have not already installed the Base software and the File System iDataAgent prior to installing the QR Agent, you will be prompted to install them.
  4. Shut down the Oracle instance to configure the disks used by Oracle as CXBF devices. Use Volume Explorer to perform this operation. If the devices are mounted, you can select the Unmount Before Configure option from the Volume Configuration dialog. The device will be mounted back automatically. Select Detect Volumes from Volume Explorer and verify that all mount points used by the Oracle instance are configured and mounted.
  5. Start up the Oracle instance on the Production Server.

Prepare the Standby Server

Before installing any software on the Standby Server, consider the following:

Install the same version of Oracle as is installed on the Production Server. You must use the same Oracle user ID, Oracle group ID, and the same installation path of Oracle as the Production Server.

  1. Set up the Oracle instance with the same instance name and configuration as that of the Production Server:
    1. Create the required directories (admin, cdump, bdump, udump, and pfile) for the instance in the Standby Server matching the Production Server.
      Example location: $ORACLE_HOME/admin/<ORACLE_SID>/cdump
    2. Oracle database may be using spfile or init file for storing the initialization parameters. Copy the init/spfile (init<ORACLE_SID.ora or spfile<ORACLE_SID>) from the Production Server to the Standby Server. Create the necessary links for the init file if required.
      Example location of these files: $ORACLE_HOME/dbs
    3. Copy the password file (example: orapw<ORACLE_SID>) of the instance from the Production Server to the Standby Server.
      Example location of this file: $ORACLE_HOME/dbs
    4. On the Production Server, note the name and location of each control file. This step is required to copy back the backup control file to each control file location when bringing up the Standby Server.
      Example: /cv_control1/control01.ctl, /cv_control2/control02.ctl, /cv_control3/control03.ctl

    NOTE:
    It is very important to follow Step 1 precisely. These files are used for bringing up the database on the Standby Server in the event the Production Server goes down. Failure to properly configure the Standby Server will result in not being able to bring up the Standby Server successfully.

  2. Install the MediaAgent and Quick Recovery Agent software on the Standby Server. If the installation detects that you have not already installed the Base software and the File System iDataAgent prior to installing the QR Agent, you will be prompted to install them.
  3. Use Volume Explorer to configure all volumes you are planning to use in the Volume Scratch Pool on the Standby Server. Detect volumes when finished.

Configure the Quick Recovery Agent

  1. Configure the scratch volume pool. The scratch volume pool for this recovery scenario should consist only of the disk resources attached to the Standby Server. The QR volumes and incremental updates will be written to these volumes. The on-line help contains detailed steps for this operation.
  2. Create a QR Policy. Select the LAN Copy Manager (LANVolCopy) on the Standby Server as the copy manager, and associate this QR Policy with the scratch volume pool that was created in the previous step. Multiple QR Policies may be associated with a given scratch volume pool.
  3. In the Authentication tab of the QR Agent Properties dialog box, add the selected Oracle instance.
  4. Create the QR Agent subclient(s) on the Production Server. Please note the following:
  5. Start or schedule the QR Volume creation job. In this particular scenario, incremental updates should be scheduled. Please note that we do not support archive logs backup with the Standby Server. The QR incremental updates already take care of archive logs that were created after the last QR update.

    From the QR Volume Creation Advanced Options dialog box, assign each volume on the Production Server to its corresponding destination volume on the Standby Server. The mount path of each standby volume should be same as its counterpart production volume. For each source raw partition on the Production Server, the corresponding destination raw partition on the Standby Server should be selected in the QR Volume Creation Advanced Options dialog box.

    If any raw partitions are used for data files or control files, you must create the necessary links for appropriate devices on the Standby Server before QR Volume creation. The soft link pointing to the destination raw partition should be the same as that of the source raw partition.

    For example, if the production database has a data file linked to a raw volume (data file) as follows:

    /ora_data/<ORACLE_SID>/raw.dbf -> /dev/cxbf/rdsk/c1t1d1s1 (source)

    then the links to the destination volume should be created on the Standby Server as follows (after selecting the destination volume in the Advanced tab and before starting the QR Volume creation process):

    /ora_data/<ORACLE_SID>/raw.dbf -> /dev/cxbf/rdsk/c2t1d1s1 (destination)

  6. To test this configuration, allow some scheduled updates to complete. If necessary, execute a few test Oracle transactions between updates to verify that changed/new blocks are being copied to the QR volumes.

Bring the Standby Server online

In cases where the Production Server suffers a failure, or requires downtime, the Standby Server can quickly be brought on-line to host the database from the QR Volumes you have created. The following steps must be taken to add the Standby Server into the network.

  1. Shut down the Production Server and/or remove it completely from the network.
  2. Rename the Standby Server to match the name of the Production Server that has been removed from the network. (Please refer to Solaris user documentation for renaming the machine). If necessary, change the IP Address to match the one on the Production Server. Using the old IP with a new name could cause problems with name resolution. Add the Standby Server to the network if necessary, and configure TCP/IP and any other applicable network settings.

    After bringing up the Standby Server, verify that the destination volumes are mounted back. These are the volumes that were present in the scratch volume pool while creating QR volumes. Also verify Oracle user is an owner of all mount points, directories and files. If necessary change the ownership to the appropriate Oracle user and group; otherwise your instance might fail to access the files.

    NOTE:
    Do not try to detect volumes in Volume Explorer on either the Production or the Standby Server. The Oracle recovery procedure is manual, and is not accomplished with Volume Explorer, the QR Agent, or the Oracle iDataAgent.

    OPTIONAL:
    Add the destination volumes and their mount points to /etc/vfstab on the Standby Server, to ensure the volumes will be mounted back in the event of a re-boot.

  3. There is a backup.ctl.galaxy (Backup Controlfile) file located in one of the archive log destinations of the instance on the Standby Server. (Example location: /ora_logs/admin/sid/arch)

    Copy the backup.ctl.galaxy file to all the control file locations as noted down in Step 1 of “Prepare the Standby Server”.

    Example:

    cp backup.ctl.galaxy /cv_control1/control01.ctl

    cp backup.ctl.galaxy /cv_control2/control02.ctl

    cp backup.ctl.galaxy /cv_control3/control03.ctl

    If the control file is on a raw device, then the above example would change to:

    dd if=backup.ctl.galaxy of=/cv_control1/controlraw.ctl

    NOTES:

  4. Start the database in the mount mode after connecting as the sysdba user (Example: sys/password as sysdba):

    SQL> startup mount

  5. Recover the database as follows:

After the successful completion of Step 5, the database is in OPEN mode and ready to use.

NOTE:
If you receive any messages that files cannot be accessed or access is denied during the startup process, check again that you have mounted all volumes, copied the files and set the ownership to Oracle user.


Appendices

Rename a Solaris Machine

There is a utility called sys-unconfig which is used for resetting the network configuration. After using it, reboot the machine and enter the new IP, hostname, etc. You can achieve the same effect by editing the listed files below and then rebooting.

/etc/hosts

This is where you specify the IP address of your hostname. Change the hostname here to match your host's new name. It should be the same as what you specify in /etc/nodename.

/etc/nodename

This is just like /etc/HOSTNAME in Linux. It defines the real name of the host. Simply change it to whatever you want to call your host.

/etc/hostname.hme0 (or other interface name)

Change these to line up with what you specified in /etc/hosts.

/etc/net/tic*/hosts

Change everything in here to line up with the files above.

/etc/resolv.conf

Just like Linux. This is where you specify your DNS servers and domain resolution information.

/etc/defaultrouter

Enter the IP address of the default router for your Solaris host. Note that Sun does not create this file by default, nor is there any other location where you may specify a default route.

Back to Top