Troubleshooting - Xen Backup

Backup

Backup jobs are failing due to unsuccessful clean-up attempts

Each backup job includes a series of clean-up tasks that are performed by the system. In some cases, these clean-up tasks may not complete successfully, which in turn causes the current backup job, as well as any subsequent backup jobs, to fail. In such scenarios, the clean-up operations must be performed manually before any other backup jobs can complete successfully.

The manual clean-up process consists of three phases.

The steps to complete each phase are described in the following sections.

Before You Begin

From the Xen Center console, determine if the proxy virtual machine contains the virtual machine disks to be backed up. These disks will appear in addition to the proxy virtual machine's own disks with a name in the following form:
_cvbackupsnpt_<Snapshot_GUID>

where <Snapshot_GUID> is a series of numbers and letters. In the following example, the snapshot's GUID is displayed in bold:

_cvbackupsnpt_491a866d-b916-9389-fa3e-0139b0d5d8d3_200911217212
Note down the snapshot's GUID, as this GUID will be needed during Phase I, below.

Phase I: Destroy the GUIDs of the Snapshot, VBDs, and VDIs

To destroy the GUIDs of the snapshot, VBDs, and VDIs:

From the Xen Server host's command shell, execute the following command:
xe template-list name-label=_cvbackupsnpt_<Snapshot_GUID> params=uuid

This command will provide the GUID of the snapshot, as displayed below:

[root@<root_name> ~]# xe template-list name-label=_cvbackupsnpt_<GUID> params=uuid

uuid ( RO) : <Snapshot_GUID>
Execute the following command with the GUID of the snapshot:
xe vbd-list vm-uuid=<Snapshot_GUID> params=uuid type=Disk

This command will provide the VBD GUIs of the snapshot, as displayed below (this example assumes 3 VBDs are present):

[root@<root_name> ~]# xe vbd-list vm-uuid=<Snapshot_GUID> params=uuid type=Disk

uuid ( RO) : <VBD_1_GUID>

uuid ( RO) : <VBD_2_GUID>

uuid ( RO) : <VBD_3_GUID>
For each VBD GUID, execute the following command:
xe vdi-list vdi-uuids=<VBD_x_GUID> params=uuid

This command will provide the VDIs of each VDB, as displayed below (this example assumes each VBD has one VDI):

[root@<root_name> ~]# xe vdi-list vbd-uuids=<VBD_1_GUID> params=uuid

uuid ( RO) : <VDI_1_GUID>

[root@<root_name> ~]# xe vdi-list vbd-uuids=<VBD_2_GUID> params=uuid

uuid ( RO) : <VDI_2_GUID>

[root@<root_name> ~]# xe vdi-list vbd-uuids=<VBD_3_GUID> params=uuid

uuid ( RO) : <VDI_3_GUID>
For each VDI GUID, execute the following command to destroy the VDI GUID:
xe vdi-destroy uuid=<VDI_x_GUID>

This command will destroy each VDI GUID. The following text is displayed for each VDI GUID:

[root@<root_name> ~]# xe vdi-destroy uuid=<VDI_x_GUID>
For each VBD GUID, execute the following command to destroy the VBD GUID:
xe vdi-destroy uuid=<VBD_x_GUID>

This command will destroy each VBD GUID. The following text is displayed for each VBD GUID:

[root@<root_name> ~]# xe vdi-destroy uuid=9<VBD_x_GUID>
Finally, execute the following command to destroy the snapshot:
xe vdi-destroy uuid=<Snapshot_GUID>

The following text is displayed. Answer yes to all questions in this command:

[root@<root_name> ~]# xe template-uninstall template-uuid=<Snapshot_GUID>

This command will destroy the snapshot. The following text is displayed:

[root@<root_name> ~]# xe template-list name-label=_cvbackupsnpt_<Snapshot_GUID> params=uuid

uuid ( RO) : <Snapshot_GUID>

Once the above steps have been completed, move on to Phase II, below.

Phase II: Delete the Mounted Disks from the Proxy Virtual Machine

To delete the mounted disks from the proxy virtual machine:

Execute the following command:
xe vbd-list vm-name-label=<Proxy_Virtual_Machine_Name>

The above command displays all the VDI UUIDs attached to the proxy virtual machine. In the following example, the proxy virtual machine contains two VDI UUIDs:

[root@<root_name> ~]# xe vbd-list vm-name-label=<Proxy_Virtual_Machine_Name>

uuid ( RO) : <UUID_1>

vm-uuid ( RO): <VM_UUID>

vm-name-label ( RO): <Proxy_Virtual_Machine_Name_Label>

vdi-uuid ( RO): <VDI_UUID_1>

empty ( RO): false

device ( RO): hdb

uuid ( RO) : <UUID_2>

vm-uuid ( RO): vm-uuid ( RO): <VM_UUID>

vm-name-label ( RO): <Proxy_Virtual_Machine_Name_Label>

vdi-uuid ( RO): <VDI_UUID_2>

empty ( RO): false

device ( RO): hda
Execute the following command for each disk you want to delete.
xe vbd-param-set uuid=<VDI_UUID_x> unpluggable=true;

xe vbd-unplug uuid=<VDI_UUID_x>

xe vbd-destroy uuid=<VDI_UUID_x>

xe vdi-destroy uuid=<VDI_UUID_x>

Do not delete any disks of type hda. For example, VDI_UUID_2 displayed in Step 1 is of type hda and, as such, should not be removed.

Once the above steps have been completed, move on to Phase III, below.

Phase III: Delete Files from the Mount Path

To delete the files from the mount path:

Navigate to the mount path. By default, this mount path is the job results directory on the proxy virtual machine.
Delete all files within the mount path.

This completes the clean-up process.

Xen Server stops responding

Symptom

During a Virtual Server iDataAgent backup, the Xen Server becomes unresponsive.

cause

If you are performing backup of more than 4 virtual machines simultaneously, the Xen server may stop responding.

Solution

If you want to backup more than 4 virtual machines, perform the sequential backups. Otherwise, install Virtual Server iDataAgent on another Xen server for the load balancing. Each server will share the load of the backup job and will not become unresponsive.

Backup Job Goes to Pending state with a Network Error

Symptom

The backup job goes to pending state with the following error message:

No network connection to the host

Cause

The virtual machine, where you have installed the Virtual Server iDataAgent has a xencenter name different than its host name.

Solution

Change the xencenter name of the virtual machine where you have installed the Virtual Server iDataAgent. Ensure that the xencenter name is same as the host name.