The Microsoft SQL Server iDataAgent
uses the following logical entities to manage backup and restore
operations from the CommCell Console.AgentFacilitates SQL instance discovery. InstanceDefines the SQL Server instance to be backed up. SubclientDefines the SQL databases to be backed up. |
By default, all databases within each SQL Server instance are automatically assigned to the default subclient. This subclient backs up the entire instance.
If you want to divide your backups into smaller groups, you can do so by creating user-defined subclients as described in the following sections.
If you want to back up groups of specific databases, you can do so by creating a user-defined subclient containing any number of databases that exist within the instance. This is useful if you want to back up a subset of databases at certain times or with a particular frequency.
When you create a user-defined subclient, the contents of the user-defined subclient will be excluded from the Default Subclient. |
In many cases, large databases may contain portions of data that require more frequent backups than others. For example, tables consisting of records entered on a daily basis may require nightly backups, whereas tables consisting of records entered on a quarterly basis may require only monthly backups. You can group such elements together by creating a user-defined subclient for files or filegroups.
By default, new instances added to the SQL Server are automatically discovered if the option to do so was enabled during the SQL Server iDataAgent installation. If this option was not enabled during installation, you can discover new instances at any time as follows:
|
If you enable automatic discovery new SQL Server instances will be discovered as follows:
This capability ensures all instances are accounted for on a daily basis for backups.
If you want to enable or disable automatic instance discovery, you can do so as follows:
|
If you want instances to be automatically discovered on a more or less frequent basis, you can do so as follows:
|
By default, new databases created on the SQL Server are automatically discovered and assigned to the default subclient. You can disable this functionality as follows:
|
If automatic discovery of databases is disabled, you can manually add databases to a subclient as follows:
|
By default, offline databases in the following states are not automatically discovered:
You can configure automatic discovery of offline databases for one or all clients as described below.
|
Example:qoperation execscript -sn SetKeyIntoGlobalParamTbl.sql -si DatabaseStateString -si y -si suspect;shutdown;standby In this example, databases in the suspect, shutdown, and standby states will be automatically discovered. |
Configuring this option will override the configuration at the CommServe level described in the For All Clients section above. |
|
In some cases, it may be necessary to exclude certain databases from backups for a period of time. For example, you may have configured an entire SQL Server to back up using a particular schedule, but do not require all databases to be backed up according to that schedule. You can exclude databases from backups by following the steps below.
|
Databases that are deleted from the SQL Server are handled differently during backups depending on how the database was discovered and the type of subclient the database was originally assigned to.
Specifically:
The following sections provide information on managing databases that have been deleted from the SQL Server.
By default, when automatically discovered databases are deleted from the SQL Server, they are removed from the default subclient content during the subsequent backup.
In some cases, it may be desirable to manage all databases deleted from the SQL Server manually. For example, if a database is accidentally deleted from the SQL Server, it may not be desirable for it to be removed from the subclient during the next backup, especially if the next backup may occur while attempting to bring the deleted database back. Follow the steps below to manage deleted databases manually for the default subclient:
|
Databases that have been manually added to the default subclient are not automatically removed from the subclient if deleted from the SQL Server. To ensure that the subsequent backup jobs complete without any errors, deleted databases from the server should be removed from the subclient.
|
By default, databases are not automatically removed from subclients when deleted from the SQL Server. If a database is deleted from the SQL Server, the next backup performed on its associated subclient will not complete successfully.
You can also configure the subclient to automatically remove such databases as follows:
|
Use the following procedure to configure and perform OnDemand backup for Database subclients:
|
You can modify the argument file (xml file) to perform different types of backups such as Full, Transaction Log and Differential backups, you can do so as follows:
|
Use the following procedure to configure and perform OnDemand backup for File and Filegroup Subclients:
|
Backup conversion rules provide the facility to convert certain types of backups to another backup type under specific circumstances. This functionality helps ensure all SQL data is protected regardless of circumstances that may cause a failure.
By default, database backups are converted as depicted in the following table.
Backup Conversion Type | Conditions for Conversion | Benefit of Conversion |
Log Backup to Differential Backup | The database recovery model is set to Simple. | Because the Simple recovery model does not support log backups, converting to a differential backup ensures both logs and data are properly backed up. This, in turn, provides the facility to restore the logs. |
Differential Backup to Full Backup | A full backups was performed using other software. | For first-time users, starting with a full backup provides complete protection as a baseline for future backups. |
All Backups to Full Backups |
|
Converting to full backups in these scenarios ensure you have complete protection of the latest state of each database. In the case of system databases, a full backup will ensure the restored database is backed up at the most recent point-in-time. |
If you want to disable this functionality, you can do so using the steps below.
Keep in mind that disabling this option for one scenario disables the option for all scenarios listed above. As such, it is recommended this option remain enabled to ensure no data is unintentionally left out of a backup. |
|
By default, log backups performed outside of the system (for example, using SQL Enterprise Manager) are automatically converted to full backups. This provides a baseline for future backups.
If necessary, you can preserve the log backups performed by previous software packages as follows:
|
If the Convert checkbox is selected, all backups convert as specified in the rules of the dialog box. However, if you want to skip the conversion of Transaction Log Backups to differential backups for subclients with databases set to simple recovery model, you can do so by configuring additional settings to set bSkipTLForSimpleRecoveryModelDB.
|
By default, if files or filegroups have been added to a database since the previous backup, the next backup will automatically be converted to a full backup. This ensures the new files or filegroups are given proper protection as quickly as possible, regardless of the type of backup originally intended.
If you do not require this functionality, you can disable it as follows:
|
By default, all backups performed on File/Filegroup subclients are automatically converted to full backups as depicted in the following table:
Backup Conversion Type | Conditions for Conversion | Benefit of Conversion |
All Backups to Full Backups |
|
Converting to full backups in these scenarios ensure you have complete protection of the latest state of each file/filegroup. |
If you do not want backups to convert to full backups under these circumstances, you can disable this option by following the steps below.
Keep in mind that disabling this option for one scenario disables the option for all scenarios listed above. As such, it is recommended this option remain enabled to ensure no data is unintentionally left out of a backup. |
|
Several options are available for enhancing backup performance reducing network bandwidth overhead. These options include:
You can configure these options as follows:
|
Full backups are necessary at regular intervals as it reduces the chance of data loss if one of log backup becomes corrupted as it will invalidate (not restorable) all other log backups performed after that. This key is used for the purpose of re-enforcing the need of a full backup after certain number of transaction log backups have run.
When this registry key is configured, a minor event will be generated in the Event Viewer to remind users to run a full backup after the configured number of transaction log backups have run.
Use the following steps to configure the number of log backups:
|
By default, a full backup is required after performing a log backup. If you do not require a full backup at the time you want to back up the logs (for example, a full backup was performed outside of the system), you can do so as follows:
|
In some cases, it may be desirable to allow write operations on SQL databases being backed up to continue while the backup runs. This is especially critical in environments where data entry to a SQL database is constant (such as databases being used daily in different parts of the world). Enabling VSS for backups provides this capability.
VSS can be enabled for backups of local volumes in both clustered and non-clustered environments. If the operating system fails to create a shadow copy of the data a traditional backup of the data will be performed, and a corresponding message will appear in the Event Viewer.
When VSS backups are performed, the following will automatically occur:
During a VSS backup, the total amount of free space depends on the size of the backup data. As such, make sure to have sufficient disk space when you perform VSS backups. |
|
By default, backup data is sent to media in two streams. This means that a database, or a portion thereof, is sent to media during a backup in two parallel waves. This results in the backup taking about half the time to complete as it otherwise would if only one stream is used.
You can increase the number of streams used for backups for a particular subclient provided the number of streams does not exceed the maximum number configured in the subclient's storage policy. Increasing the number of streams for a subclient further reduces the amount of time a backup takes to complete. For example, increasing the number of streams from 2 to 3 enhances backup time from one-half that of a single stream to one-third.
Keep in mind that the number of streams configured for backups must also be used when restoring data. For example, if you configure a subclient to use 4 streams, you must also use 4 streams to restore the data. |
|
|
The SQL Server iDataAgent requires a Windows user account that has sufficient privileges for the software to:
The following table illustrates the requirements for the user account you will need to provide:
If the SQL Server Is: | The user account Should Be: |
On a non-Domain Controller |
|
On a Domain Controller | An account other than the Domain Administrator account that has Administrator and SQL sa privileges. |
By default, the local system administrator account is used for all components. You can change the user account to be used for backups at the CommCell, Client Computer Group, Agent, and Instance levels. Accounts configured at each level will be used for all entities within that level as described in the following sections.
In order to access the SQL Server databases to perform data protection and recovery operations, the SQL sysadmin rights are required.
This user account will be used for all SQL Server iDataAgents in your CommCell. Configure the user account at this level if one person will be conducting all backup and restore operations in your organization.
|
This user account will be used for all computers within a Client Computer Group. Configure the user account at this level if different people will be conducting backup and restore operations for each Client Computer Group in your organization. This user account will override the user account configured at the CommCell level.
|
This user account will be used for all instances and associated subclients. Configure the user account at this level if one person will be conducting all backup and restore operations on the client on which the SQL Server iDataAgent is installed. This user account will override the user account configured at the CommCell and Client Computer Group levels.
|
This user account will be used for all subclients within the instance. Configure the user account at this level if backup and restore operations will be conducted by a different person for each instance. This user account will override the user account configured at the CommCell, Client Computer Group, and Agent levels.
|
You can add, modify or delete several configurable properties for SQL iDataAgent from command line.
Command line configuration enables you to:
The following sections describe the available command line configurations:
To run command line operations you must first login to the CommServe as follows:
qlogin -cs <commserve name> -u <user name>
qlogin -cs server1 -u user1
qoperation execute -af CreateSQL_Instance_Template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx
qoperation execute -af ModifySQL_Instance_Template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx
qoperation execute -af GetSQL_Instance_Properties_Template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx
qoperation execute -af DeleteSQL_Instance_Template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx
The following table displays all the parameters that you can use with the commands mentioned in the above sections. To add a parameter to your command, use the following syntax: (An example is provided at the end of the table.)
qoperation execute -af <template XML file> -<parameter name> <value>
Parameter | Description of Parameter Values |
description | A general description of the instance if needed. |
clientName | Name of the client computer. |
appName | Name of the application. In this case it would be 'SQL Server'. |
instanceName | Name of the SQL instance. If you are specifying an instance other than the default instance, the name must be entered along with the client name, e.g. client1\instance1. |
vDITimeOut | The time the system must wait for the SQL server to become ready. Time value is in seconds. |
useVss | Option to enable/disable VSS. Valid values are True/False. |
overrideGlobalAuthentication | Option to override Global Authentication. Valid values are True/False. |
useLocalSystemAccount | Option to use Local Administrator
account. Valid values are 0/1. |
userName | Name of the database administrator who has permissions to perform jobs. |
password | Password for the database user |
propertyLevel | Properties to be retrieved for the instance.
Valid values are:
|
The following example shows how to add a parameter for a command:
Enable VSS |
To enable VSS for the instance, add the 'useVSS'
parameter to the following command: qoperation execute -af ModifySQL_Instance_Template.xml -appName 'SQL Server' -clientName client1 -instanceName client1\instance1 -useVSS true |
For detailed information on the instance parameters see Instance Properties.
qoperation execute –af CreateSQL_Subclient_Template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx –subclientName xxxxx -dataBackupStoragePolicy/storagePolicyName xxxxx -logBackupStoragePolicy/storagePolicyName xxxxx
qoperation execute –af GetSQL_Subclient_Properties_Template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx –subclientName xxxxx
qoperation execute –af ModifySQL_Subclient_Template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx –subclientName xxxxx
qoperation execute –af remove_subclient_template.xml -appName 'SQL Server' -clientName xxxxx -instanceName xxxxx -subclientName xxxxx
The following table displays all the parameters you can use with the commands mentioned in the above sections. To add a parameter to your command, use the following syntax: (Some examples are provided at the end of the table.)
qoperation execute -af <template XML file> -<parameter name> <value>
Parameter | Description of Parameter Values |
appName | Name of the application. In this case it would be 'SQL Server'. |
backupsetName | Name of the backupset. In this case, it would be 'defaultBackupSet'. |
clientName | Name of the client computer. |
subclientName | Name of the SQL subclient. |
instanceName | Name of the SQL instance. If you are specifying an instance other than the default instance, the name must be entered along with the client name, e.g. client1\instance1. |
enableBackup | Option to enable backup of Subclient. Valid values are True/False. |
encryptionFlag | Option to set the encryption points during
backups. Valid values are:
|
readBuffersize | This is used for tuning SQL Server performance.
Valid values are 64, 128, 256, 512, 1024, 2048 and 4096. |
onDemandSubClient | Use this option during creation of a subclient.
Valid values are True/False. |
networkAgents | Number of Network Agents. |
softwareCompression | Option to enable compression on the
Client or MediaAgent computer. Valid values are:
|
throttleNetworkBandwidth | Enhancing backup performance by reducing network
bandwidth overhead. Valid values are 0/1. |
storagePolicyName | Name of Storage Policy to be associated for Backup. |
enableDeduplication | Option to enable deduplication on subclient. Valid values are True/False. |
generateSignature | A component of deduplication performed on the
client or MediaAgent computer. Valid values are:
|
backupRules | Option to set Backup conversion rules. Valid
values are:
|
bufferCount | Number of data blocks to use during backups to improve performance. |
maxTransferSize | Maximum number of bytes to transfer at a time to
tune performance. Valid values in bytes are 65536, 131072, 262144, 524288, 1048576, 2097152, and 4194304. |
numberOfBackupStreams | Number of Backup Streams. |
numberOfTransactionLogStreams | Number of Transaction Log Streams. |
runPostBackup | Option to run a process after backup completes. Valid values are Yes/No. |
postBackupCommand | Path to the post process script that will be run after the backup. |
preBackupCommand | Path to the pre process script that will be run before the backup. |
runAs | Option to specify the user name who has permissions to run the pre/post
process scripts. Valid values are:
|
sqlSubclientType | Type of SQL subclient. Valid values are:
|
contentOperationType | Modification type. Valid values are
|
disableLogConsistencyCheck | Option to disable log consistency check.
Valid values are True/False. |
The following examples show how to add an parameter for a command:
Enable Backup |
To enable backup for a subclient, add the 'enableBackup' parameter to
the following command: qoperation execute –af ModifySQL_Subclient_Template.xml -appName 'SQL Server' -clientName client1 -instanceName client1\instance1 –subclientName subclient1 -enableBackup true |
Setting a Storage Policy |
To assign a storage policy to a subclient, add the 'storagePolicyName'
parameter to the following command: qoperation execute –af modify_storage_policy.xml -appName 'SQL Server' -clientName client1 -instanceName client1\instance1 –subclientName subclient1 -dataBackupStoragePolicy/storagePolicyName SPdata -logBackupStoragePolicy/storagePolicyName SPlog |
Creating an On Demand Database Subclient |
To create an on demand database subclient, add the 'onDemandSubClient'
and 'sqlSubclientType'
parameters to the following command: qoperation execute –af CreateSQL_Subclient_Template.xml –appName ‘SQL Server’ –clientName client1 –instanceName client1\instance1 –subclientName subclient1 –onDemandSubClient true –sqlSubclientType DATABASE –dataBackupStoragePolicy/storagePolicyName SPData –logBackupStoragePolicy/storagePolicyName SPLog |
Creating an On Demand File/FileGroup Subclient |
To create an on demand file/filegroup subclient, add the 'onDemandSubClient'
and 'sqlSubclientType'
parameters to the following command: qoperation execute –af CreateSQL_Subclient_Template.xml –appName ‘SQL Server’ –clientName client1 –instanceName client1\instance1 –subclientName subclient1 –onDemandSubClient true –sqlSubclientType FILE_FILEGROUP –dataBackupStoragePolicy/storagePolicyName SPData –logBackupStoragePolicy/storagePolicyName SPLog |
Adding Content to a Subclient |
To add content to a subclient, add the 'contentOperationType' parameter
to the following command: qoperation execute -af modify_subclient_content.xml -appName 'SQL Server' -clientName client1 -instanceName client1\instance1 -subclientName subclient1 -sqlSubclientType DATABASE -mssqlDbContent/databaseName DB1 -contentOperationType ADD |
Overwriting Content to a Subclient |
To overwrite content to a subclient, add the 'contentOperationType'
parameter to the following command: qoperation execute -af modify_subclient_content.xml -appName 'SQL Server' -clientName client1 -instanceName client1\instance1 -subclientName subclient1 -sqlSubClientType DATABASE -mssqlDbContent/databaseName DB1 -contentOperationType OVERWRITE |
For detailed information on the Subclient parameters see Subclient Properties.
Besides using the operating system's command line interface, you can also perform the command line operations from MSSQL Management Studio interface by integrating the commands within the SQL scripts. Registration of stored procedure and execution of commands should be done from SQL Server version 2008 or higher.
Use the following steps to run command line operations from SQL Management Studio:
In order to do this, make sure the trustworthy database property for the SQL database is set to ON.
You can login to the CommServe using an encrypted or open password.
@input = N'qlogin -u "admin" -clp "admin"',
Examples:
Creating an Instance |
DECLARE @return_value int, @response nvarchar(max) EXEC @return_value = [dbo].[cv_cmdshell] @input = N'qlist instance -c client1 -a Q_MSSQL', @response = @response OUTPUT IF RTRIM(@response) LIKE N'client1\instance1%' Print 'Instance already exists' ELSE Print 'Creating instance client1\instance1' EXEC @return_value = [dbo].[cv_cmdshell] @input = N'qoperation execute -af e:\MyFolder\createSQL_Template.xml -instanceName client1\instance1 -clientName client1', @response = @response OUTPUT where, e:\MyFolder is the location where the template XML is saved. |
Performing a full backup |
DECLARE @return_value int, @response nvarchar(max) EXEC @return_value = [dbo].[cv_cmdshell] @input = N'qoperation execute -af e:\MyFolder\full.xml -subClientName subclient1 -clientName client1 -instanceName client1\instance1 -ondemandinputfile C:\MyTest\myDBsContent.txt', @response = @response OUTPUT where, myDBsContent.txt is the content file that list the databases to be backed up and e:\MyFolder is the location where the template XML is saved. |
Perform a restore |
DECLARE @return_value int, @response nvarchar(max) EXEC @return_value = [dbo].[cv_cmdshell] @input = N'qoperation execute -af e:\MyFolder\restore_template.xml -clientName client1 -instanceName client1\instance1 -toTimeValue 2011-11-28 15:40:00 -restoreSource DB1 -database DB1', @response = @response OUTPUT where e:\MyFolder is the location where the template XML is saved and the –toTimeValue is the backup finish date. |
You can un-register the stored procedure from the SQL database by executing the below query:
There are several configurable properties available for your agent that can be modified from the agent, instance, or subclient level as per need.
It is recommended that you do not modify the properties of a subclient when a job is in progress for that specific subclient. If a job is in progress, either wait for the job to complete or kill the job from the Job Controller.
The following table describes the properties that can configured from the agent, instance, and subclient levels.
Option | Description | Related topics |
Change Storage Policies |
You can modify the
storage policies in any of the following situations:
You can change the storage policies from the subclient level.
|
Refer to Storage Policies. |
Rename a Subclient |
You can rename a subclient:
|
|
Data Transfer Options |
You can efficiently configure the available resources for transferring data secured
by data protection operations from the subclient level. This includes the following:
You can configure the data transfer options.
|
Refer to Data Compression and Network Bandwidth Throttling. |
View Data Paths |
You can view the data paths associated with the primary
storage policy copy of the selected storage policy or incremental storage
policy. You can also modify the data paths including their priority from
the subclient level.
|
|
Configure a Subclient for Pre/Post Processing of Data Protection |
You can add, modify or view Pre/Post processes for the
subclient. These are batch files or shell scripts that you can run before
or after certain job phases.
|
Refer to Pre/Post Processes. |
Configure Activity Control |
You can enable backup and restore operations from the agent and subclient level.
However, you can enable restore operations only from the agent level.
|
Refer to Activity Control. |
Configure User Security |
You can configure user security from the agent or subclient
level.
You can perform the following functions:
|
Refer to User Administration and Security. |
Enable/Disable Data Encryption |
You can enable data encryption from the suclient level.Encryption must be enabled at the client level prior to configuring any
instances residing on that client.
|
Refer to Data Encryption. |
View Software Version and Installed Updates |
The Version tab, at the Agent level displays the
software version of
the component.
|
|
CommCell Configuration Report |
The CommCell Configuration Report provides the properties of the
CommServe, MediaAgents, clients, agents, SRM agents, subclients, and
storage policies within the CommCell based on the selected filter
criteria.
|
Refer to CommCell Configuration. |
The following sections describe the steps involved in deleting an agent, instance, or subclient.
When you delete an instance or backupset, the associated data is logically deleted and you can no longer access the corresponding data from CommCell Console for recovery purposes. Refer to the troubleshooting article on Recovering Data Associated with Deleted Clients and Storage Policies for information on how to recover data if you accidentally delete an entity. |
You need to uninstall or DeConfigure the agent software from the client computer before deleting from CommCell Browser. After you delete the client software, you can either leave the corresponding data intact for appropriate action or you can remove the data immediately. If you choose to remove the data immediately, you must delete the agent from the CommCell Browser. If you delete the agent, all of the agent's data is irretrievably lost.
This operation will permanently delete the data backed up from this level and it cannot be restored.
Consider the following before deleting an instance:
Consider the following before deleting a subclient:
Click No to cancel the deletion and retain the subclient, or click Yes to continue the deletion.