What Is Application-Aware Backup?
When it comes to backups, you have a wide range of options to consider for successful and fast data recovery. The options include type of backup, frequency, source, destination, and many others. Nowadays, it’s no longer enough for backup solutions to simply capture files on a disk given that most organizations use servers to run applications. “Inconsistent” file backups are not adequate for optimal recovery time objectives and recovery point objectives.
We’ve previously covered “crash consistent” backups, which are consistent for a volume backed up by using Volume Shadow Copy Service (VSS). However, you can also create consistent application backups. Most virtual and physical machine backup solutions on the market offer an option for application-aware backups, making applications available immediately after a VM or physical machine is recovered.
Application-Aware Consistent Backups: How They Work
Application-consistent backups are backups that ensure all data within an application is captured in a consistent state, guaranteeing that this data can be restored fully without corruption or integrity issues. These backups go beyond simply copying files or disk data by coordinating with the application to quiesce (temporarily pause or stabilize) its operations and ensure all in-flight data is committed to disk before the backup process starts.
Application-consistent backups take crash-consistent backups a step further. While crash consistent backups create consistent backups of files on a volume by utilizing the Volume Shadow Copy Service, they are unaware of the application data that may be living in memory as well as any pending I/O operations. Application-aware backups leverage special hooks provided by the Volume Shadow Copy Service, called VSS writers.
VSS writers are special application-specific components of Microsoft’s Volume Shadow Copy Service. They serve the special role of making sure that application data is properly flushed from memory, frozen long enough for a VSS snapshot to be taken, and then unfrozen after the snapshot has taken place. The process typically only takes a few seconds.
This process is important for applications that require transactional consistency, such as Microsoft SQL Server. The Microsoft VSS writers for SQL Server are able to flush data from memory, freeze SQL operations, and then release the freeze after the snapshot has been taken. This ensures that data living in memory and pending data I/O operations are properly flushed and taken care of before the snapshot operation takes the VSS snapshot of the disk.
This makes the backup operation “application consistent” as not only the disk but also the application is backed up in a state where it maintains transactional consistency. If we only use a crash-consistent backup without the application-aware features turned on, there is a good chance we will be left with an application that is not in a consistent state.
Application consistency is not only a concern at the time the backup operation takes place. It is and should be a concern also for recovery operations. We can successfully restore an application server such as Microsoft Exchange Server or SQL Server from a crash-consistent backup. However, we have to use the process those applications call for to bring the application up to a consistent state. This may include replaying logs among other things. This is why the time to restore those servers will include not only the time to restore the raw file resources, whether it be a VMware or Hyper-V virtual machine or a physical machine, but also the time it takes to bring the application up to a consistent state.
Creating Application-Aware Backups with NAKIVO Backup & Replication
Below, we see an example of a backup job in NAKIVO Backup & Replication. You should select App-aware mode as well as how we want the job to proceed if it encounters errors with VSS on the Options page of the wizard.
If you hover your mouse over the “question mark” icon next to the app-ware box, you can see a detailed description of the app-ware process and the dependencies. Note the mention, in the case of VMware virtual machines, of VMware Tools. VMware Tools is used for guest OS quiescing for application data.
VSS Writers and Troubleshooting
As mentioned, VSS writers are the special purpose application-specific components of Microsoft’s Volume Shadow Copy Service. These can either be Microsoft or third-party components that are typically installed with the application itself. In the case of Microsoft VSS writer components, these can be seen installed on a specific application basis. Domain controllers will have the “NTDS” writer, SQL Server will have the “SqlServerwriter”, and Exchange Server will have the “Microsoft Exchange Writer”. When it comes to troubleshooting VSS writers if and when we have issues with application-aware backups, we have several utilities and resources available to us for troubleshooting.
A great command line utility to see and troubleshoot the specific VSS writers that are active, as well as the state of these writers, is the vssadmin command. If you open a command prompt and simply type vssadmin you can see the specific commands that are available with vssadmin.
Using the vssadmin list writers command, we can see the detailed list of the special VSS writers used for various applications. Notice below we ran the command on a Microsoft SQL Server. We have the SqlServerWriter listed. Also, note the State and the Last error notations as these are very helpful to see the current state and any errors that might be present on the particular VSS writer.
The next screenshot is taken from a Windows Server 2016 domain controller. Note that for this server, the vssadmin list writers command shows the NTDS writer, which is specific to Active Directory Domain Services.
As noted above, VMware Tools is an essential part of application-aware processing inside a virtual machine. By default, logging for VMware Tools is not turned on. When troubleshooting issues with VSS on a virtual machine, we may need to enable logging for VMware Tools inside the guest operating system. To do that, we need to edit or create the file tools.conf in our virtual machine.
Below, the file was not present in our Windows Server 2016 virtual machine. Navigating to the C:\Programdata\VMware\VMware Tools directory, we created the configuration file. Make sure to have your extensions displayed to correctly name the file.
After the file is created, we need to edit the file and add the following for the corresponding operating system. Note for Windows the double backslashes for the tools data path. You can also use one forward slash for Windows.
Windows guests
[logging]
log = true
vmtoolsd.level = debug
vmtoolsd.handler = file
vmtoolsd.data = c:\\windows\\temp\\vmtoolsd.log
Linux guests
vmtoolsd.data = /tmp/vmtoolsd.log
After creating/editing the file, simply restart the VMware Tools service.
The resulting log file contains debug information, including VSS operation information. Below is a snippet of a fairly clean debug log right after enabling the debug mode of VMware Tools.
Also, it is advisable that when troubleshooting any VSS related error you look in the Windows event logs, particularly the Application and System logs. Application log entries will appear as the source VSS and System log entries will appear under the source of volsnap. There are also specific VMware VSS troubleshooting steps to go through in looking at and troubleshooting VMware tools related VSS errors.
Conclusion
Using application-aware VM backups for business-critical applications is essential to having a solid disaster recovery strategy when backing up applications that rely on transactional consistency. Application-aware backups allow for all the data living in memory as well as pending I/O operations to be flushed to disk before the consistent backup of the disk is taken. This is accomplished through special Volume Shadow Copy Service writers that speak to the specific application and properly freeze operations long enough for the application to be backed up with transactional consistency. For organizations wanting to both protect data and be able to restore application data as quickly as possible, restoring application-aware backups bypasses the need to restore the application data separately such as replaying logs, etc. This ensures that organizations can deliver on both recovery point and recovery time objectives.
