If you create and administer backups, you will have heard the term Microsoft Volume Shadow Copy Service (VSS). You may even know that VSS runs in the background while a backup takes place. Knowing that is usually enough but if you’re customizing your backups or encountering VSS problems it helps to know more. So let’s take a closer look at VSS, the virtual shadow copy service.
Creating consistent backups with VSS
Backups can take a long time, which is a problem because the data you are backing can be changing while it is being backed up. This means the backup contains data as it was at different points in time–it is not consistent. This inconsistency causes problems for data, especially application data like databases that are constantly changing. VSS solves the inconsistent data problem by creating and maintaining a point–in–time snapshot of the volume to be backed up. The backup job can then use this snapshot to get copies of the data that have changed since the backup job started.
Show more
VSS can create snapshots with two levels of consistency:
- Crash consistent: A snapshot is created of the selected data›s volume. The snapshot does not include data that is changing or in memory, so data may be missing or inconsistent.
- Application consistent: The application being backed up checks its own files in the snapshot to make sure they are correct. For example, information in memory and uncompleted database transactions are included in the snapshot, making it more accurate and consistent.
Note: BackupAssist has a VSS application selection box that you can tick on the data selection screen. If you do not select this, and only select the application’s files, the application will not be aware of the backup and will not make it application consistent.
How VSS works
The VSS is service that works with 3 subsystems to create and maintain VSS snapshots. The subsystems are called the VSS requester, the VSS writer and the VSS provider. To understand VSS you will need to understand how these 3 subsystems work.
The VSS subsystems
- The VSS requester (like in BackupAssist) asks for the snapshot to be created and, when it’s no longer needed, asks for the snapshot to be removed. VSS requesters are built into VSS-Aware applications like backup software.
- A VSS writer allows applications to ensure that their own files (in a snapshot) are application consistent. VSS writers are built into VSS-aware applications such as Exchange and SQL.
- The VSS providers create and maintain snapshots, and come in two types: A software provider and a hardware provider. A VSS software provider that creates a snapshot on the same local disk as the live data. Microsoft operating systems ship with a software VSS provider. A VSS hardware provider that uses dedicated hardware to create a snapshot. 3rd party vendors can make hardware VSS providers for their devices, like a mirrored disk on a NAS that acts as the snapshot.
The VSS process
Now that you know about the subsystems used by VSS, let’s look at how they work together to create a snapshot. Specifically, we will look at a snapshot created and maintained by a Microsoft VSS provider. The Microsoft VSS provider normally creates a snapshot on the same volume as the live data. If the live data changes, a copy of the previous version of that data is saved to a location called the shadow storage. By doing this, the snapshot is able to maintain a point–in–time version of the live data, that only requires disk space equivalent to the data that has changed.
Show more
So now that you know how the snapshot is maintained, how does the backup job use it? Simple – The backup job asks the VSS service for the selected data and the following happens:
- If the data HAS changed, the snapshot gives the backup job a copy of the data as it was before it changed, from the shadow storage.
- If the data HAS NOT changed, the snapshot will provide the backup job with the data from the live volume.
Note: The process of copying live data before it changes causes a small performance reduction for volume writes. The longer a snapshot is needed, the larger it will become; the more data changes the faster the snapshot grows. The space available for the snapshots must be sufficient to store all changes to the data while the snapshot exists
The VSS process step by step
Step 1 The VSS requester (BackupAssist) tells the Microsoft VSS provider service what it needs and to prepare a snapshot.
Step 2 The VSS service tells the volumes VSS provider(s) what the snapshot requirements are.
Step 3 The Microsoft VSS provider will then make the snapshot and maintain it, as the live data changes.
Step 4 So that backups are application consistent, any assigned VSS writers will advise on live data changes to the snapshot.
Step 5 The snapshot is complete and ready to use, and control passes back to BackupAssist, the VSS requester.
The backup job can now begin requesting data from the VSS service, and create the backup.
Once a VSS aware application like Exchange Server knows that its data has been backed up, its VSS writer will perform some clean–up activities such as clearing its database transaction logs. This can free up disk space and speed up the application.
The backup job uses a variable path that points to both the snapshot and the volume the application is on. The VSS service will determine if the file requested by the backup is in the snapshot or the live data. BackupAssist never knows if the data it backed up came from the live data or the snapshot.
How System Protection backups use VSS
System Protection uses snapshots to produce consistent backups in the way described in the previous sections. However, there are two additional functions that System Protection can use a snapshot for.
Faster incremental image backups
System Protection creates a full image backup when it first runs. If the data selection and destination do not change, the next backup will be incremental. This is achieved by reviewing the data to be backed up and the data in the destination - to see what data changed. What has changed, is what is backed up.
Determining what data has changed consumes a lot of time but BackupAssist can do this quickly using snapshots. A snapshot knows what data has changed, because that is what it maintains. So, instead of deleting the snapshot after a backup, System Protection keeps the snapshot and uses it in the next backup to help identify that data that has changed. This process is called incremental reading.
Show more
To maintain a persistent snapshot for incremental reading select the BackupAssist Setting tab > Windows Settings > Enable image tracking. With incremental reading enabled, a 20 min incremental image backup could be completed in 2 min.
Note: This only works if the destination does not change and if the data selection does not change. For that reason, we advise that the same media is used at least for a week, so the performance gains can be achieved. System Protection cannot use incremental reading to enable fast incremental backups if the data source is a ReFS formatted drive.
Multiple restore points
The limitation of image backups is that each backup overwrites data that has changed, so you can only restore data from the previous backup. The only way to create a restore point for every backup would be to create a full image backup each time the job runs, and that would waste time and disk space.
This limitation is overcome by creating a persistent historical snapshot, on the backup destination. A historical snapshot is created before the backup jobs runs. This is a snapshot of the backup itself, so when the backup job then incrementally overwrites data in the backup, the historical snapshot maintains a copy of the data that was changed. This means that when a restore is performed, either the image backup or any of these historical snapshots (of data that changed) can be used to provide a restore point for each backup job that ran.
Show more
- A historical snapshot can reference the data of previous historical snapshots, meaning they only need to save data that has changed since the last backup job ran.
- The shadow storage space used by the snapshot has a size limit. When the limit is reached, the data referenced by the oldest historical snapshot is deleted.
Note: The historical snapshots are persistent snapshots so the shadow storage space will grow. For this reason it is important that the destination is only used for these backups.
Note: It is important to understand that the historical snapshots are stored on the same disk as the backup. The space on the disk used by the snapshot to store data is called the volume shadow copy storage. When the amount of disk space allocated to the shadow copy storage reaches its limit, it will delete the oldest data to make room for new data.
For this reason, you should:
- Only use this destination for backups
- Set the disk space allocated to your shadow copy to a size that allows for it to grow as data is added.
- Only backup destinations that are seen as local devices by the computer you’re backing up can support historical snapshots. These destinations include local hard drives, iSCSI targets, external hard disk drives (that are directly attached) and BackupAssist Data containers.
To learn more about VSS, see our blog articles:
Making VSS application backups. Read the article.
Making VSS application consistent backups. Read the article.