Best-Practices: Screenshot Verification

Topic

This article explains how screenshot settings can affect the behavior of your Datto appliance, and what the best practices are for configuring these settings. You can learn more about screenshot verification in the Introduction to Screenshot Verification article.

Environment

  • Datto SIRIS
  • Datto ALTO

Description

Screenshot Scheduling and the Backup Schedule

Datto appliances have several options for scheduling screenshot verification on an agent's snapshots. These settings are under the Protect > Configure Agent Settings menu for each agent.


Figure 1: Screenshot schedule options

The available selections are tied to the agent's backup schedule. The default schedule setting is All Offsite Points; do not use this option if your Datto appliance does not replicate offsite.

Based on the selection made, the screenshot scheduler process will add the corresponding snapshot to the screenshot queue. For example, the First point of the day option will tell the screenshot scheduler to queue screenshot verification on the first scheduled backup of the day.

The automated screenshot scheduler will only queue backups taken by the automated backup scheduler. If a scheduled backup for a screenshot verification is missed, that screenshot verification will not queue, and will be considered missed by the agent.

If your device has offsite replication service, Datto recommends using the default screenshot scheduling option of All Offsite Points to ensure that you have more than one bootable recovery point available in the Datto Cloud.

Additional Wait Time

The Additional Wait Time setting determines how long the Datto appliance waits to take a screenshot after it detects a ready state in the test virtualization. This is useful for agents that take a long time to boot.


Figure 2: Additional wait time

Ready state refers to criteria detected on a running VM that indicates it has booted to an operating system. If the Datto appliance does not detect a ready state, it waits for 5 minutes, and then checks for a ready state again. When the ready state detection step ends, the Datto appliance waits for the period of time specified in the Additional Wait Time setting before taking the screenshot. The process is similar to the following example:

1. TEST VM BOOTS
2. READY STATE DETECTION BEGINS
3. READY STATE DETECTED? (Wait up to 5 minutes if no)
4. WAIT X SECONDS (Additional wait time)
5. TAKE SCREENSHOT

You should set the Additional Wait Time to the lowest possible value necessary to capture a successful boot.

Screenshot Has Not Occurred Notification

This reporting feature will send an error email if a screenshot does not run within a specific window of time. Setting the error threshold determines how much time the appliance allows to pass without a screenshot being run for an agent.


Figure 3: Screenshot has not occurred notification

Datto recommends setting the Error Threshold value to the number of hours between scheduled screenshots + 12. For example, if screenshots are scheduled for 6 PM every day, the time elapsed would be 24 hours + 12, or 36 hours. This is because at 6 PM, the backup won't exist, and the screenshot will be unable to start, resulting in an automatic failure state. More time may be necessary for devices with many more agents.

You can also choose to receive email notifications about the success or failure of screenshots. See our Screenshot Verification Settings article for more information.

Screenshot Limitations

When a device is protecting many agents, setting custom schedules becomes essential for allowing screenshot verification to complete in an orderly fashion.

Total Verifications

A Datto appliance can perform between 22 and 240 verifications per day. No more than 25 screenshot verification sessions can be in the device queue at the same time.

At most, 10 agents can be verified per hour, because of the 5 - 6 minute minimum time necessary to complete the operation, for a theoretical total of 240 verifications a day. But due to a number of variables, including backups completing at slightly different times and scheduler processing, this number is often much lower.

If all agents need to have their Additional Wait Time set to 60 minutes, with the default 5-minute delay, each verification would take 65 minutes, for a total of 22 verifications per day.

Because of these considerations, Datto recommends spreading out the screenshot verification schedule. Try to evenly distribute screenshot sessions so that each agent is scheduled for a different hourly block. If you have more than 24 agents, begin by scheduling 2 verifications per block, then 3 per block and so forth. See the Full Verification Pass section of this article for considerations when devices need more than 24 hours to verify all of the agents.

Full Verification Pass

A full verification pass represents the total time needed to complete verification for all of a device's agents.

To calculate this value, configure all of your agents to the lowest additional wait time they need to capture a successful boot. Total all of the agents' additional wait time values, and add another 5 minutes per agent. The result is the total time your appliance will need for a full verification pass.

As an example, if you have 35 agents with 2 minutes of additional wait time each:

(35 x 2) + (35 x 5) = 70 + 175 = 145 Minutes

In this example, you would not need to set up a complex schedule, as the day's verification completes quickly.

However, if this result is greater than 24 hours, then you will encounter a situation where agents add the next day's backups to the screenshot queue before the queue has finished. This can result in “Screenshot has not occurred alerts.

To fix this, you will need to set up a custom schedule that reflects this total time.

For example, if the total time is 48 hours, set a custom schedule for all agents to screenshot every other day, and configure the Screenshot Has Not Occurred Notification to 60 hours. If configured correctly, all agents will be scheduled to take a single screenshot within the window of the full verification pass.

Windows updates

Pending windows updates can cause screenshots to fail. You should install any Windows updates that have been downloaded and are pending install on your system.