Understanding the retention process

Topic

This article explains the function of the retention feature on Datto appliances.

Environment

• Datto SIRIS
• Datto ALTO
• Datto NAS
• Datto Cloud

Description

Understanding Datto's Inverse Chain backup technology is key to understanding retention settings on a Datto appliance.

The Inverse Chain Technology engine is sparse, efficient, reliable, and virtualizes in an instant. Each backup creates a recovery point on the Datto device. Incremental backups contain only the data that is unique to that snapshot's point in time. In other words, the data that has changed since the last incremental backup. For most backup engines, this is where the innovation ends. If part of a recovery point becomes lost due to data corruption, all of the recovery points after that point are useless, because they depend on a broken piece of the chain. The Datto Inverse Chain does not have this limitation. If one recovery point is lost, the recovery points after it refer to data that is elsewhere in the chain, and can rebuild themselves in an instant.

Local Retention

The local retention process runs at the end of every backup and nightly as part of Datto scheduled operations. You can also force retention to run the process in between backups. When retention runs, specific recovery points get deleted from the chain. Think of it as data pruning.

Since each backup point only contains the data that is unique to itself, this may not remove as much data as you might think, because the device moves much of the information to other recovery points rather than deleting it.

Retention only removes recovery points not flagged for offsite replication and ones that have successfully synced offsite. Any points queued for replication which have not synchronized with the cloud will not be removed. The Datto device will also not remove any recovery points that it requires to keep the integrity of the backup chain intact. See Deleting local and cloud data for more information.

Agents with a backlog of points pending replication might appear to use more space on the Datto appliance than normal until their offsite synchronization is current.

• Paused agents will continue to run retention according to a nightly job schedule, custom retention settings should be scheduled accordingly.
• Local retention will not run on archived agent datasets.
• Cloud Retention will not run on archived agent datasets. Archived agents remain subject to the Offsite Retention Policy associated with the Active Service Subscription on the device, and Datto may delete the archived agent from the Datto Cloud accordingly. See Datto Cloud Service Plan retention schedules for more information.

Retention is configured in the Device Web under Protect > Configure Agent Settings and can be changed at any time.

IMPORTANT  Available retention settings can vary depending on your billing policy. Each policy has different features, and the choice of a policy depends on your circumstances.

Intra-Daily to Dailies

Pruning of intra-daily backups removes all but one snapshot that day and leaves one point from that day in the chain for recovery purposes. This remaining backup is then the daily backup. Which point remains depends on when retention runs, but it is usually either the first point or the last point of the day.

In the above example, intra-daily backups will have pruning run on them after seven days. Up to the seven-day mark, all backups for all days remain available. After seven days, only one backup from the pruned period remains available; this point is the last backup of each day.

Dailies to Weekly

After one week (default), the daily recovery points created in the previous step prune down to one weekly recovery point; this point is the last backup of each week.

Weeklies to Monthly

After one month (default), the daily recovery points created in the previous step prune down to one monthly recovery point; this point is the last backup of each month.

Keep Local Backups

This setting schedules when to delete all local recovery points. The default local is to keep the monthly recovery points for three months.

IMPORTANT  When retention is run, the backup we keep for that interval will only include the data that existed on the server at the time that retained backup was initially taken. If a file did not exist on the protected system at the time of that backup, that file will not be recoverable.

Cloud Retention

Except where noted, for this article, the term 'Cloud' applies to both the Datto Cloud and Private Cloud storage.

Cloud Retention (sometimes called 'Offsite Retention') works much in the same way as local retention. When the retention runs, the default action for a protected system with no changes to its offsite sync or retention policies is to preserve consistent daily points for the last backup of the day, both locally and in the cloud.

• Cloud Retention runs on a regular schedule: every day at 2:00 AM EST.
• The Cloud Retention process deletes 300 recovery points at a time. Therefore, it may take some time to consolidate very large data sets.
• Retention will not run on archived datasets, however retention will continue to run on paused agent data.

With the default settings on a Datto device, only one recovery point per day replicates to the cloud. Therefore, there is no Intra-Daily to Daily retention. The result is that the only setting that affects the size of offsite data points is the Keep backups in the cloud for... option. The following shows the full Cloud Retention settings (Service plan will effect the available options):

• If you have purchased Infinite Cloud Retention, but you have specific protected systems which must retain data only for a set period, you can still customize the total retention setting for your agents. However, you can only do so within the limits of the service plan and at the level of granularity which offsite retention permits.
  

• If you have purchased Time-Based Retention, you will not be able to modify its cloud retention settings.

  NOTE  For more information about these plans, see the Datto Business Continuity and Disaster Recovery (BCDR) Product Terms of Use: 6. BCDR Retention Schedules and Termination document on the Datto Partner Portal, or contact your Datto Sales Executive.