Getting started with agentless backups
Topic
This article describes system requirements for agentless backups to a Datto appliance.
Environment
- Datto SIRIS
- Agentless backups
Description
System requirements
| Virtual OS |
Linux OS Compatability Linux OS compatibility is the same as the Datto Linux Agent. Hardware Independent Restore (HIR) is not supported for EFI Linux whether agentless of agent based. See Getting started with the Datto Linux Agent for details and a list of supported Linux distributions. For compatibility information about operating systems not listed here, see the Warnings section of this article. |
| Virtual hardware requirements |
|
| Network |
Communication between the virtual environment and the Datto appliance requires the following ports: On both the Host and the Datto Appliance:
On the Datto Appliance:
In addition, see Unified Backup Networking & Bandwidth Requirements article for general networking requirements. |
| Hypervisor |
Agentless backups run on physical SIRIS and Virtual SIRIS powered by vSphere versions 6.0, 6.5, 6.7, 7.0, and 8.0.
. |
Overview
Physical and virtual Datto SIRIS appliances can take agentless backups of virtual machines running VMware vSphere. Agentless backups use the host hypervisor's capabilities to take a snapshot of the production VM (even when shut down).
NOTE The agentless backup solution is not available for Datto ALTO or Datto NAS appliances.
You do not need to install a backup agent on the target machine for agentless backups to communicate with the Datto appliance. See the Backup Process section of this article for a technical overview of how agentless backups work.
- Communication from VM targets to Datto appliances is quicker and easier.
- Backups, done through VMware's data storage API (VADP) (external link), are more efficient.
- Agentless backups work on both Windows and Linux virtual machines.
- You can run a backup even when a VM is powered off.
- Other common SIRIS functions remain the same.
- If you prefer, you can still back up your machines using an agent-based solution. (see this article for crucial agent-based deployment considerations).
Limitations
SIRIS Virtual and agentless backups do not support the free version of vSphere.
- Agentless backups do not work on encrypted VMs
- Agentless backup support is unavailable for:
- spanned volumes
- ESXi shared volumes
- multiple volumes using the same GUID
- systems using Logical Volume Management (LVM)
- backup of deduplicated volumes is untested and may produce inconsistent results.
- file restore of deduplicated volumes is not possible. Restoration of deduplicated volumes through virtualization, Direct Restore, or Bare Metal Restore, is not supported.
- volumes formatted using Microsoft's ReFS.
- Windows systems that contain volumes that use Dynamic Disk Technology (These systems may have issues regardless of whether the Dynamic Disk volumes are included in the backup or not).
- Backup of User Profile Disks used by the Remote Desktop Service is unsupported. Remote Desktop Service (external link). These volumes should be excluded from backups.
Universal VM Backup may be a viable alternative in the above scenarios.
Agentless backups from a physical device (and failover for Virtual SIRIS) operate through the Network Block Driver Transport Method, which restricts their operating speed to a maximum of 40% of the management network interface speed.
Backup process
Unlike agent-based backups, which are VSS-dependent, in an agentless backup, the Datto appliance interacts directly with your hypervisor to snapshot and back up a virtual machine. It does so by taking the following steps:
- The Datto appliance uses the hypervisor connection to connect to the vSphere environment.
- The VMware snapshot provider (part of VMware tools) takes a snapshot of the VM.
The first two steps also occur when the Datto pairs with the target VM. - If the Datto device is a virtual appliance, it can use HotAdd (through vddk-fuse) to directly attach the VMDK from the VMware snapshot to itself. HotAdd transfers the backup data over the vSphere management network. As a failover, or if the Datto device is a physical appliance, the Datto appliance connects to the VMDK from the VMware snapshot using the Network Block Device (NBD) protocol. With this type of connection, the VM network transfers the data.
- The Datto appliance uses libguestfs to analyze the disk image to get information about the disk structure and the file system(s) on it.
- The appliance transfers the backup data from the VMware snapshot of the VMDK to itself, and takes a ZFS snapshot of the disk images in the live dataset, just like an agent-based backup.
Warnings
IMPORTANT Operating systems that are not compatible with Datto's standard agentless backup method will be protected by the Universal VM Backup solution. Universal VM Backup is a limited agentless solution that allows you to restore via Export Image or Virtualize via hypervisor methods only.
If you add a new volume to a protected agentless system after it has already been backed up by the Datto appliance, you must reboot the protected host before the new volume can begin backups.
Installation
Before you can take an agentless backup, you need to connect the Datto appliance to your hypervisor. For hypervisor connection steps, see the article Connecting A Datto Appliance To A VMware Hypervisor.
To pair a virtual machine for backup on the Datto appliance, see the Pairing a Target System for Agentless Backups article.
The host hypervisor and protected virtual machines must be in a healthy, stable state. For more information, see Agentless Backups: Best Practices.
Frequently asked questions
Are the backups saved on the Datto device thin or thick provisioned?
Agentless backups transfer to the Datto appliance as a sparse image. This type of disk image file is similar to a thin-provisioned VMDK; it takes up only as much disk space as the size of its stored contents. Unlike a thick-provisioned VMDK, a sparse image file should not take up the full provisioned size of the virtual disk.
Sparse images grow in size as the user adds data. Over time, as files are deleted or overwritten by the production VM, both a thin-provisioned disk and a sparse image file will grow because VMware cannot tell that these files have been removed or altered. Although the VM marks these blocks as free in the filesystem they will copy into the backup image.
EXAMPLE A 100 GB VMDK using 30 GB, but undergoing an additional 50 GB of disk change, could produce a backup of 80 GB. See VMware's article about Growing, thinning, and shrinking virtual disks for VMware ESXi and ESXi (external link) for information about resizing these types of disks.
Do agentless backups have any size limitations for the VMDK hosted on the hypervisor?
Virtual Datto SIRIS devices: When backing up a protected virtual machine, virtual Datto devices use the HotAdd VDDK protocol, which attaches the protected machine's disks directly to the Datto device. The amount of free disk space on the appliance's array limits the virtual disk size. Virtual Datto appliances will also failover to the NBD protocol if hot-add fails.
Physical Datto SIRIS devices: These devices use the NBD VDDK protocol. With NBD, VMware recommends using virtual disks that are no larger than 1 TB in size.
The VMkernels primary function is to orchestrate VM processes. While using the NBD protocol, the VMkernel will automatically cap each session for stability. This cap can result in lower backup throughput; this is so that NBD transfers do not bottleneck VM management and other VMkernel traffic.
To counteract bottlenecks, design your backup policies and job configurations to distribute the load over multiple ESXi hosts instead of running numerous backup jobs from the same ESXi host simultaneously.
How does the agentless solution take incremental backups without agent software running in the protected VM's guest OS?
Agentless backups use the VMware Changed Block Tracker (CBT), which keeps track of changed or newly-written blocks within a virtual machine. Every time a new backup task begins, the Datto solution uses the VMware CBT values to detect disk change and only back up the changed data.
A power failure or hard shutdown while a virtual machine is powered on could lead to CBT losing track of incremental backups. For additional information on CBT, see Changed Block Tracking (CBT) on virtual machines (external article) in the VMware Knowledge Base.
If a virtual machine's hardware is version 6 or older, or the CBT is corrupt, every backup for that machine will be a full backup until you upgrade the hardware version or repair the CBT.
What else can cause a full backup on a virtual machine?
Code issues in unpatched versions of ESXi 5.5 can cause frequent full backups. Datto recommends keeping your hypervisor versions patched and up-to-date to help avoid these issues.
Third-party agentless solutions running alongside Datto's agentless solution on the same production machine can cause full backups by corrupting the CBT. Datto does not recommend running multiple agentless backup utilities on a production machine at the same time.
How does the agentless solution take application-aware backups without agent software installed in the protected virtual machine's guest OS?
The Datto solution uses the quiesced snapshots VMware feature. When taking a quiesced snapshot, VMware pauses, then writes to the virtual machine's virtual disk to achieve a consistent state.
The Datto solution uses the virtual production machine's VSS writers to back up a Windows guest OS. If a quiesced snapshot of a VM fails to complete, a backup job will not run.
I am trying to set up a hypervisor connection to my vSphere cluster from my SIRIS. Should I connect to an individual ESXi host or the vCenter Management Server?
VMware has two values that it uses to assign each virtual machine:
- a unique identifier (vmID). vmIDs are unique at the ESXi host level.
- the Managed Object Reference ID (MoRefID). MoRefIDs are unique across an entire vCenter cluster.
You can tell the difference between a MoRefID and a vmID by their format.
The prefix vm- will precede a MoRefID. For example, vm-9463.
A vmID will not have a prefix.
If using a clustered environment: you must connect to the vCenter management server.
If using a standalone connection to an individual ESXi host: the vmID of any vMotioned virtual machine will change. Because the standalone connection only uses the vmID, backups will fail until you determine the replacement vmID value and reassociate the backup chain to it.
If you start agentless backups via a standalone connection, then switch to a vCenter cluster connection, you must change all vmIDs to MoRefIDs on the back-end of the device to get backups running again.
Are VMware snapshots a good backup strategy? Should I take some manually and keep them around just in case?
Snapshots are useful as a secondary backup method for short-term or ad-hoc backups.
While snapshots are growing in size, the Datto appliance will lock the entire LUN on which a VM resides, thereby slowing down I/O performance for that VM and all other VMs sharing the same LUN.
Keeping snapshots for an extended period will impact the performance of your production VMs.
Consolidating a virtual disk that has long-term snapshots takes a very long time, and can impact the I/O performance of that VM.
By default, the snapshots Datto appliances take persist for only as long as it takes to back up any given target virtual machine.
Why do the backups appear to be larger than I would expect?
Due to how our systems handle Agentless backups the information being read and written may be larger than expected. This is less noticeable in standard Agentless backups as the backup compression will shrink the size of the snapshot. This is more noticeable in Encrypted Agentless backups as there is little to no compression shrinking the snapshot size.
Additional Resources
- TCP and UDP Ports required to access VMware vCenter Server, VMware ESXi and ESX hosts, and other network components (external link)
- Agentless backups: Best practices
- Virtual Disk Transport Methods (external link)
- Changed Block Tracking (CBT) on virtual machines (external link)
- Changed Block Tracking is reset after a storage vMotion operation in vSphere 5.x (external link)
