Datto Linux Agent: Troubleshooting DLAD failures
Topic
This article provides resources to help troubleshoot failures of the Datto Linux Agent's DLAD service.
Environment
-
Datto Linux Agent
Description
The core of the Datto Linux Agent is a service called dlad. You can check if it is running by running the following on the protected machine:
ps -ef | grep dlad
You should see a response similar to the following, if DLAD is not running, the line will be missing from the response.
EXAMPLE root 24120 1 0 15:34 ? 00:00:00 /usr/sbin/dlad
Commands
The service can be interacted with using various commands, depending on distribution.
systemctl <command> dlad
service dlad <command>
/etc/init.d/dlad <command>
The common commands that would be useful are:
- status
- stop
- start
- restart
If DLAD fails to start, some types of errors can be reported by the service or the /etc/init.d
script.
You can see problems like missing files or permissions issues due to a bad install when you try and start the service.
If the service
command appears to work, but dlad is not running, then it is possible that the service started up and quickly exited. Check the dlad.sqlite
logs for the reason.
If the dlad.sqlite file doesn't exist, or isn't informative enough, check /var/log/syslog
or journalctl
. Some startup failures will make it to the syslog and can be identified there.
An error message similar to the following will indicate that DLAD is already running:
Mar 16 16:25:16 we dlad[27245]: Could not lock pid file: File already locked
Newer versions of DLAD check for this problem and will say:
Could not start Datto Linux Agent. An instance of the service is already running.
If you don't see anything there, another option is to run DLAD in debug mode:
sudo /usr/sbin/dlad -d -l1
This command will cause the DLAD process to run in the terminal, and any debug output available will be printed there. The output should point to the problem. If the daemon doesn't exit on its own, pressing CTRL+C will force it to exit. Once the problem is resolved, you should start DLAD normally with the previously listed options.
It is possible that another service or instance of DLAD is running and listening on port 25567. Dlad then fails because the other process has the port locked. To check this, use the netstat command.
netstat -nap | grep 25567
tcp 0 0 0.0.0.0:25567 0.0.0.0:* LISTEN 16989/dlad
This shows that the DLAD process (process id 16989) is bound to port 25567, and is already running. If you see a different process name there, then you will need to stop the program, or configure it to use a different port, so that 25567 is free for DLAD to bind to.
The installation process creates all the links and directories and installs all the files DLAD needs to be able to start. If this was interrupted, the folders can be created manually if needed. To first confirm if they are present list the contents of the directory:
ls -la /etc/datto/dla
The output should include the following folders at a minimum:
drwxr-xr-x 8 smark smark 4096 Mar 15 16:29 .
drwxr-xr-x 4 smark root 4096 Sep 8 2016 .
.
drwxrwxrwx 3 root root 4096 Jan 12 12:52 certs
drwxr-xr-x 2 root root 4096 Mar 13 15:01 pps
If DLAD has ever run, this file will exist also:
-rw-r--r-- 1 root root 5474304 Mar 15 16:29 dlad.sqlite
If the directories /etc/datto/dla/certs
and pps
don't exist, you can manually create them, and then attempt to start the service again.
After upgrading a Datto Linux Agent installation from version 1.0 to 2.0, backups fail, and the dlad service on the protected machine is unable to start. This can be caused by an out-of-date version of gnutls is preventing the service from starting.
To confirm the issue, from a Linux command prompt on the protected machine, run:
sudo /usr/sbin/dlad -d
You should see the following error message:
[CRITICAL:] HTTP server failed to start with err: 0
Once you've verified the error, update the gnutls package to the latest version for your Linux distribution.
NOTE Datto has not determined how many distributions of Linux or versions of gnutls will experience this issue. If you see "HTTP server failed to start with err: 0" after running dlad -d, you should update the gnutls package as a first step.