README.md

Agent claiming

Agent claiming allows a Netdata Agent, running on a distributed node, to securely connect to Netdata Cloud. A Space's administrator creates a claiming token, which is used to add an Agent to their Space via the Agent-Cloud link (ACLK).

Are you just starting out with Netdata Cloud? See our get started with Cloud guide for a walkthrough of the process and simplified instructions.

Claiming nodes is a security feature in Netdata Cloud. Through the process of claiming, you demonstrate in a few ways that you have administrative access to that node and the configuration settings for its Agent. By logging into the node, you prove you have access, and by using the claiming script or the Netdata command line, you prove you have write access and administrative privileges.

Only the administrators of a Space in Netdata Cloud can view the claiming token and accompanying script generated by Netdata Cloud.

The claiming process ensures no third party can add your node, and then view your node's metrics, in a Cloud account, Space, or War Room that you did not authorize.

By claiming a node, you opt-in to sending data from your Agent to Netdata Cloud via the ACLK. This data is encrypted by TLS while it is in transit. We use the RSA keypair created during claiming to authenticate the identity of the Agent when it connects to the Cloud. While the data does flow through Netdata Cloud servers on its way from Agents to the browser, we do not store or log it.

You can claim a node during the Cloud onboarding process, or after you created a Space by clicking on the USER's Space dropdown, then Manage claimed nodes.

There are two important notes regarding claiming:

• You can only claim any given node in a single Space. You can, however, add that claimed node to multiple War Rooms within that one Space.
• You must repeat the claiming process on every node you want to add to Netdata Cloud.

How to claim a node

To claim a node, select which War Rooms you want to add this node to with the dropdown, then copy and paste the script given by Cloud into your node's terminal. Hit Enter.

sudo netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud

The script should return Agent was successfully claimed.. If the claiming script returns errors, or if you don't see the node in your Space after 60 seconds, see the troubleshooting information. If you prefer not to use root privileges via sudo to run the claiming script, see the next section.

Repeat this process with every node you want to add to Cloud during onboarding. You can also add more nodes once you've finished onboarding.

Claim an agent without root privileges

If you don't want to run the claiming script with root privileges, you can discover which user is running the Agent, switch to that user, and run the claiming script.

Use grep to search your netdata.conf file, which is typically located at /etc/netdata/netdata.conf, for the run as user setting. For example:

grep "run as user" /etc/netdata/netdata.conf 
    # run as user = netdata

The default user is netdata. Yours may be different, so pay attention to the output from grep. Switch to that user and run the claiming script.

netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud

Hit Enter. The script should return Agent was successfully claimed.. If the claiming script returns errors, or if you don't see the node in your Space after 60 seconds, see the troubleshooting information.

Claim an Agent running in Docker

The claiming process works with Agents running inside of Docker containers. You can use docker exec to run the claiming script on containers already running, or append the claiming script to docker run to create a new container and immediately claim it.

Running Agent containers

Claim a running Agent container by appending the script offered by Cloud to a docker exec ... command, replacing netdata with the name of your running container:

docker exec -it netdata netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud

The script should return Agent was successfully claimed.. If the claiming script returns errors, or if you don't see the node in your Space after 60 seconds, see the troubleshooting information.

New/ephemeral Agent containers

Claim a newly-created container with docker run ....

In the example below, the last line calls the daemon binary, sets essential variables, and then executes claiming using the information after -W "claim... . You should copy the relevant token, rooms, and URL from Cloud.

docker run -d --name=netdata \
  -p 19999:19999 \
  -v netdatalib:/var/lib/netdata \
  -v netdatacache:/var/cache/netdata \
  -v /etc/passwd:/host/etc/passwd:ro \
  -v /etc/group:/host/etc/group:ro \
  -v /proc:/host/proc:ro \
  -v /sys:/host/sys:ro \
  -v /etc/os-release:/host/etc/os-release:ro \
  --restart unless-stopped \
  --cap-add SYS_PTRACE \
  --security-opt apparmor=unconfined \
  netdata/netdata \
  -W set2 cloud global enabled true -W set2 cloud global "cloud base url" "https://app.netdata.cloud" -W "claim \
  -token=TOKEN \
  -rooms=ROOM1,ROOM2 \
  -url=https://app.netdata.cloud"

The container runs in detached mode, so you won't see any output. If the node does not appear in your Space, you can run the following to find any error output and use that to guide your troubleshooting. Replace netdata with the name of your container if different.

docker logs netdata 2>&1 | grep -E --line-buffered 'ACLK|claim|cloud'

Claim through a proxy

A Space's administrator can claim a node through a SOCKS5 or HTTP(S) proxy.

You should first configure the proxy in the [cloud] section of netdata.conf. The proxy settings you specify here will also be used to tunnel the ACLK. The default proxy setting is none.

[cloud]
    proxy = none

The proxy setting can take one of the following values:

• none: Do not use a proxy, even if the system configured otherwise.
• env: Try to read proxy settings from set environment variables http_proxy/socks_proxy.
• socks5[h]://[user:pass@]host:ip: The ACLK and claiming will use the specified SOCKS5 proxy.
• http://[user:pass@]host:ip: The ACLK and claiming will use the specified HTTP(S) proxy.

For example, a SOCKS5 proxy setting may look like the following:

[cloud]
    proxy = socks5h://203.0.113.0:1080       # With an IP address
    proxy = socks5h://proxy.example.com:1080 # With a URL

You can now move on to claiming. When you claim with the netdata-claim.sh script, add the -proxy= parameter and append the same proxy setting you added to netdata.conf.

sudo netdata-claim.sh -token=MYTOKEN1234567 -rooms=room1,room2 -url=https://app.netdata.cloud -proxy=socks5h://203.0.113.0:1080

Hit Enter. The script should return Agent was successfully claimed.. If the claiming script returns errors, or if you don't see the node in your Space after 60 seconds, see the troubleshooting information.

Troubleshooting

If you're having trouble claiming a node, this may be because the ACLK cannot connect to Cloud.

With the Netdata Agent running, visit http://NODE:19999/api/v1/info in your browser, replacing NODE with the IP address or hostname of your Agent. The returned JSON contains four keys that will be helpful to diagnose any issues you might be having with the ACLK or claiming process.

	"cloud-enabled"
	"cloud-available"
	"agent-claimed"
	"aclk-available"

Use these keys and the information below to troubleshoot the ACLK.

Claiming on older distributions (Ubuntu 14.04, Debian 8, CentOS 6)

If you're running an older Linux distribution or one that has reached EOL, such as Ubuntu 14.04 LTS, Debian 8, or CentOS 6, your Agent may not be able to securely connect to Netdata Cloud due to an outdated version of OpenSSL. These old versions of OpenSSL cannot perform hostname validation, which helps securely encrypt SSL connections.

We recommend you reinstall Netdata with a static build, which uses an up-to-date version of OpenSSL with hostname validation enabled.

If you choose to continue using the outdated version of OpenSSL, your node will still connect to Netdata Cloud, albeit with hostname verification disabled. Without verification, your Netdata Cloud connection could be vulnerable to man-in-the-middle attacks.

cloud-enabled is false

If cloud-enabled is false, you probably ran the installer with --disable-cloud option.

Additionally, check that the enabled setting in var/lib/netdata/cloud.d/cloud.conf is set to true:

[global]
    enabled = true

To fix this issue, reinstall Netdata using your preferred method and do not add the --disable-cloud option.

cloud-available is false

If cloud-available is false after you verified Cloud is enabled in the previous step, the most likely issue is that Cloud features failed to build during installation.

If Cloud features fail to build, the installer continues and finishes the process without Cloud functionality as opposed to failing the installation altogether. We do this to ensure the Agent will always finish installing.

If you can't see an explicit error in the installer's output, you can run the installer with the --require-cloud option. This option causes the installation to fail if Cloud functionality can't be built and enabled, and the installer's output should give you more error details.

You may see one of the following error messages during installation:

• Failed to build libmosquitto. The install process will continue, but you will not be able to connect this node to Netdata Cloud.
• Unable to fetch sources for libmosquitto. The install process will continue, but you will not be able to connect this node to Netdata Cloud.
• Failed to build libwebsockets. The install process will continue, but you may not be able to connect this node to Netdata Cloud.
• Unable to fetch sources for libwebsockets. The install process will continue, but you may not be able to connect this node to Netdata Cloud.
• Could not find cmake, which is required to build libwebsockets. The install process will continue, but you may not be able to connect this node to Netdata Cloud.
• Could not find cmake, which is required to build JSON-C. The install process will continue, but Netdata Cloud support will be disabled.
• Failed to build JSON-C. Netdata Cloud support will be disabled.
• Unable to fetch sources for JSON-C. Netdata Cloud support will be disabled.

One common cause of the installer failing to build Cloud features is not having one of the following dependencies on your system: cmake and OpenSSL, including the devel package.

You can also look for error messages in /var/log/netdata/error.log. Try one of the following two commands to search for ACLK-related errors.

less /var/log/netdata/error.log
grep -i ACLK /var/log/netdata/error.log

If the installer's output does not help you enable Cloud features, contact us by creating an issue on GitHub with details about your system and relevant output from error.log.

agent-claimed is false

You must claim your node.

aclk-available is false

If aclk-available is false and all other keys are true, your Agent is having trouble connecting to the Cloud through the ACLK. Please check your system's firewall.

If your Agent needs to use a proxy to access the internet, you must set up a proxy for claiming.

If you are certain firewall and proxy settings are not the issue, you should consult the Agent's error.log at /var/log/netdata/error.log and contact us by creating an issue on GitHub with details about your system and relevant output from error.log.

Remove and reclaim a node

To remove a node from your Space in Netdata Cloud, delete the cloud.d/ directory in your Netdata library directory.

cd /var/lib/netdata   # Replace with your Netdata library directory, if not /var/lib/netdata/
sudo rm -rf cloud.d/

This node no longer has access to the credentials it was claimed with and cannot connect to Netdata Cloud via the ACLK. You will still be able to see this node in your War Rooms in an unreachable state.

If you want to reclaim this node into a different Space, you need to create a new identity by adding -id=$(uuidgen) to the claiming script parameters. For example, using the default claiming script:

sudo netdata-claim.sh -token=TOKEN -rooms=ROOM1,ROOM2 -url=https://app.netdata.cloud -id=$(uuidgen)

The agent must be restarted after this change.

Claiming reference

In the sections below, you can find reference material for the claiming script, claiming via the Agent's command line tool, and details about the files found in cloud.d.

The cloud.conf file

This section defines how and whether your Agent connects to Netdata Cloud using the ACLK.

setting	default	info
cloud base url	https://app.netdata.cloud	The URL for the Netdata Cloud web application. You should not change this. If you want to disable Cloud, change the enabled setting.
enabled	yes	The runtime option to disable the Agent-Cloud link and prevent your Agent from connecting to Netdata Cloud.

Claiming script

A Space's administrator can claim an Agent by directly calling the netdata-claim.sh script either with root privileges using sudo, or as the user running the Agent (typically netdata), and passing the following arguments:

-token=TOKEN
    where TOKEN is the Space's claiming token.
-rooms=ROOM1,ROOM2,...
    where ROOMX is the War Room this node should be added to. This list is optional.
-url=URL_BASE
    where URL_BASE is the Netdata Cloud endpoint base URL. By default, this is https://app.netdata.cloud.
-id=AGENT_ID
    where AGENT_ID is the unique identifier of the Agent. This is the Agent's MACHINE_GUID by default.
-hostname=HOSTNAME
    where HOSTNAME is the result of the hostname command by default.
-proxy=PROXY_URL
    where PROXY_URL is the endpoint of a SOCKS5 proxy.

For example, the following command claims an Agent and adds it to rooms room1 and room2:

netdata-claim.sh -token=MYTOKEN1234567 -rooms=room1,room2

You should then update the netdata service about the result with netdatacli:

netdatacli reload-claiming-state

This reloads the Agent claiming state from disk.

Netdata Agent command line

If a Netdata Agent is running, the Space's administrator can claim a node using the netdata service binary with additional command line parameters:

-W "claim -token=TOKEN -rooms=ROOM1,ROOM2"

For example:

/usr/sbin/netdata -D -W "claim -token=MYTOKEN1234567 -rooms=room1,room2"

If need be, the user can override the Agent's defaults by providing additional arguments like those described here.

Claiming directory

Netdata stores the Agent's claiming-related state in the Netdata library directory under cloud.d. For a default installation, this directory exists at /var/lib/netdata/cloud.d. The directory and its files should be owned by the user that runs the Agent, which is typically the netdata user.

The cloud.d/token file should contain the claiming-token and the cloud.d/rooms file should contain the list of War Rooms you added that node to.

The user can also put the Cloud endpoint's full certificate chain in cloud.d/cloud_fullchain.pem so that the Agent can trust the endpoint if necessary.