Skip to content

3. HOWTO: Secure SSH Access to Unpatched Servers Using ZTM

Jump to bottom
CaiShu edited this page Aug 2, 2024 · 1 revision

In July 2024, a critical security vulnerability, CVE-2024-6387 (RegreSSHion), was discovered in OpenSSH. If you are unable to patch this vulnerability immediately or are concerned about potential negative impacts from patching, you can use ZTM to quickly secure affected hosts while maintaining uninterrupted, secure SSH access. This approach can be applied to various vulnerabilities when time constraints or concerns about patching necessitate an alternative method to quickly mitigate risk while ensuring continuous service.

Let's consider a scenario where Host A is affected by this CVE. The following steps demonstrate how to use ZTM to provide secure SSH access without upgrading the SSH service. The approach involves running ZTM Agent on Host A and ZTM Hub on a secure Host X. The ZTM Agent on Host A connects to the ZTM Hub on Host X, establishing a tunnel (X-A tunnel) for accessing Host A's SSH service through Host X. Finally, public access to SSH on Host A is disabled, restricting it to listening on 127.0.0.1:22. After implementing this setup, administrators can access Host A via SSH from Hosts X and Y through the ZTM tunnel, while all other access methods are blocked, rendering any scanning or brute-force attempts ineffective. The configuration process is as follows:

1. Installing ZTM Agent on Host A

First, download the appropriate version of ZTM from the ZTM Releases page. In this example, we're downloading the generic_linux-x86_64 version:

admin@ip-172-31-0-137:~$ tar xzvf ztm-aio-v0.1.0-generic_linux-x86_64.tar.gz
bin/ztm
admin@ip-172-31-0-137:~$ sudo cp bin/ztm /usr/local/bin/
admin@ip-172-31-0-137:~$ ztm version
ZTM:
  Version : v0.1.0
  Commit  : 58e91bd4eb840a453cdd7929055ab5611bc4455a
  Date    : Fri, 26 Jul 2024 14:15:04 +0800
Pipy:
  Version : 1.3.0
  Commit  : e391b8da94f619b13adf9265eb42aed3cb224cf7
  Date    : Fri, 26 Jul 2024 13:13:18 +0800

To start the ZTM agent, run:

admin@ip-172-31-0-137:~$ sudo ztm start agent

Since ZTM will register as a system service, you'll need sudo to execute this command. You can verify the service status as follows:

admin@ip-172-31-0-137:~$ ps -ef | grep ztm
root         630       1  0 13:04 ?        00:00:00 /usr/local/bin/ztm run agent --data /root/.ztm --listen 127.0.0.1:7777
root         633     630  0 13:04 ?        00:00:00 /usr/local/bin/ztm --pipy repo://ztm/agent --args --data /root/.ztm --listen 127.0.0.1:7777
admin        637     586  0 13:05 pts/0    00:00:00 grep ztm
admin@ip-172-31-0-137:~$ sudo systemctl status ztm-agent
● ztm-agent.service - ztm agent service
     Loaded: loaded (/etc/systemd/system/ztm-agent.service; disabled; preset: enabled)
     Active: active (running) since Thu 2024-08-01 13:04:40 UTC; 1min 15s ago
   Main PID: 630 (ztm)
      Tasks: 6 (limit: 1137)
     Memory: 100.0M
        CPU: 261ms
     CGroup: /system.slice/ztm-agent.service
             ├─630 /usr/local/bin/ztm run agent --data /root/.ztm --listen 127.0.0.1:7777
             └─633 /usr/local/bin/ztm --pipy repo://ztm/agent --args --data /root/.ztm --listen 127.0.0.1:7777

Aug 01 13:04:40 ip-172-31-0-137 systemd[1]: Started ztm-agent.service - ztm agent service.

2. Installing ZTM Hub on Host X

The ZTM executable is an all-in-one binary, meaning that both the Hub and Agent functionalities are included in the same file, and the specific role is determined by the startup parameters. The installation process on Host X is very similar to the installation of the ZTM Agent on Host A. However, in addition to starting the ZTM Agent, you will also need to start the ZTM Hub. Here’s how to do it:

  1. Download and Extract ZTM:

    First, download the ZTM binary and extract it:

    admin@ip-172-31-8-122:~$ wget https://github.com/flomesh-io/ztm/releases/download/v0.1.0/ztm-aio-v0.1.0-generic_linux-x86_64.tar.gz
--2024-08-01 13:17:20--  https://github.com/flomesh-io/ztm/releases/download/v0.1.0/ztm-aio-v0.1.0-generic_linux-x86_64.tar.gz
Resolving github.com (github.com)... 140.82.113.3
Connecting to github.com (github.com)|140.82.113.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/786869480/40985ed2-fc06-4a8f-be56-77963583330e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240801%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240801T131720Z&X-Amz-Expires=300&X-Amz-Signature=93b4c7e8394bcdf1ccead0d5da22049a1d2abd3985eb8aedbace977939e5e572&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=786869480&response-content-disposition=attachment%3B%20filename%3Dztm-aio-v0.1.0-generic_linux-x86_64.tar.gz&response-content-type=application%2Foctet-stream [following]
--2024-08-01 13:17:20--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/786869480/40985ed2-fc06-4a8f-be56-77963583330e?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20240801%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240801T131720Z&X-Amz-Expires=300&X-Amz-Signature=93b4c7e8394bcdf1ccead0d5da22049a1d2abd3985eb8aedbace977939e5e572&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=786869480&response-content-disposition=attachment%3B%20filename%3Dztm-aio-v0.1.0-generic_linux-x86_64.tar.gz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10253106 (9.8M) [application/octet-stream]
Saving to: ‘ztm-aio-v0.1.0-generic_linux-x86_64.tar.gz’

ztm-aio-v0.1.0-generic_linux-x86_64.ta 100%[===========================================================================>]   9.78M  44.3MB/s    in 0.2s

2024-08-01 13:17:20 (44.3 MB/s) - ‘ztm-aio-v0.1.0-generic_linux-x86_64.tar.gz’ saved [10253106/10253106]

admin@ip-172-31-8-122:~$ tar xzvf ztm-aio-v0.1.0-generic_linux-x86_64.tar.gz
bin/ztm
admin@ip-172-31-8-122:~$ sudo cp bin/ztm /usr/local/bin/
admin@ip-172-31-8-122:~$ ztm version
ZTM:
  Version : v0.1.0
  Commit  : 58e91bd4eb840a453cdd7929055ab5611bc4455a
  Date    : Fri, 26 Jul 2024 14:15:04 +0800
Pipy:
  Version : 1.3.0
  Commit  : e391b8da94f619b13adf9265eb42aed3cb224cf7
  Date    : Fri, 26 Jul 2024 13:13:18 +0800

  2. Start ZTM Hub:

    To start the ZTM Hub on Host X, run the following command:

    admin@ip-172-31-8-122:~$ sudo ztm start hub --names 172.31.8.122:443 --listen 0.0.0.0:443 --permit permit.json

    This command configures the ZTM Hub to listen on port 443 for incoming connections. The --permit option creates a permit.json file, which is required for other endpoints (such as Host A) to join the mesh network.

    After running the command, the permit file is saved at /home/admin/permit.json.

  3. Joining the Mesh:

    To connect another endpoint to the mesh network, follow these steps:

    • Transfer the permit.json file to the target endpoint (e.g., Host A).

    • On the target endpoint, execute the following command:

      ztm join my-mesh --as my-first-ep --permit permit.json

This process sets up the ZTM Hub on Host X, which will manage secure SSH access to Host A through the ZTM tunnel. Let me know if you need further assistance!

2. Starting ZTM Agent on Host X

In addition to running the ZTM Hub on Host X, you also need to start the ZTM Agent on the same machine. This allows Host X to serve both as the central hub and as an agent in the ZTM network. Here’s how you can do it:

  1. Start the ZTM Agent:

    On Host X, run the following command to start the ZTM Agent:

    admin@ip-172-31-8-122:~$ sudo ztm start agent

    To verify that the agent is running, use the following command:

    admin@ip-172-31-8-122:~$ sudo systemctl status ztm-agent
● ztm-agent.service - ztm agent service
     Loaded: loaded (/etc/systemd/system/ztm-agent.service; disabled; preset: enabled)
     Active: active (running) since Thu 2024-08-01 13:26:09 UTC; 21s ago
   Main PID: 816 (ztm)
      Tasks: 6 (limit: 1137)
     Memory: 90.8M
        CPU: 258ms
     CGroup: /system.slice/ztm-agent.service
             ├─816 /usr/local/bin/ztm run agent --data /root/.ztm --listen 127.0.0.1:7777
             └─819 /usr/local/bin/ztm --pipy repo://ztm/agent --args --data /root/.ztm --listen 127.0.0.1:7777

Aug 01 13:26:09 ip-172-31-8-122 systemd[1]: Started ztm-agent.service - ztm agent service.

  2. Register the ZTM Agent with the Hub:

    After starting the ZTM Agent on Host X, it needs to be registered with the ZTM Hub running on the same machine. Use the following command:

    admin@ip-172-31-8-122:~$ ztm join hub-x --as agent-x --permit permit.json

    Verify that the agent is correctly registered and connected:

    admin@ip-172-31-8-122:~$ ztm get meshes
NAME   JOINED AS  USER  HUBS              STATUS
hub-x  agent-x    root  172.31.8.122:443  Connected

admin@ip-172-31-8-122:~$ ztm get endpoints
NAME             USER  IP            PORT   STATUS
agent-x (local)  root  172.31.8.122  56680  Online

3. Registering Agent-A with Hub-X

To connect the Agent running on Host A to the Hub-X on Host X, follow these steps:

  1. Transfer the Permit File:

    Copy the permit.json file from Host X to Host A.

  2. Join the Hub:

    On Host A, use the following command to join the Hub-X:

    admin@ip-172-31-0-137:~$ ztm join hub-x --as agent-a --permit permit-x.json

  3. Verify the Connection:

    Check the status of the mesh and the endpoints to ensure that Agent-A is connected:

    admin@ip-172-31-0-137:~$ ztm get meshes
NAME   JOINED AS  USER  HUBS              STATUS
hub-x  agent-a    root  172.31.8.122:443  Connected

admin@ip-172-31-0-137:~$ ztm get endpoints
NAME             USER  IP            PORT   STATUS
agent-x          root  172.31.8.122  56680  Online
agent-a (local)  root  172.31.0.137  34476  Online

4. Configuring the X-A Tunnel for Secure SSH Access from Host X to Host A

To secure SSH access to Host A, we will create a ZTM Tunnel. The tunnel will listen on port 10233 on Host X and forward traffic to port 22 on Host A. Here’s how to set it up:

  1. Open the Tunnel:

    First, open a tunnel on Host X that will listen on port 10233:

    admin@ip-172-31-8-122:~$ ztm tunnel open in tcp/sshd --listen 10233

    Then, on Host A, set the tunnel’s output to target the local SSH service:

    admin@ip-172-31-8-122:~$ ztm ep agent-a tunnel open out tcp/sshd --target 127.0.0.1:22

  2. Verify the Tunnel:

    To ensure the tunnel is working, try accessing Host A’s SSH service via the tunnel from Host X:

    admin@ip-172-31-8-122:~$ ssh -i .ssh/key.em -p 10233 admin@localhost

    You should see a prompt like this:

    The authenticity of host '[localhost]:10233 ([127.0.0.1]:10233)' can't be established.
ED25519 key fingerprint is SHA256:3/Fl4ovzz8vFUzjFrwMhPF89m+EOSj+hfBB2SvBob2Y.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:10233' (ED25519) to the list of known hosts.

    After confirming, you should be logged into Host A via SSH.

5. Disabling Open SSH Access on Host A

To ensure that SSH access to Host A is only possible through the ZTM Tunnel, you’ll need to restrict SSH on Host A to listen only on the localhost interface.

  1. Update SSH Configuration:

    Edit the SSH configuration file on Host A:

    admin@ip-172-31-0-137:~$ sudo vi /etc/ssh/sshd_config

    Change the ListenAddress to 127.0.0.1:

    ListenAddress 127.0.0.1

  2. Restart the SSH Service:

    After editing the configuration, restart the SSH service:

    admin@ip-172-31-0-137:~$ sudo systemctl restart sshd

    Verify that SSH is now only listening on localhost:

    admin@ip-172-31-0-137:~$ sudo grep Listen /etc/ssh/sshd_config
ListenAddress 127.0.0.1
#ListenAddress ::

    Confirm that the SSH service is running as expected:

    admin@ip-172-31-0-137:~$ sudo systemctl status sshd
● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-08-01 13:50:16 UTC; 41s ago
       Docs: man:sshd(8)
             man:sshd_config(5)
    Main PID: 758 (sshd)
       Tasks: 1 (limit: 1137)
      Memory: 1.4M
         CPU: 22ms
      CGroup: /system.slice/ssh.service
              └─758 "sshd: /usr/sbin/sshd -D [listener] 0 of 10-100 startups"

  3. Verify Access Restrictions:

    Try accessing Host A through the tunnel on Host X to ensure it works:

    admin@ip-172-31-8-122:~$ ssh -i .ssh/key.em -p 10233 admin@localhost

    You should be able to log in without issues. Then, try accessing Host A directly via its IP:

    admin@ip-172-31-8-122:~$ ssh -i .ssh/key.em admin@172.31.0.137
ssh: connect to host 172.31.0.137 port 22: Connection refused

    As expected, direct SSH access is now blocked, and SSH is only accessible via the ZTM Tunnel.

With these steps, you’ve securely configured Host A to only allow SSH access through the ZTM Tunnel, effectively protecting it from unauthorized access.

Clone this wiki locally