Using a Single Port for OpenZiti Components

Ziti Overlay
1

I am testing the configuration of an OpenZiti environment with the goal of having all components (such as the controller, routers, etc.) communicate over a single port to avoid exposing multiple ports.

In the process, I have used the ssl_sni method for matching, which seems to work for HTTP/HTTPS protocols, but when using it for SSH, it fails. Instead, I successfully used req.payload to match the TPKT header.

Is there any plan to provide TPKT header-based identification for different components in future versions?

Below is my relevant test setup:

HAProxy Configuration File:

global
  # Global Configuration

defaults
  timeout connect 5000
  timeout client 50000
  timeout server 50000

frontend main
  mode tcp
  bind *:443

  tcp-request inspect-delay 3s

  # Matching HTTP using payload: (Uncomment the line below if needed)
  # acl is_http req.payload(0,3) -m bin 474554 504f53 505554 44454c 4f5054 484541 434f4e 545241
  # Matching SSH using payload (TPKT header match, not ssl_sni):
  # acl is_ssh req.payload(0,3) -m bin 535348
  # Matching RDP using payload (Uncomment the line below if needed):
  # acl is_rdp req.payload(0,3) -m bin 030000

  acl is_ctrl req.ssl_sni -i ctrl.zer0.eu.org
  acl is_route req.ssl_sni -i route.zer0.eu.org
  # For 4-layer applications like SSH, matching using ssl_sni failed, switched to TPKT header matching
  # acl is_ssh req.ssl_sni -i ssh.zer0.eu.org
  acl is_ssh req.payload(0,3) -m bin 535348

  tcp-request content accept if is_ctrl
  tcp-request content accept if is_route
  tcp-request content accept if is_ssh
  tcp-request content accept

  use_backend ctrl if is_ctrl
  use_backend route if is_route
  use_backend ssh if is_ssh

backend ctrl
  mode tcp
  server ctrl 127.0.0.1:8441

backend route
  mode tcp
  server route 127.0.0.1:8442

backend ssh
  mode tcp
  server ssh 127.0.0.1:22

Below are some simple tests I conducted.

root@ip-10-111-0-5:/etc/haproxy# curl https://route.zer0.eu.org
curl: (35) error:0A000438:SSL routines::tlsv1 alert internal error
root@ip-10-111-0-5:/etc/haproxy# curl https://ctrl.zer0.eu.org
{"data":{"apiVersions":{"edge":{"v1":{"apiBaseUrls":["https://X.X.X.X:8441/edge/client/v1"],"path":"/edge/client/v1"}},"edge-client":{"v1":{"apiBaseUrls":["https://X.X.X.X:8441/edge/client/v1"],"path":"/edge/client/v1"}},"edge-management":{"v1":{"apiBaseUrls":["https://X.X.X.X:8441/edge/management/v1"],"path":"/edge/management/v1"}}},"buildDate":"2024-10-02T12:59:41Z","capabilities":[],"revision":"0eec47ce3c80","runtimeVersion":"go1.23.1","version":"v1.1.15"},"meta":{}}
root@ip-10-111-0-5:/etc/haproxy# ssh ssh.zer0.eu.org -p 443
The authenticity of host '[ssh.zer0.eu.org]:443 ([X.X.X.X]:443)' can't be established.
ED25519 key fingerprint is SHA256:QuFOmgQ5GDSuE642zVAH4YIusjNDWB9LTn6oWo2rgro.
This host key is known by the following other names/addresses:
    ~/.ssh/known_hosts:1: [hashed name]
Are you sure you want to continue connecting (yes/no/[fingerprint])? ^C```
2

I'm not sure how SSH comes into this equation but there's no plans for TPKT as far as I know.

I tried HA Proxy out and found it relatively easy to configure a single port behind a TLS-passthrough proxy like HA Proxy. You can find my whole little project over on my github here:

I put a new video on our YouTube channel too walking through this exact setup (minus the wildcard DNS setup which I didn't think was too relevant)

3

Your setup with HAProxy on port 5443 won’t be effective since nothing will actually use HAProxy’s 5443 port. OpenZiti still operates on ports 10443 and 11443, so if you only open port 5443 and don’t allow 10443 and 11443, OpenZiti won’t function properly. The only way to ensure functionality is to open ports 10443 and 11443 on the firewall. In this case, whether HAProxy’s port 5443 is open becomes irrelevant, as OpenZiti’s core services rely on the other two ports.

4

The original intent of port multiplexing is to reduce exposed ports, but your current setup actually adds an unnecessary HAProxy port. Services like ZAC could potentially work through HAProxy, but if I’ve already opened the firewall for the control port 10443, there’s little reason to use HAProxy to access ZAC. Client access still relies on ports 10443 and 11443, and won’t utilize the HAProxy port.

5

Well yes indeed that is the intent. I went back and double checked and GAH i forgot to edit the config in the video when I recorded it! That's on me for sure.

There are two steps to make this work - step one is to emit the configuration and the PKI. That was covered in the video and by the repo. step 2 is to modify the config files so that the advertised addresses are all pointing through haproxy and the ip's are all bound to 0.0.0.0 --> 127.0.0.1.

Extra Update Needed For Contoller:

<     advertiseAddress: tls:hapctrl.clint.demo.openziti.org:5443
---
>     advertiseAddress: tls:hapctrl.clint.demo.openziti.org:10443
95c95
<   listener:             tls:127.0.0.1:10443
---
>   listener:             tls:0.0.0.0:10443
133c133
<     address: hapctrl.clint.demo.openziti.org:5443
---
>     address: hapctrl.clint.demo.openziti.org:10443
173c173
<       - interface: 127.0.0.1:10443
---
>       - interface: 0.0.0.0:10443
177c177
<         address: hapctrl.clint.demo.openziti.org:5443
---
>         address: hapctrl.clint.demo.openziti.org:10443

Extra Updates to Router

<   endpoint:             tls:hapctrl.clint.demo.openziti.org:5443
---
>   endpoint:             tls:hapctrl.clint.demo.openziti.org:10443
20,21c20,21
<       bind:             tls:127.0.0.1:11443
<       advertise:        tls:haper.clint.demo.openziti.org:5443
---
>       bind:             tls:0.0.0.0:11443
>       advertise:        tls:haper.clint.demo.openziti.org:11443
28c28
<     address: tls:127.0.0.1:11443
---
>     address: tls:0.0.0.0:11443
30c30
<       advertise: haper.clint.demo.openziti.org:5443
---
>       advertise: haper.clint.demo.openziti.org:11443

You'll see via ss that the processes are now only bound to 127.0.0.1:

ss -lntp
State                   Recv-Q                   Send-Q                                     Local Address:Port                                      Peer Address:Port                  Process
LISTEN                  0                        4096                                             0.0.0.0:5443                                           0.0.0.0:*
LISTEN                  0                        4096                                           127.0.0.1:11443                                          0.0.0.0:*                      users:(("ziti",pid=2042080,fd=7))
LISTEN                  0                        4096                                           127.0.0.1:10443                                          0.0.0.0:*                      users:(("ziti",pid=2042027,fd=9))

When done, you should only require port 5443 (or whatever port) to be open.

I'll make a change to the script so that it sed's the files accordingly. I will probably record anew video demonstrating this too.

Maybe that will clear up the confusion?

2 Likes
6

I've gone ahead and added the appropriate sed commands to the bottom of the script. If you're interested:

echo "  - sed'ing controller config file for 0.0.0.0->127.0.0.1"
sed -i 's/0\.0\.0\.0:/127.0.0.1:/g' "${ZITI_HOME}/${ZITI_NETWORK}.yaml"
echo "  - sed'ing controller config file for ${controller_port}->${ha_proxy_port}"
sed -i "s/org:${controller_port}/org:${ha_proxy_port}/g" "${ZITI_HOME}/${ZITI_NETWORK}.yaml"

echo "  - sed'ing router config file for 0.0.0.0->127.0.0.1"
sed -i 's/0\.0\.0\.0:/127.0.0.1:/g' "${ZITI_HOME}/${ZITI_NETWORK}-edge-router.yaml"
echo "  - sed'ing router config file for ${controller_port}->${ha_proxy_port}"
sed -i "s/org:${controller_port}/org:${ha_proxy_port}/g" "${ZITI_HOME}/${ZITI_NETWORK}-edge-router.yaml"
echo "  - sed'ing router config file for ${router_port}->${ha_proxy_port}"
sed -i "s/org:${router_port}/org:${ha_proxy_port}/g" "${ZITI_HOME}/${ZITI_NETWORK}-edge-router.yaml"

If you try again, you'll see the difference. I'll record a video later on with the small tweak, thanks for checking it out and finding that! cheers

1 Like
7

I’ve considered modifying the configuration, but I haven’t tried it yet. In my view, a Layer 7 HTTP protocol can use header information for routing, but with Layer 4 TCP, it wouldn’t inspect domain headers. When I tested SSH, it didn’t detect the SNI header—unless you’ve done extra processing to support ALPN with Layer 4 protocols. I’ll try the changes you suggested tomorrow and will share any relevant feedback. Thank you for your dedication; you’re one of the most responsive maintainers I’ve encountered!

8

Full disclosure -- I didn't get ssh to work sadly. but it does that feel "out of scope". I was able to connect to the sshd but never successfully authenticate - i didn't dig too much to understand why.

If it were me, i would just ssh using the OpenZiti overlay anyway :wink: