Router as local Gateway

Hi there,

I'm already using Ziti for external access of individuals but I'm currently trying to replace a site-to-site VPN following this guide: Use a Router as a Local Gateway | OpenZiti

This works as long as I set a route on the client machine to send traffic to the foreign subnet over the local ziti router. So I assume the general overlay configuration is working (when intercepting the IP address). Just like the guide shows for the http service.

But settings the client's DNS settings to point to the local ziti router doesn't work.
The router doesn't expose a DNS resolver as far as I see.

root@adw-ztna01:~# resolvectl status
Global
         Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
  resolv.conf mode: stub

Link 2 (eth0)
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.210.22
       DNS Servers: 192.168.210.22 192.168.210.254

The router runs in tproxy mode.
I have enabled the CAP_NET_BIND_SERVICE and CAP_NET_ADMIN capabilities for systemd service:

root@adw-ztna01:~# cat /etc/systemd/system/ziti-router.service.d/override.conf
[Service]

# allow binding low ports, e.g., 443/tcp; required when ZITI_ROUTER_MODE=tproxy or ZITI_ROUTER_PORT <= 1024
AmbientCapabilities=CAP_NET_BIND_SERVICE

# allow adding IP routes and iptables rules; required when ZITI_ROUTER_MODE=tproxy
AmbientCapabilities=CAP_NET_ADMIN

#
## Optional Parameters
#
# you must re-initialize with an empty ExecStartPre or ExecStart value before redefining
# ExecStartPre=
# ExecStartPre=/opt/openziti/etc/router/entrypoint.bash check alt_config.yml
# ExecStart=
# ExecStart=/opt/openziti/bin/ziti router run alt_config.yml ${ZITI_ARGS}

Btw, I set it all up using the deployment guide Router Deployment | OpenZiti and not using quickstart.

Any help is greatly appreciated. Thank you!

So, your client is using the IP of your ziti-router as the resolver?
The port 53 has to be open on the ziti-router's firewall.

As you pointed out, all the setting and config were specified in the document "Use a Router as a Local Gateway".

What OS is your client running under?

Yes, the client is using the ip of the router as resolver. No firewall active atm.

The client is running on windows.

On Windows, you will need to add this route:
route add 100.64.0.0 mask 255.192.0.0

then you can check the ip address of the intended destination by doing:
nslookup

But I should be able to resolve the overlay IP without setting the route.
The problem is that no dns resolver is exposed and therefore I don’t get an IP address of the overlay network 100.64…. So the route wouldn’t help with that.

can you resolve that IP on the router?

@pgross

I'm curious to know how you configured the edge router to run in tproxy mode. You mentioned that you followed the Router Deployment & I'd like to know the exact steps, so we can improve the docs.

Based on the above conversation my guess is tproxy/resolver is misconfigured. If you wouldn't mind providing a snippet of the config, specifically the listeners/- binding: tunnel that configured to use tproxy, we can verify.

If you used the package to install the ziti-router, then the configuration should be available here:

sudo cat /var/lib/ziti-router/config.yml

You'll likely need to adjust the resolver to listen on something other than 127.0.0.1

  - binding: tunnel
    options:
      mode: tproxy
      resolver: udp://10.200.2.192:53 #Resolver listener
      lanIf: enp0s5 #Optional Lan interface to manage inbound iptables rules for services
      dnsSvcIpRange: 100.64.0.0/10 # Resolver IP assignment range

This example would start a resolver to listen on IP 10.200.2.192 & port 53/udp.

Let me know if that helps.

@emoscardini thank you for the advice.

My router's config looked like this indeed.

  - binding: tunnel
    options:
      mode: tproxy #tproxy|host

I have now added resolver: udp://192.168.210.21:53 as suggested.

Furthermore I have looked into the ziti_router_auto_enroll.py script and did the following:

mkdir -p /usr/lib/systemd/resolved.conf.d
echo > /usr/lib/systemd/resolved.conf.d/01-ziti.conf << EOL
#Ziti Added file
[Resolve]
DNS=192.168.210.21
EOL

The local resolver is working correctly now.
I'll now add a route to 100.64.0.0/10 on my local ISP gateway router pointing to the ziti router and continue testing.

I would agree that the tproxy mode is rather undocumented. For example the service capabilities I mentioned above or the additional configuration you posted. Later could be comments in the standard configuration file shipped with the package.

Thank you so far for the help!

2 Likes

Hi there, again,

something isn't working as it should and I'm feeling kind of lost now.

On a windows machine, using the windows tunneler, I get the following:

PS C:\Users\p.gross> Resolve-DnsName test.ziti

Name                                           Type   TTL   Section    IPAddress
----                                           ----   ---   -------    ---------
test.ziti                                      A      60    Answer     100.64.0.3

On one of my router - configured in tproxy mode - I get no result:

root@ad-ztna01:~# dig @127.0.0.1 test.ziti

; <<>> DiG 9.18.30-0ubuntu0.24.04.2-Ubuntu <<>> @127.0.0.1 test.ziti
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: REFUSED, id: 46665
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;test.ziti.                     IN      A

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1) (UDP)
;; WHEN: Fri Feb 14 19:57:28 UTC 2025
;; MSG SIZE  rcvd: 27

config.yml:

  - binding: tunnel
    options:
      mode: tproxy #tproxy|host
      resolver: udp://127.0.0.1:53
      dnsSvcIpRange: 100.64.0.1/10
root@ad-ztna01:~# cat /usr/lib/systemd/resolved.conf.d/01-ziti.conf
#Ziti Added file
[Resolve]
DNS=127.0.0.1
root@ad-ztna01:~# resolvectl 
Global
         Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
  resolv.conf mode: stub
Current DNS Server: 127.0.0.1
       DNS Servers: 127.0.0.1

Link 2 (eth0)
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 8.8.8.8
       DNS Servers: 8.8.8.8 8.8.4.4

nevermind, I figured it out.

I forgot to link the router identity to the service so the router wasn't allowed to use the service and didn't resolve it.

Hi @pgross,

I'm glad you figured it out, I was just about to ask if the Edge Router identity was setup with a dial policy to that service.

Also, could you clarify this statement? Was this output from a windows client that has the ZDEW(Ziti Desktop Edge for Windows) installed? I was unsure how this related to using an Edge Router as a gateway.

hi @emoscardini

yes, I used ZDEW. It should simply show that the service was properly configured to intercept the DNS name.

Thanks for all the help.
I'm currently facing another issue when resolving SRV records for active directory but I guess this needs a new thread.