How to for self hosting behind NAT?

Hello, I’m currently using opnsense and wireguard for routing and VPN.
Am looking into trying out this new Zero Trust way.
So far I managed to install it using the ‘Local with Docker’, have the ZAC working but on attempt to import an identity JWT on Android client that is on 5G (not LAN with docker) it cannot reach the controller, it’s trying to connect on the docker network alias instead of FQDN.

From android log:

10-19 20:34:04.011 6128 6156 W ziti-sdk:ziti_ctrl.c:602 ctrl_next_ep(): ctrl[https://ziti-edge-controller:1280] no controllers are online

Which hosts should be accessible publicly and what kind of port forwarding should be set up in my scenario?
I have my own domain with wildcard Letsencrypt, using nginx as reverse proxy for several services.
Managed to put ZAC behind Authelia, now to learn how it all actually works!

Hi @ZerotrustExplorer, welcome to the community and to OpenZiti!

It sounds to me that you didn't setup the advertised host properly when starting everything up. You can correct this a few ways but by far the easiest would be to dump what you have and start again. With a zero trust overlay you must have your security setup right from the start properly or else things won't work correctly as they can't form proper mTLS links. This means a couple of key points:

  • no tls termination/inspections at any load balancer/proxies, you'll need to pass TLS through to controllers/routers
  • the certs that are served have the proper SANS (DNS/IP but DNS is preferable)
  • the "advertised" addresses are all accessible from whatever networks you plan to use (which often is "the" internet but not always)

In your controller and router config files there are 'advertised' addresses. These addresses will be pointing to that docker hostname and that's what's causing your problem here.

You can "fix" the PKI by finding each server cert, learning the ziti pki command to 'fix' the pki and repair everything if you like. I just think it'll be faster to dump it all and make sure you setup the external dns entry properly from the start.

You want an external FQDN and port for the controller and for one router. I think this video is still pretty relevant the big difference is that since that video we've implement SNI so that you don't need four ports (although you still have the option of having all four ports) https://www.youtube.com/watch?v=Fk2sE0ydVo8 The ports referenced there are the ones that the quickstarts use but of course you can change those too...

hopefully that helps

Thanks for the full reply! Will have a go on a fresh VM and will report back.

Oh you might find this video useful too since you're proxying your own stuff. Uses HA proxy but i'm sure you'll figure out the differences between HA Proxy and nginx. This shows you how to use SNI and a proxy to only expose one port...

Looking around for guides I found this post from you from 2023 with a full docker-compose yaml and env file, is it still relevant today?

That or should I use this guide? Deploying with Docker | NetFoundry Documentation

I assume there is no openziti repository for use with apt for debian/ubuntu?

I would think it's still relevant. There is a newer "deployment" docker image that doesn't install a bunch of dev tools if you'd rather go that approach. It also has a small "bootstrapper" that tries to make it easy to install/configure.

You can add the repo yes. See the controller for example Controller Deployment | NetFoundry Documentation

That'll install the ziti and ziti-controller package. You'll see this mentioned:

The openziti package provides the ziti CLI and is installed as a dependency.

So am trying with the below:

VM IP: 192.168.1.250

/etc/hosts on the VM:

mydomainname.tld <WAN_ADDRESS>

.env:
ZITI_IMAGE=openziti/quickstart
ZITI_VERSION=latest
ZITI_CONTROLLER_RAWNAME=ziti-controller

ZITI_CTRL_PORT=8440
ZITI_EDGE_CONTROLLER_PORT=8441
ZITI_EDGE_ROUTER_PORT=8442
ZITI_EDGE_ROUTER_LISTENER_BIND_PORT=10080
ZITI_ZAC_PORTTLS=8443

EXTERNAL_DNS=mydomainname.tld
ZITI_NETWORK_NAME=${EXTERNAL_DNS}
ZITI_CONTROLLER_HOSTNAME=${EXTERNAL_DNS}

ZITI_EDGE_ROUTER_RAWNAME=${EXTERNAL_DNS}
ZITI_EDGE_ROUTER_DESIRED_RAWNAME=${EXTERNAL_DNS}
ZITI_EDGE_ROUTER_HOSTNAME=${EXTERNAL_DNS}

ZITI_EDGE_ROUTER_ROLES=public

docker-compose-yaml from: Connect Desktop Tunneler to Docker Quickstart on seperate host - #3 by TheLumberjack

router port forwards:

incoming on WAN ADDRESS: 8440 - 8442 TCP to 192.168.250

incoming on WAN ADDRESS: 6262 TCP to 192.168.250

incoming on WAN ADDRESS:10080 TCP to 192.168.250

I can use nginx to reverse proxy zac so am not including 8443 here.

I would use things like SNI but I already am using mydomainname.tld for a webserver, on nginx and pihole I specify hostnames.

On running docker-compose up it remains stuck at:

ziti-console-1 | waiting for server key to exist...
ziti-edge-router-1 | [ 25.387] ERROR ziti/router/env.(*networkControllers).connectToControllerWithBackoff.func2: {error=[error connecting ctrl (dial tcp <WAN_ADDRESS>:6262: i/o timeout)] endpoint=[tls:ziti:6262
]} unable to connect controller

Not sure how to proceed.

6262 is a port in the config file that is for the controller's control plane. There must be a missing env var that wasn't set. I think it'd be ZITI_CTRL_ADVERTISED_PORT. I can see I do set that for the HA Proxy demo (see here).

I had an old compose file and .env file on a public machine so I stood it up to make sure it still works. It uses the quickstart image if that matters to you. Here's the full compose and full .env I used so you can adapt to your needs.

docker compose file

services:
  ziti-controller:
    image: "${ZITI_IMAGE}:${ZITI_VERSION}"
    env_file:
      - ./.env
    ports:
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}:${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_CTRL_ADVERTISED_PORT:-6262}:${ZITI_CTRL_ADVERTISED_PORT:-6262}
    environment:
      - ZITI_CTRL_NAME=${ZITI_CTRL_NAME:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
      - ZITI_CTRL_EDGE_IP_OVERRIDE=${ZITI_CTRL_EDGE_IP_OVERRIDE:-127.0.0.1}
      - ZITI_CTRL_ADVERTISED_PORT=${ZITI_CTRL_ADVERTISED_PORT:-6262}
      - ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=${ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION}
      - ZITI_ROUTER_ENROLLMENT_DURATION=${ZITI_ROUTER_ENROLLMENT_DURATION}
      - ZITI_USER=${ZITI_USER:-admin}
      - ZITI_PWD=${ZITI_PWD}
    networks:
      ziti:
        aliases:
          - ziti-edge-controller
    volumes:
      - ziti-fs:/persistent
    entrypoint:
      - "/var/openziti/scripts/run-controller.sh"

  ziti-controller-init-container:
    image: "${ZITI_IMAGE}:${ZITI_VERSION}"
    depends_on:
      - ziti-controller
    environment:
      - ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
    env_file:
      - ./.env
    networks:
      ziti:
    volumes:
      - ziti-fs:/persistent
    entrypoint:
      - "/var/openziti/scripts/run-with-ziti-cli.sh"
    command:
      - "/var/openziti/scripts/access-control.sh"

  ziti-edge-router:
    image: "${ZITI_IMAGE}:${ZITI_VERSION}"
    env_file:
      - ./.env
    depends_on:
      - ziti-controller
    ports:
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_ROUTER_PORT:-3022}:${ZITI_ROUTER_PORT:-3022}
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}:${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}
    environment:
      - ZITI_CTRL_ADVERTISED_ADDRESS=${ZITI_CTRL_ADVERTISED_ADDRESS:-ziti-controller}
      - ZITI_CTRL_ADVERTISED_PORT=${ZITI_CTRL_ADVERTISED_PORT:-6262}
      - ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
      - ZITI_ROUTER_NAME=${ZITI_ROUTER_NAME:-ziti-edge-router}
      - ZITI_ROUTER_ADVERTISED_ADDRESS=${ZITI_ROUTER_ADVERTISED_ADDRESS:-ziti-edge-router}
      - ZITI_ROUTER_PORT=${ZITI_ROUTER_PORT:-3022}
      - ZITI_ROUTER_LISTENER_BIND_PORT=${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}
      - ZITI_ROUTER_ROLES=public
    networks:
      - ziti
    volumes:
      - ziti-fs:/persistent
    entrypoint: /bin/bash
    command: "/var/openziti/scripts/run-router.sh edge"


networks:
  ziti:

volumes:
  ziti-fs:

.env file

# OpenZiti Variables
ZITI_IMAGE=openziti/quickstart
ZITI_VERSION=latest

# the user and password to use
# Leave password blank to have a unique value generated or set the password explicitly
ZITI_USER=admin
ZITI_PWD=password

ZITI_INTERFACE=0.0.0.0

# controller name, address/port information
ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=ec2-3-18-113-172.us-east-2.compute.amazonaws.com
ZITI_CTRL_ADVERTISED_ADDRESS=ec2-3-18-113-172.us-east-2.compute.amazonaws.com
#ZITI_CTRL_EDGE_IP_OVERRIDE=10.10.10.10

ZITI_CTRL_ADVERTISED_PORT=8700
ZITI_CTRL_EDGE_ADVERTISED_PORT=8700

# The duration of the enrollment period (in minutes), default if not set. shown - 7days
ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=110080
ZITI_ROUTER_ENROLLMENT_DURATION=110080

ZITI_ROUTER_LISTENER_BIND_PORT=8701

If you want to add the ZAC to that compose you'll need to download a zac of your choosing and update the config accordingly. Me, I exec into the container as root:

docker compose exec -it --user root ziti-controller bash

grab a script to fetch zac i use:

curl -fsSL https://raw.githubusercontent.com/dovholuknf/openziti-scripts/refs/heads/main/fetch-zac.sh -o fetch-zac.sh
chmod +x fetch-zac.sh
#install unzip as it's necessary for fetch-zac
apt update && apt install unzip
./fetch-zac.sh

which will show you:

Latest release found: app-ziti-console-v3.12.6
Downloading artifact from: https://github.com/openziti/ziti-console/releases/download/app-ziti-console-v3.12.6/ziti-console.zip
Artifact saved to: /persistent/zac/app-ziti-console-v3.12.6.zip
Extracting /persistent/zac/app-ziti-console-v3.12.6.zip to /persistent/zac/ziti-console-v3.12.6
./fetch-zac.sh: line 35: unzip: command not found
Extraction complete.
      - binding: zac
        options:
          location: "/persistent/zac/ziti-console-v3.12.6"
          indexFile: index.html

Then add that to the bottom of the controller config file:

vi /persistent/ziti-edge-controller.yaml

restart the controller:

docker compose restart ziti-controller

And now you'll have the ZAC available if you want

Sorry @ZerotrustExplorer - I wasn't online for the weekend. Last I'd checked you had things working but clearly not fully yet. Can we reset here and can you let me know what your end goal is? I know you posted a lot but it'd be helpful to me and let me help you better if I knew your end goal and how i can help best. I'd also be interested in which doc wasn't ready to use? Thanks

That was embarassing: I messed up my port forwarding.

Your example works as intended!

next up:

Console Configuration | NetFoundry Documentation to put a proper cert in place for the console.

And adding 2fa to the console, how does that work without Authelia?
Could only change my password in console.

Slowly making progress, now have zac running on a different port, so I can stick it behind Authelia.

One issue, the zac page remains blank, it doesn’t want to do anything with css or js?

Anything ziti is on openziti.domain.tld, except zac is on zac.domain.tld

Errors in browser console:

Uncaught SyntaxError: Unexpected token '<' (at ziti.js:1:1)

main.89ee42078e487765.js:1 Failed to load module script: Expected a JavaScript-or-Wasm module script but the server responded with a MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.

ziti-edge-controller.yaml

v: 3

db:                     "/persistent/db/ctrl.db"

identity:
cert:        "/persistent/pki/openziti.domain.tld-intermediate/certs/openziti.domain.tld-client.chain.pem"
server_cert: "/persistent/pki/openziti.domain.tld-intermediate/certs/openziti.domain.tld-server.chain.pem"
key:         "/persistent/pki/openziti.domain.tld-intermediate/keys/openziti.domain.tld-server.key"
ca:          "/persistent/pki/cas.pem"

ctrl:
options:
advertiseAddress: tls:openziti.domain.tld:8700
listener:             tls:0.0.0.0:8700

healthChecks:
boltCheck:
interval: 30s
timeout: 20s
initialDelay: 30s

edge:
api:
address: openziti.domain.tld:8700
enrollment:
signingCert:
cert: /persistent/pki/signing.pem
key:  /persistent/pki/ziti-signing-intermediate/keys/ziti-signing-intermediate.key
edgeIdentity:
duration: 110080m
edgeRouter:
duration: 110080m

web:

name: client
bindPoints:

interface: 0.0.0.0:8700
address: openziti.domain.tld:8700
identity:
ca:          "/persistent/pki/ziti-edge-controller-root-ca/certs/ziti-edge-controller-root-ca.cert"
key:         "/persistent/pki/ziti-edge-controller-intermediate/keys/openziti.domain.tld-server.key"
server_cert: "/persistent/pki/ziti-edge-controller-intermediate/certs/openziti.domain.tld-server.chain.pem"
cert:    ziti-edge-controller.yaml
    "/persistent/pki/ziti-edge-controller-intermediate/certs/openziti.domain.tld-client.chain.pem"
options:
idleTimeout: 5000ms  #http timeouts, new
readTimeout: 5000ms
writeTimeout: 100000ms
minTLSVersion: TLS1.2
maxTLSVersion: TLS1.3
apis:

binding: edge-client
options: { }

name: management
bindPoints:

interface: 0.0.0.0:9000
address: openziti.domain.tld:9000
identity:
ca:          "/persistent/pki/ziti-edge-controller-root-ca/certs/ziti-edge-controller-root-ca.cert"
key:         "/persistent/pki/ziti-edge-controller-intermediate/keys/openziti.domain.tld-server.key"
server_cert: "/persistent/pki/ziti-edge-controller-intermediate/certs/openziti.domain.tld-server.chain.pem"
cert:        "/persistent/pki/ziti-edge-controller-intermediate/certs/openziti.domain.tld-client.chain.pem"
options:
idleTimeout: 5000ms  #http timeouts, new
readTimeout: 5000ms
writeTimeout: 100000ms
minTLSVersion: TLS1.2
maxTLSVersion: TLS1.3
apis:

binding: edge-management
options: { }

binding: fabric
options: { }

binding: edge-oidc
options: { }

binding: zac
options:
location: "/persistent/zac/ziti-console-v3.12.6"
indexFile: index.html

docker-compose.yml:

services:
ziti-controller:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
env_file:
- ./.env
ports:
- ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}:${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
- ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_CTRL_ADVERTISED_PORT:-6262}:${ZITI_CTRL_ADVERTISED_PORT:-6262}
- 9000:9000 

environment:
- ZITI_CTRL_NAME=${ZITI_CTRL_NAME:-ziti-edge-controller}
- ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
- ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
- ZITI_CTRL_EDGE_IP_OVERRIDE=${ZITI_CTRL_EDGE_IP_OVERRIDE:-127.0.0.1}
- ZITI_CTRL_ADVERTISED_PORT=${ZITI_CTRL_ADVERTISED_PORT:-6262}
- ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=${ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION}
- ZITI_ROUTER_ENROLLMENT_DURATION=${ZITI_ROUTER_ENROLLMENT_DURATION}
- ZITI_USER=${ZITI_USER:-admin}
- ZITI_PWD=${ZITI_PWD}
networks:
ziti:
aliases:
- ziti-edge-controller
volumes:
- ziti-fs:/persistent
entrypoint:
- "/var/openziti/scripts/run-controller.sh"
restart: always

ziti-controller-init-container:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
depends_on:
- ziti-controller
environment:
- ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
- ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
env_file:
- ./.env
networks:
ziti:
volumes:
- ziti-fs:/persistent
entrypoint:
- "/var/openziti/scripts/run-with-ziti-cli.sh"
command:
- "/var/openziti/scripts/access-control.sh"

ziti-edge-router:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
env_file:
- ./.env
depends_on:
- ziti-controller
ports:
- ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_ROUTER_PORT:-3022}:${ZITI_ROUTER_PORT:-3022}
- ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}:${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}
environment:
- ZITI_CTRL_ADVERTISED_ADDRESS=${ZITI_CTRL_ADVERTISED_ADDRESS:-ziti-controller}
- ZITI_CTRL_ADVERTISED_PORT=${ZITI_CTRL_ADVERTISED_PORT:-6262}
- ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
- ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
- ZITI_ROUTER_NAME=${ZITI_ROUTER_NAME:-ziti-edge-router}
- ZITI_ROUTER_ADVERTISED_ADDRESS=${ZITI_ROUTER_ADVERTISED_ADDRESS:-ziti-edge-router}
- ZITI_ROUTER_PORT=${ZITI_ROUTER_PORT:-3022}
- ZITI_ROUTER_LISTENER_BIND_PORT=${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}
- ZITI_ROUTER_ROLES=public
networks:
- ziti
volumes:
- ziti-fs:/persistent
entrypoint: /bin/bash
command: "/var/openziti/scripts/run-router.sh edge"
restart: always

networks:
ziti:

volumes:
ziti-fs:

Relevant nginx bit that does the forwarding with authelia:

    error_page 401 =302 https://auth.domain.tld?rd=$target_url; 
    proxy_pass "https://192.168.1.250:9000/zac/login"; 
    proxy_ssl_verify              off; 
    proxy_set_header Upgrade $http_upgrade; 
    proxy_set_header Connection "upgrade";

May be an Authelia bug?
I’ll just stick it behind an IP range filter for my LAN for now.
Any OpenZiti way to put 2FA in front of ZAC?

Edit:
Same with just IP range filtering in nginx, so not Authelia?

So ignoring the zac 2fa bit I tried setting up a first connection but no joy there either.

Following the minecraft tutorial somewhat by doing the same steps in zac:

-create identity for phone
-create identify for webserver (vm running nginx in default mode on port 80)

enroll jwt on both phone and webserver

on webserver: install systemd-resolved, configure my LAN’s dns servers, confirm DNS works

copy over jwt to webserver
import by: sudo ziti-edge-tunnel add --jwt "$(< ./webserver.jwt)" --identity webserver

check on zac that webserver is now online

for intercept I used address webserver.ziti and port 80, but no joy yet when attempting to connect to http://webserver.ziti

Looking at the zac identities, the phone remains offline, visualizer shows an error for the link between phone and edge router.

Edit:
Tracked down to an issue with GrapeheneOS where I cannot ‘Tap to connect’, the button remains in the off setting.
If I try on my spare phone with LineageOS it’s happy to connect.

No luck yet with connecting through openzit, some more tinkering required.

I give up until there is ready to use documentation, this is taking me too long.

Hi @ZerotrustExplorer, sorry I wasn't online over the weekend. Could you summarize what your end goal is and where you're stuck? It will help me help you better. Also can you let me know what doc wasn't ready to use so I can look at it and see if we can make it better?

No worries, rest first, appreciate the reply!

End goal:

  • At home setup, don’t want to use cloud or VPS
  • Replace Wireguard VPN with OpenZiti
  • Do away with port forwards of existing services
  • 2fa for zac, I thought I could by splitting binding away and then using Authelia but I’m running into JavaScript/CSS errors, am accessing locally by IP address and ignoring cert errors for now
  • Create a short howto from my notes once finished

What I have:

  • DNS setup, linking openziti.domain.tld to my WAN address
  • OpenZiti services running, forwarded ports 8700, 8701 and 3022 WAN address to OpenZiti VM local IP (192.168.1.250)
  • zac on different port, not publicly accessible
  • Test server VM running nginx on default port 80, for testing VPN functionality, local IP 192.168.1.61
    This vm has ziti-edge-tunnel installed, is online in zac
  • Test client VM with Windows 11, trying to connect to webserver via ziti
  • Pixel phone with GrapheneOS
  • Phone with LineageOS

Don’t have yet:

  • Connect by OpenZiti android client on GrapheneOS, in android app, ‘Tap to connect’ briefly shows the button green then nothing
  • Can connect on other phone that has LineageOS but cannot reach webserver.ziti
  • Windows VM Ziti service log shows:
    [2025-10-27T14:19:29.527Z] ERROR ziti-sdk:channel.c:976 on_tls_connect() ch[0] failed to connect to ER[ziti-edge-router] [-3008/unknown node or service]
    [2025-10-27T14:19:34.994Z] ERROR ziti-sdk:channel.c:976 on_tls_connect() ch[0] failed to connect to ER[ziti-edge-router] [-3008/unknown node or service]
    [2025-10-27T14:19:45.160Z] ERROR ziti-sdk:channel.c:976 on_tls_connect() ch[0] failed to connect to ER[ziti-edge-router] [-3008/unknown node or service]
    Is offline in zac

Screenshot of me adding the service in zac:

All clients I want to access the webserver have the webserver.clients attribute.

Forgot to answer the doc question: a how to involving split api to keep zac private, containing the complete .env file, docker-compose, dns settings, port forwards and instructions on how to setup using the zac, for an example service like default nginx.

I think your biggest problem is the router not advertising the proper address. It seems like it's still got the hostname and OMG I see that I didn't put that into the .env file I shared with you. SHAME ON ME.... I wrote the rest of this (below) before realizing this and decided to move this to the top... UGH. That's totally my bad, I've led you astray...

I'll cook up a new .env file example and I WILL run the verify command (see my advice below) to make sure it all works :slight_smile:


Looks like you're pretty close to me. A few notes.

Personally I like and recommend zac be deployed with the controller and then I will have two sets of services. Purely 'public' services: edge-client, oidc and purely private services: fabric, edge-management, zac. I like to bind the private services to 127.0.0.1 but any internal IP would be fine. If you're in docker, I'd scope it entirely to within docker. I also like to use a co-located router with my controller and then I will make a service in ziti that allows me to offload from that router back to the controller (coming soon though, you'll be able to bind these directly via an identity instead of 127.0.0.1).

I don't know if GrapheneOS or LineageOS will 'matter'. I don't use these os'es and I don't spend a lot of time in the Android tunneler other than as a user myself. I'm not sure I'll be much help there.

The error the windows vm is showing makes me think the advertised address of the router is wrong. It looks like that's probably the hostname from the docker container. That will be "a problem". It needs to advertise (and have certs) an address that anyone "anywhere" can access it otherwise you'll never be able to connect from outside that network. Getting that working is the single most important thing right now because without a router, you're 100% dead in the water when it comes to ziti.

The ziti cli ships with ziti ops verify traffic. Before splitting the services, you should run this command from a machine that is mimic'ing being over the internet. This command will test that the overlay network itself is properly setup.

I haven't done a ziti TV for a long while but this sounds like the sort of thing that would make for a good Ziti TV. I used to live stream those Fridays at 11 am, I'm leaning towards going through all this on Friday at 11. It's recorded so you won't have to watch it live but if you do, you'll be able to chat and get questions answered if you want. I think I'm going to try to solve your problem with a video/gist on a ziti tv. That will help with the doc too, hopefully.

1 Like

Video walkthrough:

Readme:

.env file:

compose file:

Hopefully that tells you what you need :slight_smile:

This is brilliant!
I’m making my own little bash script to do this all in one go on a local debian 13 VM, will post once it’s finished.

The traffic test shows:

ziti ops verify traffic --controller-url ``https://domain.tld``:port --username admin --password password
WARNING no prefix and mode [] is not 'both'. default prefix of 2025-10-28-1602 will be used
Untrusted certificate authority retrieved from server
Verified that server supplied certificates are trusted by server
Server supplied 5 certificates
Trust server provided certificate authority [Y/N]: y
Server certificate chain written to /home/username/.config/ziti/certs/domain.tld
Token: 82218e0a-83a7-4e6f-8c0c-f18b954c4c9c
Saving identity 'default' to /home/username/.config/ziti/ziti-cli.json
INFO generating P-384 EC key
INFO generating P-384 EC key
INFO waiting 10s for terminator for service: 2025-10-28-1602.traffic
INFO successfully bound service: 2025-10-28-1602.traffic.

INFO Server is listening for a connection and will exit when one is received.
INFO found terminator for service: 2025-10-28-1602.traffic
INFO found service named: 2025-10-28-1602.traffic
INFO Server has accepted a connection and will exit soon.
INFO successfully dialed service: 2025-10-28-1602.traffic.
INFO traffic test successfully detected
INFO Server complete.

The above looks..happy?
In the video I did not see the warning about the prefix

Next up is to put the api split/secure tunnel bit into the bash file.

Looks good yes. And the warning is there at 1:51 and again at 2:20 :wink: (at least)
image

quickconfig.zip (3.2 KB)

Finished my quick setup scripts for use on a Debian 13 VM, see attached for zip or below for script files in full.

Guide for starters

Install Debian 13 in a VM, create a snapshot of the VM after installation.

Initial VM prep
Install docker by following these instructions: Debian | Docker Docs .
Add your user to the docker group: sudo usermod -a -G docker yourusername
Log off/log in for the rights to apply.
Ensure the docker service autostart is enabled: sudo systemctl enable docker
Create a VM snapshot.
Give the VM a static IP address. NetworkConfiguration - Debian Wiki
sudo nano /etc/network/interfaces, something like below:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug ens18
#iface ens18 inet dhcp
iface ens18 inet static
address 192.168.1.123
netmask 255.255.255.0
gateway 192.168.1.1
dns-nameservers 8.8.8.8 8.8.4.4

Restart VM to apply, or restart networking service.

OpenZiti
Put all these files in one folder, make the .sh files executable (chmod +x filename.sh)
Edit the installserver.sh file with your preferred domain name, ports, password, check domain and portforwarding is OK, you will want to forward these ports mentioned in installserver.sh, as well as check if the DNS settings are pointing to the right WAN address:

DOMAIN=openziti.mydomain.tld
ROUTERPORT=3022
CONTROLLERPORT=8700

Remove exit from installserver.sh and run. (./installserver.sh)
Transfer ziti-admin.jwt from VM to your client, I like python3 -m http.server on VM, then browsing to http://VMIPADDRESS:8000 on a client PC, don’t forget to close python once done.
For a more secure option, use WinSCP to connect to your VM by SSH.
Import the JWT into your OpenZiti client of choice, the instructions per client can be found here: Tunnelers | NetFoundry Documentation
Test https://secured-apis.ziti:$ZACPORT/zac/dashboard from the client
Login with admin for username and the password specified in installserver.sh.
Once working, create another VM snapshot/backup.

I hope this all makes sense, thank you @TheLumberjack!

Troubleshooting

In case you lose your client config with the admin JWT, login to your OpenZiti VM, run below commands, transfer the ziti-admin-help.jwt from VM to your client:

docker compose exec -it ziti-controller bash

ziti edge login ${ZITI_NETWORK}:$ZITI_CTRL_SECURE_PORT \
  --username admin --password yourpassword --yes

ziti edge create identity ziti-admin-help -a admins -o ziti-admin-help.jwt

exit

docker compose cp ziti-controller:/persistent/ziti-admin-help.jwt .

adminsetup.sh

#!/bin/bash
source /persistent/ziti.env

ziti edge login ${ZITI_NETWORK}:$ZITI_CTRL_SECURE_PORT 
--username $ZITI_USER --password $ZITI_PWD --yes

ziti edge create config secure-apis-host.v1 host.v1 
'{"protocol":"tcp", "address":"'"${ZITI_NETWORK}"'","port":'"${ZITI_CTRL_SECURE_PORT}"'}'
ziti edge create config secure-apis-intercept.v1 intercept.v1 
'{"protocols":["tcp"],"addresses":["secured-apis.ziti"], "portRanges":[{"low":'${ZITI_CTRL_SECURE_PORT}', "high":'${ZITI_CTRL_SECURE_PORT}'}]}'
ziti edge create service secure-apis --configs "secure-apis-host.v1","secure-apis-intercept.v1" -a admin-services

ziti edge create service-policy "secured-apis-bind" Bind 
--service-roles "#admin-services" 
--identity-roles "@${ZITI_ROUTER_NAME}" 
--semantic "AnyOf"
ziti edge create service-policy "secured-apis-dial" Dial 
--service-roles "#admin-services" 
--identity-roles "#admins" 
--semantic "AnyOf"

ziti edge create identity ziti-admin -a admins -o ziti-admin.jwt

apisplit.sh

#!/bin/bash
source ziti.env

sed '/^web/,$d' ${ZITI_NETWORK}.yaml > temp.yaml && \
  mv temp.yaml ${ZITI_NETWORK}.yaml

cat >> ${ZITI_NETWORK}.yaml <<HERE
web:
  - name: public-apis
    bindPoints:
      - interface: 0.0.0.0:8841
        address: ec2-3-18-113-172.us-east-2.compute.amazonaws.com:8841
    options:
      idleTimeout: 5000ms
      readTimeout: 5000ms
      writeTimeout: 100000ms
      minTLSVersion: TLS1.2
      maxTLSVersion: TLS1.3
    apis:
      - binding: edge-client
        options: { }
      - binding: edge-oidc
        options: { }
  - name: secured-apis
    bindPoints:
      - interface: ${ZITI_NETWORK}:${ZITI_CTRL_SECURE_PORT}
        address: ${ZITI_NETWORK}:${ZITI_CTRL_SECURE_PORT}
    options:
      idleTimeout: 5000ms
      readTimeout: 5000ms
      writeTimeout: 100000ms
      minTLSVersion: TLS1.2
      maxTLSVersion: TLS1.3
    apis:
      - binding: edge-management
        options: { }
      - binding: fabric
        options: { }
      - binding: zac
        options:
          location: /zac
          indexFile: index.html
HERE

docker-compose.yml

services:
  ziti-controller:
    image: "${ZITI_IMAGE}:${ZITI_VERSION}"
    env_file:
      - ./.env
    ports:
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}:${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_CTRL_ADVERTISED_PORT:-6262}:${ZITI_CTRL_ADVERTISED_PORT:-6262}
    environment:
      - ZITI_CTRL_NAME=${ZITI_CTRL_NAME:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
      - ZITI_CTRL_EDGE_IP_OVERRIDE=${ZITI_CTRL_EDGE_IP_OVERRIDE:-127.0.0.1}
      - ZITI_CTRL_ADVERTISED_PORT=${ZITI_CTRL_ADVERTISED_PORT:-6262}
      - ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=${ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION}
      - ZITI_ROUTER_ENROLLMENT_DURATION=${ZITI_ROUTER_ENROLLMENT_DURATION}
      - ZITI_USER=${ZITI_USER:-admin}
      - ZITI_PWD=${ZITI_PWD}
    networks:
      ziti:
        aliases:
          - ziti-edge-controller
    volumes:
      - ziti-fs:/persistent
    entrypoint:
      - "/var/openziti/scripts/run-controller.sh"
    restart: always

  ziti-controller-init-container:
    image: "${ZITI_IMAGE}:${ZITI_VERSION}"
    depends_on:
      - ziti-controller
    environment:
      - ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
    env_file:
      - ./.env
    networks:
      ziti:
    volumes:
      - ziti-fs:/persistent
    entrypoint:
      - "/var/openziti/scripts/run-with-ziti-cli.sh"
    command:
      - "/var/openziti/scripts/access-control.sh"

  ziti-edge-router:
    image: "${ZITI_IMAGE}:${ZITI_VERSION}"
    env_file:
      - ./.env
    depends_on:
      - ziti-controller
    ports:
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_ROUTER_PORT:-3022}:${ZITI_ROUTER_PORT:-3022}
      - ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}:${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}
    environment:
      - ZITI_CTRL_ADVERTISED_ADDRESS=${ZITI_CTRL_ADVERTISED_ADDRESS:-ziti-controller}
      - ZITI_CTRL_ADVERTISED_PORT=${ZITI_CTRL_ADVERTISED_PORT:-6262}
      - ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS:-ziti-edge-controller}
      - ZITI_CTRL_EDGE_ADVERTISED_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT:-1280}
      - ZITI_ROUTER_NAME=${ZITI_ROUTER_NAME:-ziti-edge-router}
      - ZITI_ROUTER_ADVERTISED_ADDRESS=${ZITI_ROUTER_ADVERTISED_ADDRESS:-ziti-edge-router}
      - ZITI_ROUTER_PORT=${ZITI_ROUTER_PORT:-3022}
      - ZITI_ROUTER_LISTENER_BIND_PORT=${ZITI_ROUTER_LISTENER_BIND_PORT:-10080}
      - ZITI_ROUTER_ROLES=public
    networks:
      - ziti
    volumes:
      - ziti-fs:/persistent
    entrypoint: /bin/bash
    command: "/var/openziti/scripts/run-router.sh edge"
    restart: always


networks:
  ziti:

volumes:
  ziti-fs:

installserver.sh

#!/bin/bash
#!!add bits for installing docker for debian 13 and adding user to docker group!
echo "Ensure ports, password and domain in this file are set, ports forwarded and exit removed, otherwise press CTRL+C!"
read -sr -n 1 -p "Press any key to continue..."
exit
DOMAIN=openziti.mydomain.tld
ROUTERPORT=3022
CONTROLLERPORT=8700
ZACPORT=9000
PASSWORD=clintisawesome

echo "install openziti repo debian"
sudo apt update && sudo apt install jq unzip gpg -y
curl -sSLf https://get.openziti.io/tun/package-repos.gpg | sudo gpg --dearmor --output /usr/share/keyrings/openziti.gpg
sudo chmod a+r /usr/share/keyrings/openziti.gpg
sudo tee /etc/apt/sources.list.d/openziti-release.list >/dev/null <<EOF
deb [signed-by=/usr/share/keyrings/openziti.gpg] https://packages.openziti.org/zitipax-openziti-deb-stable debian main
EOF
echo "openziti debian repo install complete"
echo "install openziti apt package"
sudo apt update && sudo apt install openziti -y
echo "openziti apt package installation done"
echo "getting openziti script files"
wget https://raw.githubusercontent.com/dovholuknf/openziti-scripts/refs/heads/main/fetch-zac.sh
wget https://raw.githubusercontent.com/dovholuknf/openziti-scripts/refs/heads/main/discourse/5255/docker-compose.yml
wget https://raw.githubusercontent.com/dovholuknf/openziti-scripts/refs/heads/main/discourse/5255/.env
echo "script files retrieved"
echo "adding in our values"
sed -i "s|ec2-3-18-113-172.us-east-2.compute.amazonaws.com|$DOMAIN|" .env
sed -i "s|8841|$CONTROLLERPORT|" .env
sed -i "s|8842|$ROUTERPORT|" .env
sed -i "s|8888|$ZACPORT|" .env
sed -i "s|discourse5255|$PASSWORD|" .env
echo "values added"
echo "setting up zac"
chmod +x fetch-zac.sh
./fetch-zac.sh
sed -i "s|\/home\/ubuntu\/discourse\/discourse5255\/zac\/ziti-console-v3.12.6|\/home\/$USER\/zac\/ziti-console-v3.12.6|g" docker-compose.yml
echo "zac setup"
echo "starting containers, give it a minute"
docker compose up -d
echo "sleeping for 30s to give services time to start, please wait..."
sleep 30
echo "should be started now, testing traffic"
ziti ops verify traffic --controller-url https://$DOMAIN:$CONTROLLERPORT --username admin --password $PASSWORD -y
read -sr -n 1 -p "Does above look happy? Press any key to continue or press Ctrl+C to cancel."
echo -e
echo "splitting API"
sed -i "s|ec2-3-18-113-172.us-east-2.compute.amazonaws.com|$DOMAIN|" apisplit.sh
sed -i "s|8841|$CONTROLLERPORT|" apisplit.sh
sed -i "s|8842|$ROUTERPORT|" apisplit.sh
chmod +x apisplit.sh
docker compose cp apisplit.sh ziti-controller:/persistent
docker compose exec -i ziti-controller /persistent/apisplit.sh && docker compose restart ziti-controller
#echo "waiting 10s for ziti-controller to restart, please wait..."
sleep 20s
echo "getting adminsetup and running"
chmod +x adminsetup.sh
docker compose cp adminsetup.sh ziti-controller:/persistent
docker compose exec --user ziti -i ziti-controller /persistent/adminsetup.sh && docker compose restart ziti-controller
echo "waiting 10s for ziti-controller to restart, please wait..."
sleep 10s
docker compose cp ziti-controller:/persistent/ziti-admin.jwt .
echo "import ziti-admin.jwt file into your workstation tunneler!"
echo "check if https://secured-apis.ziti:$ZACPORT/zac/dashboard works!"

1 Like