I had a Zrok instance, shares, and accesses containers configured in a working state six months ago. However, it broke after not using it, and I spent the last few days remaking my configurations from scratch based on the Zrok website's docker-based tutorials like Self-hosting guide for Docker | Zrok. Like my previous configuration, the server hosting my Zrok instance uses Traefik and thus does not have port 80 open for Caddy. Therefore, my goal has been to use the Traefik labels I use with other containers to secure my Zrok instance. Unfortunately, I can't get Ziti to work with Traefik labels. I've only gotten my Zrok instance working by opening the container ports for Ziti. That is problematic because the data does not go through Traefik for TLS.
Here is my Zrok instance compose.yml file:
services:
ziti-quickstart:
image: ${ZITI_CLI_IMAGE:-docker.io/openziti/ziti-cli}:${ZITI_CLI_TAG:-latest}
restart: unless-stopped
# Labels won't work... must temporarily open insecure ports for Ziti
labels:
- "traefik.enable=true"
- "traefik.docker.network=servnet"
# Ziti CTRL
- "traefik.http.services.zitictrl.loadbalancer.server.port=1280"
- "traefik.http.routers.zitictrl.rule=Host(`ziti.libregalaxy.org`)"
- "traefik.http.routers.zitictrl.entrypoints=websecure"
- "traefik.http.routers.zitictrl.tls.certresolver=production"
# Ziti Data
- "traefik.http.services.zitidata.loadbalancer.server.port=3022"
- "traefik.http.routers.zitidata.rule=Host(`ziti.libregalaxy.org`)"
- "traefik.http.routers.zitidata.entrypoints=websecure"
- "traefik.http.routers.zitidata.tls.certresolver=production"
ports:
- ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_CTRL_ADVERTISED_PORT:-1280}:${ZITI_CTRL_ADVERTISED_PORT:-1280}
- ${ZITI_INTERFACE:-0.0.0.0}:${ZITI_ROUTER_PORT:-3022}:${ZITI_ROUTER_PORT:-3022}
expose:
- ${ZITI_CTRL_ADVERTISED_PORT:-1280}
- ${ZITI_ROUTER_PORT:-3022}
networks:
servnet:
aliases:
- ziti.${ZROK_DNS_ZONE}
#zrok-instance:
# this allows other containers to use the same external DNS name to reach the quickstart container from within the
# Docker network that clients outside the Docker network use to reach the quickstart container via port forwarding
#aliases:
#- ziti.${ZROK_DNS_ZONE}
entrypoint:
- bash
- -euc
- |
ZITI_CMD+=" --ctrl-address ziti.${ZROK_DNS_ZONE}"\
" --ctrl-port ${ZITI_CTRL_ADVERTISED_PORT:-1280}"\
" --router-address ziti.${ZROK_DNS_ZONE}"\
" --router-port ${ZITI_ROUTER_PORT:-3022}"\
" --password ${ZITI_PWD:-admin}"
echo "DEBUG: run command is: ziti $${@} $${ZITI_CMD}"
exec ziti "$${@}" $${ZITI_CMD}
command: -- edge quickstart --home /home/ziggy/quickstart
user: ${ZIGGY_UID:-1000}
environment:
HOME: /home/ziggy
PFXLOG_NO_JSON: "${PFXLOG_NO_JSON:-true}"
ZITI_ROUTER_NAME: ${ZITI_ROUTER_NAME:-quickstart-router}
volumes:
# store the quickstart state in a named volume "ziti_home" or store the quickstart state on the Docker host in a
# directory, ZITI_HOME
#- ${ZITI_HOME:-ziti_home}:/home/ziggy
- ./ziti_home:/home/ziggy
depends_on:
ziti-quickstart-init:
condition: service_completed_successfully
healthcheck:
test:
- CMD
- ziti
- agent
- stats
interval: 3s
timeout: 3s
retries: 5
start_period: 30s
# this service is used to initialize the ziti_home volume by setting the owner to the UID of the user running the
# quickstart container
ziti-quickstart-init:
image: busybox
command: chown -Rc ${ZIGGY_UID:-1000} /home/ziggy
user: root
environment:
HOME: /home/ziggy
volumes:
# store the quickstart state in a named volume "ziti_home" or store the quickstart state on the Docker host in a
# directory, ZITI_HOME
#- ${ZITI_HOME:-ziti_home}:/home/ziggy
- ./ziti_home:/home/ziggy
# add a health check for the quickstart network
ziti-quickstart-check:
image: busybox
command: echo "Ziti is cooking"
depends_on:
ziti-quickstart:
condition: service_healthy
zrok-permissions:
image: busybox
command:
- /bin/sh
- -euxc
- |
chown -Rc ${ZIGGY_UID:-2171} /var/lib/zrok-*;
chmod -Rc ug=rwX,o-rwx /var/lib/zrok-*;
volumes:
- ./zrok_ctrl:/var/lib/zrok-controller
- ./zrok_frontend:/var/lib/zrok-frontend
zrok-controller:
depends_on:
zrok-permissions:
condition: service_completed_successfully
build:
context: .
dockerfile: ./zrok-controller.Dockerfile
args:
ZROK_CLI_IMAGE: ${ZROK_CLI_IMAGE:-openziti/zrok}
ZROK_CLI_TAG: ${ZROK_CLI_TAG:-latest}
ZROK_DNS_ZONE: ${ZROK_DNS_ZONE} # e.g., "example.com" or "127.0.0.1.sslip.io"
ZITI_CTRL_ADVERTISED_PORT: ${ZITI_CTRL_ADVERTISED_PORT:-1280}
ZROK_ADMIN_TOKEN: ${ZROK_ADMIN_TOKEN} # zrok controller admin password
ZROK_CTRL_PORT: ${ZROK_CTRL_PORT:-18080}
ZITI_PWD: ${ZITI_PWD} # ziti controller admin password
user: ${ZIGGY_UID:-2171}
command: zrok controller /etc/zrok-controller/config.yml --verbose
volumes:
- ./zrok_ctrl:/var/lib/zrok-controller
networks:
servnet:
aliases:
- zrok.${ZROK_DNS_ZONE}
#zrok-instance:
#aliases:
#- zrok.${ZROK_DNS_ZONE}
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.services.zrok.loadbalancer.server.port=18080"
- "traefik.docker.network=servnet"
- "traefik.http.routers.zrok.rule=Host(`zrok.libregalaxy.org`)"
- "traefik.http.routers.zrok.entrypoints=websecure"
- "traefik.http.routers.zrok.tls.certresolver=production"
expose:
- ${ZROK_CTRL_PORT:-18080} # (not published)
#ports:
#- ${ZROK_INSECURE_INTERFACE:-127.0.0.1}:${ZROK_CTRL_PORT:-18080}:${ZROK_CTRL_PORT:-18080}
environment:
ZROK_USER_PWD: ${ZROK_USER_PWD} # admin account password (initial user account)
ZROK_USER_EMAIL: ${ZROK_USER_EMAIL} # login email address (initial user account)
ZROK_ADMIN_TOKEN: ${ZROK_ADMIN_TOKEN} # zrok controller admin password
ZROK_API_ENDPOINT: http://zrok-controller:${ZROK_CTRL_PORT:-18080} # bridge address of the zrok controller
# Not in use
zrok-frontend:
depends_on:
zrok-permissions:
condition: service_completed_successfully
build:
context: .
dockerfile: zrok-frontend.Dockerfile
args:
ZROK_CLI_IMAGE: ${ZROK_CLI_IMAGE:-openziti/zrok}
ZROK_CLI_TAG: ${ZROK_CLI_TAG:-latest}
ZROK_DNS_ZONE: ${ZROK_DNS_ZONE} # e.g., "example.com" or "127.0.0.1.sslip.io"
ZROK_FRONTEND_PORT: ${ZROK_FRONTEND_PORT:-8080}
ZROK_OAUTH_PORT: ${ZROK_OAUTH_PORT:-8081}
ZROK_OAUTH_HASH_KEY: ${ZROK_OAUTH_HASH_KEY-noop}
ZROK_OAUTH_GOOGLE_CLIENT_ID: ${ZROK_OAUTH_GOOGLE_CLIENT_ID:-noop}
ZROK_OAUTH_GOOGLE_CLIENT_SECRET: ${ZROK_OAUTH_GOOGLE_CLIENT_SECRET:-noop}
ZROK_OAUTH_GITHUB_CLIENT_ID: ${ZROK_OAUTH_GITHUB_CLIENT_ID:-noop}
ZROK_OAUTH_GITHUB_CLIENT_SECRET: ${ZROK_OAUTH_GITHUB_CLIENT_SECRET:-noop}
user: ${ZIGGY_UID:-2171}
command: zrok access public /etc/zrok-frontend/config.yml --verbose
volumes:
- ./zrok_frontend:/var/lib/zrok-frontend
networks:
servnet:
#zrok-instance:
restart: unless-stopped
labels:
#- "traefik.enable=true"
- "traefik.http.services.zrokfe.loadbalancer.server.port=8080"
- "traefik.docker.network=servnet"
- "traefik.http.routers.zrokfe.rule=Host(`zrokfe.libregalaxy.org`)"
- "traefik.http.routers.zrokfe.entrypoints=websecure"
- "traefik.http.routers.zrokfe.tls.certresolver=production"
expose:
- ${ZROK_FRONTEND_PORT:-8080} # (not published)
- ${ZROK_OAUTH_PORT:-8081} # (not published)
#ports:
#- ${ZROK_INSECURE_INTERFACE:-127.0.0.1}:${ZROK_FRONTEND_PORT:-8080}:${ZROK_FRONTEND_PORT:-8080}
#- ${ZROK_INSECURE_INTERFACE:-127.0.0.1}:${ZROK_OAUTH_PORT:-8081}:${ZROK_OAUTH_PORT:-8081}
environment:
HOME: /var/lib/zrok-frontend
ZROK_DNS_ZONE: ${ZROK_DNS_ZONE} # e.g., "example.com" or "127.0.0.1.sslip.io"
ZROK_ADMIN_TOKEN: ${ZROK_ADMIN_TOKEN} # zrok controller admin password
ZROK_API_ENDPOINT: http://zrok-controller:${ZROK_CTRL_PORT:-18080} # bridge address of the zrok controller
ZROK_FRONTEND_SCHEME: http
ZROK_FRONTEND_PORT: ${ZROK_FRONTEND_PORT:-8080}
ZITI_CTRL_ADVERTISED_PORT: ${ZITI_CTRL_ADVERTISED_PORT:-1280}
ZITI_PWD: ${ZITI_PWD} # ziti controller admin password
#volumes:
#ziti_home: # this will not be used if you switch from named volume to bind mount volume
#zrok_ctrl:
#zrok_frontend:
# define a custom network so that we can also define DNS aliases
networks:
servnet:
driver: bridge
external: true
#zrok-instance:
#driver: bridge
When the container ports are open, "https://ziti.libregalaxy.org:1280/edge/client/v1/authenticate?method=cert" can be connected to, albeit only with HTTP. However, Zrok shares/accesses can't establish a connection when the ports are closed. I understand this is a problem with my Docker Traefik labels; I'd appreciate your help fixing them.
P.S.
I changed from a Docker volume to ./mnt in the compose.yml file because the permission-changing container did not work while it was a volume, but it did after changing it to mount locally. Any idea what is causing that behavior? The server is running Fedora Linux.
