Hello, I’ve encountered a possible bug while playing with the Docker Compose setup.
It seems that the Ziti Edge Router can enroll correctly the first times, but after some hours, if you run docker compose down then docker compose up -d, the edge router ends up not connecting.
At first I thought it was something that I was doing wrong, but now I’m not so sure since I could reproduce it with only the Ziti Controller and the Ziti Edge Router.
Here are my files. These files are stored in a folder called zititest. There are no other running containers. Ports are open as expected and first connections work correctly.
As a small disclaimer, I made a small modification to the docker compose, the containers have a healthcheck which is just a pretty dumb check for listening ports so this way Docker won’t start the edge router container if the controller is not yet listening on the expected ports.
docker-compose.yml
services:
ziti-controller:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
ports:
- ${ZITI_EDGE_CONTROLLER_PORT:-1280}:${ZITI_EDGE_CONTROLLER_PORT:-1280}
- ${ZITI_CTRL_PORT:-6262}:${ZITI_CTRL_PORT:-6262}
environment:
- ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=${ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION}
- ZITI_EDGE_ROUTER_ENROLLMENT_DURATION=${ZITI_EDGE_ROUTER_ENROLLMENT_DURATION}
env_file:
- $MAIN_DIR/.env
networks:
ziti:
aliases:
- ziti-edge-controller
volumes:
- ziti-fs:/persistent
entrypoint:
- "/var/openziti/scripts/run-controller.sh"
healthcheck:
test: ["CMD", "bash", "-c", "lsof -i -P -n | grep -q 'TCP.*:6262' && lsof -i -P -n | grep -q 'TCP.*:1280'"]
interval: 10s
timeout: 5s
retries: 10
ziti-controller-init-container:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
depends_on:
ziti-controller:
condition: service_healthy
environment:
- ZITI_CONTROLLER_RAWNAME="${ZITI_CONTROLLER_RAWNAME}"
- ZITI_EDGE_CONTROLLER_RAWNAME="${ZITI_EDGE_CONTROLLER_RAWNAME}"
env_file:
- $MAIN_DIR/.env
networks:
ziti:
aliases:
- ziti-edge-controller-init-container
volumes:
- ziti-fs:/persistent
entrypoint:
- "/var/openziti/scripts/run-with-ziti-cli.sh"
command:
- "/var/openziti/scripts/access-control.sh"
ziti-edge-router:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
depends_on:
ziti-controller:
condition: service_healthy
ziti-controller-init-container:
condition: service_completed_successfully
environment:
- ZITI_CONTROLLER_RAWNAME="${ZITI_CONTROLLER_RAWNAME}"
- ZITI_EDGE_CONTROLLER_RAWNAME="${ZITI_EDGE_CONTROLLER_RAWNAME}"
- ZITI_EDGE_ROUTER_RAWNAME=${ZITI_EDGE_ROUTER_RAWNAME:-ziti-edge-router}
- ZITI_EDGE_ROUTER_ROLES=public
env_file:
- $MAIN_DIR/.env
ports:
- ${ZITI_EDGE_ROUTER_PORT:-3022}:${ZITI_EDGE_ROUTER_PORT:-3022}
networks:
- ziti
volumes:
- ziti-fs:/persistent
entrypoint: /bin/bash
command: "/var/openziti/scripts/run-router.sh edge"
healthcheck:
test: ["CMD", "bash", "-c", "lsof -i -P -n | grep -q 'TCP.*:3022'"]
interval: 10s
timeout: 5s
retries: 10
networks:
ziti:
volumes:
ziti-fs:
.env
# Generic
MAIN_DIR=/home/ubuntu/zititest
# OpenZiti Variables
ZITI_IMAGE=openziti/quickstart
ZITI_VERSION=0.27.9
# The duration of the enrollment period (in minutes), default if not set
# shown - 7days
ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=10080
ZITI_EDGE_ROUTER_ENROLLMENT_DURATION=10080
# controller address/port information
ZITI_CONTROLLER_RAWNAME=ziti-controller
ZITI_CONTROLLER_HOSTNAME=mymachine.mydomain.com
ZITI_EDGE_CONTROLLER_RAWNAME=ziti-edge-controller
ZITI_EDGE_CONTROLLER_HOSTNAME=mymachine.mydomain.com
ZITI_EDGE_CONTROLLER_IP_OVERRIDE=11.22.33.44
# router address/port information
ZITI_EDGE_ROUTER_RAWNAME=mymachine.mydomain.com
ZITI_EDGE_ROUTER_IP_OVERRIDE=11.22.33.44
Upon first running docker compose up -d the containers start nicely, you can check the logs here (some logs are big so I am using pastebin, hope that’s okay)
Ziti Controller logs
Ziti Init Controller Container logs
Ziti Edge Router logs
I left them running for a while. I’ve made no connections at all, nor have I created any identities. All with just the defaults.
Then I performed the following
docker compose down at 13:28
docker compose up -d at 13:30 → logs looked good
docker compose down at 13:50
docker compose down at 13:51 → logs looked good
And then I left it running for a couple hours, and I ran
docker compose down at 16:02
docker compose up -d at 16:03
And then the following happened:
Ziti Controller logs (check errors at the end)
Ziti Init Controller Container Logs (adding them just for completion, but no relevant info here I think)
Ziti Edge Router logs
It seems like the edge router failed executing some Ziti command because the request got Unauthorized.
Could this be related to the ZITI_PWD issue we discussed in another post? Or am I doing something wrong?
Thanks in advance for your time 
Though I noticed that the image version didn’t change, it’s still 0.27.9, wouldn’t it be better to have 0.27.10 or any different version to differentiate the one with the bug and the one without it? I’m just thinking out loud, it doesn’t affect me but I thought it was strange.
^