# Cannot enroll Docker tunneler instance **URL:** \ **Category:** Ziti Edge Tunnel\ **Created:** [August 24, 2023, 3:15pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551 "2023-08-24T15:15:18Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![mkuhlmann](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@mkuhlmann](https://openziti.discourse.group/u/mkuhlmann)\ **Post date:** [August 24, 2023, 3:15pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/1 "2023-08-24T15:15:18Z")
Hi all, Im having a hard time enrolling a docker instance of the edge tunneler. It seems that the image cannot contact the controller, but I have verified the connection in numerous ways. Any idea and assistance is highly appreciated. Thanks in advance! Heres the error log from the image: > [root@docker ziti]# docker run --name ziti-host --rm --network=host --env ZITI\_IDENTITY\_BASENAME="somename" --env ZITI\_IDENTITY\_WAIT=10 --volume /root/ziti/somename.jwt:/ziti-edge-tunnel/somename.jwt openziti/ziti-host > WARN: the identities directory is only available inside this container because /ziti-edge-tunnel is not a mounted volume. Be careful to not publish this image with identity inside or lose access to the identity by removing the image prematurely. > INFO: setting NF\_REG\_NAME to ${ZITI\_IDENTITY\_BASENAME} (somename) > INFO: setting NF\_REG\_WAIT to ${ZITI\_IDENTITY\_WAIT} (10) > DEBUG: waiting 10s for /ziti-edge-tunnel/somename.json (or token) to appear > INFO: identity file /ziti-edge-tunnel/somename.json does not exist > INFO: looking for /var/run/secrets/netfoundry.io/enrollment-token/somename.jwt > INFO: looking for /enrollment-token/somename.jwt > INFO: looking for /ziti-edge-tunnel/somename.jwt > INFO: enrolling /ziti-edge-tunnel/somename.jwt > (8)[0.000] INFO ziti-sdk:utils.c:188 ziti\_log\_set\_level() set log level: root=3/INFO > (8)[0.000] INFO ziti-sdk:utils.c:188 ziti\_log\_set\_level() set log level: root=3/INFO > (8)[0.000] INFO ziti-sdk:ziti\_enroll.c:92 ziti\_enroll() Ziti C SDK version 0.33.4 @27bac90(HEAD) starting enrollment at (2023-08-24T15:05:10.441) > (8)[0.065] ERROR ziti-sdk:ziti\_ctrl.c:154 ctrl\_resp\_cb() ctrl[controlleradress] request failed: -103(software caused connection abort) > (8)[0.065] ERROR ziti-sdk:ziti\_enroll.c:235 enroll\_cb() failed to enroll with controller: [https://controlleradress:1280](https://controlleradress:1280) CONTROLLER\_UNAVAILABLE (software caused connection abort) > (8)[0.065] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:2137 enroll\_cb() enrollment failed: CONTROLLER\_UNAVAILABLE(-3) > ERROR: failed to enroll with token from /ziti-edge-tunnel/somename.jwt (888B) ---
**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\ **Post date:** [August 24, 2023, 3:41pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/2 "2023-08-24T15:41:50Z")
I see the `ziti-host` container uses the Docker host's `host` network mode, which is useful for the container to serve as a reverse-proxy to servers reachable by the Docker host. For this to work, [https://controlleradress:1280/](https://controlleradress:1280/) needs to be reachable from the Docker host. Can you confirm that is the case? ---
**Author:** ![mkuhlmann](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@mkuhlmann](https://openziti.discourse.group/u/mkuhlmann)\ **Post date:** [August 24, 2023, 3:59pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/3 "2023-08-24T15:59:41Z")
Hi @qrkourier, thanks for your quick reply. Yes, the address is resolvable and reachable from the docker host. I have verified this via curl: > [root@docker ziti]# curl -k [https://someaddress:1280](https://someaddress:1280) > {"data":{"apiVersions":{"edge":{"v1":{"apiBaseUrls":["[https://someaddress:1280/edge/client/v1"],"path":"/edge/client/v1"}},"edge-client":{"v1":{"apiBaseUrls":["https://someaddress:1280/edge/client/v1"],"path":"/edge/client/v1"}},"edge-management":{"v1":{"apiBaseUrls":["https://someaddress:1280/edge/management/v1"],"path":"/edge/management/v1"}}},"buildDate":"2023-04-12T18:24:17Z","revision":"e4520fd2c0ed","runtimeVersion":"go1.20.2","version":"v0.27.9"},"meta](https://someaddress:1280/edge/client/v1%22%5D,%22path%22:%22/edge/client/v1%22%7D%7D,%22edge-client%22:%7B%22v1%22:%7B%22apiBaseUrls%22:%5B%22https://someaddress:1280/edge/client/v1%22%5D,%22path%22:%22/edge/client/v1%22%7D%7D,%22edge-management%22:%7B%22v1%22:%7B%22apiBaseUrls%22:%5B%22https://someaddress:1280/edge/management/v1%22%5D,%22path%22:%22/edge/management/v1%22%7D%7D%7D,%22buildDate%22:%222023-04-12T18:24:17Z%22,%22revision%22:%22e4520fd2c0ed%22,%22runtimeVersion%22:%22go1.20.2%22,%22version%22:%22v0.27.9%22%7D,%22meta)":{}} That the container uses the host network is actually already part of my debugging process. Initially I wanted to create a network shared by some of the containers on the host and the tunneler, to make the services available via ziti. ---
**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\ **Post date:** [August 24, 2023, 5:20pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/4 "2023-08-24T17:20:54Z")
That cURL test shows a good result from the network perspective of the Docker host. That is strange the container is unable to either resolve the same name or reach the resolved IP address. What result do you get with a busybox container that's using the same `host` network mode? ```bash docker run --rm --network=host busybox wget -O- https://someaddress:1280/ ``` ---
**Author:** ![mkuhlmann](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@mkuhlmann](https://openziti.discourse.group/u/mkuhlmann)\ **Post date:** [August 24, 2023, 6:06pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/5 "2023-08-24T18:06:59Z")
Hi @qrkourier, thanks for the hint. This is the output of the command you mentioned: > [root@docker ~]# docker run --rm --network=host busybox wget -O- [https://someaddress:1280/](https://someaddress:1280/) > Connecting to someaddress:1280 (192.168.11.52:1280) > wget: note: TLS certificate validation not implemented > wget: TLS error from peer (alert code 40): handshake failure > wget: error getting response: Connection reset by peer I guess this proves that the controller is reachable. The failed certificate check was expected, right? Addendum: This command yields the same output without the "--network=host" flag. ---
**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\ **Post date:** [August 24, 2023, 6:45pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/6 "2023-08-24T18:45:34Z")
Good, so it's reachable from the busybox container. I forgot to add `--no-check-certificate`. You should see the JSON response. I suppose Docker wouldn't let you create a bridge network named "host," so it must be using the Docker host's network. It works from a bridge too so that concern is moot. Let's turn up the log level on the ziti-host container. Set these command args: `run-host --verbose=4` e.g. ```bash docker run \ --name ziti-host \ --rm \ --network=host \ --env ZITI_IDENTITY_BASENAME="somename" \ --env ZITI_IDENTITY_WAIT=10 \ --volume /root/ziti/somename.jwt:/ziti-edge-tunnel/somename.jwt \ openziti/ziti-host \ run-host \ --verbose=4 ``` ---
**Author:** ![mkuhlmann](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@mkuhlmann](https://openziti.discourse.group/u/mkuhlmann)\ **Post date:** [August 28, 2023, 2:40pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/7 "2023-08-28T14:40:13Z")
Hi, @qrkourier, there seems to be something strange with my controller. Adding the --no-check-certificate flag does not yield the JSON. The handshake still fails. I added the DNS option for debugging purposes. Raising the debug level did not yield additional information. Any help is appreciated. > [root@docker ~]# docker run --rm --network=host --dns=8.8.8.8 busybox wget --no-check-certificate -O- [https://controller:1280](https://controller:1280) > Connecting to controller:1280 (IPADDRESS:1280) > wget: TLS error from peer (alert code 40): handshake failure > wget: error getting response: Connection reset by peer I scanned the available cipher suites on the controller with nmap and found a warning. Might this be part of the issue? > [root@docker ~]# nmap --script ssl-enum-ciphers -p 1280 controller > Starting Nmap 7.70 ( [https://nmap.org](https://nmap.org) ) at 2023-08-28 16:42 CEST > Nmap scan report for controller (IPADDRESS) > Host is up (0.00057s latency). > > PORT STATE SERVICE > 1280/tcp open pictrography > | ssl-enum-ciphers: > | TLSv1.2: > | ciphers: > | TLS\_ECDHE\_RSA\_WITH\_AES\_128\_GCM\_SHA256 (secp256r1) - A > | TLS\_ECDHE\_RSA\_WITH\_AES\_256\_GCM\_SHA384 (secp256r1) - A > | TLS\_ECDHE\_RSA\_WITH\_CHACHA20\_POLY1305\_SHA256 (secp256r1) - A > | compressors: > | NULL > | cipher preference: server > | warnings: > | Key exchange (secp256r1) of lower strength than certificate key > |\_ least strength: A > > Nmap done: 1 IP address (1 host up) scanned in 1.08 seconds From outside of the container, wget receives the JSON just fine: > [root@docker ~]# wget -nv --no-check-certificate -O- [https://controller:1280](https://controller:1280) > WARNING: The certificate of ‘controller’ is not trusted. > WARNING: The certificate of ‘controller’ hasn't got a known issuer. > {"data":{"apiVersions":{"edge":{"v1":{"apiBaseUrls":["[https://controller:1280/edge/client/v1"],"path":"/edge/client/v1"}},"edge-client":{"v1":{"apiBaseUrls":["https://controller:1280/edge/client/v1"],"path":"/edge/client/v1"}},"edge-management":{"v1":{"apiBaseUrls":["https://controller:1280/edge/management/v1"],"path":"/edge/management/v1"}}},"buildDate":"2023-04-12T18:24:17Z","revision":"e4520fd2c0ed","runtimeVersion":"go1.20.2","version":"v0.27.9"},"meta](https://controller:1280/edge/client/v1%22%5D,%22path%22:%22/edge/client/v1%22%7D%7D,%22edge-client%22:%7B%22v1%22:%7B%22apiBaseUrls%22:%5B%22https://controller:1280/edge/client/v1%22%5D,%22path%22:%22/edge/client/v1%22%7D%7D,%22edge-management%22:%7B%22v1%22:%7B%22apiBaseUrls%22:%5B%22https://controller:1280/edge/management/v1%22%5D,%22path%22:%22/edge/management/v1%22%7D%7D%7D,%22buildDate%22:%222023-04-12T18:24:17Z%22,%22revision%22:%22e4520fd2c0ed%22,%22runtimeVersion%22:%22go1.20.2%22,%22version%22:%22v0.27.9%22%7D,%22meta)":{}} > 2023-08-28 16:46:01 URL:[https://controller:1280/](https://controller:1280/) [469/469] -\> "-" [1] ---
**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\ **Post date:** [August 28, 2023, 2:45pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/8 "2023-08-28T14:45:02Z")
To isolate the cause, will you perform the same `docker run` with the IP address of the controller instead of the domain name? Also, do you see how to add the `run-host --verbose=4` command args to turn-up the log level on the `ziti-host` container? Maybe that will tell us more about the problem from Ziti's perspective. It appears that containers with network mode "host" can't resolve the controller's domain name in DNS. ---
**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\ **Post date:** [August 28, 2023, 2:55pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/9 "2023-08-28T14:55:27Z")
> [@mkuhlmann](#): > > I scanned the available cipher suites on the controller with nmap and found a warning. Might this be part of the issue? No, I don't believe so. That appears to be an informational message from Nmap warning the user that the key exchange algorithm's elliptic curve has fewer bits than the curve used by the certificate, the implications of which are beyond my immediate grasp. ---
**Author:** ![mkuhlmann](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@mkuhlmann](https://openziti.discourse.group/u/mkuhlmann)\ **Post date:** [August 28, 2023, 3:21pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/10 "2023-08-28T15:21:17Z")
Thanks @qrkourier, Having a dedicated network solved the enrollment problem. Now I have a different issue. The docker container seems to be unable to open up the tun device. Im using Alma Linux 8, but SELINUX is temporarily disabled. > [root@docker ~]# docker run --name ziti-host --rm --network=ziti-host-net --env ZITI\_IDENTITY\_BASENAME="controller" --env ZITI\_IDENTITY\_WAIT=10 --volume /root/ziti/controller.jwt:/ziti-edge-tunnel/controller.jwt --volume /root/ziti:/ziti-edge-tunnel --cap-add=NET\_ADMIN --privileged openziti/ziti-host --verbose=4 > INFO: setting NF\_REG\_NAME to ${ZITI\_IDENTITY\_BASENAME} (controller) > INFO: setting NF\_REG\_WAIT to ${ZITI\_IDENTITY\_WAIT} (10) > WARN: the identities directory is only available inside this container because /ziti-edge-tunnel is not a mounted volume. Be careful to not publish this image with identity inside or lose access to the identity by removing the image prematurely. > DEBUG: waiting 10s for /ziti-edge-tunnel/controller.json (or token) to appear > INFO: found identity file /ziti-edge-tunnel/controller.json > DEBUG: evaluating positionals: --verbose=4 > INFO: running ziti-edge-tunnel > (8)[0.000] INFO ziti-sdk:utils.c:188 ziti\_log\_set\_level() set log level: root=4/DEBUG > (8)[0.000] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:1542 run\_tunnel() failed to open network interface: failed to open tun device:Operation not permitted > About to run tunnel service... ziti-edge-tunnel/docker-entrypoint.sh: line 25: kill: (8) - No such process ---
**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\ **Post date:** [August 28, 2023, 7:10pm UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/11 "2023-08-28T19:10:33Z")
Oh, if you're still wanting to use this `ziti-host` container as a reverse-proxy only you may drop the `--cap-add=NET_ADMIN --privileged` options. Instead of `--verbose=4` you need `run-host --verbose=4`. For example: ```bash docker run \ --name ziti-host \ --rm \ --network=ziti-host-net \ --env ZITI_IDENTITY_BASENAME="controller" \ --env ZITI_IDENTITY_WAIT=10 \ --volume /root/ziti/controller.jwt:/ziti-edge-tunnel/controller.jwt \ openziti/ziti-host \ run-host \ --verbose=4 ``` ---
**Author:** ![mkuhlmann](https://avatars.discourse-cdn.com/v4/letter/m/5f9b8f/32.png) [@mkuhlmann](https://openziti.discourse.group/u/mkuhlmann)\ **Post date:** [August 29, 2023, 8:40am UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/12 "2023-08-29T08:40:53Z")
Thank you @qrkourier , thats the missing piece. adding run-host works. I wonder why this is not described in [Containers | OpenZiti](https://openziti.io/docs/reference/tunnelers/linux/container/#use-case-hosting-openziti-services). ---
**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\ **Post date:** [August 29, 2023, 11:20am UTC](https://openziti.discourse.group/t/cannot-enroll-docker-tunneler-instance/1551/13 "2023-08-29T11:20:50Z")
Excellent. We could add a troubleshooting heading under the hosting use case (reverse proxy). That'd be a good place for how to obtain the debug log. Proposed: [document verbose container args by qrkourier · Pull Request #723 · openziti/ziti-tunnel-sdk-c · GitHub](https://github.com/openziti/ziti-tunnel-sdk-c/pull/723)