# Using public CA certificates with Docker Compose?

**URL:** <https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529>\
**Category:** General Questions\
**Created:** [August 18, 2023, 3:08pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529 "2023-08-18T15:08:38Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Reverse5431](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Reverse5431](https://openziti.discourse.group/u/Reverse5431)\
**Post date:** [August 18, 2023, 3:08pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/1 "2023-08-18T15:08:38Z")

</div>

Hi everyone,

I was wondering if there was an easy way to provide the “alternative” certificates issued by a public CA such as Let’s Encrypt via the environment file when using Docker Compose to deploy OpenZiti.

An alternative I thought about is mounting the folder /persistent, and editing the files as explained in [Using Public CA Certificates | OpenZiti](https://openziti.io/docs/guides/alt-server-certs/), but being able to provide the certificates via .env file would be better.

Thank you

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [August 18, 2023, 4:55pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/2 "2023-08-18T16:55:17Z")

</div>

Hi @Reverse5431. Welcome to the community and to OpenZiti!

I have actually been working towards that from a quickstart perspective. I have some WIP docker compose files but haven’t had the opportunity to finish it off yet.

You can most definitely do this and I do it all the time when I install browZer or when I self-host zrok. Like I said, we just don’t have decent (or any) guide for it yet. There are a couple of minor ‘gotchas’ when using LE, most notably is that if you mount the certs to the filesystem you’ll have to figure out how to grant the container permissions to the files.

If you’re ok with what I have and don’t need too much help, here’s the most basic of steps. I don’t know if this is an ok level of detail or not though (since it’s not much):

- make a folder somewhere, we’ll call it `$HOME/docker-compose-alt-certs`: `mkdir -p $HOME/docker-compose-alt-certs`

- make a file named (this is important, docker ‘things’) `.env` in that directory: `vi $HOME/docker-compose-alt-certs/.env`  
**NOTE:** “obviously” `YOUR_PASSWORD_HERE`, `YOUR_EXT_DNS_HERE`, `YOUR_EXT_IP_HERE`, etc

- make a file named `docker-compose.yml` in that directory: `vi $HOME/docker-compose-alt-certs/docker-compose.yml`

- Make the letsencrypt files available to the docker images…

After making the `.env` and `docker-compose.yml` files, you should be able to just `docker compose up` that compose file.

I **THINK** this will work…

---

<div class="post-metadata">

**Author:** ![Reverse5431](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Reverse5431](https://openziti.discourse.group/u/Reverse5431)\
**Post date:** [August 19, 2023, 3:30pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/3 "2023-08-19T15:30:32Z")

</div>

Hi @TheLumberjack, thank you very much for your help.

I have generated a wildcard certificate using Certbot (DNS Challenge), and followed all the instructions. I have changed the ZITI\_ROUTER\_ADVERTISED\_HOST variable to ZITI\_ROUTER\_ADVERTISED\_ADDRESS as I notice this commit: [Updating ZITI\_ROUTER\_ADVERTISED\_HOST to use ZITI\_ROUTER\_ADVERTISED\_AD… · openziti/ziti@6e7f3d0 · GitHub](https://github.com/openziti/ziti/commit/6e7f3d0fb2f80c5b49fa54f089c5c81c49384bd0).

Despite having followed your instructions step by step, I am failing to have OpenZiti up and running. This is the .env file (I've replaced sensitive information such as password, domain and IP):

> **.env**
>
> # OpenZiti Variables
> 
> ZITI\_IMAGE=openziti/quickstart  
> ZITI\_VERSION=latest
> 
> # Ziti default user credentials
> 
> ZITI\_USER=admin  
> ZITI\_PWD=password
> 
> # controller name, address/port information
> 
> ZITI\_CTRL\_NAME=ziti-controller  
> ZITI\_CTRL\_EDGE\_ADVERTISED\_ADDRESS=[edge.example.com](http://edge.example.com)  
> ZITI\_CTRL\_ADVERTISED\_ADDRESS=[ctrl.example.com](http://ctrl.example.com)  
> ZITI\_CTRL\_EDGE\_IP\_OVERRIDE=xxx.xxx.xxx.xxx  
> ZITI\_CTRL\_EDGE\_ADVERTISED\_PORT=8441  
> ZITI\_CTRL\_ADVERTISED\_PORT=8440
> 
> # The duration of the enrollment period (in minutes), default if not set. shown - 7days
> 
> ZITI\_EDGE\_IDENTITY\_ENROLLMENT\_DURATION=10080  
> ZITI\_ROUTER\_ENROLLMENT\_DURATION=10080
> 
> # router address/port information
> 
> ZITI\_ROUTER\_NAME=ziti-edge-router  
> ZITI\_ROUTER\_ADVERTISED\_ADDRESS=[router.example.com](http://router.example.com)  
> ZITI\_ROUTER\_PORT=8442  
> ZITI\_ROUTER\_IP\_OVERRIDE=xxx.xxx.xxx.xxx  
> ZITI\_ROUTER\_LISTENER\_BIND\_PORT=8444  
> #ZITI\_ROUTER\_ROLES=public,aws
> 
> # pki setup
> 
> ZITI\_PKI\_ALT\_SERVER\_CERT=/etc/letsencrypt/live/example.com/fullchain.pem  
> ZITI\_PKI\_ALT\_SERVER\_KEY=/etc/letsencrypt/live/example.com/privkey.pem

_Splitting this commend into multiple bits to overcome the "maximum 10 links" limit_

---

<div class="post-metadata">

**Author:** ![Reverse5431](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Reverse5431](https://openziti.discourse.group/u/Reverse5431)\
**Post date:** [August 19, 2023, 3:31pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/4 "2023-08-19T15:31:09Z")

</div>

The controller initialization container starts, but keeps waiting for the edge controller on port 8441 endlessly.

> **ziti-controller-init-container**
>
> NOT OVERRIDING: env var ZITI\_BIN\_DIR already set. using existing value  
> NOT OVERRIDING: env var ZITI\_BIN\_ROOT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ENV\_FILE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_HOME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_NETWORK already set. using existing value  
> NOT OVERRIDING: env var ZITI\_SCRIPTS already set. using existing value  
> NOT OVERRIDING: env var ZITI\_SHARED already set. using existing value
> 
> adding /var/openziti/ziti-bin to the path  
> waiting for [https://edge.example.com:8441](https://edge.example.com:8441)

---

<div class="post-metadata">

**Author:** ![Reverse5431](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Reverse5431](https://openziti.discourse.group/u/Reverse5431)\
**Post date:** [August 19, 2023, 3:32pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/5 "2023-08-19T15:32:24Z")

</div>

From what I can see, the controller still creates their own CA, to issue self-signed certificates, despite the Let’s Encrypt certificates have been provided.

> **ziti-controller**
>
> system has not been initialized. initializing…  
> Populating environment variables  
> ZITI\_NETWORK overridden: ziti  
> ZITI\_HOME overridden: /persistent  
> ZITI\_USER overridden: admin  
> ZITI\_PWD overridden: password  
> ZITI\_BIN\_DIR overridden: /var/openziti/ziti-bin  
> ZITI\_CTRL\_NAME overridden: ziti-controller  
> ZITI\_CTRL\_EDGE\_ADVERTISED\_PORT overridden: 8441  
> ZITI\_CTRL\_EDGE\_ADVERTISED\_ADDRESS overridden: [edge.example.com](http://edge.example.com)  
> ZITI\_CTRL\_ADVERTISED\_ADDRESS overridden: [ctrl.example.com](http://ctrl.example.com)  
> ZITI\_CTRL\_ADVERTISED\_PORT overridden: 8440  
> ZITI\_ROUTER\_NAME overridden: ziti-edge-router  
> ZITI\_ROUTER\_PORT overridden: 8442  
> ZITI\_ROUTER\_LISTENER\_BIND\_PORT overridden: 8444  
> ZITI\_HOME overridden: /persistent  
> ZITI\_ENV\_FILE overridden: /persistent/ziti.env  
> Your OpenZiti environment has been set up successfully.
> 
> A file with all pertinent environment values was created here: /persistent/ziti.env
> 
> NOT OVERRIDING: env var ZITI\_ARCH already set. using existing value  
> NOT OVERRIDING: env var ZITI\_BINARIES\_FILE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_BINARIES\_VERSION already set. using existing value  
> NOT OVERRIDING: env var ZITI\_BIN\_DIR already set. using existing value  
> NOT OVERRIDING: env var ZITI\_BIN\_ROOT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_CTRL\_ADVERTISED\_ADDRESS already set. using existing value  
> NOT OVERRIDING: env var ZITI\_CTRL\_ADVERTISED\_PORT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_CTRL\_EDGE\_ADVERTISED\_ADDRESS already set. using existing value  
> NOT OVERRIDING: env var ZITI\_CTRL\_EDGE\_ADVERTISED\_PORT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_CTRL\_EDGE\_IP\_OVERRIDE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_CTRL\_EDGE\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_CTRL\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_EDGE\_IDENTITY\_ENROLLMENT\_DURATION already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ENV\_FILE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_HOME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_IMAGE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_NETWORK already set. using existing value  
> NOT OVERRIDING: env var ZITI\_OSTYPE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_ALT\_SERVER\_CERT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_ALT\_SERVER\_KEY already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_CA already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_CERT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_EDGE\_INTERMEDIATE\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_EDGE\_ROOTCA\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_INTERMEDIATE\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_KEY already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_ROOTCA\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_CTRL\_SERVER\_CERT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_EDGE\_CA already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_EDGE\_CERT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_EDGE\_KEY already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_EDGE\_SERVER\_CERT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_SIGNER\_CERT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_SIGNER\_CERT\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_SIGNER\_INTERMEDIATE\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_SIGNER\_KEY already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PKI\_SIGNER\_ROOTCA\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_PWD already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ROUTER\_ADVERTISED\_ADDRESS already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ROUTER\_ENROLLMENT\_DURATION already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ROUTER\_IP\_OVERRIDE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ROUTER\_LISTENER\_BIND\_PORT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ROUTER\_NAME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ROUTER\_PORT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_SCRIPTS already set. using existing value  
> NOT OVERRIDING: env var ZITI\_SHARED already set. using existing value  
> NOT OVERRIDING: env var ZITI\_USER already set. using existing value  
> NOT OVERRIDING: env var ZITI\_VERSION already set. using existing value
> 
> adding /var/openziti/ziti-bin to the path  
> Populating environment variables  
> ZITI\_HOME overridden: /persistent  
> ZITI\_USER overridden: admin  
> ZITI\_PWD overridden: passwordnano  
> ZITI\_PKI overridden: /persistent/pki  
> ZITI\_PKI\_SIGNER\_CERT\_NAME overridden: ziti-signing  
> ZITI\_PKI\_SIGNER\_ROOTCA\_NAME overridden: ziti-signing-root-ca  
> ZITI\_PKI\_SIGNER\_INTERMEDIATE\_NAME overridden: ziti-signing-intermediate  
> ZITI\_PKI\_SIGNER\_CERT overridden: /persistent/pki/ziti-signing-intermediate/certs/ziti-signing-intermediate.cert  
> ZITI\_PKI\_SIGNER\_KEY overridden: /persistent/pki/ziti-signing-intermediate/keys/ziti-signing-intermediate.key  
> ZITI\_BIN\_DIR overridden: /var/openziti/ziti-bin  
> ZITI\_CTRL\_NAME overridden: ziti-controller  
> ZITI\_CTRL\_EDGE\_NAME overridden: ziti-edge-controller  
> ZITI\_CTRL\_EDGE\_ADVERTISED\_PORT overridden: 8441  
> ZITI\_CTRL\_EDGE\_ADVERTISED\_ADDRESS overridden: [edge.example.com](http://edge.example.com)  
> ZITI\_CTRL\_ADVERTISED\_ADDRESS overridden: [ctrl.example.com](http://ctrl.example.com)  
> ZITI\_CTRL\_ADVERTISED\_PORT overridden: 8440  
> ZITI\_PKI\_CTRL\_ROOTCA\_NAME overridden: ctrl.example.com-root-ca  
> ZITI\_PKI\_CTRL\_INTERMEDIATE\_NAME overridden: ctrl.example.com-intermediate  
> ZITI\_PKI\_CTRL\_EDGE\_ROOTCA\_NAME overridden: ziti-edge-controller-root-ca  
> ZITI\_PKI\_CTRL\_EDGE\_INTERMEDIATE\_NAME overridden: ziti-edge-controller-intermediate  
> ZITI\_PKI\_CTRL\_SERVER\_CERT overridden: /persistent/pki/ctrl.example.com-intermediate/certs/ctrl.example.com-server.chain.pem  
> ZITI\_PKI\_CTRL\_KEY overridden: /persistent/pki/ctrl.example.com-intermediate/keys/ctrl.example.com-server.key  
> ZITI\_PKI\_CTRL\_CA overridden: /persistent/pki/cas.pem  
> ZITI\_PKI\_CTRL\_CERT overridden: /persistent/pki/ctrl.example.com-intermediate/certs/ctrl.example.com-client.cert  
> ZITI\_PKI\_EDGE\_CERT overridden: /persistent/pki/ziti-edge-controller-intermediate/certs/edge.example.com-client.cert  
> ZITI\_PKI\_EDGE\_SERVER\_CERT overridden: /persistent/pki/ziti-edge-controller-intermediate/certs/edge.example.com-server.chain.pem  
> ZITI\_PKI\_EDGE\_KEY overridden: /persistent/pki/ziti-edge-controller-intermediate/keys/edge.example.com-server.key  
> ZITI\_PKI\_EDGE\_CA overridden: /persistent/pki/ziti-edge-controller-intermediate/certs/ziti-edge-controller-intermediate.cert  
> ZITI\_ROUTER\_NAME overridden: ziti-edge-router  
> ZITI\_ROUTER\_PORT overridden: 8442  
> ZITI\_ROUTER\_LISTENER\_BIND\_PORT overridden: 8444  
> ZITI\_HOME overridden: /persistent  
> ZITI\_ENV\_FILE overridden: /persistent/ziti.env  
> Your OpenZiti environment has been set up successfully.
> 
> A file with all pertinent environment values was created here: /persistent/ziti.env
> 
> Generating PKI  
> Creating CA: ctrl.example.com-root-ca  
> Success
> 
> Creating CA: ziti-edge-controller-root-ca  
> Success
> 
> Creating CA: ziti-signing-root-ca  
> Success
> 
> Creating intermediate: ctrl.example.com-root-ca ctrl.example.com-intermediate 1  
> Using CA name: ctrl.example.com-root-ca  
> Success
> 
> Creating intermediate: ziti-edge-controller-root-ca ziti-edge-controller-intermediate 1  
> Using CA name: ziti-edge-controller-root-ca

---

<div class="post-metadata">

**Author:** ![Reverse5431](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Reverse5431](https://openziti.discourse.group/u/Reverse5431)\
**Post date:** [August 19, 2023, 3:33pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/6 "2023-08-19T15:33:06Z")

</div>

The Admin Panel uses the wildcard certificate, and starts without any problem. A question regarding this, will the control panel be accessible by visiting [ctrl.example.com:8443](http://ctrl.example.com:8443), or should a proxy be used to reach it?

> **ZAC**
>
> running ZAC  
> ZAC will use this key for TLS: /etc/letsencrypt/live/example.com/privkey.pem  
> ZAC will present this pem for TLS: /etc/letsencrypt/live/example.com/fullchain.pem  
> emitting settings.json  
> Loading Settings File From: /usr/src/app/…/ziti/settings.json  
> {  
> edgeControllers: [  
> {  
> name: ‘ziti-controller’,  
> url: ‘[https://edge.example.com:8441](https://edge.example.com:8441)’,  
> default: true  
> }  
> ],  
> editable: true,  
> update: false,  
> location: ‘…/ziti’,  
> port: 1408,  
> portTLS: 8443,  
> logo: ‘’,  
> primary: ‘’,  
> secondary: ‘’,  
> allowPersonal: true,  
> rejectUnauthorized: false,  
> mail: { host: ‘’, port: 25, secure: false, auth: { user: ‘’, pass: ‘’ } },  
> from: ‘’,  
> to: ‘’  
> }  
> TLS initialized on port: 8443  
> Ziti Admin Console is now listening on port 1408

The edge router, like the init container, keeps waiting for the edge controller on port 8441 endlessly.

> **ziti-edge-router**
>
> \_ZITI\_ROUTER\_NAME set to: ziti-edge-router  
> NOT OVERRIDING: env var ZITI\_BIN\_DIR already set. using existing value  
> NOT OVERRIDING: env var ZITI\_BIN\_ROOT already set. using existing value  
> NOT OVERRIDING: env var ZITI\_ENV\_FILE already set. using existing value  
> NOT OVERRIDING: env var ZITI\_HOME already set. using existing value  
> NOT OVERRIDING: env var ZITI\_NETWORK already set. using existing value  
> NOT OVERRIDING: env var ZITI\_SCRIPTS already set. using existing value  
> NOT OVERRIDING: env var ZITI\_SHARED already set. using existing value
> 
> adding /var/openziti/ziti-bin to the path  
> waiting for [https://edge.example.com:8441](https://edge.example.com:8441)

Is there something I have missed? Let me know if I can provide any other information to help understanding the problem.

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [August 19, 2023, 7:54pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/7 "2023-08-19T19:54:44Z")

</div>

If it gets stuck: `waiting for https://edge.example.com:8441`, that usually means only one of two things:

- the controller failed to start
- the firewall doesn't have that port open

Since docker is in this mix, it could be that there's an exposed port mismatch too, but I can clearly see this in your output so I'm pretty sure that's not the case:

```auto
ZITI_CTRL_EDGE_ADVERTISED_PORT overridden: 8441

```

When this happens to me, I **think** it was because the container couldn't read the configured cert/key and failed at startup. Let's look at the docker logs for the controller using:

```auto
# cd to $HOME/docker-compose-alt-certs
docker compose logs ziti-controller

```

I just went to my own server, changed the permissions on the certs/keys from LE like this: `chown -R root:root /etc/letsencrypt` and started my docker compose... My controller fails to start, I see the same "waiting for ...." message you see but in my logs I see this the controller doesn't have permissions to open the LE certs/key and the controller throws a fatal error:

> **panic: unable to load identity (open /etc/letsencrypt/live/clint.demo.openziti.org/fullchain.pem: permission denied)**

```auto
docker-compose-alt-certs-ziti-controller-1 | [0.018] FATAL edge/controller/subcmd.configureController: {error=[unable to load identity (open /etc/letsencrypt/live/clint.demo.openziti.org/fullchain.pem: permission denied)]} could not read configuration file [/persistent/ziti-controller.yaml]
docker-compose-alt-certs-ziti-controller-1 | controller initialized. unsetting ZITI_USER/ZITI_PWD from env
docker-compose-alt-certs-ziti-controller-1 | [0.022] ERROR ziti/ziti/controller.run: {go-version=[go1.20.5] version=[v0.0.0] os=[linux] arch=[amd64] error=[unable to load identity (open /etc/letsencrypt/live/clint.demo.openziti.org/fullchain.pem: permission denied)] revision=[local] build-date=[2020-01-01 01:01:01]} error starting ziti-controller
docker-compose-alt-certs-ziti-controller-1 | panic: unable to load identity (open /etc/letsencrypt/live/clint.demo.openziti.org/fullchain.pem: permission denied)

```

So I'm guessing that's the issue?

As for the network running it's own PKI, don't worry about the that. **YOU WANT THAT** (and need it). OpenZiti will still use/maintain/operate it's own full PKI for it's own mTLS connections. Alternative certs are primarily useful for the rest API, for "websocket" support in routers (to support BrowZer) and for ZAC. The overlay network itself will maintain its own PKI.

If you want to use **YOUR** PKI for identities, that's actually a different feature called ["Third Pary CAs"](https://openziti.io/docs/learn/core-concepts/security/authentication/third-party-cas/)

Anyway, hopefully the issue is just the permission issue. That's why I included that last blurb cause it's kind of a pain... As a reminder, here's what I run to make docker happy.

```auto
sudo groupadd -g 2171 zitiweb
sudo useradd -u 2171 -M ziggy
sudo usermod -aG zitiweb ziggy
sudo usermod -aG zitiweb $USER
sudo chown -R root:zitiweb /etc/letsencrypt/
sudo chmod -R g+rX /etc/letsencrypt/

```

Hope that helps

---

<div class="post-metadata">

**Author:** ![Reverse5431](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Reverse5431](https://openziti.discourse.group/u/Reverse5431)\
**Post date:** [August 19, 2023, 9:02pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/8 "2023-08-19T21:02:26Z")

</div>

Really sorry about this. Turns out the permissions were already set properly. As I am using a very low-power and low-end device for the network, it was just very slow. Once the controller finishes setting up the network, both the controller-init and the edge router successfully connect to [edge.example.com:8441](http://edge.example.com:8441).

However, something still doesn’t work properly. The controller fails to complete the handshake due to bad certificate. It also fails to send and receive heartbeats.

> **controller**
>
> [7.111] INFO xweb/v2.(\*Server).Start: starting ApiConfig to listen and serve tls on 0.0.0.0:8441 for server client-management with APIs: [edge-management edge-client fabric]  
> [10.806] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[remote error: tls: bad certificate]} handshake failed  
> [11.689] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[remote error: tls: bad certificate]} handshake failed  
> …  
> [80.869] ERROR channel/v2.(\*heartbeater).sendHeartbeat: {error=[context deadline exceeded] channelId=[ch{ID\_a}-\>u{classic}-\>i{ID-b}]} failed to send heartbeat  
> [200.438] ERROR fabric/controller/handler\_ctrl.(\*heartbeatCallback).CheckHeartBeat: {channelType=[router] channelId=[ID\_a]} heartbeat not received in time, closing link  
> [204.119] ERROR edge/controller/handler\_edge\_ctrl.(\*createApiSessionHandler).createApiSession: {error=[channel closed]} failed to send response  
> [1002.883] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[remote error: tls: unknown certificate authority]} handshake failed  
> [1004.724] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.738] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.752] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.767] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.783] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.797] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.813] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.826] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.843] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed  
> [1004.863] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[tls: first record does not look like a TLS handshake]} handshake failed

The Edge Router fails to get the interface for port 8444, but succeeds for port 8442. (No other service running on port 8444) It then fails to send/receive heartbeats as well, to finally throw a fatal error.

> **edge router**
>
> [0.435] WARNING fabric/router/xlink\_transport.loadListenerConfig: {error=[no network interface found for 0.0.0.0] addr=[tls:0.0.0.0:8444]} unable to get interface for address  
> [0.439] INFO edge/router/xgress\_edge.(\*listener).Listen: {address=[tls:0.0.0.0:8442]} starting channel listener  
> [0.481] WARNING edge/tunnel/dns.flushDnsCaches: {error=[exec: “resolvectl”: executable file not found in $PATH]} unable to find systemd-resolve or resolvectl in path, consider adding a dns flush to your restart process  
> …  
> [141.663] FATAL edge/router/xgress\_edge\_tunnel.(\*servicePoller).pollServices: {error=[timeout waiting for message reply: context deadline exceeded]} xgress\_edge\_tunnel unable to authenticate to controller. ensure tunneler mode is enabled for this router or disable tunnel listener. exiting

The controller init finishes initializing the docker volume successfully. However, it says that the certificate authority retrieved from the server is untrusted. Is it referring to the ones generated by the controller, or to the let’s encrypt generated ones?

> **controller init**
>
> * * *
> 
> Untrusted certificate authority retrieved from server  
> Verified that server supplied certificates are trusted by server  
> Server supplied 5 certificates  
> Server certificate chain written to /home/ziti/.config/ziti/certs/edge.example.com  
> Token: TOKEN  
> Saving identity ‘default’ to /home/ziti/.config/ziti/ziti-cli.json
> 
> * * *
> 
> * * *
> 
> New edge router policy all-endpoints-public-routers created with id: ID  
> New service edge router policy all-routers-all-services created with id: ID  
> This docker volume has been initialized.

Any idea what might be causing this?

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [August 19, 2023, 9:25pm UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/9 "2023-08-19T21:25:45Z")

</div>

> [@Reverse5431](#):
>
> . The controller fails to complete the handshake due to bad certificate.

This actually "might be ok" tbh depending on what is going on... For example, if you have a tunneler running somewhere, and you're tearing down the network (`docker compose down -v`) as I suspect you might be doing, it's **quite common** for that older tunneler to keep trying to connect the _NEW_ overlay network, using an _OLD_ PKI (which is dead, because a whole new one was generated), and fail. If that's the situation, you'll see this `{error=[local error: tls: bad record MAC]} handshake failed` over and over and over again on a relatively periodic interval. Now, the errors in your log all look to be coming in the same second, so that doesn't **seem** to be the problem here.

Can we just do this, how about we just stop the `ziti-router`, and maybe stop the `ziti-console` if you have that running to and take it down to the bare minimum of _just_ the `ziti-controller`.

Let's add each piece in one at a time and let's see if there's some kind of race condition I've never experienced because for me it's not been this slow? That sound ok?

So basically in that compose file I referenced before, comment out the whole section for the `ziti-edge-rotuer` and `ziti-console`... dump the whole network using `docker compose down -v`, up the network again using `docker compose up -d` (daemon mode)...

Tail the docker logs using `docker compose logs -f` and let them settle down, make sure you see NO log messages... If you do, that's an indicator that a ziti-edge-tunnel (or _some_ process) is trying to attach to the controller.. Find that process and stop it...

Once things seem ok, let's just add back the `ziti-edge-router` service, and then run `docker compose up -d` again... The router **should** come online....

Assuming it _DOES_... exec to the controller:

```auto
docker compose exec -it ziti-controller bash

```

run `zitiLogin` and let's just see the status of:

```auto
ziti edge list edge routers

```

There should be one and it should be online:

```auto
ziti@eb1988feb641:/persistent$ ziti edge list edge-routers
╭────────────┬──────────────────┬────────┬───────────────┬──────┬────────────╮
│ ID │ NAME │ ONLINE │ ALLOW TRANSIT │ COST │ ATTRIBUTES │
├────────────┼──────────────────┼────────┼───────────────┼──────┼────────────┤
│ bHPyFPKDKB │ ziti-edge-router │ true │ true │ 0 │ public │
╰────────────┴──────────────────┴────────┴───────────────┴──────┴────────────╯
results: 1-1 of 1

```

 ![image](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/b/ba3771a4fe8db46fa3ada83435a3e7b263741cc7.png)

Can we try that?

---

<div class="post-metadata">

**Author:** ![Reverse5431](https://avatars.discourse-cdn.com/v4/letter/r/82dd89/32.png) [@Reverse5431](https://openziti.discourse.group/u/Reverse5431)\
**Post date:** [August 20, 2023, 12:50am UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/10 "2023-08-20T00:50:17Z")

</div>

Alright, so I had to regenerate the certificate as something was wrong with it. Then I did run “docker compose down -v” which deleted removed the containers, network and volume. I commented out the edge-router and the console, and then run again docker compose up -d. The handshake failed still occurs, and is the only error that keeps occurring post initialization. No other logs indicating that any process is attempting to connect to the controller.

> **handshake**
>
> open-ziti-controller-1 | [9.647] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[remote error: tls: bad certificate]} handshake failed  
> …  
> open-ziti-controller-init-container-1 | This docker volume has been initialized.  
> open-ziti-controller-init-container-1 exited with code 0  
> open-ziti-controller-1 | [50.514] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[remote error: tls: unknown certificate authority]} handshake failed

I noticed that visiting [edge.example.com:8441](http://edge.example.com:8441) provides a NetFoundry issued certificate, rather than the Let’s Encrypt one provided via .env file. Is this intended, as part of the mTLS connections you mentioned before? Also shot port 8441 be changed to 443, so to bypass some network restrictions that block almost any port other than port 443?  
 ![image](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/7/77eec2711c04808dff6f7b1a6a501a4dd262231a.png)

In then uncommented the ziti-edge-router, and run docker compose up -d again. There are a few error/warning here and there, but the router manages to connect to the controller. I have verified it with “ziti edge list edge-routers”.  
 ![image](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/4/4ba64267db5c74bacd166b2ed754d2ef45bdcd63.png)

> **with the edge router**
>
> open-ziti-edge-router-1 | [0.426] WARNING edge/router/internal/edgerouter.(\*Config).LoadConfigFromMap: Invalid heartbeat interval [0] (min: 60, max: 10), setting to default [60]  
> open-ziti-edge-router-1 | [0.440] WARNING fabric/router/xlink\_transport.loadListenerConfig: {addr=[tls:0.0.0.0:8444] error=[no network interface found for 0.0.0.0]} unable to get interface for address  
> open-ziti-edge-router-1 | [0.477] WARNING edge/tunnel/dns.flushDnsCaches: {error=[exec: “resolvectl”: executable file not found in $PATH]} unable to find systemd-resolve or resolvectl in path, consider adding a dns flush to your restart process  
> open-ziti-edge-router-1 | [13.947] ERROR channel/v2.(\*heartbeater).sendHeartbeat: {error=[context deadline exceeded] channelId=[ch{ctrl}-\>u{reconnecting}-\>i{xxxx}]} failed to send heartbeat  
> open-ziti-controller-1 | [732.309] ERROR transport/v2/tls.(\*sharedListener).processConn [tls:0.0.0.0:8441]: {error=[remote error: tls: bad certificate]} handshake failed

Also, I can now access ZAC without any problem, but when visiting it for the first time for a browser window, ZAC prints this error five times:

> **console error**
>
> open-ziti-console-1 | [session-file-store] will retry, error on last attempt: Error: ENOENT: no such file or directory, open ‘sessions/random\_alphanumerical\_lf.json’

Are these errors negligible, and not to worry about?

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [August 20, 2023, 2:24am UTC](https://openziti.discourse.group/t/using-public-ca-certificates-with-docker-compose/1529/11 "2023-08-20T02:24:44Z")

</div>

### The Handshake Issue

> [@Reverse5431](#):
>
> The handshake failed still occurs, and is the only error that keeps occurring post initialization.

So it seems to never settle down? that's what you're saying, right? If that's the case, there's something very strange happening. If you have a browser tab open to "[https://edge.example.com:8441](https://edge.example.com:8441)", or if you have a `ziti-edge-tunnel`/or Ziti Desktop Edge for Windows/Mac running, or maybe a `ziti router` process running somewhere? You can run that out **entirely** by redoing the entire setup again and changing the `.env` to use something like:

```auto
ZITI_CTRL_EDGE_ADVERTISED_PORT=18441

```

You could also use `netstat` or `ss` if you're familiar with those or tcpdump to track down exactly what process from what IP is attaching to your :8441 port. **Something** seems to be attempting to access it and is presenting a certificate, trying to create an mTLS connection that the controller is rejecting. That's most likely a router, I'd think.

> [@Reverse5431](#):
>
> I noticed that visiting [edge.example.com:8441](http://edge.example.com:8441) provides a NetFoundry issued certificate,

### Alternative Server Certs - Boring Details

> [@Reverse5431](#):
>
> I noticed that visiting [edge.example.com:8441](http://edge.example.com:8441) provides a NetFoundry issued certificate

This is the helpful hint I needed. With this, I think I finally understand the difference between what you're doing and what I always do and how I've tested/verified this whole thing in the past and why this is failing for you. It comes down to our notion of "Alternate Server Certs". Technically, I think what you're trying to do is **replace** the server certs, so they would be the **primary** certs and not use an alternate set. Let me explain...

When I use Alternate Server Certs, it's because I always use AWS and AWS provides a DNS name for it's VMs. So for example I'll have a DNS entry of "ec2-1.2.3.4" that AWS provided for my VM. That's my **primary** DNS entry. Then I'll also register something like "[controller.clint.demo.openziti.org](http://controller.clint.demo.openziti.org)" and that's my **alternate** DNS entry. I have **TWO** , but you only want to use one. Our notion of "alternate" server certs is to indicate that you want more than one DNS entry to find/locate your server.... But here, you're only using one. That changes things enough where we might need to do more work to get it working "right" if you want to use the quickstart. It wasn't built for this particular use case.

### Easiest Way to Address this Situation

Right now, imo, the easiest way to deal with this situation and use the quickstart would be to provision a second DNS name **just** for OpenZiti. For example, I just made `nf.clint.demo.openziti.org`. It points to `13.58.222.94`. You can access it at [https://clintnf.demo.openziti.org:8441/](https://clintnf.demo.openziti.org:8441/) and you'll see it uses the self-signed CA the quickstart creates. There's also [https://ctrl.clint.demo.openziti.org:8441/](https://ctrl.clint.demo.openziti.org:8441/) which points to the exact same place and uses docker as we've been discussing here. The _only_ difference is I have changed these two lines in my `.env` file:

```auto
ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=clintnf.demo.openziti.org
ZITI_CTRL_ADVERTISED_ADDRESS=clintnf.demo.openziti.org

```

### Conclusion

Are you willing to make two DNS entries, one for the self-signed PKI and one for the LE PKI? If you do that, change the `.env` file as shown just above, it should just fire up.

Sorry that this is complex, but there's many different ways to accomplish all these things and without knowing exactly what you were doing and how, my own biases got in the way a little bit from giving you better guidance at the start.

Let me know how you get on! Hopefully you'll be good now.