Encount a problem when Install BrowZer with docker

Hello, I tried to install BrowZer with Docker, but encountered an error. Here is the error log:

ubuntu@ip-10-1-2-216:~$ sudo docker run \
  --name ziti-browzer-bootstrapper \
  --rm -v /etc/letsencrypt:/etc/letsencrypt \
  --user "${UID}:2171" \
  -p ${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT}:${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT} \
  -e NODE_ENV="${NODE_ENV}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_LOGLEVEL="${ZITI_BROWZER_BOOTSTRAPPER_LOGLEVEL}" \
  -e ZITI_BROWZER_RUNTIME_LOGLEVEL="${ZITI_BROWZER_RUNTIME_LOGLEVEL}" \
  -e ZITI_BROWZER_RUNTIME_HOTKEY="${ZITI_BROWZER_RUNTIME_HOTKEY}" \
  -e ZITI_CONTROLLER_HOST="${ZITI_CONTROLLER_HOST}" \
  -e ZITI_CONTROLLER_PORT="${ZITI_CONTROLLER_PORT}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_HOST="${ZITI_BROWZER_BOOTSTRAPPER_HOST}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_SCHEME="${ZITI_BROWZER_BOOTSTRAPPER_SCHEME}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_CERTIFICATE_PATH="${ZITI_BROWZER_BOOTSTRAPPER_CERTIFICATE_PATH}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_KEY_PATH="${ZITI_BROWZER_BOOTSTRAPPER_KEY_PATH}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT="${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_TARGETS="${ZITI_BROWZER_BOOTSTRAPPER_TARGETS}" \
  ghcr.io/openziti/ziti-browzer-bootstrapper:latest
{"timestamp": "2024-05-30T09:07:39.448Z", "level": "info", "message":  "ZITI_BROWZER_BOOTSTRAPPER_LOG_PATH is null"}
/home/node/ziti-browzer-bootstrapper/index.js:921
    logger.error( e );
           ^

TypeError: Cannot read properties of undefined (reading 'error')
    at process.<anonymous> (/home/node/ziti-browzer-bootstrapper/index.js:921:12)
    at process.emit (node:events:513:28)
    at process._fatalException (node:internal/process/execution:149:25)
ubuntu@ip-10-1-2-216:~$ echo $NODE_ENV
production
ubuntu@ip-10-1-2-216:~$ echo $ZITI_BROWZER_BOOTSTRAPPER_LOGLEVEL
debug
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_RUNTIME_LOGLEVEL}
debug
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_RUNTIME_HOTKEY}
alt+F12
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_CONTROLLER_HOST}
ctrl.${wildcard_url}
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_CONTROLLER_PORT}
8441
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_BOOTSTRAPPER_HOST}
browzer.${wildcard_url}
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_BOOTSTRAPPER_SCHEME}
https
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_BOOTSTRAPPER_CERTIFICATE_PATH}
/etc/letsencrypt/live/rootwang.link/fullchain.pem
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_BOOTSTRAPPER_KEY_PATH}
/etc/letsencrypt/live/rootwang.link/privkey.pem
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT}
8446
ubuntu@ip-10-1-2-216:~$ echo ${ZITI_BROWZER_BOOTSTRAPPER_TARGETS}
{ "targetArray": [ { "vhost": "brozac.${wildcard_url}", "service": "brozac", "path": "/", "scheme": "http", "idp_issuer_base_url": "https://${AUTH0_DOMAIN}", "idp_client_id": "${AUTH0_CLIENTID}" } ] }

I replaced key information in the environment with variables, but all parameters have values.

Hi @rootwang,

Just to be clear, you have followed all the instructions from here, is this correct? Example Enabling BrowZer | OpenZiti

Did you try to pull that image? docker pull ghcr.io/openziti/ziti-browzer-bootstrapper:latest to make sure you have the actual latest? Maybe you could try a specific version?

We'll probably need @curt to comment on this one, he's THE browzer authority! :slight_smile:

Yes, I just followed this course step by step, everything was correctly until this step.
https://openziti.io/docs/learn/quickstarts/browzer/example/

I did everything you said and got same error.

ubuntu@ip-10-1-2-216:~$ sudo docker images
REPOSITORY                                   TAG       IMAGE ID       CREATED       SIZE
ghcr.io/openziti/ziti-browzer-bootstrapper   0.61.0    72b0be96ebe3   7 days ago    364MB
ghcr.io/openziti/ziti-browzer-bootstrapper   latest    72b0be96ebe3   7 days ago    364MB
certbot/certbot                              latest    a909abdebd11   8 weeks ago   110MB

At first, I thought it was because ZITI_BROWZER_BOOTSTRAPPER_LOG_PATH was not defined, but after looking at the source code, it allowed to be empty.

did you find this previous answer from the forum?

Could it be that the docker container can't write that log location? That does sort of alighn with your hint of looking at ZITI_BROWZER_BOOTSTRAPPER_LOG_PATH. You might be onto something.

Yes, I have read that, but I am not sure what permissions are needed to create the log directory, i tried ziggy, but not work. and if i want to create it in advance what the path for this directory should be.

@rootwang whatever is wrong happens so early in the bootstrappers start-up process that its logger hasn't been created.

To help get more trouble-shooting info, I just published a branch build that I hope will dump more info about what is happening.

When you have a moment, instead of pulling the latest bootstrapper image, please pull pr272.650.

Then run your same start process. Then post the output here.

Thanks!

I reinstalled it from the beginning later, and this time there were all right. The difference is i execute the "Copy PKI From Controller" step that i forgot last time. I wonder if that has anything to do with it, thank you.

This is the process of ziti-browzer-bootstrapper installed this time:

root@ip-10-1-2-73:~# docker run \
  --name ziti-browzer-bootstrapper \
  --rm -v /etc/letsencrypt:/etc/letsencrypt \
  --user "${UID}:2171" \
  -p ${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT}:${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT} \
  -e NODE_ENV="${NODE_ENV}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_LOGLEVEL="${ZITI_BROWZER_BOOTSTRAPPER_LOGLEVEL}" \
  -e ZITI_BROWZER_RUNTIME_LOGLEVEL="${ZITI_BROWZER_RUNTIME_LOGLEVEL}" \
  -e ZITI_BROWZER_RUNTIME_HOTKEY="${ZITI_BROWZER_RUNTIME_HOTKEY}" \
  -e ZITI_CONTROLLER_HOST="${ZITI_CONTROLLER_HOST}" \
  -e ZITI_CONTROLLER_PORT="${ZITI_CONTROLLER_PORT}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_HOST="${ZITI_BROWZER_BOOTSTRAPPER_HOST}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_SCHEME="${ZITI_BROWZER_BOOTSTRAPPER_SCHEME}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_CERTIFICATE_PATH="${ZITI_BROWZER_BOOTSTRAPPER_CERTIFICATE_PATH}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_KEY_PATH="${ZITI_BROWZER_BOOTSTRAPPER_KEY_PATH}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT="${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT}" \
  -e ZITI_BROWZER_BOOTSTRAPPER_TARGETS="${ZITI_BROWZER_BOOTSTRAPPER_TARGETS}" \
  ghcr.io/openziti/ziti-browzer-bootstrapper:PR272.650
Unable to find image 'ghcr.io/openziti/ziti-browzer-bootstrapper:PR272.650' locally
PR272.650: Pulling from openziti/ziti-browzer-bootstrapper
7d97e254a046: Pull complete
b323a342d564: Pull complete
5817cb8b5fae: Pull complete
8535a7dc8bd9: Pull complete
a5e8db731e10: Pull complete
2b6307250b3f: Pull complete
9c324bded5ad: Pull complete
4f4fb700ef54: Pull complete
Digest: sha256:0c482decd90ac9b4f83b9ae4c852dbf930394d4884ebfbebda75e130f1d10794
Status: Downloaded newer image for ghcr.io/openziti/ziti-browzer-bootstrapper:PR272.650
{"timestamp": "2024-06-03T05:35:42.592Z", "level": "info", "message":  "ZITI_BROWZER_BOOTSTRAPPER_LOG_PATH is null"}
{"level":"info","message":"ziti-browzer-bootstrapper initializing","timestamp":"2024-06-03T05:35:52.137Z","version":"0.61.1"}
{"field":"idp_type","level":"warn","message":"obsolete config field encountered - ignored","timestamp":"2024-06-03T05:35:52.145Z","version":"0.61.1"}
{"host":"ctrl.rootwang.link","level":"info","message":"contacting specified controller","port":"8441","timestamp":"2024-06-03T05:35:52.150Z","version":"0.61.1"}
{"level":"debug","message":"configured target service(s)","targets":{"targetArray":[{"idp_client_id":"J5rNgir9YjD7BKkmbkNiRua1NdmKW1uj","idp_issuer_base_url":"https://dev-8fczl84d262geely.us.auth0.com","path":"/","scheme":"http","service":"brozac","vhost":"brozac.rootwang.link"}]},"timestamp":"2024-06-03T05:35:54.852Z","version":"0.61.1"}
{"certificate_path":"/etc/letsencrypt/live/rootwang.link/fullchain.pem","key_path":"/etc/letsencrypt/live/rootwang.link/privkey.pem","level":"info","message":"new tlsContext created","timestamp":"2024-06-03T05:35:54.970Z","version":"0.61.1"}
{"level":"info","message":"listening","port":"8446","scheme":"https","timestamp":"2024-06-03T05:35:55.038Z","version":"0.61.1"}
{"controllerVersion":"1.0.0","level":"info","message":"attached controller version","timestamp":"2024-06-03T05:35:55.326Z","version":"0.61.1"}

@TheLumberjack @curt I just followed the video and got a big process, the only difference with the video is the zac that installed with docker, but when access the https://${ZITI_BROWZER_VHOST}:${ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT}, i got response like this.

ubuntu@ip-10-1-2-109:~$ curl https://brozac.rootwang.link:8446

<!doctype html>
<html>
  <head>

<!-- load JSPI Origin Trial Token -->
<meta http-equiv="origin-trial" id="ziti-browzer-origin-trial" content="AtUaUNUk/l7kBpCXAyB6W9KWymYJb8ydLcPLTkENXw88AJB0bXMUzwkwcPKy796N2h0KaCkq1zaAycQJ21jtpwIAAACAeyJvcmlnaW4iOiJodHRwczovL2Jyb3d6ZXIuY2xvdWR6aXRpLmlvOjQ0MyIsImZlYXR1cmUiOiJXZWJBc3NlbWJseUpTUHJvbWlzZUludGVncmF0aW9uIiwiZXhwaXJ5IjoxNzMwMjQ2Mzk5LCJpc1N1YmRvbWFpbiI6dHJ1ZX0=">
<!-- load Ziti browZer Runtime -->
<script id="from-ziti-browzer-bootstrapper" type="text/javascript" src="https://brozac.rootwang.link:8446/ziti-browzer-runtime-9cab69a5.js"></script>


<!-- load JSPI Origin Trial Token -->
<meta http-equiv="origin-trial" id="ziti-browzer-origin-trial" content="AtUaUNUk/l7kBpCXAyB6W9KWymYJb8ydLcPLTkENXw88AJB0bXMUzwkwcPKy796N2h0KaCkq1zaAycQJ21jtpwIAAACAeyJvcmlnaW4iOiJodHRwczovL2Jyb3d6ZXIuY2xvdWR6aXRpLmlvOjQ0MyIsImZlYXR1cmUiOiJXZWJBc3NlbWJseUpTUHJvbWlzZUludGVncmF0aW9uIiwiZXhwaXJ5IjoxNzMwMjQ2Mzk5LCJpc1N1YmRvbWFpbiI6dHJ1ZX0=">
<!-- load Ziti browZer Runtime -->
<script id="from-ziti-browzer-bootstrapper" type="text/javascript" src="https://brozac.rootwang.link:8446/ziti-browzer-runtime-9cab69a5.js"></script>

    <title>OpenZiti BrowZer Bootstrapper</title>
  </head>
  <body></body>
</html>

The url of response is

https://brozac.rootwang.link:8446/browzer_error?browzer_error_data==%7B%22status%22:409,%22code%22:1014,%22title%22:%22OriginTrial%20subdomain%20mismatch%20for%20feature%20%5BWebAssemblyJSPromiseIntegration%5D%22,%22message%22:%22Expected%20origin%20%5B*.cloudziti.io%5D%20but%20is%20accessed%20from%20%5B*.rootwang.link%5D%22,%22myvar%22:%7B%22type%22:%22zbr%22%7D%7D

and decode it

{"status":409,"code":1014,"title":"OriginTrial subdomain mismatch for feature [WebAssemblyJSPromiseIntegration]","message":"Expected origin [*.cloudziti.io] but is accessed from [*.rootwang.link]","myvar":{"type":"zbr"}}

and i looked this url and did clear site data, but not worked. Then i enable JSPI, but still did not work.


Finally i set the ZITI_BROWZER_RUNTIME_ORIGIN_TRIAL_TOKEN variable and restarted browzer-bootstrapper with "sudo systemctl restart browzer-bootstrapper", but still not work.

which step i was wrong. thanks.

@TheLumberjack when i was run

source /dev/stdin <<< "$(wget -qO- https://get.openziti.io/ziti-cli-functions.sh)"; expressInstall

finally i got

PKI generated successfully

********         Setting Up Controller        ********
adding controller root CA to ca bundle: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/pki/ip-10-1-2-25-root-ca/certs/ip-10-1-2-25-root-ca.cert
adding signing root CA to ZITI_PKI_CTRL_CA: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/pki/cas.pem
wrote CA file to: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/pki/cas.pem
adding parent intermediate CA to ZITI_PKI_SIGNER_CERT: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/pki/signing.pem
adding grandparent intermediate CA to ZITI_PKI_SIGNER_CERT: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/pki/signing.pem
wrote signer cert file to: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/pki/signing.pem
Controller configuration file written to: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/ip-10-1-2-25.yaml
ZITI_HOME overridden: /home/ubuntu/.ziti/quickstart/ip-10-1-2-25
  --- There was an error while initializing the controller, check the logs at /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/ip-10-1-2-25-init.log ---

ubuntu@ip-10-1-2-25:~/.ziti$ more /home/ubuntu/.ziti/quickstart/ip-10-1-2-25/ip-10-1-2-25-init.log 
[   0.006]   FATAL ziti/controller/subcmd.configureController: {error=[unable to load identity (open /etc/letsencrypt/live/rootwang2023.link/fullchain.pem: permission denied)]} could not read configuration file [/home/ubuntu/.ziti/quickstart/ip-10-1-2-25

I found the problem aboat permission denied, but the permission is right

ubuntu@ip-10-1-2-25:~/.ziti$ sudo ls -l /etc/letsencrypt/live/${wildcard_url}
total 4
-rw-r--r-- 1 root zitiweb 692 Jun  4 15:10 README
lrwxrwxrwx 1 root zitiweb  41 Jun  4 15:10 cert.pem -> ../../archive/rootwang2023.link/cert1.pem
lrwxrwxrwx 1 root zitiweb  42 Jun  4 15:10 chain.pem -> ../../archive/rootwang2023.link/chain1.pem
lrwxrwxrwx 1 root zitiweb  46 Jun  4 15:10 fullchain.pem -> ../../archive/rootwang2023.link/fullchain1.pem
lrwxrwxrwx 1 root zitiweb  44 Jun  4 15:10 privkey.pem -> ../../archive/rootwang2023.link/privkey1.pem

As root (or whatever runs your controller), can you cat the file? It's possible that the link is broken/wrong or that you're not running as root (or a user in zitiweb)?

sudo head -c10 /etc/letsencrypt/live/rootwang2023.link/fullchain.pem

I follow these instructions routinely, I'm pretty sure they're still valid. It might be worthwhile to start clean. If you started the controller as "you" that might explain the issue

I am so sure that the link is right and had no problem if i login with root, but the error is reported if i login with ubuntu. I saw the BrowZer enabled video, it is also not use root.
One more thing, it doesn't happen every time with the same OS and operator, which is quite strange. I'll keep looking into the cause. Thank you.

Is there anything about your install that's different? That video was made back when we only had the quickstart type deployment guides. Did you follow that guide or did you do it any other way? Getting the permissions right with letsencrypt can sometimes feel tricky. If you're not running the quickstart like that page shows, maybe that's the difference?

Are there any other possible differences?

The reason I have already found is that an additional variable ZITI_PWD was set, but I don't know why. If you have time, could you please take a look at this issue for me? Thank you.

There's no way to set an "additional" variable. I'm not sure what you mean. That env var is only used when you bootstrap the overlay for the first time, and when you try to use the alias that is put into your shell when you source the .env file named zitiLogin.

You don't ever actually need that variable at all, once you setup your overlay. I'm not sure what you're asking.

The problem i have already resloved, actually, what I would like you to help me with is another issue.