K8s browzer deployment: TypeError: Cannot read properties of undefined (reading 'error')

Support General Questions
1

Hi,
I am currently trying to run browzer on my K8s (OKD) cluster.
The deployment configuration looks like this:

kind: Deployment
apiVersion: apps/v1
metadata:
  name: ziti-browzer-bootstrapper
  namespace: demo-ziti-browzer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ziti-browzer-bootstrapper
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: ziti-browzer-bootstrapper
        deployment: ziti-browzer-bootstrapper
    spec:
      volumes:
        - name: logs
          persistentVolumeClaim:
            claimName: browzer-logs
        - name: certs
          secret:
            secretName: ziti-demo-tld-cert
            defaultMode: 420
      containers:
        - resources: {}
          terminationMessagePath: /dev/termination-log
          name: ziti-browzer-bootstrapper
          env:
            - name: NODE_ENV
              value: production
            - name: ZITI_BROWZER_RUNTIME_HOTKEY
              value: alt+F12
            - name: ZITI_CONTROLLER_HOST
              value: demo-openziti-controller-client.demo-ziti-controller.svc
            - name: ZITI_CONTROLLER_PORT
              value: '443'
            - name: ZITI_BROWZER_BOOTSTRAPPER_LOGLEVEL
              value: debug
            - name: ZITI_BROWZER_RUNTIME_LOGLEVEL
              value: debug
            - name: ZITI_BROWZER_BOOTSTRAPPER_HOST
              value: connect.demo.tld
            - name: ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT
              value: '1443'
            - name: ZITI_BROWZER_BOOTSTRAPPER_SCHEME
              value: https
            - name: ZITI_BROWZER_BOOTSTRAPPER_CERTIFICATE_PATH
              value: /certs/tls.crt
            - name: ZITI_BROWZER_BOOTSTRAPPER_KEY_PATH
              value: /certs/tls.key
            - name: ZITI_BROWZER_BOOTSTRAPPER_LOG_PATH
              value: /logs/browzer.logs
          ports:
            - containerPort: 8000
              protocol: TCP
            - containerPort: 8443
              protocol: TCP
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: logs
              mountPath: /logs
            - name: certs
              mountPath: /certs
          terminationMessagePolicy: File
          envFrom:
            - configMapRef:
                name: demo-ziti-browzer-targets
          image: 'ghcr.io/openziti/ziti-browzer-bootstrapper@sha256:cf220f73b9f3712e2b2e4bf632411b4c2bd1decbbca2241b4261fe24bf415da8'
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 25%
      maxSurge: 25%
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

kind: ConfigMap
apiVersion: v1
metadata:
  name: demo-ziti-browzer-targets
  namespace: demo-ziti-browzer
data:
  ZITI_BROWZER_BOOTSTRAPPER_TARGETS: |-
    {
      "targetArray": [
        {
          "vhost": "wiki.demo.tld",
          "service": "outline.demo-wiki.svc",
          "path": "/",
          "scheme": "http",
          "idp_issuer_base_url": "https://sso.demo.tld/realms/demo-internal",
          "idp_client_id": "openziti"
        }
      ]
    }

The starating pod instantly crashes - the only log line outputted to stdout/stderr is

{"timestamp": "2024-03-22T00:32:08.882Z", "level": "info", "message": "ZITI_BROWZER_BOOTSTRAPPER_LOG_PATH is /logs/browzer.logs"}

Inside of /logs/browzer.logs there following is logged

TypeError: Cannot read properties of undefined (reading 'error')
    at process.<anonymous> (/home/node/ziti-browzer-bootstrapper/index.js:751:12)
    at process.emit (node:events:513:28)
    at process._fatalException (node:internal/process/execution:149:25)
/home/node/ziti-browzer-bootstrapper/index.js:751
    logger.error( e );
           ^

TypeError: Cannot read properties of undefined (reading 'error')
    at process.<anonymous> (/home/node/ziti-browzer-bootstrapper/index.js:751:12)
    at process.emit (node:events:513:28)
    at process._fatalException (node:internal/process/execution:149:25)
/home/node/ziti-browzer-bootstrapper/index.js:751
    logger.error( e );
           ^

TypeError: Cannot read properties of undefined (reading 'error')
    at process.<anonymous> (/home/node/ziti-browzer-bootstrapper/index.js:751:12)
    at process.emit (node:events:513:28)
    at process._fatalException (node:internal/process/execution:149:25)

(Ziti was configured as described here.)

My debugging stopped here as I tested some things out but couldn't produce a different error message and the error itself isn't quite helpful for further investigations

Do you have any ideas/hints what to try next?

Thanks in advance!

1 Like
2

Hi,

I've invested some time in remote debugging and found out that this here was the actual error:

Uncaught Error Error: EACCES: permission denied, mkdir 'log'
    at handleErrorFromBinding (node:internal/fs/utils:346:5)
    at mkdirSync (node:fs:1382:3)
    at eval (repl:1:4)

Which is thrown on this line: ziti-browzer-bootstrapper/index.js at 2c7890f0d9c1d04cb7e07472a15116f80c0c29ee · openziti/ziti-browzer-bootstrapper · GitHub

Are there config properties which allow me to switch to stdout/stderr logging only? This would be quite neat for me as K8s folks are propably gonna end up redirecting these logs into Loki (or an equivalent solution) using a K8s operator.

Gonna need more sleep to directly correlate this error to my infra setup but I was kinda happy to find this error so I wanted to share it right away.

3

If you prefer to write logs to a file, not stdout, then you probably need an init container to set owner and mode on the mounted volume aligning with the configured log path.

@elysweyr The log path option variable ZITI_BROWZER_BOOTSTRAPPER_LOG_PATH is used in one place, the Docker image's entrypoint script. Its only purpose is to redirect output to the configured file in the optional variable. If you stop defining the variable then logs will flow to stdout.

EDIT: found the answer, deleted my exploratory remarks