K8s Deployment not recognizing Web-client-identiy cert is expired

1

Hi,

I’ve ran into an issue with our deployment. Our web client identity cert has a short expiry config (lets say 10 days), when the cert expires the deployment doesn’t recognize this and issue a new CSR to the issuer. Effectively blocking connections until I manually replace the secret in k8s. Is this intended behavior?

Should I increase the cert expiration from a short window to a couple years? or do I need to have automation that kicks the secret before expiration and then reboots the controller pod so it loads the new cert/secret.

Let me know! And thanks for your time :slight_smile:

2

You’re correct, there are several components in a Ziti network that must have a writable identity so they can auto-renew, or you must managed the identity configuration file out-of-band, e.g., replace the certificate in the k8s secret.

This will be critical for networks in clustered mode because the list of controller nodes will change and must be saved in the same file, the identity configuration file, usually JSON.

Some deployments have the option to use a writable volume or a read-only k8s secret mounted on the pod. Which Helm chart are you using? Does the user guide describe a writable volume option?

3

Hi,

I am on chart version 2.0.0 and appVersion 1.6.2.

4

Which Helm chart are you using? I'm unsure which deployment model to look at.

5

Hi,

This is my controller chart file

apiVersion: v2
appVersion: 1.6.2
dependencies:
- condition: cert-manager.enabled
  name: cert-manager
  repository: 'file://charts/cert-manager/'
  version: ~1.14.0
- condition: trust-manager.enabled
  name: trust-manager
  repository: 'file://charts/trust-manager/'
  version: ~0.7.0
- condition: ingress-nginx.enabled
  name: ingress-nginx
  repository: 'file://charts/ingress-nginx/'
  version: ~4.10.1
description: Host an OpenZiti controller in Kubernetes
name: ziti-controller
type: application
version: 2.0.0
6

Which certificate is expiring prematurely? Is it the server certificate for the controller's web PKI that is presented by the client API with a name like <release>-web-identity-cert (helm-charts/charts/ziti-controller/templates/ca-web-identity.yaml at ziti-controller-2.0.0 · openziti/helm-charts · GitHub)?

7

the issue was not that the cert was expiring prematurely, but that when cert-manager issued a new cert, the pod didn’t get restarted to load in the new secret.

I added a cronjob to the helm deployment to rollout restart the controller pod 1 day before the 10 day cert expiry (so every 9 days) and this resolved my issue. Its probably a hacky workaround but for now it does the job lol