Hi there,
I've set up a new instance of OpenZiti to test some of the newer features next to our working prod environment.
I'm very impressed by the HA work - cool stuff! 
After understanding the PKI generation and how the clusters work together I think it's very straightforward. I found some missing parts in the documentation which I like to submit as issues, but that shouldn't be part of this topic
I tried using ext JWT signers combined with Browzer and HA but didn't manage to get it to work, I would love some help here, please find my deployment information below.
When trying to set up the JWT signer from scratch, I've come across this error:
ziti@ctrl1:~$ ziti edge delete auth-policy browzer-authentik-auth-policy
delete of auth-policy with id 6VCrqCTukacdje2mWY8XoV: FAIL
error: error deleting auth-policies/6VCrqCTukacdje2mWY8XoV instance in Ziti Edge Controller at https://alt.ctrl1.zt.test.deltasecure.de:8441/edge/management/v1. Status code: 500 Internal Server Error, Server returned: {
"error": {
"cause": {
"code": "UNHANDLED",
"message": "{\"LocalId\":\"6VCrqCTukacdje2mWY8XoV\",\"LocalType\":\"authPolicy\",\"RemoteField\":\"authPolicyId\",\"RemoteIds\":[\"iXiPMlFtHX\"],\"RemoteType\":\"identity\"}"
},
"code": "UNHANDLED",
"message": "An unhandled error occurred",
"requestId": "WWYP1lcpn"
},
"meta": {
"apiEnrollmentVersion": "0.0.1",
"apiVersion": "0.0.1"
}
}
[ble: exit 1]
Edit: I just found out this happens when trying to delete an auth-policy that's still referenced by an identity. I assume an error is expected behaviour, but also this error?
I can't manage to get Browzer and Ext JWT to work.
Here's the Browser Browzer Console Logs:
Ziti BrowZer Runtime is now Bootstrapping
ziti-browzer-runtime-CbL5am6h.js:216254 ZBR Logging Begins...
ziti-browzer-logo.svg:1
Failed to load resource: the server responded with a status of 500 (ServerError)Understand this error
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-info Controller Version acquired: v1.5.4
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-info externalJwtSigners acquired: Array(1)
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-debug ################ SW register completed ################
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-debug sending msg: ZBR_INIT_COMPLETE
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-debug sending msg: SET_CONFIG
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-debug ZBSW: Got response from network.
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-debug sending msg: SET_COOKIE - __ziti-browzer-config eyJjb250cm9sbGVyIjp7ImFwaSI6Imh0dHBzOi8vYWx0LmN0cmwxLnp0LnRlc3QuZGVsdGFzZWN1cmUuZGU6ODQ0MS9lZGdlL2NsaWVudC92MSJ9LCJicm93emVyIjp7ImJvb3RzdHJhcHBlciI6eyJzZWxmIjp7InNjaGVtZSI6Imh0dHBzIiwiaG9zdCI6ImJyb3d6ZXIuenQudGVzdC5kZWx0YXNlY3VyZS5kZSIsInBvcnQiOiI4NDQ2IiwidmVyc2lvbiI6IjAuODcuNCJ9LCJ0YXJnZXQiOnsic2VydmljZSI6ImJyb3d6ZXIuenQudGVzdCIsInBhdGgiOiIvIiwic2NoZW1lIjoiaHR0cHMifX0sInN3Ijp7ImxvY2F0aW9uIjoieml0aS1icm93emVyLXN3LmpzIiwidmVyc2lvbiI6IjAuODMuMCIsImxvZ0xldmVsIjoiZGVidWcifSwicnVudGltZSI6eyJzcmMiOiJ6aXRpLWJyb3d6ZXItcnVudGltZS1DYkw1YW02aC5qcyIsImNzcyI6InppdGktYnJvd3plci1jc3MtQ1FzSVNTZFQuY3NzIiwibG9nTGV2ZWwiOiJkZWJ1ZyIsIm9yaWdpblRyaWFsVG9rZW4iOiJBajQyOGxFUFJLb2paMzNJbGNkaWsrQ1M2SjdzTngwcndLMlBWMU1MbGJLd014LzJ6cEZSUWdZdWtTMzRNeGg0YVdiSEJhbHdyaUl3bnpua2dUL3NjUTRBQUFDTmV5SnZjbWxuYVc0aU9pSm9kSFJ3Y3pvdkwySnlMV2RoZEdWM1lYa3RkR1Z6ZEMxNmRDNWtaV3gwWVhObFkzVnlaUzVrWlRvME5ETWlMQ0ptWldGMGRYSmxJam9pVjJWaVFYTnpaVzFpYkhsS1UxQnliMjFwYzJWSmJuUmxaM0poZEdsdmJpSXNJbVY0Y0dseWVTSTZNVGMxTXpFME1qUXdNQ3dpYVhOVGRXSmtiMjFoYVc0aU9uUnlkV1Y5In0sImxvYWRiYWxhbmNlciI6e30sIndoaXRlbGFiZWwiOnsiZW5hYmxlIjp7ImJyb3daZXJCdXR0b24iOnRydWUsImJyb3daZXJUb2FzdE1lc3NhZ2VzIjp0cnVlLCJicm93WmVyU3BsYXNoU2NyZWVuIjp0cnVlLCJicm93WmVyVGhyb3VnaHB1dENoYXJ0Ijp0cnVlfSwiYnJhbmRpbmciOnsiYnJvd1plck5hbWUiOiJPcGVuWml0aSBCcm93WmVyIiwiYnJvd1plclNwbGFzaE1lc3NhZ2UiOiJTdGFuZCBieSB3aGlsZSBPcGVuWml0aSBCcm93WmVyIEJvb3RzdHJhcHMgeW91ciBwcml2YXRlIHdlYiBhcHAiLCJicm93WmVyQnV0dG9uSWNvblN2Z1VybCI6Imh0dHBzOi8vYnJvd3plci56dC50ZXN0LmRlbHRhc2VjdXJlLmRlL3ppdGktYnJvd3plci1sb2dvLnN2ZyIsImJyb3daZXJDU1MiOiJodHRwczovL2Jyb3d6ZXIuenQudGVzdC5kZWx0YXNlY3VyZS5kZS96aXRpLWJyb3d6ZXItY3NzLUNRc0lTU2RULmNzcyJ9fX0sImlkcCI6eyJob3N0IjoiaHR0cHM6Ly9sb2dpbi5kZWx0YXNlY3VyZS5kZS9hcHBsaWNhdGlvbi9vL2V4dGp3dC10ZXN0LyIsImNsaWVudElkIjoiT2FrbFFLcUpCOG5xak5GaTdGUVIydTFUSnBKTkZMbjYzV0tldzZ3WiJ9fQ%3D%3D
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-debug ################ loadedViaBootstrapper detected -- doing page reload in 1sec ################
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-info ZBSW: Controller Version acquired:
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-info ZBSW: externalJwtSigners acquired:
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-error ZBSW: TypeError: Failed to fetch
log @ ziti-browzer-runtime-CbL5am6h.js:172615Understand this error
ziti-browzer-sw--ziti-browzer-core--c02849a7.js:113400 Uncaught (in promise) Error: The request could not be completed. The session is not authorized or the credentials are invalid
at ZitiContext.fetchServices (ziti-browzer-sw--ziti-browzer-core--c02849a7.js:113400:15)
at async ZitiContext.getServiceConfigByName (ziti-browzer-sw--ziti-browzer-core--c02849a7.js:113506:9)
at async ZitiContext.getConnectAppDataByServiceName (ziti-browzer-sw--ziti-browzer-core--c02849a7.js:113613:20)
at async ZitiContext.initialize (ziti-browzer-sw--ziti-browzer-core--c02849a7.js:112308:37)
at async ziti-browzer-sw--ziti-browzer-sw-workbox-strategies--5e6e634c.js:460:27Understand this error
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-error ZBSW: The request could not be completed. The session is not authorized or the credentials are invalid
log @ ziti-browzer-runtime-CbL5am6h.js:172615Understand this error
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-debug ZBSW: doing raw internet fetch for [https://browzer.zt.test.deltasecure.de/ziti-browzer-css-CQsISSdT.css]
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-error ZBSW: Got error: [TypeError: Failed to fetch]
log @ ziti-browzer-runtime-CbL5am6h.js:172615Understand this error
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-warn ZBSW: Unable to get a response from the network. Will respond with a cached response.
log @ ziti-browzer-runtime-CbL5am6h.js:172615Understand this warning
ziti-browzer-runtime-CbL5am6h.js:172615 ZBR-error ZBSW: no-response when trying to reach URL [https://browzer.zt.test.deltasecure.de/ziti-browzer-css-CQsISSdT.css]
log @ ziti-browzer-runtime-CbL5am6h.js:172615Understand this error
/ziti-browzer-css-CQsISSdT.css:1
Failed to load resource: the server responded with a status of 500 (ServerError)
browzer.env
ziti@br-gateway-test-zt:~/ziti-browzer-bootstrapper$ cat browzer.env
ZITI_BROWZER_BOOTSTRAPPER_HOST="browzer.zt.test.deltasecure.de"
ZITI_BROWZER_BOOTSTRAPPER_LOGLEVEL="debug"
ZITI_BROWZER_RUNTIME_LOGLEVEL="debug"
ZITI_BROWZER_RUNTIME_HOTKEY="alt+F12"
ZITI_CONTROLLER_HOST="alt.ctrl1.zt.test.deltasecure.de"
ZITI_CONTROLLER_PORT="8441"
ZITI_BROWZER_BOOTSTRAPPER_SCHEME="https"
ZITI_BROWZER_BOOTSTRAPPER_CERTIFICATE_PATH="/etc/letsencrypt/live/browzer.zt.test.deltasecure.de/fullchain.pem"
ZITI_BROWZER_BOOTSTRAPPER_KEY_PATH="/etc/letsencrypt/live/browzer.zt.test.deltasecure.de/privkey.pem"
ZITI_BROWZER_BOOTSTRAPPER_LISTEN_PORT="8446"
export ZITI_BROWZER_BOOTSTRAPPER_TARGETS='{
"targetArray": [
{
"vhost": "browzer.zt.test.deltasecure.de",
"service": "browzer.zt.test",
"path": "/",
"scheme": "https",
"idp_issuer_base_url": "https://login.deltasecure.de/application/o/extjwt-test/",
"idp_client_id": "myclientid"
}
]
}'
NODE_EXTRA_CA_CERTS=node_modules/node_extra_ca_certs_mozilla_bundle/ca_bundle/ca_intermediate_root_bundle.pem
ctrl1.yaml
---
v: 3
cluster:
dataDir: /home/ziti/cluster_data
identity:
cert: /home/ziti/pki/ctrl1/certs/client.chain.pem
server_cert: /home/ziti/pki/ctrl1/certs/server.chain.pem
key: /home/ziti/pki/ctrl1/keys/server.key
ca: /home/ziti/pki/root_ca/certs/cas.pem
alt_server_certs:
- server_cert: /etc/letsencrypt/live/alt.ctrl1.zt.test.deltasecure.de/fullchain.pem
server_key: /etc/letsencrypt/live/alt.ctrl1.zt.test.deltasecure.de/privkey.pem
edge:
api:
sessionTimeout: 30m
address: ctrl1.zt.test.deltasecure.de:8441
enrollment:
signingCert:
cert: /home/ziti/pki/ctrl1/certs/ctrl1.cert
key: /home/ziti/pki/ctrl1/keys/ctrl1.key
edgeIdentity:
duration: 180m
edgeRouter:
duration: 180m
ctrl:
listener: tls:0.0.0.0:8440
options:
advertiseAddress: tls:ctrl1.zt.test.deltasecure.de:8440
web:
- name: all-apis-localhost
bindPoints:
- interface: 0.0.0.0:8441
address: ctrl1.zt.test.deltasecure.de:8441
options:
minTLSVersion: TLS1.2
maxTLSVersion: TLS1.3
apis:
- binding: health-checks
- binding: fabric
- binding: edge-management
- binding: zac
options:
location: ./console
indexFile: index.html
- binding: edge-client
- binding: edge-oidc
Anything coming to mind already from the config? The alt_server_cert is issued to alt.ctrl1.zt.test.deltasecure.de
I didn't try issuing alt certs to ctrl2 and ctrl3 yet. Would that be necessary since I'm also only able to specify ctrl1 within the browzer bootstrapper?
Thanks!
