I got a hiccup with my carrier and, as a result, there was a change in the public IP assigned to my home router (through DHCP). I removed and reinstalled openziti in my Debian and my laptops worked fine again after that (the identities created and jwt files downloaded into the laptops + services associated to those identities performed properly from the laptops viewpoint.).
What stopped working (and was working before the hiccup with the carrier) was after I had the Ziti mobile Edge reinstalled into my mobile. When I try to enroll an identity into the mobile I get "Unable to enroll QRScanxxxxxxxx.jwt CONTROLLER_UNAVAILABLE".
I tried with my mobile inside my network and out of the Internet with the same error.
Here is the log from my mobile ...
Edge:AppDelegate.swift:28 init() io.netfoundry.ZitiMobilePacketTunnel Version: 2.47 (525), OS: Version 15.8.3 (Build 19H386)
[2024-11-08T18:39:57:290Z] INFO Ziti Mobile Edge:TunnelMgr.swift:127 loadFromPreferences() Updating log level to 3 (INFO)
[2024-11-08T18:39:57:294Z] INFO Ziti Mobile Edge:Logger.swift:242 updateRotateSettings() Updating log rotate config to daily:true, count:2, sizeMB:5
[2024-11-08T18:40:17:294Z] INFO Ziti Mobile Edge:UserNotifications.swift:94 requestAuth() Auth request authorized? true
[2024-11-08T18:40:17:310Z] INFO Ziti Mobile Edge:TunnelMgr.swift:102 loadFromPreferences() Saved successfully. Re-loading preferences
[2024-11-08T18:40:17:346Z] ERROR Ziti Mobile Edge:TunnelMgr.swift:105 loadFromPreferences() Re-loaded preferences, error=false
2024-11-08 12:40:20.395 Ziti Mobile Edge[42726:21835713] invalid mode 'kCFRunLoopCommonModes' provided to CFRunLoopRunSpecific - break on _CFRunLoopError_RunCalledWithInvalidMode to debug. This message will only appear once per execution.
[2024-11-08T18:40:20:398Z] INFO Ziti Mobile Edge:TunnelMgr.swift:193 startTunnel() starting tunnel
[2024-11-08T18:40:20:398Z] INFO Ziti Mobile Edge:TunnelMgr.swift:195 startTunnel() start tunnel called with no error
[2024-11-08T18:40:45:070Z] INFO Ziti Mobile Edge:Array+ZitiIdentity.swift:25 updateIdentity() QRScan1731091243.jwt:y0RtjiBjA CHANGED
(42726)[2024-11-08T18:40:56.985Z] INFO ziti-sdk:utils.c:198 ziti_log_set_level() set log level: root=3/INFO
(42726)[2024-11-08T18:40:56.985Z] INFO ziti-sdk:utils.c:169 ziti_log_init() Ziti C SDK version 1.1.5 @g2120296(HEAD) starting at (2024-11-08T18:40:56.988)
(42726)[2024-11-08T18:40:56.985Z] INFO ziti-sdk:ziti_enroll.c:91 ziti_enroll() Ziti C SDK version 1.1.5 @g2120296(HEAD) starting enrollment at (2024-11-08T18:40:56.988)
(42726)[2024-11-08T18:40:56.985Z] INFO ziti-sdk:ziti_ctrl.c:593 ziti_ctrl_init() ctrl[(null):] using https://ziti:8441
(42726)[2024-11-08T18:40:57.027Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T18:40:57.027Z] WARN ziti-sdk:ziti_ctrl.c:319 internal_version_cb() ctrl[ziti:8441] CONTROLLER_UNAVAILABLE(unknown node or service)
(42726)[2024-11-08T18:40:57.027Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T18:40:57.027Z] INFO ziti-sdk:ziti_ctrl.c:183 ctrl_resp_cb() ctrl[ziti:8441] attempting to switch endpoint
(42726)[2024-11-08T18:40:57.027Z] WARN ziti-sdk:ziti_ctrl.c:566 ctrl_next_ep() ctrl[ziti:8441] no controllers are online
(42726)[2024-11-08T18:40:57.027Z] ERROR ziti-sdk:ziti_enroll.c:249 well_known_certs_cb() /Users/runner/work/ziti-sdk-swift/ziti-sdk-swift/deps/ziti-tunnel-sdk-c/build-iphoneos-arm64/_deps/ziti-sdk-c-src/library/ziti_enroll.c:144 - ZITI_JWT_VERIFICATION_FAILED => -7 (JWT verification failed)
[2024-11-08T18:40:57:028Z] ERROR CZiti:ZitiEnroller.swift:213 on_enroll() CONTROLLER_UNAVAILABLE
[2024-11-08T18:40:57:029Z] ERROR CZiti:Ziti.swift:327 enroll() Optional(Error Domain=ZitiError Code=-7 "CONTROLLER_UNAVAILABLE" UserInfo={NSLocalizedDescription=CONTROLLER_UNAVAILABLE})
[2024-11-08T18:40:58:134Z] INFO Ziti Mobile Edge:Array+ZitiIdentity.swift:25 updateIdentity() QRScan1731091243.jwt:y0RtjiBjA CHANGED
(42726)[2024-11-08T19:00:37.984Z] INFO ziti-sdk:ziti_enroll.c:91 ziti_enroll() Ziti C SDK version 1.1.5 @g2120296(HEAD) starting enrollment at (2024-11-08T19:00:37.983)
(42726)[2024-11-08T19:00:37.984Z] INFO ziti-sdk:ziti_ctrl.c:593 ziti_ctrl_init() ctrl[(null):] using https://ziti:8441
(42726)[2024-11-08T19:00:38.018Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T19:00:38.018Z] WARN ziti-sdk:ziti_ctrl.c:319 internal_version_cb() ctrl[ziti:8441] CONTROLLER_UNAVAILABLE(unknown node or service)
(42726)[2024-11-08T19:00:38.018Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T19:00:38.018Z] INFO ziti-sdk:ziti_ctrl.c:183 ctrl_resp_cb() ctrl[ziti:8441] attempting to switch endpoint
(42726)[2024-11-08T19:00:38.018Z] WARN ziti-sdk:ziti_ctrl.c:566 ctrl_next_ep() ctrl[ziti:8441] no controllers are online
(42726)[2024-11-08T19:00:38.018Z] ERROR ziti-sdk:ziti_enroll.c:249 well_known_certs_cb() /Users/runner/work/ziti-sdk-swift/ziti-sdk-swift/deps/ziti-tunnel-sdk-c/build-iphoneos-arm64/_deps/ziti-sdk-c-src/library/ziti_enroll.c:144 - ZITI_JWT_VERIFICATION_FAILED => -7 (JWT verification failed)
[2024-11-08T19:00:38:019Z] ERROR CZiti:ZitiEnroller.swift:213 on_enroll() CONTROLLER_UNAVAILABLE
[2024-11-08T19:00:38:020Z] ERROR CZiti:Ziti.swift:327 enroll() Optional(Error Domain=ZitiError Code=-7 "CONTROLLER_UNAVAILABLE" UserInfo={NSLocalizedDescription=CONTROLLER_UNAVAILABLE})
[2024-11-08T19:00:40:311Z] INFO Ziti Mobile Edge:Array+ZitiIdentity.swift:25 updateIdentity() QRScan1731091243.jwt:y0RtjiBjA CHANGED
[2024-11-08T19:08:48:327Z] INFO Ziti Mobile Edge:Array+ZitiIdentity.swift:25 updateIdentity() QRScan1731092927.jwt:iAB.k7Tyl CHANGED
(42726)[2024-11-08T19:08:56.337Z] INFO ziti-sdk:ziti_enroll.c:91 ziti_enroll() Ziti C SDK version 1.1.5 @g2120296(HEAD) starting enrollment at (2024-11-08T19:08:56.337)
(42726)[2024-11-08T19:08:56.337Z] INFO ziti-sdk:ziti_ctrl.c:593 ziti_ctrl_init() ctrl[(null):] using https://ziti:8441
(42726)[2024-11-08T19:08:56.351Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T19:08:56.351Z] WARN ziti-sdk:ziti_ctrl.c:319 internal_version_cb() ctrl[ziti:8441] CONTROLLER_UNAVAILABLE(unknown node or service)
(42726)[2024-11-08T19:08:56.351Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T19:08:56.351Z] INFO ziti-sdk:ziti_ctrl.c:183 ctrl_resp_cb() ctrl[ziti:8441] attempting to switch endpoint
(42726)[2024-11-08T19:08:56.351Z] WARN ziti-sdk:ziti_ctrl.c:566 ctrl_next_ep() ctrl[ziti:8441] no controllers are online
(42726)[2024-11-08T19:08:56.351Z] ERROR ziti-sdk:ziti_enroll.c:249 well_known_certs_cb() /Users/runner/work/ziti-sdk-swift/ziti-sdk-swift/deps/ziti-tunnel-sdk-c/build-iphoneos-arm64/_deps/ziti-sdk-c-src/library/ziti_enroll.c:144 - ZITI_JWT_VERIFICATION_FAILED => -7 (JWT verification failed)
[2024-11-08T19:08:56:352Z] ERROR CZiti:ZitiEnroller.swift:213 on_enroll() CONTROLLER_UNAVAILABLE
[2024-11-08T19:08:56:353Z] ERROR CZiti:Ziti.swift:327 enroll() Optional(Error Domain=ZitiError Code=-7 "CONTROLLER_UNAVAILABLE" UserInfo={NSLocalizedDescription=CONTROLLER_UNAVAILABLE})
[2024-11-08T19:08:57:399Z] INFO Ziti Mobile Edge:Array+ZitiIdentity.swift:25 updateIdentity() QRScan1731092927.jwt:iAB.k7Tyl CHANGED
[2024-11-08T19:12:31:692Z] INFO Ziti Mobile Edge:AdvancedViewController.swift:91 prepare() No connection data available
[2024-11-08T19:12:45:330Z] INFO Ziti Mobile Edge:LogLevelViewController.swift:33 tableView() selected row at 4
[2024-11-08T19:12:45:330Z] INFO Ziti Mobile Edge:TunnelMgr.swift:251 updateLogLevel() Updating log level to DEBUG
(42726)[2024-11-08T19:12:45.330Z] INFO ziti-sdk:utils.c:198 ziti_log_set_level() set log level: root=4/DEBUG
[2024-11-08T19:12:45:339Z] INFO Ziti Mobile Edge:TunnelMgr.swift:263 updateLogLevel() Updated providerConfiguration: ["interceptMatchedDns": 1, "logRotateCount": 2, "lowPowerMode": 0, "logRotateSizeMB": 5, "subnet": 255.192.0.0, "dns": 100.64.0.2, "logRotateDaily": 1, "mtu": 4000, "ip": 100.64.0.1, "fallbackDns": 1.1.1.1, "fallbackDnsEnabled": 0, "logLevel": "4"]
[2024-11-08T19:12:45:416Z] INFO Ziti Mobile Edge:TunnelMgr.swift:270 updateLogLevel() Sending logLevel DEBUG to provider
[2024-11-08T19:12:45:417Z] DEBUG Ziti Mobile Edge:IpcAppClient.swift:73 sendToAppex() SetLogLevel: {"meta":{"msgId":"EDD12A01-A2E8-4BC0-8AD9-B09ADD79CD9A","msgType":3},"logLevel":4}
(42726)[2024-11-08T19:12:45.330Z] INFO ziti-sdk:ziti_enroll.c:91 ziti_enroll() Ziti C SDK version 1.1.5 @g2120296(HEAD) starting enrollment at (2024-11-08T19:12:55.784)
(42726)[2024-11-08T19:12:45.330Z] DEBUG ziti-sdk:jwt.c:82 load_jwt() filename is: /private/var/mobile/Containers/Shared/AppGroup/E33F80D8-6E9E-4814-BFA4-99D14CD2C3B5/iAB.k7Tyl.jwt
(42726)[2024-11-08T19:12:45.330Z] DEBUG ziti-sdk:jwt.c:75 load_jwt_file() jwt file content is:
eyJhbGciOiJSUzI1NiIsImtpZCI6IjVkNjBkY2I2NDQ2NzZmMDc3YTZiMDRiMmEzMzk5YWM2Zjc3MmI2ZWYiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL3ppdGk6ODQ0MSIsInN1YiI6ImlBQi5rN1R5bCIsImF1ZCI6WyIiXSwiZXhwIjoxNzMxMTAzNjM1LCJqdGkiOiIyMWJjYTgwMC0yOWUxLTQzZjAtYWJjYS02N2I4NjBhNTZlM2UiLCJlbSI6Im90dCIsImN0cmxzIjpudWxsfQ.PEx6EvgLcJWpHKn4B9Y0l2PX28KauUBKpTE9OhVuA2dh41HIs8PR-dHXbVGzK0QIt7sLqaF89jby9HTQNmK8mSmnGyNGSka4d7tqmCsSOb8V6v8RXJuXVrb4l4qFrFY9AUh9av7VtOgleHf4snRyV4A5MSxZ2aYVEFwI1jl9xIVOG9UANzIeEHxBoY0sH3HtqJKnid_0r-tDmKGOX-LKaZRMaqfAetTfHgDUSIHos6SSFXvDUJ2rxJr3HAgENPy9ALziptQry4SuT8LrJ8se3XIcoHyTurb4nBvt7JR7v4nazivbrUzb-hzlu-Hzm7-xaEgp_NKF0qU_MA043KBz0vtogouBqUOBUa4zaKMBSAvbm187hAaMTOAoDAlgb4qPF3w4rxNZLh3AJ28b1paQCLJRm26w6mwG80peAqVFyjghig8XXEgTenXsm4aEo2SN2sEJvR7N0dKFOMAPFBufEzQXIZCyHv2G2r1eUo2nndvDDdgHeBb6pH9RtWZ4eszusOLy4EcxcC-p1J_tQXGzT1Nffbr7c0_NbIoV1z59HQVUL3JSTrTHJn6JNeRu7wDaX9ZGKv_hLQDn_H7XUdw3NdkX-JEzMmSWwaDJ2R6js0QUCaxpwCVYB-kJy3wzpOkor1sfEsIwA7Lg3UmbOyEdHE2qVtwbNNqhi2bp0rTZvZY
(42726)[2024-11-08T19:12:45.330Z] DEBUG ziti-sdk:jwt.c:36 parse_jwt_content() ecfg->jwt_signing_input is:
eyJhbGciOiJSUzI1NiIsImtpZCI6IjVkNjBkY2I2NDQ2NzZmMDc3YTZiMDRiMmEzMzk5YWM2Zjc3MmI2ZWYiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL3ppdGk6ODQ0MSIsInN1YiI6ImlBQi5rN1R5bCIsImF1ZCI6WyIiXSwiZXhwIjoxNzMxMTAzNjM1LCJqdGkiOiIyMWJjYTgwMC0yOWUxLTQzZjAtYWJjYS02N2I4NjBhNTZlM2UiLCJlbSI6Im90dCIsImN0cmxzIjpudWxsfQ
(42726)[2024-11-08T19:12:45.330Z] INFO ziti-sdk:ziti_ctrl.c:593 ziti_ctrl_init() ctrl[(null):] using https://ziti:8441
(42726)[2024-11-08T19:12:45.330Z] DEBUG ziti-sdk:ziti_ctrl.c:607 ziti_ctrl_init() ctrl[ziti:8441] ziti controller client initialized
(42726)[2024-11-08T19:12:55.787Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T19:12:55.787Z] WARN ziti-sdk:ziti_ctrl.c:319 internal_version_cb() ctrl[ziti:8441] CONTROLLER_UNAVAILABLE(unknown node or service)
(42726)[2024-11-08T19:12:55.787Z] WARN ziti-sdk:ziti_ctrl.c:180 ctrl_resp_cb() ctrl[ziti:8441] request failed: -3008(unknown node or service)
(42726)[2024-11-08T19:12:55.787Z] INFO ziti-sdk:ziti_ctrl.c:183 ctrl_resp_cb() ctrl[ziti:8441] attempting to switch endpoint
(42726)[2024-11-08T19:12:55.787Z] WARN ziti-sdk:ziti_ctrl.c:566 ctrl_next_ep() ctrl[ziti:8441] no controllers are online
(42726)[2024-11-08T19:12:55.787Z] DEBUG ziti-sdk:ziti_enroll.c:143 well_known_certs_cb() err->message is: unknown node or service
(42726)[2024-11-08T19:12:55.787Z] ERROR ziti-sdk:ziti_enroll.c:249 well_known_certs_cb() /Users/runner/work/ziti-sdk-swift/ziti-sdk-swift/deps/ziti-tunnel-sdk-c/build-iphoneos-arm64/_deps/ziti-sdk-c-src/library/ziti_enroll.c:144 - ZITI_JWT_VERIFICATION_FAILED => -7 (JWT verification failed)
[2024-11-08T19:12:55:788Z] ERROR CZiti:ZitiEnroller.swift:213 on_enroll() CONTROLLER_UNAVAILABLE
[2024-11-08T19:12:55:788Z] ERROR CZiti:Ziti.swift:327 enroll() Optional(Error Domain=ZitiError Code=-7 "CONTROLLER_UNAVAILABLE" UserInfo={NSLocalizedDescription=CONTROLLER_UNAVAILABLE})
[2024-11-08T19:12:58:336Z] INFO Ziti Mobile Edge:Array+ZitiIdentity.swift:25 updateIdentity() QRScan1731092927.jwt:iAB.k7Tyl CHANGED
Hi,
It looks like the enrollment failed because the controller's hostname ziti could not be resolved:
Is this really the controller's hostname, and do you expect your phone to be able to resolve it? I'd think you'd need custom DNS settings on the phone for a bare hostname to work - either a DNS server that knows how to answer for "ziti" or the domain for the "ziti" host is specified in your search domains.
Can you browse to your controller at https://ziti:8441 from Safari?
During the reinstallation of openziti in Debian I exported the DNS name used for my home router (on the WAN side) -> export $EXTERNAL_DNS="DNS name for my home router on the WAN side". I used that to browse https://:8441/zac from my laptop (inside my WiFi network) and it worked. I can see in ziti.yaml file two lines with reference to ziti:8441. After changing them to the DNS name assigned to the WAN side of my router and restarting the controller I could enroll the identity in my mobile w/o problems.
Not sure how ziti:8441 was in the file instead of the DNS name assigned to my home router on the WAN side. Any wrong step I might have taken during reinstallation of openziti package in my Debian? I did remove .ziti directory, .config/ziti/directory, /etc/systemd/system/ziti* files and killed -9 controller and router processes before reinstalling ... Did I forget anything in the cleanup?
By the way, "ziti" is the hostname seen in /etc/hosts file in my Debian ...
The issuer in the jwt ultimately comes from the controller's config file. I'm not sure but I think it's edge -> api -> address
I managed to fix the problem. By changing the hostname in my Debian to the DNA name assigned to the WAN side of my home router it worked as a charm. I could enroll the identity to my mobile and my laptops inside my WiFi at home were still working.
Thanks for the pointer.
1 Like