I’ve managed to set up a few services!
On the other hand, I seem to have broken the zac secure link, client log shows:
[2025-10-29T20:58:47.745Z] ERROR tunnel-cbs:ziti_tunnel_cbs.c:103 on_ziti_connect() ziti dial failed: connection is closed
[2025-10-29T20:58:47.906Z] ERROR ziti-sdk:connect.c:1068 connect_reply_cb() conn1.50/rQ8p8WHr/Connecting failed to connect, reason=invalid session
How should I troubleshoot on the controller/edge side?
Look at the logs from the router and controller. There should be something relevant and helpful in there. You're hosting the ZAC on the controller like I showed, right? Can you get to the management api itself? (i would presume no?)
Haven’t changed how the containers run, just some editing through zac.
http://secured-apis.ziti:$zacport from the client with tunneler I cannot reach.
curl https://ziti-controller:$zacport/edge/management/v1 --insecure
from inside the ziti-controller returns expected output
some bits from edge-router:
[1934.144] INFO ziti/router/xgress_edge.(*hostedServiceRegistry).evaluateEstablishQueue: {state=[establishing] serviceSessionTokenId=[cmhcjw5d401zw0a9grpr5jgss] terminatorId=[4mWqXHtF1HW6nao8J0ZNLx]} queuing terminator to send create
[1934.144] INFO ziti/router/xgress_edge.(*hostedServiceRegistry).establishTerminator: {terminatorId=[4mWqXHtF1HW6nao8J0ZNLx] serviceSessionToken=[map[apiSessionId:cmhciom3y000r0b9g1kbwhewb identityId:eBQaOXuEDm serviceId:43goV7ylfrfVLQAsiuGvko tokenId:cmhcjw5d401zw0a9grpr5jgss type:JWT]] apiSessionToken=[map[apiSessionId:cmhciom3y000r0b9g1kbwhewb certFingerprints:[e286670ecb8496d2e1dde2d61598e384e70c9941] identityId:eBQaOXuEDm tokenId:gR2u8Q465CKiUwlScrWi1Jno1FA type:legacyProtobuf]] routerId=[.B7V21VKp8]} sending create terminator v2 request
[1934.147] INFO ziti/router/xgress_edge.(*hostedServiceRegistry).Remove: {terminatorId=[4mWqXHtF1HW6nao8J0ZNLx] reason=[invalid session]} terminator removed from router set
[1934.968] ERROR ziti/router/xgress_edge.(*edgeClientConn).processBind [ch{edge}->u{classic}->i{ziti-sdk-c[0]@jellyfin/Z3OZ}]: {edgeSeq=[0] routerId=[.B7V21VKp8] connId=[0] type=[EdgeBindType] error=[api session id (cmhciofy500030b9guk7u7myk) does not match service session api session id (cmhcgei6b0cwp0c9gb1w4n4mn)] chSeq=[197]} unable to verify service session token
[1935.421] INFO ziti/router/xgress_edge.(*edgeClientConn).processBindV2 [ch{edge}->u{classic}->i{ziti-sdk-c[0]@zititest/ZQdA}]: {bindConnId=[5] terminatorId=[3tqAqk5p6MLNzNClQocK9g] serviceSessionToken=[map[apiSessionId:cmhciom3y000r0b9g1kbwhewb identityId:eBQaOXuEDm serviceId:43goV7ylfrfVLQAsiuGvko tokenId:cmhcjw6co01zz0a9gisi0uilc type:JWT]] edgeSeq=[0] apiSessionToken=[map[apiSessionId:cmhciom3y000r0b9g1kbwhewb certFingerprints:[e286670ecb8496d2e1dde2d61598e384e70c9941] identityId:eBQaOXuEDm tokenId:gR2u8Q465CKiUwlScrWi1Jno1FA type:legacyProtobuf]] type=[EdgeBindType] listenerId=[QB'>�
52H��C5�JV����`f�q����] chSeq=[799] connId=[5] routerId=[.B7V21VKp8]} establishing terminator
[1935.421] INFO ziti/router/xgress_edge.(*hostedServiceRegistry).evaluateEstablishQueue: {terminatorId=[3tqAqk5p6MLNzNClQocK9g] state=[establishing] serviceSessionTokenId=[cmhcjw6co01zz0a9gisi0uilc]} queuing terminator to send create
[1935.421] INFO ziti/router/xgress_edge.(*hostedServiceRegistry).establishTerminator: {apiSessionToken=[map[apiSessionId:cmhciom3y000r0b9g1kbwhewb certFingerprints:[e286670ecb8496d2e1dde2d61598e384e70c9941] identityId:eBQaOXuEDm tokenId:gR2u8Q465CKiUwlScrWi1Jno1FA type:legacyProtobuf]] routerId=[.B7V21VKp8] terminatorId=[3tqAqk5p6MLNzNClQocK9g] serviceSessionToken=[map[apiSessionId:cmhciom3y000r0b9g1kbwhewb identityId:eBQaOXuEDm serviceId:43goV7ylfrfVLQAsiuGvko tokenId:cmhcjw6co01zz0a9gisi0uilc type:JWT]]} sending create terminator v2 request
[1935.423] INFO ziti/router/xgress_edge.(*hostedServiceRegistry).Remove: {terminatorId=[3tqAqk5p6MLNzNClQocK9g] reason=[invalid session]} terminator removed from router set
some bits from controller:
[1952.416] ERROR ziti/controller/handler_edge_ctrl.(*createTerminatorV2Handler).returnError [ch{.B7V21VKp8}->u{classic}->i{.B7V21VKp8/3X6V}]: {terminatorId=[2PKg0DrWUpuFnQqFafh3gg] error=[invalid session] routerId=[.B7V21VKp8]} responded with error
[1952.495] ERROR ziti/controller/handler_edge_ctrl.(*baseSessionRequestContext).loadFromBolt: {operation=[create.terminator] error=[invalid session]} invalid session
[1952.495] ERROR ziti/controller/handler_edge_ctrl.(*createTerminatorV2Handler).returnError [ch{.B7V21VKp8}->u{classic}->i{.B7V21VKp8/3X6V}]: {error=[invalid session] routerId=[.B7V21VKp8] terminatorId=[6LvLeyLMH7qiVys2FTOajr]} responded with error
[1957.789] ERROR ziti/controller/handler_edge_ctrl.(*baseSessionRequestContext).loadFromBolt: {operation=[create.terminator] error=[invalid session]} invalid session
[1957.789] ERROR ziti/controller/handler_edge_ctrl.(*createTerminatorV2Handler).returnError [ch{.B7V21VKp8}->u{classic}->i{.B7V21VKp8/3X6V}]: {terminatorId=[7Cqztry2Vnq5JMCGuLZlwI] error=[invalid session] routerId=[.B7V21VKp8]} responded with erro
Hmmm. To be honest, I'm not sure I have a good picture of what might be going on.
It looks to me like you're running ZAC using the controller as an API like I showed in the video/compose. Is it possible the location the ZAC was unzipped to was deleted? That ZAC is an SPA so it seems odd you can get to the controller mgmt api but not ZAC unless the files are simply gone.
The non-printable chars in the listener seems - strange too. I'd be interested to konw if you restart the router if it works again. It's possible there some sort of strange bug that's manifesting.
I'm not sure what's up with those
[1952.416] ERROR ziti/controller/handler_edge_ctrl.(*createTerminatorV2Handler).returnError [ch{.B7V21VKp8}->u{classic}->i{.B7V21VKp8/3X6V}]: {terminatorId=[2PKg0DrWUpuFnQqFafh3gg] error=[invalid session] routerId=[.B7V21VKp8]} responded with error
[1952.495] ERROR ziti/controller/handler_edge_ctrl.(*baseSessionRequestContext).loadFromBolt: {operation=[create.terminator] error=[invalid session]} invalid session
[1952.495] ERROR ziti/controller/handler_edge_ctrl.(*createTerminatorV2Handler).returnError [ch{.B7V21VKp8}->u{classic}->i{.B7V21VKp8/3X6V}]: {error=[invalid session] routerId=[.B7V21VKp8] terminatorId=[6LvLeyLMH7qiVys2FTOajr]} responded with error
I'll see if I point someone else there. This thread is getting long in the tooth though. It would probably make sense to start a new thread. If restarting the router 'fixes' it obviously that's good AND bad... I'd start there and see where to go after that
I’ll mark your post as solution and will create a new thread for this specific issue, thanks.