It’s a weird bug. It’s hard to catch them.
Connections to Ziti services intermittently fail after ~5 minutes of stable connectivity. The issue is random and affects different services at different times. Restarting the Ziti Edge Desktop client temporarily resolves the problem.
What I have?
- 3 public ziti-routers
- ziti-edge-tunnel as DaemonSet in my EKS.
- ~ 30 ziti services
- ~ 30 ziti host v1
- ~ 30 ziti dial
- ~ 30 ziti bind
- ~ 30 ziti intercept v1
- ~ 30 ziti users identities for ziti edge desktop ( mac and windows )
Steps to Reproduce
-
Connect to Ziti using Ziti Edge Desktop.
-
Verify that services are reachable (Service A and Service B).
-
Wait approximately 5 minutes.
-
Observe random connection failures:
- Example: Connection refused for Service A, Service B remains accessible.
-
Restart Ziti Edge Desktop. All services become reachable again.
-
Wait ~5 minutes. Another service may fail (e.g., Service B fails, Service A OK).
-
Sometimes both services fail.
Note: The failures are random; no specific service consistently fails first.
Ziti edge desktop logs (TRACE)
(31125)[2026-01-02T12:27:37.500Z] ERROR ziti-sdk:connect.c:1070 connect_reply_cb() conn[1.285/9K7-Ptrh/Connecting](hello.ziti) failed to connect, reason=invalid session
(31125)[2026-01-02T12:27:37.500Z] VERBOSE ziti-sdk:connect.c:129 conn_set_state() conn[1.285/9K7-Ptrh/Connecting](hello.ziti) transitioning Connecting => Disconnected
(31125)[2026-01-02T12:27:37.500Z] DEBUG ziti-sdk:connect.c:323 complete_conn_req() conn[1.285/9K7-Ptrh/Disconnected](hello.ziti) Disconnected failed: connection is closed
(31125)[2026-01-02T12:27:37.500Z] VERBOSE ziti-sdk:connect.c:129 conn_set_state() conn[1.285/9K7-Ptrh/Disconnected](hello.ziti) transitioning Disconnected => Disconnected
(31125)[2026-01-02T12:27:37.500Z] TRACE ziti-sdk:channel.c:461 ziti_channel_send() ch[3] => ct[StateClosedType] seq[1226] len[0]
(31125)[2026-01-02T12:27:37.500Z] TRACE ziti-sdk:channel.c:435 ziti_channel_send_message() ch[3] => ct[StateClosedType] seq[1226] len[0]
(31125)[2026-01-02T12:27:37.500Z] TRACE ziti-sdk:channel.c:405 on_channel_send() ch[3] write delay = 0.000d q=1 qs=32
(31125)[2026-01-02T12:27:37.500Z] VERBOSE tunnel-cbs:ziti_tunnel_cbs.c:93 on_ziti_connect() on_ziti_connect status: -24
(31125)[2026-01-02T12:27:37.500Z] ERROR tunnel-cbs:ziti_tunnel_cbs.c:103 on_ziti_connect() ziti dial failed: connection is closed
(31125)[2026-01-02T12:27:37.500Z] VERBOSE ziti-sdk:connect.c:129 conn_set_state() conn[1.285/9K7-Ptrh/Disconnected](hello.ziti) transitioning Disconnected => Closed
(31125)[2026-01-02T12:27:37.500Z] TRACE ziti-sdk:connect.c:810 flush_connection() conn[1.285/9K7-Ptrh/Closed](hello.ziti) starting flusher
(31125)[2026-01-02T12:27:37.500Z] TRACE ziti-sdk:channel.c:932 on_channel_data() ch[3] read no data
(31125)[2026-01-02T12:27:37.500Z] TRACE tlsuv:tlsuv.c:418 finished reading after 2 iterations
(31125)[2026-01-02T12:27:37.500Z] DEBUG ziti-sdk:connect.c:893 flush_to_client() conn[1.285/9K7-Ptrh/Closed](hello.ziti) no data_cb: can't flush, 0 bytes available
(31125)[2026-01-02T12:27:37.500Z] TRACE ziti-sdk:connect.c:879 flush_to_service() conn[1.285/9K7-Ptrh/Closed](hello.ziti) flushed 0 messages
(31125)[2026-01-02T12:27:37.500Z] TRACE ziti-sdk:connect.c:803 on_flush() conn[1.285/9K7-Ptrh/Closed](hello.ziti) stopping flusher
(31125)[2026-01-02T12:27:37.500Z] DEBUG ziti-sdk:connect.c:184 close_conn_internal() conn[1.285/9K7-Ptrh/Closed](hello.ziti) removing
(31125)[2026-01-02T12:27:37.500Z] TRACE tunnel-cbs:ziti_tunnel_cbs.c:602 ziti_conn_close_cb() ziti_conn[0x12f711fc0] is closed
(31125)[2026-01-02T12:27:37.500Z] DEBUG tunnel-sdk:ziti_tunnel.c:446 ziti_tunneler_close() closing connection: client[tcp:100.64.0.1:62188] service[hello.ziti]
(31125)[2026-01-02T12:27:37.500Z] DEBUG tunnel-sdk:tunnel_tcp.c:252 tunneler_tcp_close() closing src[tcp:100.64.0.1:62188] dst[tcp:100.64.0.5:443] state[3/SYN_RCVD] flags[0x100] service[hello.ziti]
Ziti edge router logs
{"_context":"ch{edge}-\u003eu{classic}-\u003ei{ziti-sdk-c[4]@mac-pro-14.local/pVNP}","chSeq":275,"connId":373,"edgeSeq":0,"error":"invalid session","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:199","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2026-01-02T12:46:00.039Z","token":"eyJhbGciOiJSUzI1NiIsImtpZCI6IjFhZWVjMmY5Mzc4MWMzZTMyOGI1ZjNhMzk2NDA3NmMwYWJmNzQxMmIiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2VkZ2Uueml0aS5tYWd4LmFwcDo0NDMiLCJzdWIiOiIxRVVWUzJ4VlV5Q3A0bFk2dlJkUUV3IiwiYXVkIjpbIm9wZW56aXRpIl0sImV4cCI6MTc5ODg5MTkxOSwiaWF0IjoxNzY3MzU1OTE5LCJqdGkiOiJjbWp3dTJ6YW14ZDl6MGM4NjMxcmt1b2Z6Iiwiel9hc2lkIjoiY21qd3UycTh5eGQ4YTBjODZ2MDJvbzZ4cyIsInpfaWlkIjoibnRVdEk3Ull1Iiwiel90IjoicyIsInpfc3QiOiJEaWFsIiwiel9sZWciOnRydWV9.uvIGp7v8znaonP4BPQGBWTJOC8_LKPzF_Sl59lWqr4q4AOREIWs8gHdHv5SF3STEFC-igVOVRForNQU7QdijK1T-qmR7VhNM9-4mSlEGWRN7u_o-1skKABczNxJs-TYO5vGi8mrd2uVUphdn2TinEROcp_nE5vEaU1I_rYSvM_CcUficjmV9nsZWKCCrNovWXUbaVdU8A1ZE8cpj9kZNOXCD2CBTLRB7eH-5gNrMhwxVhTYEAkRGIbex7-aZXg1lz0p_SIHr_e6gC59RmgBE4tbH-CIqF_wKbZ-SrlMk7QjY1V6CDV86lW8oPw4OaPJkm-Jjs1HYGrV20KCH7N1cTaR0O8D0B-c-FrSbgpqgablZL6nOv7Pu-r_0IIgA_av4UoykTOYKN_iUEtJzuSsJiz554yVDbpOYCnchCqXWUrJxOz0WffVFmJOkvSPEXMcLCVTBeaWBE386JOa1RVWXRpqaW3DsHV4CqcS7AeTyqLlmue2mxTenvhqq0-dfqGQJivR97fOs1CcCidxHUJCPhCtMyxQXKzSsnWfJiLhzVkX4mcWWvAkpJfhPNlNjiOa3y-7wM-9u6JCaCjd8ZMD7QZCqf7HSaUdstasgAJwInBBIdFRG4P0nnNIIZ7g_VDmXq-hC70in5r4QtjfAHRPmRPl7wFGG2L24B_KcdfwX0xk","type":"EdgeConnectType"}
Ziti edge controller logs
{"error":"invalid session","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:334","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseSessionRequestContext).loadFromBolt","level":"error","msg":"invalid session","operation":"create.terminator","time":"2026-01-02T12:48:29.512Z"}
{"_context":"ch{geJiU9KJr}-\u003eu{classic}-\u003ei{geJiU9KJr/OvAd}","error":"invalid session","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/create_terminator_v2.go:206","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*createTerminatorV2Handler).returnError","level":"error","msg":"responded with error","routerId":"geJiU9KJr","terminatorId":"5NQobnMm964fdwJHHXK3Wo","time":"2026-01-02T12:48:29.512Z"}
Observed Behavior
-
Connections work for a short period (~5 minutes).
-
After ~5 minutes, connections randomly fail with
invalid session. -
Restarting the Ziti Edge Desktop client temporarily restores connectivity.
-
Failures can affect different services each time; sometimes multiple services fail simultaneously.
I don’t know why it’s working only for ~5 minutes?
This bug makes it hard to maintain stable connectivity to services through Ziti Edge Tunnel. Further investigation into session management and possible race conditions in the Ziti Edge SDK or Router is needed.