# Failure of Ziti-Edge-Tunnel Gateway Mode on Linux **URL:** \ **Category:** Uncategorized\ **Created:** [March 6, 2024, 5:34pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219 "2024-03-06T17:34:35Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 6, 2024, 5:34pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/1 "2024-03-06T17:34:35Z")
In the same environment, I attempted to use different agents, where both macOS and Windows could successfully expose services under the Tunnel network as gateways. However, when trying the Linux version of the Tunnel, I couldn't access it in any way. To address this, I tried switching between multiple cloud providers (such as AWS and Aliyun) for testing, but the issue persisted. How should I troubleshoot this? Because I haven't seen any useful error logs. Also, there shouldn't be any issues with the controller's configuration, as it works fine on Windows or macOS. ---
**Author:** ![scareything](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/scareything/32/106_2.png) [@scareything](https://openziti.discourse.group/u/scareything)\ **Post date:** [March 6, 2024, 6:36pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/2 "2024-03-06T18:36:17Z")
Hello and welcome to the OpenZiti discourse! Could you please share the logs from ziti-edge-tunnel? The logs will be in different places depending on how you started it: if you start it from the command line then you see the logs on the terminal, and if you start it as a systemd service then you'll see the logs with `journalctl -u ziti-edge-tunnel | cat`. Could you also show me the OpenZiti service(s) that you're trying to use with this `ziti-edge-tunnel`, as well as the related service policies? My _guess_ is that you need to give the ziti-edge-tunnel's identity `bind` permission for the service(s) that you're trying to use, but the logs and service info will help us narrow it down for certain. Thanks! ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 7, 2024, 4:08am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/3 "2024-03-07T04:08:35Z")
**Version information is as follows:** Kernel ：6.5.0-1014-aws #14~22.04.1-Ubuntu x86\_64 GNU/Linux Controller/Route: v0.32.2 ZAC: 2.9.4 ziti-edge-tunnel: v0.22.24 Mac ziti desktop edge: v0.22.24 **ziti-edge-tunnel logs** : ```shell # journalctl -u ziti-edge-tunnel Mar 05 03:42:35 ip-172-31-21-121 systemd[1]: Starting Ziti Edge Tunnel... Mar 05 03:42:35 ip-172-31-21-121 ziti-edge-tunnel.sh[5416]: NOTICE: no new JWT files in /opt/openziti/etc/identities/*.jwt Mar 05 03:42:35 ip-172-31-21-121 systemd[1]: Started Ziti Edge Tunnel. Mar 05 03:42:35 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[0.000] ERROR ziti-edge-tunnel:instance-config.c:61 load_config_from_file() The config file No such file or directory cannot be opened due to /var/lib/ziti/config.json. This is normal if this is a new install or if the c> Mar 05 03:42:35 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[0.000] ERROR ziti-edge-tunnel:instance-config.c:61 load_config_from_file() The config file No such file or directory cannot be opened due to /var/lib/ziti/config.json.backup. This is normal if this is a new install or i> Mar 05 03:42:35 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[0.000] WARN ziti-edge-tunnel:instance-config.c:98 load_tunnel_status_from_file() Config files /var/lib/ziti/config.json and the backup file cannot be read or they do not exist, will create a new config file or the old > Mar 05 03:42:35 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[0.064] ERROR ziti-edge-tunnel:instance-config.c:136 save_tunnel_status_to_file() Could not copy config file [/var/lib/ziti/config.json] to backup config file, the config might not exists at the moment Mar 05 03:42:40 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[4.888] ERROR ziti-sdk:ziti_enroll.c:123 ziti_enroll() /github/workspace/build/_deps/ziti-sdk-c-src/library/ziti_enroll.c:108 - load_jwt_content(ecfg, &ecfg->zejh, &ecfg->zej) => -4 (JWT has invalid format) Mar 05 03:42:40 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[4.888] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:319 tunnel_enroll_cb() enrollment failed: enroll failed(-4) Mar 05 03:42:40 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[4.889] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:629 on_cmd() received from client - EOF. Closing connection. Mar 05 03:42:40 ip-172-31-21-121 ziti-edge-tunnel[5417]: (5417)[4.889] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:641 on_cmd() IPC client connection closed, count: 0 Mar 05 03:42:52 ip-172-31-21-121 systemd[1]: ziti-edge-tunnel.service: Main process exited, code=dumped, status=11/SEGV Mar 05 03:42:52 ip-172-31-21-121 systemd[1]: ziti-edge-tunnel.service: Failed with result 'core-dump'. Mar 05 03:42:55 ip-172-31-21-121 systemd[1]: ziti-edge-tunnel.service: Scheduled restart job, restart counter is at 1. Mar 05 03:42:55 ip-172-31-21-121 systemd[1]: Stopped Ziti Edge Tunnel. Mar 05 03:42:55 ip-172-31-21-121 systemd[1]: Starting Ziti Edge Tunnel... Mar 05 03:42:55 ip-172-31-21-121 systemd[1]: Started Ziti Edge Tunnel. Mar 05 03:42:55 ip-172-31-21-121 ziti-edge-tunnel.sh[5456]: NOTICE: no new JWT files in /opt/openziti/etc/identities/*.jwt Mar 05 03:42:55 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[0.000] WARN ziti-edge-tunnel:instance.c:40 find_tunnel_identity() Identity ztx[/opt/openziti/etc/identities/HTTP Sever.json] is not loaded yet or already removed. Mar 05 03:43:17 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[21.351] ERROR ziti-sdk:ziti_enroll.c:123 ziti_enroll() /github/workspace/build/_deps/ziti-sdk-c-src/library/ziti_enroll.c:108 - load_jwt_content(ecfg, &ecfg->zejh, &ecfg->zej) => -4 (JWT has invalid format) Mar 05 03:43:17 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[21.351] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:319 tunnel_enroll_cb() enrollment failed: enroll failed(-4) Mar 05 03:43:17 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[21.352] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:629 on_cmd() received from client - EOF. Closing connection. Mar 05 03:43:17 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[21.352] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:641 on_cmd() IPC client connection closed, count: 0 Mar 05 03:44:34 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[98.638] ERROR ziti-sdk:ziti_enroll.c:123 ziti_enroll() /github/workspace/build/_deps/ziti-sdk-c-src/library/ziti_enroll.c:108 - load_jwt_content(ecfg, &ecfg->zejh, &ecfg->zej) => -4 (JWT has invalid format) Mar 05 03:44:34 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[98.638] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:319 tunnel_enroll_cb() enrollment failed: enroll failed(-4) Mar 05 03:44:34 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[98.639] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:629 on_cmd() received from client - EOF. Closing connection. Mar 05 03:44:34 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[98.639] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:641 on_cmd() IPC client connection closed, count: 0 Mar 05 03:45:18 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[142.765] ERROR ziti-sdk:ziti_enroll.c:123 ziti_enroll() /github/workspace/build/_deps/ziti-sdk-c-src/library/ziti_enroll.c:108 - load_jwt_content(ecfg, &ecfg->zejh, &ecfg->zej) => -4 (JWT has invalid format) Mar 05 03:45:18 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[142.765] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:319 tunnel_enroll_cb() enrollment failed: enroll failed(-4) Mar 05 03:45:18 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[142.766] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:629 on_cmd() received from client - EOF. Closing connection. Mar 05 03:45:18 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[142.766] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:641 on_cmd() IPC client connection closed, count: 0 Mar 05 03:45:39 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[164.068] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:629 on_cmd() received from client - EOF. Closing connection. Mar 05 03:45:39 ip-172-31-21-121 ziti-edge-tunnel[5457]: (5457)[164.068] WARN ziti-edge-tunnel:ziti-edge-tunnel.c:641 on_cmd() IPC client connection closed, count: 0 Mar 05 03:47:26 ip-172-31-21-121 systemd[1]: Stopping Ziti Edge Tunnel... Mar 05 03:47:26 ip-172-31-21-121 systemd[1]: ziti-edge-tunnel.service: Deactivated successfully. Mar 05 03:47:26 ip-172-31-21-121 systemd[1]: Stopped Ziti Edge Tunnel. Mar 05 03:47:26 ip-172-31-21-121 systemd[1]: Starting Ziti Edge Tunnel... Mar 05 03:47:26 ip-172-31-21-121 ziti-edge-tunnel.sh[5699]: (5699)[0.000] INFO ziti-sdk:utils.c:199 ziti_log_set_level() set log level: root=3/INFO Mar 05 03:47:26 ip-172-31-21-121 ziti-edge-tunnel.sh[5699]: (5699)[0.000] INFO ziti-sdk:utils.c:168 ziti_log_init() Ziti C SDK version 0.36.7 @11a8db3(HEAD) starting at (2024-03-05T03:47:26.105) Mar 05 03:47:26 ip-172-31-21-121 ziti-edge-tunnel.sh[5699]: (5699)[0.000] INFO ziti-sdk:ziti_enroll.c:88 ziti_enroll() Ziti C SDK version 0.36.7 @11a8db3(HEAD) starting enrollment at (2024-03-05T03:47:26.105) Mar 05 03:47:26 ip-172-31-21-121 ziti-edge-tunnel.sh[5695]: INFO: enrolled HTTP_Sever.jwt in /opt/openziti/etc/identities/HTTP_Sever.json Mar 05 03:47:26 ip-172-31-21-121 systemd[1]: Started Ziti Edge Tunnel. Mar 05 03:47:26 ip-172-31-21-121 ziti-edge-tunnel[5707]: (5707)[0.000] WARN ziti-edge-tunnel:instance.c:40 find_tunnel_identity() Identity ztx[/opt/openziti/etc/identities/HTTP Server.json] is not loaded yet or already removed. Mar 05 03:47:26 ip-172-31-21-121 ziti-edge-tunnel[5707]: (5707)[0.000] WARN ziti-edge-tunnel:instance.c:40 find_tunnel_identity() Identity ztx[/opt/openziti/etc/identities/HTTP.json] is not loaded yet or already removed. Mar 05 03:47:26 ip-172-31-21-121 ziti-edge-tunnel[5707]: (5707)[0.000] WARN ziti-edge-tunnel:instance.c:40 find_tunnel_identity() Identity ztx[/opt/openziti/etc/identities/HTTP_Sever.json] is not loaded yet or already removed. Mar 05 05:18:31 ip-172-31-21-121 ziti-edge-tunnel[5707]: (5707)[5465.527] WARN ziti-sdk:bind.c:192 session_cb() server[0.2] failed to get session for service[HTTP Service]: -29/NOT_FOUND Mar 05 05:18:32 ip-172-31-21-121 ziti-edge-tunnel[5707]: (5707)[5466.332] WARN ziti-sdk:bind.c:192 session_cb() server[0.2] failed to get session for service[HTTP Service]: -29/NOT_FOUND Mar 05 05:18:32 ip-172-31-21-121 ziti-edge-tunnel[5707]: (5707)[5466.332] ERROR tunnel-cbs:ziti_hosting.c:528 on_hosted_client_connect() hosted_service[HTTP Service] incoming connection failed: service not available Mar 05 05:18:58 ip-172-31-21-121 ziti-edge-tunnel[5707]: (5707)[5492.256] WARN ziti-sdk:bind.c:192 session_cb() server[0.3] failed to get session for service[HTTP Service]: -29/NOT_FOUND Mar 05 05:18:59 ip-172-31-21-121 systemd[1]: ziti-edge-tunnel.service: Main process exited, code=dumped, status=11/SEGV Mar 05 05:18:59 ip-172-31-21-121 systemd[1]: ziti-edge-tunnel.service: Failed with result 'core-dump'. ``` **Below is the log information from the controller:** ```auto Mar 7 04:19:26 ip-172-31-47-206 ziti[2195]: {"_context":"ch{RBepMJmcil}-\u003eu{classic}-\u003ei{Ra3W}","error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:74","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseRequestHandler).returnError","level":"error","msg":"responded with error","operation":"create.circuit","routerId":"RBepMJmcil","time":"2024-03-07T04:19:26.297Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273"} Mar 7 04:19:26 ip-172-31-47-206 ziti[2231]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{nyAn}","chSeq":12,"connId":13,"edgeSeq":0,"error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:179","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-03-07T04:19:26.298Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273","type":"EdgeConnectType"} Mar 7 04:19:26 ip-172-31-47-206 ziti[2195]: {"_context":"ch{RBepMJmcil}-\u003eu{classic}-\u003ei{Ra3W}","error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:74","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseRequestHandler).returnError","level":"error","msg":"responded with error","operation":"create.circuit","routerId":"RBepMJmcil","time":"2024-03-07T04:19:26.518Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273"} Mar 7 04:19:26 ip-172-31-47-206 ziti[2231]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{nyAn}","chSeq":13,"connId":14,"edgeSeq":0,"error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:179","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-03-07T04:19:26.519Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273","type":"EdgeConnectType"} Mar 7 04:19:26 ip-172-31-47-206 ziti[2195]: {"_context":"ch{RBepMJmcil}-\u003eu{classic}-\u003ei{Ra3W}","error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:74","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseRequestHandler).returnError","level":"error","msg":"responded with error","operation":"create.circuit","routerId":"RBepMJmcil","time":"2024-03-07T04:19:26.737Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273"} Mar 7 04:19:26 ip-172-31-47-206 ziti[2231]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{nyAn}","chSeq":14,"connId":15,"edgeSeq":0,"error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:179","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-03-07T04:19:26.738Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273","type":"EdgeConnectType"} Mar 7 04:19:27 ip-172-31-47-206 ziti[2195]: {"_context":"ch{RBepMJmcil}-\u003eu{classic}-\u003ei{Ra3W}","error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:74","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseRequestHandler).returnError","level":"error","msg":"responded with error","operation":"create.circuit","routerId":"RBepMJmcil","time":"2024-03-07T04:19:27.546Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273"} Mar 7 04:19:27 ip-172-31-47-206 ziti[2231]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{nyAn}","chSeq":15,"connId":16,"edgeSeq":0,"error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:179","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-03-07T04:19:27.547Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273","type":"EdgeConnectType"} Mar 7 04:19:27 ip-172-31-47-206 ziti[2195]: {"_context":"ch{RBepMJmcil}-\u003eu{classic}-\u003ei{Ra3W}","error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:74","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseRequestHandler).returnError","level":"error","msg":"responded with error","operation":"create.circuit","routerId":"RBepMJmcil","time":"2024-03-07T04:19:27.548Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273"} Mar 7 04:19:27 ip-172-31-47-206 ziti[2231]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{nyAn}","chSeq":16,"connId":17,"edgeSeq":0,"error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:179","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-03-07T04:19:27.549Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273","type":"EdgeConnectType"} Mar 7 04:19:32 ip-172-31-47-206 ziti[2195]: {"_context":"ch{RBepMJmcil}-\u003eu{classic}-\u003ei{Ra3W}","error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:74","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseRequestHandler).returnError","level":"error","msg":"responded with error","operation":"create.circuit","routerId":"RBepMJmcil","time":"2024-03-07T04:19:32.780Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273"} Mar 7 04:19:32 ip-172-31-47-206 ziti[2231]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{nyAn}","chSeq":17,"connId":18,"edgeSeq":0,"error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:179","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-03-07T04:19:32.781Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273","type":"EdgeConnectType"} Mar 7 04:19:32 ip-172-31-47-206 ziti[2195]: {"_context":"ch{RBepMJmcil}-\u003eu{classic}-\u003ei{Ra3W}","error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/controller/handler_edge_ctrl/common.go:74","func":"github.com/openziti/ziti/controller/handler_edge_ctrl.(*baseRequestHandler).returnError","level":"error","msg":"responded with error","operation":"create.circuit","routerId":"RBepMJmcil","time":"2024-03-07T04:19:32.782Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273"} Mar 7 04:19:32 ip-172-31-47-206 ziti[2231]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{nyAn}","chSeq":18,"connId":19,"edgeSeq":0,"error":"service 5NjcJOToxs5YFKGcceqYTX has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:179","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-03-07T04:19:32.783Z","token":"8f28c678-fdbc-4923-ae1c-21a6a776b273","type":"EdgeConnectType"} ``` It's strange that with the old version installed last year (Controller/Route: v0.31.0), it can actually access normally. I only switched the controller and the relevant router, but all versions of the agent remain the same. With the same configuration, everything works fine, so I feel the issue is not on the side of the Ziti edge tunnel. ---
**Author:** ![JamminSoleng](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/jamminsoleng/32/314_2.png) [@JamminSoleng](https://openziti.discourse.group/u/JamminSoleng)\ **Post date:** [March 7, 2024, 4:29am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/4 "2024-03-07T04:29:27Z")
"no terminators" means your hosting identity was either not up, or not able to connect to a edge router. can you list terminators from the controller. And also policy-advisor will help also. ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 7, 2024, 4:57am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/5 "2024-03-07T04:57:45Z")
I feel there's no issue with the configuration because the same configuration works fine on older versions or on macOS and Windows. ```shell root@ip-172-31-47-206:~# ziti edge list terminators # In the older versions, there is information here, but in the newly installed version, it's empty. ╭────┬─────────┬────────┬─────────┬─────────┬──────────┬──────┬────────────┬──────────────╮ │ ID │ SERVICE │ ROUTER │ BINDING │ ADDRESS │ IDENTITY │ COST │ PRECEDENCE │ DYNAMIC COST │ ├────┼─────────┼────────┼─────────┼─────────┼──────────┼──────┼────────────┼──────────────┤ ╰────┴─────────┴────────┴─────────┴─────────┴──────────┴──────┴────────────┴──────────────╯ results: none root@ip-172-31-47-206:~# ziti edge list identities ╭────────────┬──────────────────────────────┬─────────┬────────────┬─────────────╮ │ ID │ NAME │ TYPE │ ATTRIBUTES │ AUTH-POLICY │ ├────────────┼──────────────────────────────┼─────────┼────────────┼─────────────┤ │ -DJWS-Yjqw │ zsf │ Default │ │ Default │ │ 5RDYpVmPM │ Default Admin │ Default │ │ Default │ │ RBepMJmcil │ ip-172-31-47-206-edge-router │ Router │ │ Default │ │ Y4VQS-YKqw │ us │ Default │ │ Default │ │ dNQgcQfjq │ hk │ Default │ │ Default │ ╰────────────┴──────────────────────────────┴─────────┴────────────┴─────────────╯ results: 1-5 of 5 root@ip-172-31-47-206:~# ziti edge list services ╭────────────────────────┬───────────┬────────────┬─────────────────────┬────────────╮ │ ID │ NAME │ ENCRYPTION │ TERMINATOR STRATEGY │ ATTRIBUTES │ │ │ │ REQUIRED │ │ │ ├────────────────────────┼───────────┼────────────┼─────────────────────┼────────────┤ │ 5NjcJOToxs5YFKGcceqYTX │ us-server │ true │ smartrouting │ │ ╰────────────────────────┴───────────┴────────────┴─────────────────────┴────────────╯ results: 1-1 of 1 root@ip-172-31-47-206:~# ziti edge list service-policies ╭────────────────────────┬────────────────┬──────────┬───────────────┬────────────────┬─────────────────────╮ │ ID │ NAME │ SEMANTIC │ SERVICE ROLES │ IDENTITY ROLES │ POSTURE CHECK ROLES │ ├────────────────────────┼────────────────┼──────────┼───────────────┼────────────────┼─────────────────────┤ │ 1htoqUiwcbiFqjYG7J6e41 │ us-server-dial │ AnyOf │ @us-server │ @zsf │ │ │ 5RU5ee2i89zycHHfBZ6rpH │ us-server-bind │ AnyOf │ @us-server │ @us │ │ ╰────────────────────────┴────────────────┴──────────┴───────────────┴────────────────┴─────────────────────╯ results: 1-2 of 2 ``` ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 7, 2024, 9:02am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/6 "2024-03-07T09:02:31Z")
Are we considering adding a variable for specifying the version in the expressInstall installation method? ---
**Author:** ![JamminSoleng](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/jamminsoleng/32/314_2.png) [@JamminSoleng](https://openziti.discourse.group/u/JamminSoleng)\ **Post date:** [March 7, 2024, 1:40pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/7 "2024-03-07T13:40:38Z")
There is no information on the Terminators because terminators are not setup. You can do: ziti edge policy-advisor services that should show you two lines for your service, one dial and one bind. At beginning of that, it should say "OKAY", then the common routers should be at least 1. you can also display the router status for us: ziti fabric list routers ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 8, 2024, 1:36am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/8 "2024-03-08T01:36:46Z")
I initially thought it might be an issue with the AWS kernel being too high, so I redeployed an OpenZiti instance on Alibaba Cloud. Unfortunately, the result remains the same. ```bash root@iZj6cfbiqa5u3hduq973olZ:~# ziti edge policy-advisor services -q OKAY : hk (1) -> hk-server (1) Common Routers: (1/1) Dial: N Bind: Y OKAY : zsf (1) -> hk-server (1) Common Routers: (1/1) Dial: Y Bind: N root@iZj6cfbiqa5u3hduq973olZ:~# ziti fabric list routers ╭────────────┬─────────────────────────────────────┬────────┬──────┬──────────────┬──────────┬────────────────────────┬────────────────────────────────╮ │ ID │ NAME │ ONLINE │ COST │ NO TRAVERSAL │ DISABLED │ VERSION │ LISTENERS │ ├────────────┼─────────────────────────────────────┼────────┼──────┼──────────────┼──────────┼────────────────────────┼────────────────────────────────┤ │ EYtg-9WhDH │ iZj6cfbiqa5u3hduq973olZ-edge-router │ true │ 0 │ false │ false │ v0.32.2 on linux/amd64 │ 1: tls:zt1. *****.xx:10080 │ ╰────────────┴─────────────────────────────────────┴────────┴──────┴──────────────┴──────────┴────────────────────────┴────────────────────────────────╯ results: 1-1 of 1 root@iZj6cfbiqa5u3hduq973olZ:~# ``` ---
**Author:** ![JamminSoleng](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/jamminsoleng/32/314_2.png) [@JamminSoleng](https://openziti.discourse.group/u/JamminSoleng)\ **Post date:** [March 8, 2024, 2:27am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/9 "2024-03-08T02:27:23Z")
You have opened the port 10080 on the firewall ? ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 8, 2024, 4:53am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/10 "2024-03-08T04:53:14Z")
I did not turn on any firewall for testing, and the security group on the cloud also allowed all traffic. And I did a test verification of port 10080 from the outside, which was normal. In addition, I used mac as the tunnel gateway and I could see the normal "terminators". So I'm very curious why this is the case. **Here are some test information I did:** **OpenZiti（Contoller/Route:）** ```bash root@iZj6cfbiqa5u3hduq973olZ:~# ziti edge policy-advisor services -q OKAY : hk (1) -> hk-server (1) Common Routers: (1/1) Dial: N Bind: Y OKAY : zsf (1) -> hk-server (1) Common Routers: (1/1) Dial: Y Bind: N OKAY : hk (1) -> mac-server (1) Common Routers: (1/1) Dial: Y Bind: N OKAY : zsf (1) -> mac-server (1) Common Routers: (1/1) Dial: Y Bind: Y root@iZj6cfbiqa5u3hduq973olZ:~# ziti fabric list routers ╭────────────┬─────────────────────────────────────┬────────┬──────┬──────────────┬──────────┬────────────────────────┬────────────────────────────────╮ │ ID │ NAME │ ONLINE │ COST │ NO TRAVERSAL │ DISABLED │ VERSION │ LISTENERS │ ├────────────┼─────────────────────────────────────┼────────┼──────┼──────────────┼──────────┼────────────────────────┼────────────────────────────────┤ │ EYtg-9WhDH │ iZj6cfbiqa5u3hduq973olZ-edge-router │ true │ 0 │ false │ false │ v0.32.2 on linux/amd64 │ 1: tls:zt1.***.com:10080 │ ╰────────────┴─────────────────────────────────────┴────────┴──────┴──────────────┴──────────┴────────────────────────┴────────────────────────────────╯ results: 1-1 of 1 root@iZj6cfbiqa5u3hduq973olZ:~# ziti fabric list terminators ╭────────────────────────┬────────────┬─────────────────────────────────────┬─────────┬────────────────────────┬──────────┬──────┬────────────┬──────────────┬───────────╮ │ ID │ SERVICE │ ROUTER │ BINDING │ ADDRESS │ INSTANCE │ COST │ PRECEDENCE │ DYNAMIC COST │ HOST ID │ ├────────────────────────┼────────────┼─────────────────────────────────────┼─────────┼────────────────────────┼──────────┼──────┼────────────┼──────────────┼───────────┤ │ 4HfJthuD78DUFQpb9vaeWn │ mac-server │ iZj6cfbiqa5u3hduq973olZ-edge-router │ edge │ 4HfJthuD78DUFQpb9vaeWn │ │ 0 │ default │ 0 │ DV3Mj5mHl │ │ gJxkPVNlsUdzDhgKs2Gzc │ mac-server │ iZj6cfbiqa5u3hduq973olZ-edge-router │ edge │ gJxkPVNlsUdzDhgKs2Gzc │ │ 0 │ default │ 0 │ DV3Mj5mHl │ ╰────────────────────────┴────────────┴─────────────────────────────────────┴─────────┴────────────────────────┴──────────┴──────┴────────────┴──────────────┴───────────╯ results: 1-2 of 2 ``` **Linux Client（ziti-edge-tunnel）** ```bash # telnet zt1.**.com 10080 Trying x.x.x.x... Connected to zt1.xx.com. Escape character is '^]'. ~# curl -i mac.xx.ziti HTTP/1.0 200 OK Server: SimpleHTTP/0.6 Python/3.11.6 Date: Fri, 08 Mar 2024 04:41:30 GMT Content-type: text/html; charset=utf-8 Content-Length: 838 mac server ``` MAC Client(Ziti Desktop Edge) ```bash Mac# telnet zzt1.xx.com 10080 Trying 47.76.105.4... Connected to zt1.xx.com. Escape character is '^]'. Mac# python3 -m http.server 8000 Serving HTTP on :: port 8000 (http://[::]:8000/) ... ``` The test access is the same, but mac-----\>hk is not accessible, but the other way around hk-----\>mac is accessible. ---
**Author:** ![JamminSoleng](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/jamminsoleng/32/314_2.png) [@JamminSoleng](https://openziti.discourse.group/u/JamminSoleng)\ **Post date:** [March 8, 2024, 3:05pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/11 "2024-03-08T15:05:54Z")
The terminator gets created if the endpoint (hosting the service) can connect to a ER into the fabric. From your output, i assume you did an update on the service? Since you verified the port 10080 is working, so that shouldn't be the issue. So, the issue most likely is in the ziti-tunnel side. So i just looked carefully with the your ziti-edge-tunnel log, that is not running correctly. (JWT has invalid format), did you create an Edge router or identity for this? ---
**Author:** ![scareything](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/scareything/32/106_2.png) [@scareything](https://openziti.discourse.group/u/scareything)\ **Post date:** [March 8, 2024, 6:29pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/12 "2024-03-08T18:29:41Z")
I suspect you may be running into a bug that was introduced with 0.32.2 and affects clients using the latest ziti-sdk-c (including ziti-edge-tunnel). Can you please check your edge router logs for messages like this? ```auto INFO ziti/router/xgress_edge.(*edgeTerminator).close: {terminatorId=[1Bn5tvEtTos8WIki0s4k2Z] token=[388f6ead-1194-45cd-99b9-5ab13b67fd9b] reason=[terminator invalid]} removing terminator on controller ``` Thanks! ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 11, 2024, 1:53am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/13 "2024-03-11T01:53:11Z")
But when the latest version of Ziti Edge Tunnel is connected to the older version of the OpenZiti network, it works fine. So, I feel like it's not an issue with Ziti Edge Tunnel. However, if it's related to the new version of the controller or router, my MAC or WIN client works fine. Anyway, when I did research testing last year with the old version, everything was normal. But this year, when I tried to deploy it in production, I encountered this issue. I'm not sure which part went wrong. At least I can guarantee that my configuration operations and some basic network settings should be fine. **OpenZiti（Contoller/Route）:** ```bash root@iZj6cfbiqa5u3hduq973olZ:~# cat /var/log/syslog*|grep -i 'xgress_edge'|grep -i removing|head Mar 10 00:00:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"7FcPQlzw34IsP36Z7uLqiv","time":"2024-03-10T00:00:12.560Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:00:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"FVnAjjqLcvNOGUajH2bYf","time":"2024-03-10T00:00:12.561Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:05:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"3BLNJ4IRHEqoncjKfULnmV","time":"2024-03-10T00:05:12.589Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:05:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"7TocCdkEugXaA5QAPBCo36","time":"2024-03-10T00:05:12.590Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:10:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"56zT9enftAodqfpLeoTGnR","time":"2024-03-10T00:10:12.618Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:10:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"1fe8V9UsIuAeJ8Rxt5th4n","time":"2024-03-10T00:10:12.618Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:15:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"6bnhqe5kHeU27knIz46Izk","time":"2024-03-10T00:15:12.645Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:15:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"7dYfgBA2NLTk7nNNW6J4Dh","time":"2024-03-10T00:15:12.645Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:20:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"zKaI1f6Pl4w4uQcaVvWPx","time":"2024-03-10T00:20:12.673Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} Mar 10 00:20:12 iZj6cfbiqa5u3hduq973olZ ziti[25010]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:157","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","reason":"terminator invalid","terminatorId":"2TLKfIvvRdT9K6V7ohP5RO","time":"2024-03-10T00:20:12.674Z","token":"13cf36a7-9f4f-474a-89fd-e155105dd2f8"} root@iZj6cfbiqa5u3hduq973olZ:~# cat /var/log/syslog*|grep -i 'xgress_edge'|grep -i removing|wc -l 2154 ``` **Linux Client（ziti-edge-tunnel）** ```bash root@iZj6ccikg76btt2lsaiw50Z:/var/log# cat /var/log/syslog*|grep -i terminator Mar 6 11:35:40 iZj6ccikg76btt2lsaiw50Z ziti-edge-tunnel[8217]: (8217)[675.570] ERROR ziti-sdk:connect.c:963 connect_reply_cb() conn[0.2/Gnu0ETfR/Connecting] failed to connect, reason=service Ril3mxCSfmkVWphkwC6ke has no terminators Mar 6 11:36:48 iZj6ccikg76btt2lsaiw50Z ziti-edge-tunnel[12086]: (12086)[4.531] ERROR ziti-sdk:connect.c:963 connect_reply_cb() conn[0.1/PCqoU3Ic/Connecting] failed to connect, reason=service Ril3mxCSfmkVWphkwC6ke has no terminators Mar 6 11:56:24 iZj6ccikg76btt2lsaiw50Z ziti-edge-tunnel[19490]: (19490)[89.651] ERROR ziti-sdk:connect.c:963 connect_reply_cb() conn[0.1/iqQDs2vp/Connecting] failed to connect, reason=service Ril3mxCSfmkVWphkwC6ke has no terminators Mar 6 12:32:19 iZj6ccikg76btt2lsaiw50Z ziti-edge-tunnel[20803]: (20803)[116.807] ERROR ziti-sdk:connect.c:963 connect_reply_cb() conn[0.1/XZM3iXcG/Connecting] failed to connect, reason=service Ril3mxCSfmkVWphkwC6ke has no terminators Mar 6 12:32:50 iZj6ccikg76btt2lsaiw50Z ziti-edge-tunnel[20803]: (20803)[148.064] ERROR ziti-sdk:connect.c:963 connect_reply_cb() conn[0.2/jR8hmLBk/Connecting] failed to connect, reason=service Ril3mxCSfmkVWphkwC6ke has no terminators Mar 8 10:16:35 iZj6ccikg76btt2lsaiw50Z ziti-edge-tunnel[59265]: (59265)[64452.128] ERROR ziti-sdk:connect.c:963 connect_reply_cb() conn[0.2/U_Mk3WHL/Connecting] failed to connect, reason=exceeded maximum [2] retries creating circuit [c/2JeSDvv5x]: error creating route for [s/2JeSDvv5x] on [r/EYtg-9WhDH] (error creating route for [c/2JeSDvv5x]: failed to establish connection with terminator address 67szA3tS2TY1LlLBW2AByv. error: (rejected by application)) root@iZj6ccikg76btt2lsaiw50Z:/var/log# cat /var/log/syslog*|grep -i 'xgress_edge' root@iZj6ccikg76btt2lsaiw50Z:/var/log# ``` ---
**Author:** ![scareything](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/scareything/32/106_2.png) [@scareything](https://openziti.discourse.group/u/scareything)\ **Post date:** [March 11, 2024, 2:25pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/14 "2024-03-11T14:25:56Z")
> [@toadzhou](#): > > when the latest version of Ziti Edge Tunnel is connected to the older version of the OpenZiti network, it works fine. Thanks for sharing the router logs. Those error messages make me more confident that your ziti-edge-tunnel is affected by the 0.32.2 changes. Your Mac and windows clients are using older versions of ziti-sdk-c, which are not affected by those changes. The fix for this issue should be released early this week. I'll post here when it's ready. ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 12, 2024, 2:37am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/15 "2024-03-12T02:37:22Z")
Thank you for your efforts, and may success follow you in the future. ---
**Author:** ![scareything](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/scareything/32/106_2.png) [@scareything](https://openziti.discourse.group/u/scareything)\ **Post date:** [March 13, 2024, 10:29am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/16 "2024-03-13T10:29:50Z")
OpenZiti 0.33.0 includes the fix for the issue that prevents ziti-sdk-c clients from binding (hosting) services. The release is now available at [Release v0.33.0 · openziti/ziti · GitHub](https://github.com/openziti/ziti/releases/tag/v0.33.0) ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 14, 2024, 3:15am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/18 "2024-03-14T03:15:30Z")
Because I couldn't find a standard upgrade method, I had to rely on my understanding to perform the upgrade, which may not be conventional, but after testing, the previous issue has been resolved. **OpenZiti（Contoller/Route）:** ```shell # mkdir tmp # wget -P tmp/ wget https://github.com/openziti/ziti/releases/download/v0.33.0/ziti-linux-amd64-0.33.0.tar.gz # mkdir -pv ~/.ziti/quickstart/$(hostname -s)/ziti-bin/ziti-v0.33.0 # tar zxvf tmp/ziti-linux-amd64-0.33.0.tar.gz -C ~/.ziti/quickstart/$(hostname -s)/ziti-bin/ziti-v0.33.0 # sed -i 's/v0.32.2/v0.33.0/g' /etc/systemd/system/ziti-{controller,router}.service # systemctl daemon-reload # systemctl restart ziti-{controller,router}.service # rm -rf tmp ``` **Linux Client（ziti-edge-tunnel）** ```shell # apt update # apt install ziti-edge-tunnel -y # update deb # systemctl restart ziti-edge-tunnel.service ``` My MAC seems to be restarting frequently, and during the restart process, the icon is yellow. It's only accessible when the icon is green. Below are some related logs. **OpenZiti（Contoller/Route）Logs:** ```auto Mar 14 11:17:14 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"_context":"tls:0.0.0.0:8441","error":"context deadline exceeded","file":"github.com/openziti/transport/v2@v2.0.122/tls/listener.go:228","func":"github.com/openziti/transport/v2/tls.(*sharedListener).processConn","level":"error","msg":"handshake failed","remote":"58.33.29.26:50696","time":"2024-03-14T11:17:14.263Z"} Mar 14 11:17:17 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:139","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).setState","level":"info","msg":"updated state","newState":2,"oldState":1,"reason":"replacing existing terminator","terminatorId":"2IPX94Kku8Frj1WKwe8mCy","time":"2024-03-14T11:17:17.726Z"} Mar 14 11:17:17 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"file":"github.com/openziti/ziti/router/xgress_edge/fabric.go:139","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeTerminator).setState","level":"info","msg":"updated state","newState":3,"oldState":2,"reason":"newer terminator found for listener id","terminatorId":"2IPX94Kku8Frj1WKwe8mCy","time":"2024-03-14T11:17:17.726Z"} Mar 14 11:17:17 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{dPwj}","connId":0,"existingTerminatorId":"2IPX94Kku8Frj1WKwe8mCy","file":"github.com/openziti/ziti/router/xgress_edge/hosted.go:861","func":"github.com/openziti/ziti/router/xgress_edge.(*findMatchingEvent).handle","level":"info","listenerId":"\ufffd\ufffdc\ufffd\ufffdqp\ufffd7\ufffd\ufffd\ufffd7\ufffd[\ufffd\ufffdU\ufffd\u001a\ufffdE\ufffd\ufffdCưr\ufffd\ufffd","msg":"taking over terminator from existing bind","routerId":"EYtg-9WhDH","terminatorId":"1BUqJKT5qk6QKufrz6cr07","time":"2024-03-14T11:17:17.726Z","token":"35d9bb39-b978-4754-95d0-99faa392fee9"} Mar 14 11:17:17 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"_context":"ch{edge}-\u003eu{classic}-\u003ei{dPwj}","bindConnId":0,"chSeq":12,"connId":0,"edgeSeq":0,"file":"github.com/openziti/ziti/router/xgress_edge/listener.go:486","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processBindV2","level":"info","listenerId":"\ufffd\ufffdc\ufffd\ufffdqp\ufffd7\ufffd\ufffd\ufffd7\ufffd[\ufffd\ufffdU\ufffd\u001a\ufffdE\ufffd\ufffdCưr\ufffd\ufffd","msg":"sending replacement terminator success to sdk","routerId":"EYtg-9WhDH","terminatorId":"1BUqJKT5qk6QKufrz6cr07","time":"2024-03-14T11:17:17.726Z","token":"35d9bb39-b978-4754-95d0-99faa392fee9","type":"EdgeBindType"} Mar 14 11:17:29 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"_context":"tls:0.0.0.0:8441","error":"context deadline exceeded","file":"github.com/openziti/transport/v2@v2.0.122/tls/listener.go:228","func":"github.com/openziti/transport/v2/tls.(*sharedListener).processConn","level":"error","msg":"handshake failed","remote":"58.33.29.26:50751","time":"2024-03-14T11:17:29.340Z"} Mar 14 11:17:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitId":"NBVb38J3N","ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:85","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","idleThreshold":60000000000,"idleTime":94858000000,"level":"warning","msg":"circuit exceeds idle threshold","time":"2024-03-14T11:17:46.574Z"} Mar 14 11:17:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitCount":1,"ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:100","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","level":"warning","msg":"sent confirmation for circuits","time":"2024-03-14T11:17:46.574Z"} Mar 14 11:17:46 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"circuitCount":1,"file":"github.com/openziti/ziti/controller/handler_ctrl/circuit_confirmation.go:46","func":"github.com/openziti/ziti/controller/handler_ctrl.(*circuitConfirmationHandler).HandleReceive","level":"info","msg":"received circuit confirmation request","routerId":"EYtg-9WhDH","time":"2024-03-14T11:17:46.574Z"} Mar 14 11:17:54 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"_context":"tls:0.0.0.0:8441","error":"context deadline exceeded","file":"github.com/openziti/transport/v2@v2.0.122/tls/listener.go:228","func":"github.com/openziti/transport/v2/tls.(*sharedListener).processConn","level":"error","msg":"handshake failed","remote":"58.33.29.26:50838","time":"2024-03-14T11:17:54.780Z"} Mar 14 11:18:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitId":"NBVb38J3N","ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:85","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","idleThreshold":60000000000,"idleTime":154859000000,"level":"warning","msg":"circuit exceeds idle threshold","time":"2024-03-14T11:18:46.575Z"} Mar 14 11:18:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitCount":1,"ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:100","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","level":"warning","msg":"sent confirmation for circuits","time":"2024-03-14T11:18:46.575Z"} Mar 14 11:18:46 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"circuitCount":1,"file":"github.com/openziti/ziti/controller/handler_ctrl/circuit_confirmation.go:46","func":"github.com/openziti/ziti/controller/handler_ctrl.(*circuitConfirmationHandler).HandleReceive","level":"info","msg":"received circuit confirmation request","routerId":"EYtg-9WhDH","time":"2024-03-14T11:18:46.575Z"} Mar 14 11:19:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitId":"NBVb38J3N","ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:85","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","idleThreshold":60000000000,"idleTime":214860000000,"level":"warning","msg":"circuit exceeds idle threshold","time":"2024-03-14T11:19:46.576Z"} Mar 14 11:19:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitCount":1,"ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:100","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","level":"warning","msg":"sent confirmation for circuits","time":"2024-03-14T11:19:46.576Z"} Mar 14 11:19:46 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"circuitCount":1,"file":"github.com/openziti/ziti/controller/handler_ctrl/circuit_confirmation.go:46","func":"github.com/openziti/ziti/controller/handler_ctrl.(*circuitConfirmationHandler).HandleReceive","level":"info","msg":"received circuit confirmation request","routerId":"EYtg-9WhDH","time":"2024-03-14T11:19:46.576Z"} Mar 14 11:20:22 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"_context":"tls:0.0.0.0:8441","error":"context deadline exceeded","file":"github.com/openziti/transport/v2@v2.0.122/tls/listener.go:228","func":"github.com/openziti/transport/v2/tls.(*sharedListener).processConn","level":"error","msg":"handshake failed","remote":"58.33.29.26:51309","time":"2024-03-14T11:20:22.689Z"} Mar 14 11:20:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitId":"NBVb38J3N","ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:85","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","idleThreshold":60000000000,"idleTime":274860000000,"level":"warning","msg":"circuit exceeds idle threshold","time":"2024-03-14T11:20:46.576Z"} Mar 14 11:20:46 iZj6cfbiqa5u3hduq973olZ ziti[181899]: {"circuitCount":1,"ctrlId":"iZj6cfbiqa5u3hduq973olZ","file":"github.com/openziti/ziti/router/forwarder/scanner.go:100","func":"github.com/openziti/ziti/router/forwarder.(*Scanner).scan","level":"warning","msg":"sent confirmation for circuits","time":"2024-03-14T11:20:46.576Z"} Mar 14 11:20:46 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"circuitCount":1,"file":"github.com/openziti/ziti/controller/handler_ctrl/circuit_confirmation.go:46","func":"github.com/openziti/ziti/controller/handler_ctrl.(*circuitConfirmationHandler).HandleReceive","level":"info","msg":"received circuit confirmation request","routerId":"EYtg-9WhDH","time":"2024-03-14T11:20:46.577Z"} Mar 14 11:20:48 iZj6cfbiqa5u3hduq973olZ ziti[181900]: {"_context":"tls:0.0.0.0:8441","error":"context deadline exceeded","file":"github.com/openziti/transport/v2@v2.0.122/tls/listener.go:228","func":"github.com/openziti/transport/v2/tls.(*sharedListener).processConn","level":"error","msg":"handshake failed","remote":"58.33.29.26:51360","time":"2024-03-14T11:20:48.671Z"} ``` **Linux Client（ziti-edge-tunnel）Logs** ```auto root@iZj6ccikg76btt2lsaiw50Z:~# tail -f /var/log/syslog Mar 14 11:25:40 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2216 of User root. Mar 14 11:25:41 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2216.scope: Deactivated successfully. Mar 14 11:25:41 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2217 of User root. Mar 14 11:25:42 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2217.scope: Deactivated successfully. Mar 14 11:25:44 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2218 of User root. Mar 14 11:25:44 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2218.scope: Deactivated successfully. Mar 14 11:25:45 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2219 of User root. Mar 14 11:25:45 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2219.scope: Deactivated successfully. Mar 14 11:25:46 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2220 of User root. Mar 14 11:25:46 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2220.scope: Deactivated successfully. Mar 14 11:25:50 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2221 of User root. Mar 14 11:25:50 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2221.scope: Deactivated successfully. Mar 14 11:25:51 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2222 of User root. Mar 14 11:25:51 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2222.scope: Deactivated successfully. Mar 14 11:25:52 iZj6ccikg76btt2lsaiw50Z systemd[1]: Started Session 2223 of User root. Mar 14 11:25:52 iZj6ccikg76btt2lsaiw50Z systemd[1]: session-2223.scope: Deactivated successfully. ``` **MAC Client Logs** ```auto excludeRoute() excludeRoute xx.xx.105.4 => xx.xx.105.4, 255.255.255.255 [2024-03-14T03:26:28:644Z] INFO PacketTunnelProvider:ZitiTunnelDelegate.swift:222 tunnelEventCallback() ZitiTunnelEvent: identity: zsf:"DV3Mj5mHl" status: OK name: zsf version: v0.33.0 controller: https://zt1.xxx:8441 code: 0 [2024-03-14T03:26:28:648Z] INFO PacketTunnelProvider:UserNotifications.swift:100 post() Attempting to post Info notification, subitile:Optional("Controller: Available"), body:Optional("zsf\nhttps://zt1.xxx:8441"), zid:Optional("zsf") [2024-03-14T03:26:28:648Z] WARN PacketTunnelProvider:UserNotifications.swift:107 post() Not authorized to send notifications (4552)[2024-03-14T03:38:51.635Z] INFO ziti-sdk:ziti.c:1561 ziti_set_api_session() ztx[0] api session set, setting api_session_timer to 1740s ``` ---
**Author:** ![scareything](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/scareything/32/106_2.png) [@scareything](https://openziti.discourse.group/u/scareything)\ **Post date:** [March 14, 2024, 2:52pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/19 "2024-03-14T14:52:15Z")
I'd like to get a better look at what's going on with your Mac client. Can you please share the full appex.log? You can email it to [help@openziti.org](mailto:help@openziti.org) or DM a file sharing link to me here. ---
**Author:** ![toadzhou](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/toadzhou/32/1829_2.png) [@toadzhou](https://openziti.discourse.group/u/toadzhou)\ **Post date:** [March 15, 2024, 1:51am UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/20 "2024-03-15T01:51:16Z")
Hello, I have sent the appex.log file through my GMAIL. Please check. Thank you! ---
**Author:** ![scareything](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/scareything/32/106_2.png) [@scareything](https://openziti.discourse.group/u/scareything)\ **Post date:** [March 15, 2024, 1:12pm UTC](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219/21 "2024-03-15T13:12:11Z")
Thanks! You can see the connections to controller are flapping. One attempt fails: ```auto 2024-03-15T01:28:45.543Z] ERROR ziti-sdk:ziti.c:1100 update_services() ztx[0] failed to get service updates err[CONTROLLER_UNAVAILABLE/software caused connection abort] from ctrl[https://zt1.xxx:8441] [2024-03-15T01:28:45.543Z] WARN tunnel-cbs:ziti_tunnel_ctrl.c:781 on_ziti_event() ziti_ctx controller connections failed: ziti controller is not available ``` And the very next attempt just 10 seconds later succeeds: ```auto [2024-03-15T01:28:56.552Z] INFO tunnel-cbs:ziti_tunnel_ctrl.c:767 on_ziti_event() ziti_ctx[zsf] connected to controller ``` And the next request fails as above, and so on... until the end of the log, where it looks like you changed the log level to debug. From that point on we see several successful connections with the controller, and no more errors - although it is only the last few minutes of the log so maybe the connections started failing again? I'm not sure if those connection failures relate to the ssl handshake failures that you see in the controller logs. Did this start happening when you updated the controller to 0.33.0? I noticed your Mac client is at version 2.37, a couple of revs behind. Do you still see these problems after updating to 2.39? [Next page](https://openziti.discourse.group/t/failure-of-ziti-edge-tunnel-gateway-mode-on-linux/2219.md?page=2)