This is an odd one, I did a post about this before but now I actually have gathered the logs from a couple of tunneler clients.
First I would like to show you that the links are up,
All Edge routers up and online
ERP
ziti edge policy-advisor services
Identities
all should be working fine, no?
and it is! only on andoird phones.... I've tested this over and over with multiple devices as you can see withe all the identities I have. Check the logs below.
Android
=== Services ===
name: Hamadhome Service id: 3K077FJ37no4eq14exozIU permissions: Dial intercept: [TCP, UDP]:[CIDRBlock(ip=/xx.xx.xx.xx, bits=32), CIDRBlock(ip=/xx.xx.xx.xx, bits=32), CIDRBlock(ip=/xx.xx.xx.xx, bits=32)]:[1-65535]
name: fahad pc access id: X2CbbNaoe2OqXLyAyhVtT permissions: Dial intercept: [TCP, UDP]:[CIDRBlock(ip=/xx.xx.xx.xx, bits=32)]:[1-65535]
=== Available Edge Routers[2] ===
EdgeRouter(name=ziti-edge-router-2, hostname=, supportedProtocols={tls=tls://edgerouter02.redacteddomain.com:8442}, urls=null)
EdgeRouter(name=ER-Pub-AZ cloud vm azure fabric router, hostname=, supportedProtocols={tls=tls://edgerouter01.redacteddomain.com:3022}, urls=null)
=== Channels[2] ===
ER: tls://edgerouter02.redacted.com:8442 status: Connecting
ER: tls://edgerouter01.redacteddomain.com:3022 status: Connected(latency=131)
=== Connections[10] ===
conn[16]: ZitiSocketChannel[connected](Session(id=clx3zwofi9imdhuqene860oeg, service=Hamadhome Service, callerId=null, appData=null) -> null)
conn[17]: ZitiSocketChannel[connected](Session(id=clx3zwofi9imdhuqene860oeg, service=Hamadhome Service, callerId=null, appData=null) -> null)
conn[22]: ZitiSocketChannel[connected](Session(id=clx3zwofi9imdhuqene860oeg, service=Hamadhome Service, callerId=null, appData=null) -> null)
Windows
2024-06-07T09:49:11.118Z] ERROR ziti-sdk:channel.c:899 on_channel_connect_internal() ch[3] failed to connect to ER[ER-Pub-AZ cloud vm azure fabric router] [-4079/software caused connection abort]
[2024-06-07T09:49:11.118Z] INFO ziti-sdk:channel.c:775 reconnect_channel() ch[3] reconnecting in 118155ms (attempt = 16)
[2024-06-07T09:49:54.430Z] ERROR ziti-sdk:channel.c:709 ch_connect_timeout() ch[2] connect timeout
[2024-06-07T09:49:54.430Z] INFO ziti-sdk:channel.c:775 reconnect_channel() ch[2] reconnecting in 29040ms (attempt = 13)
[2024-06-07T09:49:54.430Z] ERROR ziti-sdk:channel.c:899 on_channel_connect_internal() ch[2] failed to connect to ER[ziti-edge-router-2] [-4081/operation canceled]
[2024-06-07T09:50:43.495Z] ERROR ziti-sdk:channel.c:709 ch_connect_timeout() ch[2] connect timeout
[2024-06-07T09:50:43.495Z] INFO ziti-sdk:channel.c:775 reconnect_channel() ch[2] reconnecting in 7736ms (attempt = 14)
[2024-06-07T09:50:43.495Z] ERROR ziti-sdk:channel.c:899 on_channel_connect_internal() ch[2] failed to connect to ER[ziti-edge-router-2] [-4081/operation canceled]
[2024-06-07T09:51:09.694Z] ERROR ziti-sdk:channel.c:899 on_channel_connect_internal() ch[3] failed to connect to ER[ER-Pub-AZ cloud vm azure fabric router] [-4079/software caused connection abort]
[2024-06-07T09:51:09.694Z] INFO ziti-sdk:channel.c:775 reconnect_channel() ch[3] reconnecting in 85151ms (attempt = 17)
[2024-06-07T09:51:11.242Z] ERROR ziti-sdk:channel.c:709 ch_connect_timeout() ch[2] connect timeout
[2024-06-07T09:51:11.242Z] INFO ziti-sdk:channel.c:775 reconnect_channel() ch[2] reconnecting in 134325ms (attempt = 15)
[2024-06-07T09:51:11.242Z] ERROR ziti-sdk:channel.c:899 on_channel_connect_internal() ch[2] failed to connect to ER[ziti-edge-router-2] [-4081/operation canceled]
iOS
=================
Services:
Hamadhome Service: id[3K077FJ37no4eq14exozIU] perm(dial=true,bind=false)
config[intercept.v1]={"addresses":["xx.xx.xx.xx","xx.xx.xx.xx","xx.xx.xx.xx"],"portRanges":[{"high":65535,"low":1}],"protocols":["tcp","udp"]}
config[host.v1]={"allowedAddresses":["xx.xx.xx.xx","xx.xx.xx.xx","xx.xx.xx.xx"],"allowedPortRanges":[{"high":65535,"low":1}],"allowedProtocols":["tcp","udp"],"forwardAddress":true,"forwardPort":true,"forwardProtocol":true,"listenOptions":{"bindUsingEdgeIdentity":false,"precedence":"default"}}
posture queries[1]: posture query set[dummy dial policy: no posture checks defined]
fahad pc access : id[X2CbbNaoe2OqXLyAyhVtT] perm(dial=true,bind=false)
config[intercept.v1]={"addresses":["xx.xx.xx.xx"],"portRanges":[{"high":65535,"low":1}],"protocols":["tcp","udp"]}
config[host.v1]={"allowedAddresses":["FahadPC.shabakti","xx.xx.xx.xx"],"allowedPortRanges":[{"high":65535,"low":1}],"allowedProtocols":["tcp","udp"],"forwardAddress":true,"forwardPort":true,"forwardProtocol":true}
posture queries[1]: posture query set[dummy dial policy: no posture checks defined]
==================
Sessions:
==================
Channels:
ch[6](ER-Pub-AZ cloud vm azure fabric router@tls://edgerouter01.redacteddomain.com:3022) Disconnected
ch[5](ER9Hamad@tls://hamad:8442) Disconnected
ch[2](ER3 At Fahad Home@tls://fahadzitirouter:3022) Disconnected
ch[0](ziti-edge-router-2@tls://edgerouter02.redacteddomain.com:8442) Disconnected
==================
Connections:
conn[9/Vx-oCgzb]: state[Connecting] service[Hamadhome Service] using ch[-1/(none)]
connect_time[0] idle_time[94059393] sent[0] recv[0] recv_buff[0]
conn[8/QbQpYDOM]: state[Connecting] service[Hamadhome Service] using ch[-1/(none)]
connect_time[0] idle_time[94059393] sent[0] recv[0] recv_buff[0]
conn[7/w2Hx2Zk3]: state[Connecting] service[Hamadhome Service] using ch[-1/(none)]
connect_time[0] idle_time[94059393] sent[0] recv[0] recv_buff[0]
conn[6/ynsaLlZk]: state[Connecting] service[Hamadhome Service] using ch[-1/(none)]
Not sure what is going on
send help please
Also special thanks for @TheLumberjack for helping me collect these logs and solving all the other shenanigans I had in my broken environment.