Hi Team,
I have setup the docker-compose setup as per the steps available at Local - Docker Compose | Ziti and
followed the steps showed in Wildcard DNS with OpenZiti using Ziti Desktop Edge for Windows - YouTube
But with a minor change, which is instead of assigning the ziti-edge-router to accept all identities, I have made it specific to currently 2 identities (Blue linux docker tunneller and Windows Client)
But after that the steps are pretty much the same. The Blue tunneller can access the blue web service.
But the problem I am facing is that the windows client can’t access the simple web service.
I am attaching the screenshots taken from ZAC. BTB, I used ZAC to configure all the resources.
Identities:
ER Policy:
Service policy:
The error messages showing up in Windows desktop edge shows up are these
[2022-04-27T11:56:22.566Z] [0;37m INFO[39m SDKi: tunnel-sdk:tunnel_tcp.c:368 recv_tcp() intercepted address[tcp:100.64.2.1:80] client[tcp:100.64.0.1:55673] service[DemoBasicWebTestService]
[2022-04-27T11:56:22.572Z] [0;33m WARN[39m SDKw: ziti-sdk:connect.c:871 connect_reply_cb() conn[1.1] session for service[DemoBasicWebTestService] became invalid
[2022-04-27T11:56:22.582Z] [0;31mERROR[39m SDKe: ziti-sdk:connect.c:438 connect_get_net_session_cb() conn[1.1] failed to get session for service[DemoBasicWebTestService]: NO_EDGE_ROUTERS_AVAILABLE(No edge routers are assigned and online to handle the requested connection)
[2022-04-27T11:56:22.582Z] [0;31mERROR[39m SDKe: tunnel-cbs:ziti_tunnel_cbs.c:47 on_ziti_connect() ziti dial failed: Ziti Edge Router is not available
[2022-04-27T11:56:22.582Z] [0;37m INFO[39m SDKi: tunnel-sdk:ziti_tunnel.c:414 ziti_tunneler_close() closing connection: service=DemoBasicWebTestService, client=tcp:100.64.0.1:55673
Kindly let me know where I may be going wrong.
The ziti CLI tool has a sub command that will be helpful here. Can you try running:
ziti edge policy-advisor identities -q
and
ziti edge policy-advisor services -q
(take the -q off for a banner that explains the results)
I can see in your errors: Ziti Edge Router is not available. Is there any chance you recreated the identities after you made the service policy?
From the screen shots - it does look like you have it correct… run policy advisor and post back please
Hi Clint,
This time i have executed exactly the same commands as mentioned at the Youtube tutorial, where the service ER policy and ER policy are given #public and #all attributes.
Thought that there could be something to verify first.
But in this case also i get the same error in my windows desktop edge endpoint.
No i didn’t create the identities after service policy creation.
Here is the advisor command output for your reference:
ziti@9756e3dfa046:/openziti$ ziti edge policy-advisor services -q
ERROR: DemoZitiWinEndpoint (0) → DemoSimpleWebTestService (5) Common Routers: (0/0) Dial: Y Bind: N
- Identity has no edge routers assigned. Adjust edge router policies.
ERROR: DemoZitiLinuxTunneller (0) → DemoSimpleWebTestService (5) Common Routers: (0/0) Dial: N Bind: Y
- Identity has no edge routers assigned. Adjust edge router policies.
ziti@9756e3dfa046:/openziti$ ziti edge policy-advisor identities -q
ERROR: ziti-edge-router-wss
- Identity does not have access to any services. Adjust service policies.
ERROR: Default Admin
- Identity does not have access to any services. Adjust service policies.
ERROR: ziti-private-blue
- Identity does not have access to any services. Adjust service policies.
ERROR: DemoZitiWinEndpoint (0) → DemoSimpleWebTestService (5) Common Routers: (0/0) Dial: Y Bind: N
- Identity has no edge routers assigned. Adjust edge router policies.
ERROR: SamTest2
- Identity does not have access to any services. Adjust service policies.
ERROR: DemoZitiLinuxTunneller (0) → DemoSimpleWebTestService (5) Common Routers: (0/0) Dial: N Bind: Y
- Identity has no edge routers assigned. Adjust edge router policies.
ERROR: ziti-private-red
- Identity does not have access to any services. Adjust service policies.
ERROR: DemoLinuxBlueNwTunneller
- Identity does not have access to any services. Adjust service policies.
ERROR: ziti-edge-router
- Identity does not have access to any services. Adjust service policies.
ERROR: ziti-fabric-router-br
- Identity does not have access to any services. Adjust service policies.
ziti@9756e3dfa046:/openziti$
There is another thing which bothers me now, is the identities and edge router connectivity. ZAC shows it offline.
Hi Clint,
Well i just changed the edge router policy, in which i removed the #public and #all. Rather added @ and “@ziti-edge-router”. It removed the advisor errors.
ziti@9756e3dfa046:/openziti$ ziti edge policy-advisor identities -q
ERROR: ziti-edge-router-wss
- Identity does not have access to any services. Adjust service policies.
ERROR: Default Admin
- Identity does not have access to any services. Adjust service policies.
ERROR: ziti-private-blue
- Identity does not have access to any services. Adjust service policies.
OKAY : DemoZitiWinEndpoint (1) → DemoSimpleWebTestService (5) Common Routers: (1/1) Dial: Y Bind: N
ERROR: SamTest2
- Identity does not have access to any services. Adjust service policies.
OKAY : DemoZitiLinuxTunneller (1) → DemoSimpleWebTestService (5) Common Routers: (1/1) Dial: N Bind: Y
ERROR: ziti-private-red
- Identity does not have access to any services. Adjust service policies.
ERROR: DemoLinuxBlueNwTunneller
- Identity does not have access to any services. Adjust service policies.
ERROR: ziti-edge-router
- Identity does not have access to any services. Adjust service policies.
ERROR: ziti-fabric-router-br
- Identity does not have access to any services. Adjust service policies.
ziti@9756e3dfa046:/openziti$ ziti edge policy-advisor services -q
OKAY : DemoZitiWinEndpoint (1) → DemoSimpleWebTestService (5) Common Routers: (1/1) Dial: Y Bind: N
OKAY : DemoZitiLinuxTunneller (1) → DemoSimpleWebTestService (5) Common Routers: (1/1) Dial: N Bind: Y
But still when i try to access the service, it isn’t able to fetch it.
And below are the logs from window edge endpoint.
[2022-04-27T17:18:34.772Z] e[0;37m INFOe[39m SDKi: tunnel-sdk:tunnel_tcp.c:368 recv_tcp() intercepted address[tcp:100.64.2.1:80] client[tcp:100.64.0.1:52328] service[DemoSimpleWebTestService]
[2022-04-27T17:18:34.787Z] e[0;37m INFOe[39m SDKi: ziti-sdk:channel.c:223 new_ziti_channel() ch[0] (ziti-edge-router@tls://ziti-edge-router:3022) new channel for ztx[0] identity[DemoZitiWinEndpoint]
[2022-04-27T17:18:34.787Z] e[0;37m INFOe[39m SDKi: ziti-sdk:channel.c:734 reconnect_channel() ch[0] reconnecting NOW
[2022-04-27T17:18:34.849Z] e[0;37m INFOe[39m SDKi: ziti-sdk:channel.c:635 hello_reply_cb() ch[0] connected. EdgeRouter version: v0.25.4|ea528df682bb|2022-04-04T14:37:25Z|linux|amd64
[2022-04-27T17:18:34.849Z] e[0;37m INFOe[39m router[ziti-edge-router@tls://ziti-edge-router:3022]: connected to v0.25.4|ea528df682bb|2022-04-04T14:37:25Z|linux|amd64
[2022-04-27T17:18:34.859Z] e[0;31mERRORe[39m SDKe: ziti-sdk:connect.c:893 connect_reply_cb() conn[0.1] failed to connect, reason=service bVRLSnDYFE has no terminators
[2022-04-27T17:18:34.860Z] e[0;31mERRORe[39m SDKe: tunnel-cbs:ziti_tunnel_cbs.c:47 on_ziti_connect() ziti dial failed: connection is closed
[2022-04-27T17:18:34.860Z] e[0;37m INFOe[39m SDKi: tunnel-sdk:ziti_tunnel.c:414 ziti_tunneler_close() closing connection: service=DemoSimpleWebTestService, client=tcp:100.64.0.1:52328
That’s a lot of “Common Routers: (0/0)” - which seems to be the problem. I can see we’ll need some kind of doc/flowchart/tooling to help with this particular issue.
“Common Routers: (0/0)” tells me that your router is not online or that the edge-router-policies don’t seem like they are effective.
The ZAC is showing you that those two identities are online (the first green bubble) but they do not have active sessions (the second green bubble).
I also see that DemoLinuxBlueNwTunneller → Identity does not have access to any services. Adjust service policies.
Every identity needs an edge-router-policy which states “these identities can access these edge routers”.
Every service needs a service-edge-router-policy which states “these services can access these edge routers”
Can you run :
ziti edge list service-edge-router-policies
and
ziti edge list edge-router-policies
on my install i see:
ziti edge list service-edge-router-policies
id: aSMAPlsZHI name: allSvcPublicRouters edge router roles: [#public] service roles: [#all]
ziti edge list edge-router-policies
id: DAMAylsZHI name: allEdgeRouters edge router roles: [#public] identity roles: [#all]
id: mrK1ZCNGSu name: edge-router-mrK1ZCNGSu-system edge router roles: [@ip-172-31-42-64-edge-router] identity roles: [@ip-172-31-42-64-edge-router]
One other thing - if you do a docker-compose down make sure you give it a “-v” without the -v you won’t get a fully clean environment.
I will often just dump the full thing:
docker-compose down -v
ziti@9756e3dfa046:/openziti$ ziti edge list service-edge-router-policies
id: DFKkCMDZJE name: DemoAllERAllServices edge router roles: [#all] service roles: [#all]
results: 1-1 of 1
ziti@9756e3dfa046:/openziti$
ziti@9756e3dfa046:/openziti$ ziti edge list edge-router-policies
id: -1VWmLuAjJ name: edge-router--1VWmLuAjJ-system edge router roles: [@ziti-edge-router-wss] identity roles: [@ziti-edge-router-wss]
id: 4s7QmLuTjJ name: edge-router-4s7QmLuTjJ-system edge router roles: [@ziti-private-blue] identity roles: [@ziti-private-blue]
id: IlI.mmuTyJ name: edge-router-IlI.mmuTyJ-system edge router roles: [@ziti-private-red] identity roles: [@ziti-private-red]
id: PU2QLLuAyJ name: edge-router-PU2QLLuAyJ-system edge router roles: [@ziti-edge-router] identity roles: [@ziti-edge-router]
id: W.2QLmUTy name: edge-router-W.2QLmUTy-system edge router roles: [@ziti-fabric-router-br] identity roles: [@ziti-fabric-router-br]
id: iqoMsnDYJE name: DemoAllEndpointPublicER edge router roles: [@ziti-edge-router] identity roles: [@DemoZitiWinEndpoint @DemoZitiLinuxTunneller]
results: 1-6 of 6
ziti@9756e3dfa046:/openziti$
Also one another thing, When i launched the tunneller, i got this error in its logs:
root@04bbfaebdc52:/openziti# ./ziti-edge-tunnel run -i DemoZitiLinuxTunneller &
[1] 211
root@04bbfaebdc52:/openziti# [ 0.000] INFO tunnel-sdk:ziti_tunnel.c:53 ziti_tunneler_init() Ziti Tunneler SDK (v0.17.31)
[ 0.000] INFO tunnel-cbs:ziti_dns.c:147 seed_dns() DNS configured with range 100.64.0.0 - 100.127.255.255
[ 0.049] INFO ziti-edge-tunnel:resolvers.c:72 init_libsystemd() Initializing libsystemd
[ 0.049] ERROR ziti-edge-tunnel:tun.c:174 find_dns_updater() could not find a way to configure system resolver. Ziti DNS functionality will be impaired
[ 0.049] ERROR ziti-edge-tunnel:utils.c:30 run_command_va() cmd{grep -q ‘^nameserver 100.64.0.2’ /etc/resolv.conf} failed: 256/2/No such file or directory
Is there anything which could be done to solve this error ? And why it is complaining, i can see /etc/resolv.conf file present.
I'll have to fire up my docker env and update the policy like you have. It sure seems like at first look. I will look with a more keen eye in a while.
I would have expected this policy to provide that 'common router':
DemoAllEndpointPublicER edge router roles: [@ziti-edge-router] identity roles: [@DemoZitiWinEndpoint @DemoZitiLinuxTunneller]
could not find a way to configure system resolver. Ziti DNS functionality will be impaired
could you start a separate thread for that? that's a different topic 
i'll check back in a bit
Hi Clint,
I have completely cleaned up the setup and using -v option.
Recreated the setup again from scratch and exactly executed the same steps as mentioned in Wildcard DNS with OpenZiti using Ziti Desktop Edge for Windows - YouTube.
The issue still persists that my Windows edge identity isn’t able to curl http://simple.web.test.
Could it be due to the Ziti DNS issue which i raised ?
I’ll have to acquire a legit Linux environment and try to replicate the issue. It definitely could be that dns issue. I’ll look into it soon and post back
Our ziti-edge-tunneler requieres you have in place resolvectl configured.
We have a troubleshooting guide (How to Use the Linux Tunneler – NetFoundry) it may helps you.
At the same time, the error looks like it can’t find your /etc/resolv.conf this can be because the way you have configured your endpoint, so we’ll need to understand how do you configure you linux DNS clients as well.
Can you help us:
- Validate you have DNS resolution
- Tell us how you configured that one.
With that we can provide you a couple options.
Hi @natashell ,
The ziti-edge-tunneler is running within a docker container. The docker container is launched using this command “sudo docker run --cap-add=NET_ADMIN --device /dev/net/tun --name ziti-tunneler-blue --user root --network zitinw_zitiblue -v zitinw_ziti-fs:/openziti --rm -it openziti/quickstart /bin/bash”
openziti/quickstart is a openziti provided docker image, hence i assume that this images already has DNS issues sorted out.
For your reference: The above way of launching the ziti-edge-tunneler is mentioned in this video “Wildcard DNS with OpenZiti using Ziti Desktop Edge for Windows - YouTube”.
I will check about resolvectl configured within the container.
Also an information, while running ziti-edge-tunneler with openziti/quickstart i found that ip utilities were also missing in the image and i had to install it. Without it obviously for known reasons ziti-edge-tunneler wouldn’t work as it won’t be able to update the iptables.
I just went back and looked at the video. I can see in the video I'm not trying to intercept anything from within that container. I just used it to provide access into the network. When I went frame by frame, I could see that it fails with command ip not found
I also see the nameserver bits fail too:
I think the reply @qrkourier made here Ziti Tunneller complaining "not find a way to configure system resolver" - #8 by qrkourier is probably what you'll need to do.
I think it'd be more useful to continue learning how to intercept packets inside docker on that thread.
Hi @TheLumberjack ,
If in your setup as well there was this DNS issue, then it means that it didn’t cause the problem.
So why in my case, even thou following step by step which you did in the explanatory video Wildcard DNS with OpenZiti using Ziti Desktop Edge for Windows - YouTube, i am still facing challenge in fetching the service ?
Also, Can we provide network access using the running ziti-private-blue ER directly? Then there is not need for tunneller at all. If it is possible, what config changes needs to be done.
I’ll redo that video right now with private routers → edge routers and eliminate the ziti-edge-tunnel deployment. Give me a bit and I’ll put that together…
Finally got my linux VM working. HyperV caused me a bit of pain today. Here’s the demo. This runs docker-compose with a modified docker compose file as mentioned in the other post (changed private to edge in the top left screen). The top right screen is where I exec into the docker controller to run ziti CLI commands. Bottom left screen is “outside” of docker running ziti-edge-tunnel. Bottom right is where I leave the steps up I am running, and then where the curls happen.
If that doesn’t work for you - let us know what particular step you have problems with.
Hi Clint,
This solution worked fine with all linux.
I freshly downloaded ziti-edge-tunnel binary.
I was able to run the Ziti-edge-tunnel directly on my ubuntu 18.04 virtual box, I didn’t see the DNS errors which i was observing previously.
After DIAL configuration via the tunneller proxy, i was able to fetch both http://simple.web.test and http://web-test.blue:8000 & http://web.test.blue:8000.
But i observed a weird issue, when i configured a DIAL for a window ziti-desktop-edge.
I could fetch http://simple.web.test but it wasn’t able to fetch either of http://web.test.blue:8000 or http://web-test.blue:8000
Here is the services listed on the client
Is there any issue with windows ziti-desktop-edge client ?
I would not expect that, no. In fact that was what I used for the first video. I wonder if powershell is in your way. Can you try in a browser with the ZDEW running? It looks to me like powershell/iwr isn't using the NRPT properly - which would be 'news' to me. does "Resolve-DNSName web.test.blue" work?
