Also, I couldn’t figure out, why i am not able to open the ZAC UI.
I tried fetching it like https://localhost:8443, even thou it is listening on port 8443
ubuntu@ubuntuv:~$ netstat -tulpn
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:3022 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:3023 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:8443 0.0.0.0: LISTEN -*
tcp 0 0 0.0.0.0:1280 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:1408 0.0.0.0:* LISTEN -
tcp6 0 0 :::3022 :::* LISTEN -
tcp6 0 0 :::3023 :::* LISTEN -
tcp6 0 0 :::80 :::* LISTEN -
tcp6 0 0 ::1:631 :::* LISTEN -
tcp6 0 0 :::8443 :::* LISTEN -
tcp6 0 0 :::1280 :::* LISTEN -
tcp6 0 0 :::1408 :::* LISTEN -
udp 0 0 0.0.0.0:631 0.0.0.0:* -
udp 0 0 224.0.0.251:5353 0.0.0.0:* 10931/chrome --enab
udp 0 0 224.0.0.251:5353 0.0.0.0:* 10931/chrome --enab
udp 0 0 224.0.0.251:5353 0.0.0.0:* 10931/chrome --enab
udp 0 0 224.0.0.251:5353 0.0.0.0:* 10931/chrome --enab
udp 0 0 0.0.0.0:5353 0.0.0.0:* -
udp 0 0 0.0.0.0:55275 0.0.0.0:* -
udp 0 0 127.0.0.53:53 0.0.0.0:* -
udp 0 0 0.0.0.0:68 0.0.0.0:* -
udp6 0 0 :::58276 :::* -
udp6 0 0 :::5353 :::* -
That might be the compose script being broken. Does http://localhost:1408 (non tls) work?
Yes, This worked. Thanks
i’ll put a fix in for the ZAC but you can add this to the compose file for the ziti-console container and get 8443 working for ZAC:
working_dir: /usr/src/app
As for the ZDEW - that is REALLY strange. if you run:
Get-DnsClientNrptRule | findstr blue
do you get results? You should…
I expect you will NOT get them. Can you turn the ZDEW off and on one time just to see if that ‘fixes’ it? Feels like whatever happened in there might be some bug. If turning it off/on fixes it, i’d wonder if any errors are in the logs for a clue as to what happened
I’d filed the ZAC bug for TLS 12 days ago - ZAC for docker-compose · Issue #691 · openziti/ziti · GitHub PR to fix it is here fix zac not presenting https by dovholuknf · Pull Request #702 · openziti/ziti · GitHub
I did try to restart the ziti-desktop-edge, it didn't work. But the output you asked me does show some results
Logs from ziti-desktop-edge restart
[2022-05-02T16:55:42.369Z] e[0;37m INFOe[39m shutting down events…
[2022-05-02T16:55:42.369Z] e[0;37m INFOe[39m Closing connections and removing tun interface: ZitiTUN
[2022-05-02T16:55:42.369Z] e[0;37m INFOe[39m Closing native tun: ZitiTUN
[2022-05-02T16:55:42.610Z] e[0;37m INFOe[39m Closed native tun: ZitiTUN
[2022-05-02T16:55:42.610Z] e[0;37m INFOe[39m Removing existing interface: ZitiTUN
[2022-05-02T16:55:42.612Z] e[0;37m INFOe[39m Successfully removed interface: ZitiTUN
[2022-05-02T16:55:42.612Z] e[0;37m INFOe[39m ============================== service ends ==============================
[2022-05-02T16:55:42.612Z] e[0;37m INFOe[39m normal shutdown complete
[2022-05-02T16:55:42.615Z] e[0;37m INFOe[39m service has completed
[2022-05-02T16:55:45.251Z] e[0;37m INFOe[39m ============================================================================
[2022-05-02T16:55:45.255Z] e[0;37m INFOe[39m Logger initialization
[2022-05-02T16:55:45.255Z] e[0;37m INFOe[39m - initialized at : 2022-05-02 22:25:45.2552817 +0530 IST m=+0.011717901
[2022-05-02T16:55:45.255Z] e[0;37m INFOe[39m - log file location: C:\Program Files (x86)\NetFoundry, Inc\Ziti Desktop Edge\logs\service\ziti-tunneler.log
[2022-05-02T16:55:45.255Z] e[0;37m INFOe[39m ============================================================================
[2022-05-02T16:55:45.256Z] e[0;37m INFOe[39m running as service. version: C:\Program Files (x86)\NetFoundry, Inc\Ziti Desktop Edge\ziti-tunnel.exe version: 1.11.5, revision: a13994031f87, branch: main, build-by: , built-on: 2022-04-09T00:00:57Z
[2022-05-02T16:55:45.258Z] e[0;37m INFOe[39m ============================== service begins ==============================
[2022-05-02T16:55:49.265Z] e[0;37m INFOe[39m Invoking ZitiTun adapter cleanup script
[2022-05-02T16:55:49.273Z] e[0;37m INFOe[39m reading config file located at: C:\Windows\system32\config\systemprofile\AppData\Roaming\NetFoundry\config.json
[2022-05-02T16:55:49.274Z] e[0;37m INFOe[39m creating TUN device: ZitiTUN
[2022-05-02T16:55:49.701Z] e[0;37m INFOe[39m setting TUN interface address to [100.64.0.1]
[2022-05-02T16:55:49.706Z] e[0;37m INFOe[39m checking TUN dns servers
[2022-05-02T16:55:49.845Z] e[0;37m INFOe[39m TUN dns servers set to: [fec0:0:0:ffff::1 fec0:0:0:ffff::2 fec0:0:0:ffff::3]
[2022-05-02T16:55:49.846Z] e[0;37m INFOe[39m setting routes for cidr: 100.64.0.0/10. Next Hop: 100.64.0.0
[2022-05-02T16:55:49.847Z] e[0;37m INFOe[39m routing applied
[2022-05-02T16:55:54.353Z] e[0;37m INFOe[39m DNS is applied to the TUN interface, because Ziti policies test result in this client is false
[2022-05-02T16:55:58.282Z] e[0;37m INFOe[39m starting ziti-sdk-c 0.26.26(f138ac8)[Sat-04/09/2022-00:01:24-+00]
[2022-05-02T16:55:58.282Z] e[0;37m INFOe[39m Setting cziti log level to: 3
[2022-05-02T16:55:58.282Z] e[0;37m INFOe[39m SDKi: ziti_log_set_level set log level: ziti_log_lvl=3 &ziti_log_lvl = 000000006eda5000
[2022-05-02T16:55:58.286Z] e[0;37m INFOe[39m SDKi: tunnel-sdk:ziti_tunnel.c:58 ziti_tunneler_init() Ziti Tunneler SDK (v0.15.25)
[2022-05-02T16:55:58.291Z] e[0;37m INFOe[39m flushing DNS cache using ipconfig /flushdns
[2022-05-02T16:55:58.293Z] e[0;37m INFOe[39m DNS listening at: 100.64.0.1:53
[2022-05-02T16:56:00.686Z] e[0;37m INFOe[39m connecting identity: zdewclint[777a1090befee78583a0866cf4440c50b8f4a319]
[2022-05-02T16:56:00.686Z] e[0;37m INFOe[39m loading identity zdewclint[777a1090befee78583a0866cf4440c50b8f4a319]
[2022-05-02T16:56:00.686Z] e[0;37m INFOe[39m SDKi: ziti_log_set_level set log level: ziti_log_lvl=3 &ziti_log_lvl = 000000006eda5000
[2022-05-02T16:56:00.687Z] e[0;37m INFOe[39m SDKi: ziti-sdk:ziti.c:378 ziti_init_async() ztx[0] Ziti C SDK version 0.26.26 @f138ac8(HEAD) starting at (.687)
[2022-05-02T16:56:00.688Z] e[0;37m INFOe[39m SDKi: ziti-sdk:ziti.c:379 ziti_init_async() ztx[0] Loading from config[C:\Windows\system32\config\systemprofile\AppData\Roaming\NetFoundry\777a1090befee78583a0866cf4440c50b8f4a319.json] controller[https://ziti-edge-controller:1280]
[2022-05-02T16:56:00.688Z] e[0;37m INFOe[39m Ziti Tunneler status set to running. starting cancel loop
[2022-05-02T16:56:00.688Z] e[0;37m INFOe[39m SDKi: ziti-sdk:ziti_ctrl.c:371 ziti_ctrl_init() ctrl[ziti-edge-controller] ziti controller client initialized
[2022-05-02T16:56:00.688Z] e[0;33m WARNe[39m SDKw: ziti-sdk:ziti.c:743 ziti_re_auth_with_cb() ztx[0] starting to re-auth with ctlr[https://ziti-edge-controller:1280] api_session_status[0] api_session_expired[TRUE]
[2022-05-02T16:56:00.690Z] e[0;37m INFOe[39m beginning metric collection
[2022-05-02T16:56:00.751Z] e[0;37m INFOe[39m starting DNS proxy upstream: [10.146.14.19 10.149.14.9 10.150.10.9 100.64.0.1 192.168.3.1 fec0:0:0:ffff::1 fec0:0:0:ffff::2 fec0:0:0:ffff::3], local: [100.64.0.1]
[2022-05-02T16:56:00.823Z] e[0;37m INFOe[39m SDKi: ziti-sdk:ziti.c:1385 version_cb() ztx[0] connected to controller https://ziti-edge-controller:1280 version v0.25.4(ea528df682bb 2022-04-04T14:37:25Z)
[2022-05-02T16:56:00.851Z] e[0;31mERRORe[39m zitiContextEvent failed to connect[api session is partially authenticated, waiting for auth query resolution] to controller for C:\Windows\system32\config\systemprofile\AppData\Roaming\NetFoundry\777a1090befee78583a0866cf4440c50b8f4a319.json
[2022-05-02T16:56:00.851Z] e[0;37m INFOe[39m successfully loaded @https://ziti-edge-controller:1280
[2022-05-02T16:56:00.851Z] e[0;37m INFOe[39m connecting identity completed: zdewclint[777a1090befee78583a0866cf4440c50b8f4a319] false/false
[2022-05-02T16:56:00.851Z] e[0;37m INFOe[39m SDKi: ziti-sdk:ziti.c:1274 ziti_set_api_session() ztx[0] api session set, setting api_session_timer to 540s
[2022-05-02T16:56:00.852Z] e[0;37m INFOe[39m successfully loaded zdewclint@https://ziti-edge-controller:1280
[2022-05-02T16:56:00.852Z] e[0;37m INFOe[39m connecting identity completed: zdewclint[777a1090befee78583a0866cf4440c50b8f4a319] false/false
[2022-05-02T16:56:00.918Z] e[0;37m INFOe[39m SDKi: ziti-sdk:channel.c:223 new_ziti_channel() ch[0] (ziti-edge-router@tls://ziti-edge-router:3022) new channel for ztx[0] identity[zdewclint]
[2022-05-02T16:56:00.918Z] e[0;37m INFOe[39m SDKi: ziti-sdk:channel.c:734 reconnect_channel() ch[0] reconnecting NOW
[2022-05-02T16:56:00.921Z] e[0;37m INFOe[39m service update: wildcard.web.test.service, id: cToQC1YsoR, portocols:[tcp], addresses:[{true .blue 0}], portRanges: [{8000 8000}]
[2022-05-02T16:56:00.924Z] e[0;37m INFOe[39m service update: basic.web.test.service, id: 6NE-IsY1zR, portocols:[tcp], addresses:[{true simple.web.test 0}], portRanges: [{80 80}]
[2022-05-02T16:56:01.014Z] e[0;37m INFOe[39m SDKi: ziti-sdk:channel.c:635 hello_reply_cb() ch[0] connected. EdgeRouter version: v0.25.4|ea528df682bb|2022-04-04T14:37:25Z|linux|amd64
[2022-05-02T16:56:01.014Z] e[0;37m INFOe[39m router[ziti-edge-router@tls://ziti-edge-router:3022]: connected to v0.25.4|ea528df682bb|2022-04-04T14:37:25Z|linux|amd64
[2022-05-02T16:56:02.872Z] e[0;37m INFOe[39m new event client connected - sending current status
[2022-05-02T16:56:02.873Z] e[0;37m INFOe[39m status sent. listening for new events
[2022-05-02T16:56:02.902Z] e[0;37m INFOe[39m new event client connected - sending current status
[2022-05-02T16:56:02.902Z] e[0;37m INFOe[39m status sent. listening for new events
[2022-05-02T16:56:03.952Z] e[0;37m INFOe[39m mapped the following hostnames: map[.blue:true simple.web.test:true]
[2022-05-02T16:56:04.052Z] e[0;37m INFOe[39m ConnectionSpecificDomains detected: [vsnl.co.in. vsnl.co.in. intl.vsnl.co.in.]
[2022-05-02T16:56:06.025Z] e[0;37m INFOe[39m Added connection specific domains to NRPT: map[.intl.vsnl.co.in:true .vsnl.co.in:true]
[2022-05-02T16:56:06.025Z] e[0;37m INFOe[39m establishing links to all upstream DNS. total detected upstream DNS: 8
[2022-05-02T16:56:06.025Z] e[0;37m INFOe[39m adding upstream dns server: 10.146.14.19
[2022-05-02T16:56:06.025Z] e[0;37m INFOe[39m adding upstream dns server: 10.149.14.9
[2022-05-02T16:56:06.026Z] e[0;37m INFOe[39m adding upstream dns server: 10.150.10.9
[2022-05-02T16:56:06.026Z] e[0;37m INFOe[39m adding upstream dns server: 192.168.3.1
[2022-05-02T16:56:06.026Z] e[0;37m INFOe[39m starting goroutines for all connected DNS proxies. Total goroutines to spawn: 4 of 8 detected DNS
[2022-05-02T16:56:20.851Z] e[0;33m WARNe[39m SDKw: ziti-sdk:posture.c:194 ziti_send_posture_data() ztx[0] first run or potential controller restart detected
How much ever times i hit “curl http://web.test.blue:8000”, there isn’t anything in the log file.
Looks like even thou the dns entry is added by ziti for wild card, for some reason wild card isn’t getting intercepted.
Sorry @sameersarkar-tcl - I lost track of this issue. Are you all set? I don’t want to leave you hanging if you didn’t get this working?
Hi @TheLumberjack ,
I finally got it working but still the wild card isn’t working with ziti-desktop-edge (windows version). For this i need your help in identifying the problem area.
On another hand, I was able to de-construct the setup where the individual components could run in different VM instances. On server/service side had 2 setups which used ziti-edge-router(edge) or ziti-tunnel + ziti-edge-router(private) and identity (client) was able to fetch the service/app. But i have specific question on the later combination.
Below is the setup diagram and question from it:
Questions:
Q1. Ziti-Tunnel is clearly using “ziti-edge-router (200.155.43.70) to onboard the web app traffic on the ZTN fabric. So, is it correct to say that Ziti-edge-router “private”, not required to be deployed along with Ziti-Tunnel ? Please confirm.
Q2. If Ziti-edge-router private is required, what roles does it play in this network ?
Thanks & Regards,
Sameer
Oh no, I think I know the issue. I've been running a beta build for a while now to do longevity testing. I think I need to make a new release to enable this functionality. There was one bug that held the release up, but I think we are through it. I'll work with the team and see if we can get a release out soon.
Yes, correct. You just need any edge client that knows how to offload traffic. That can be either ziti-edge-tunnel or an edge router. As an aside, in the latest versions of ziti, we changed the "private" routers to be edge routers too. That means you wouldn't need ziti-edge-tunnel in the latest stuff because you'd have an edge router in the blue network. This answer should also cover your second question too.
Hopefully that makes sense.
This would definitely help. Please do let me know once you made an official release.
This sounds great. But then in that case why having 2 options "edge" and "private". If both are serving the same purpose, then we can do away with both options.
There is a difference between private and edge, a router can be an edge router while still being private. To explain in more detail, here is what was changed in the private (non-edge) blue router to make it accessible from the edge while maintaining its "private" status.
The following listeners section was previously commented out, but by uncommenting this, we allow the ziti-private-blue router to host services that are accessible from the edge.
listeners:
# bindings of edge and tunnel requires an "edge" section below
- binding: edge
address: tls:0.0.0.0:3022
options:
advertise: ziti-private-blue:3022
connectTimeoutMs: 1000
getSessionTimeout: 60s
- binding: tunnel
options:
mode: host #tproxy|host
What makes this still a private router is the fact that, in the link section, we still have the listeners sub-section commented out. With this section remaining commented out, the router is not advertising any link listeners and therefore not exposing its existence to the network but it still has a link dialer which allows it to reach out to the network.
link:
dialers:
- binding: transport
# listeners:
# - binding: transport
# bind: tls:0.0.0.0:10080
# advertise: tls:ziti-private-blue:10080
# options:
# outQueueSize: 4
There is no “private” option per se right? What makes the edge router private is that it is deployed in a private network.
Thanks @gberl002 for the explanation.
There is an "option" if you can call it that. It's not simply a flag in the config to call it private but rather when you create a config such as with ziti create config router edge there is a --private option available which will modify the config so that it creates a private edge router. If you leave that --private option off of the cli command it will create a public edge router.
So, just having the router on your private network doesn't make it a private router, the particular config values/format will determine if it is private or public. You could have a public edge router on a private network.
Thank you Geoff, that is a very good information.
Do you have a documentation on this --private option? I sure would like to learn more about it.
Thank you,
James
We didn’t do a fantastic job documenting this. We are trying to get better at doc… 
I checked the changelog and it’s not clearly spelled out in there. I also see that it doesn’t seem to be mentioned in the help output:
ziti create config router
Error: required flag(s) "routerName" not set
Usage:
ziti create config router [flags]
ziti create config router [command]
Aliases:
router, rtr
Available Commands:
edge Create an edge router config
fabric Create a fabric router config
Flags:
-h, --help help for router
-o, --output string designated output destination for config, use "stdout" or a filepath. (default "stdout")
-n, --routerName string name of the router
-v, --verbose Enable verbose logging. Logging will be sent to stdout if the config output is sent to a file. If output is sent to stdout, logging will be sent to stderr
Use "ziti create config router [command] --help" for more information about a command.
required flag(s) "routerName" not set
@gberl002 probably worth filing an issue to make sure it’s listed as a flag at this level?
Oh I forgot you CAN get the help and explain it by running:
ziti create config router edge --help
Creates the edge router config
Usage:
ziti create config router edge [flags]
Aliases:
edge, edge
Examples:
# Create the edge router config for a router named my_router
ziti create config router edge --routerName my_router
Flags:
-h, --help help for edge
--private Create a private router config
--wss Create an edge router config with wss enabled
Global Flags:
-o, --output string designated output destination for config, use "stdout" or a filepath. (default "stdout")
-n, --routerName string name of the router
-v, --verbose Enable verbose logging. Logging will be sent to stdout if the config output is sent to a file. If output is sent to stdout, logging will be sent to stderr
clint_private_aws: ~$
but it doesn’t explain what it means that it’s ‘private’