FYI: I have created an issue: Incorrect default ports on docker .env file · Issue #986 · openziti/ziti (github.com) to resolve the non-default ports defined in the default .env file.
Looking at the commands that you have written, I think this could be an issue as well.
ziti edge create service-policy "${DEVICE_NAME}.bind" Bind --service-roles "@${DEVICE_NAME}" --identity-roles "#${DEVICE_NAME}.hosts"
The bind identity should be the client identity, ie @${DEVICE_NAME}.
Here is an example of one I have:
@fp.csat.svc is the name of the service (${DEVICE_NAME}) by the looks
@cf.csat.identity is the identity of the device that I want the traffic to exit out of
#fp.csat.tag is what you add onto the identity that wants to access the service.
1 Like
Running that command inside the docker container for the edge router returns:
ERROR: Default Admin
- Identity does not have access to any services. Adjust service policies.
ERROR: my.domain.jruiz.com
- Identity does not have access to any services. Adjust service policies.
OKAY : jruiz.player (1) -> jruiz.server (1) Common Routers: (1/1) Dial: Y Bind: N
OKAY : jruiz.server (1) -> jruiz.server (1) Common Routers: (1/1) Dial: N Bind: Y
Seems okay to me? At least the server/client for the Minecraft endpoints, i don’t know about the others
I’ve sent the logs for client and server applications, where can I find the logs for the edge router inside the container? I’ve sent you the logs from the docker container but I’m not so sure they are helpful. Also I think the problem might be at the server side? I see some weird errors with the controller.
I’m not so sure about that, because in your example you’re using an identity but I’m using an attribute assigned to the identity.
This is how it looks in my ZAC
Btw I forgot to say this before but thank you so much for the support 
I’m learning a lot while debugging this.
Yes. All that looks perfect. In fact, thanks for the logs! I am pretty sure I see the problem. I’m transcribing my response via email here for everyone else to see too…
In the logs I could see this:
[2023-02-02T18:25:39.736Z] INFO tunnel-cbs:ziti_hosting.c:610 on_hosted_client_connect() hosted_service[jruiz.server], client[caca] dst_addr[udp:jruiz.server.ziti:57775]: incoming connection
So from that ONE line… you can see:
-
the service is named “jruiz.server” from this bit:
hosted_service[jruiz.server] -
the client connecting was named “caca” from this bit:
client[caca] -
the “destination” address from this bit:
dst_addr[udp:jruiz.server.ziti:57775]
The destination appears to be wrong. When you filled out the “host.v1” configuration, did you use “localhost” or did you use jruiz.server.ziti? I am 99% sure you used jruiz.server.ziti or there’s a bug somewhere. Instead, you want to use “localhost”. The host.v1 config is telling the tunneller running on the server to offload traffic ‘somewhere’. Since it’s the ziti desktop edge for windows, and that’s where Minecraft is running, you want that to be “localhost”…
Can you show us what the config looks like in ZAC? Does it use localhost like this?
Okay so first of all, sorry for not answering sooner. I had to deal with some personal/work stuff.
Second, about this:
Im sorry, this might have been misleading to you… but this client you mention is a dummy client (with a silly spanish name… facepalm) i made to do some tests on another machine…
I’ve been testing with so many tokens i’ve lost count and I think the logs are contaminated with unrelated stuff (for example, before posting the question here, I’ve deleted and recreated the docker compose stack a couple times, and the Ziti edge apps tried to connect with an old enrollment token and there is some noise in the logs regarding that)
I think the best way to go here is to do a fresh start. Tear down every docker container and volume and start from scratch. Also, I’m going to use the default Minecraft Bedrock UDP port 19132 instead of a custom one just for the sake of simplicity.
I’ve uninstalled completely the Ziti Edge Desktop and mobile apps. Also, and again, for the sake of simplicity, I’m going to use a Windows laptop as the client instead of the Android one. The reason for this is that it’s more comfortable to extract logs from there.
So, I know im going to repeat myself but better be 100% clear, starting from the oracle cloud instance, the firewall has the following open ports
And the firewalld configuration is as follows:
ubuntu@jruizmachine:~$ sudo firewall-cmd --list-all
You're performing an operation over default zone ('public'),
but your connections/interfaces are in zone 'docker' (see --get-active-zones)
You most likely need to use --zone=docker option.
public
target: default
icmp-block-inversion: no
interfaces:
sources:
services: dhcpv6-client ssh
ports: 8441/tcp 8442/tcp 6262/tcp 10080/tcp 8443/tcp 1280/tcp 3022/tcp
protocols:
forward: yes
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
NOTE: I just noticed the message saying that my connections interfaces are in zone ‘docker’… I’m unsure if that’s related to my issue…
I’ve done the following to delete and recreate the docker compose stack
docker compose down
docker system prune -a && docker volume rm $(docker volume ls -q)
This should clear everything from containers/images to volumes (if not, let me know)
Then goes docker compose up
Then I’ve entered the ziti-edge-router container to run some commands.
The commands I posted before were copied and pasted from the Minecraft tutorial, and some variable names were a little bit confusing, so I’ve made some changes.
These are the commands I’ve run inside the container
zitiLogin
export SERVER_DEVICE_NAME="jruiz.server"
export CLIENT_DEVICE_NAME="jruiz.client"
export PORT=19132
ziti edge create identity device ${SERVER_DEVICE_NAME} -o ${SERVER_DEVICE_NAME}.jwt -a "${SERVER_DEVICE_NAME}.hosts"
ziti edge create identity user ${CLIENT_DEVICE_NAME} -o ${CLIENT_DEVICE_NAME}.jwt -a "${SERVER_DEVICE_NAME}.clients"
ziti edge create config ${SERVER_DEVICE_NAME}.hostv1 host.v1 '{"protocol":"udp", "address":"localhost","port":'${PORT}'}'
ziti edge create config ${SERVER_DEVICE_NAME}.interceptv1 intercept.v1 '{"protocols":["udp"],"addresses":["'${SERVER_DEVICE_NAME}'.ziti"], "portRanges":[{"low":'${PORT}', "high":'${PORT}'}]}'
ziti edge create service ${SERVER_DEVICE_NAME} --configs "${SERVER_DEVICE_NAME}.hostv1,${SERVER_DEVICE_NAME}.interceptv1"
ziti edge create service-policy "${SERVER_DEVICE_NAME}.bind" Bind --service-roles "@${SERVER_DEVICE_NAME}" --identity-roles "#${SERVER_DEVICE_NAME}.hosts"
ziti edge create service-policy "${SERVER_DEVICE_NAME}.dial" Dial --service-roles "@${SERVER_DEVICE_NAME}" --identity-roles "#${SERVER_DEVICE_NAME}.clients"
Then I’ve installed Ziti Desktop Edge on the Server machine, and enrolled the JWT token. Done the same on the client machine.
This is how everything looks in the ZAC
In the image below, the port says 19132 but it doesn’t show the last digit (i’ve checked)
I’ve started the Minecraft Bedrock Server on port 19132 on the server machine
Tried to reach that using the client machine, no success. Just to double check, trying to access localhost:19132 on the Windows desktop works (i tested just in case…)
@TheLumberjack I’m sending you the logs again via email because it doesn’t let me attach them to the forum post
Hopefully I don’t forget to censor my custom domain and name (and no silly spanish names…)
I know i repeat myself a lot, but thanks a ton for the patience and the support
Hi @jruiz94. I would expect you’re running the java version of minecraft right? I had to go through the whole process myself… Java Minecraft server runs on tcp not udp. I think that’ll fix your problem up. I ran all the same commands you supplied and it worked fine once i checked udp vs tcp
I gather you have tried the following command and it is returning as per previous posts?
On the windows client, if you ping jruiz.server.ziti I gather it is resolving to a 100.x.x.x address - this confirms that the intercept in pushing traffic over the overlay.
Hi again. No, I was using Bedrock edition. Bedrock edition uses UDP AFAIK.
However, I should’ve tried with Minecraft Java because… you’re absolutely right. It works. I mean, I have created another service to leave the UDP one as it was just in case, so I added an intercept for jruiz.server.tcp.ziti with a TCP port and the policy dial/bind and BOOM it worked instantly and flawlessly.
Soooo I don’t know, seems like everything was good all along (except perhaps the closed port 3022 I had that @gooseleggs mentioned) and I got hard trolled by Microsoft and its Bedrock Minecraft thing or something… I don’t know.
I’m still kinda puzzled on what might be wrong about the UDP Minecraft Bedrock setup. Like I want to know if it’s a firewall thing or something? I don’t know. But anyway, that’s a non-OpenZiti related issue and I’m super happy to have this fixed and working on Docker!
Thanks a lot again to both of you
oh so you were using Bedrock! I didn’t think you could host your own!? Can you point me at how you started it up?? I’ll give that a shot.
Sounds like you got TCP / Java working though, that’s cool
I was able to get Bedrock running over UDP as well. Here’s the exact set of steps I ran to accomplish it:
zitiLogin
export SERVER_DEVICE_NAME="jruiz.bedrock.server"
export PORT=19132
ziti edge create identity device ${SERVER_DEVICE_NAME} -o ${SERVER_DEVICE_NAME}.jwt -a "${SERVER_DEVICE_NAME}.clients,${SERVER_DEVICE_NAME}.hosts"
ziti edge create config ${SERVER_DEVICE_NAME}.hostv1 host.v1 '{"protocol":"udp", "address":"localhost","port":'${PORT}'}'
ziti edge create config ${SERVER_DEVICE_NAME}.interceptv1 intercept.v1 '{"protocols":["udp"],"addresses":["'${SERVER_DEVICE_NAME}'.ziti"], "portRanges":[{"low":'${PORT}', "high":'${PORT}'}]}'
ziti edge create service ${SERVER_DEVICE_NAME} --configs "${SERVER_DEVICE_NAME}.hostv1,${SERVER_DEVICE_NAME}.interceptv1"
ziti edge create service-policy "${SERVER_DEVICE_NAME}.bind" Bind --service-roles "@${SERVER_DEVICE_NAME}" --identity-roles "#${SERVER_DEVICE_NAME}.hosts"
ziti edge create service-policy "${SERVER_DEVICE_NAME}.dial" Dial --service-roles "@${SERVER_DEVICE_NAME}" --identity-roles "#${SERVER_DEVICE_NAME}.clients"
You’ll see I used YOUR commands exactly with the one difference that I have used the SAME identity as both the “host” and the “client” for testing. OpenZiti can allow you to do that, and it makes it a lot easier to test as long as you don’t try to intercept the same address you’re sending the traffic to. Since I used “localhost” - like you did, that’s not a problem.
Some things I noticed when doing it:
- I like to
tailmy logs (using WSL in windows) to see what is happening. When I do that, when it works, you clearly will see these two log messages:- [2023-02-04T00:46:59.170Z] INFO tunnel-cbs:ziti_tunnel_ctrl.c:686 on_service() starting intercepting for service[jruiz.bedrock.server]
- [2023-02-04T00:48:11.713Z] INFO tunnel-cbs:ziti_hosting.c:610 on_hosted_client_connect() hosted_service[jruiz.bedrock.server], client[jruiz.bedrock.server] dst_addr[udp:jruiz.bedrock.server.ziti:19132]: incoming connection
This is showing me/you that the tunneler (ZDEW) intercepted the serivce Jruiz.bedrock.server, it went out to the OpenZiti overlay network, and came back down to the same client (the ‘incoming connection’ log)
When at the “Additional Servers” menu, a “ziti-enabled” Minecraft server won’t have a ping shown I found that both pretty neat, but also somewhat of a bummer since ping time is important. OpenZiti doesn’t support ICMP at this time, and it seems minecraft is using ICMP ping to do ‘ping time’
After Testing with One Identity…
After making sure it worked with a single identity, I then removed the ‘dial’ ability from the server using:
ziti edge update identity ${SERVER_DEVICE_NAME} -a ${SERVER_DEVICE_NAME}.hosts
Then I created a new “client” identity:
export CLIENT_DEVICE_NAME="jruiz.bedrock.client"
ziti edge create identity user ${CLIENT_DEVICE_NAME} -o ${CLIENT_DEVICE_NAME}.jwt -a "${SERVER_DEVICE_NAME}.clients"
enrolled the jwt into my local ZDEW:
Turned Minecraft on, and it all kept working.
So we know it’ll work, we’re now back to diagnosing other problem. I had to run CheckNetIsolation.exe LoopbackExempt –a –p=S-1-15-2-1958404141-86561845-1752920682-3514627264-368642714-62675701-733520436 to get the app to work.
Perhaps you didn’t run that on the ‘other’ windows machine?
1 Like
That’s quite clever I didn’t think of that, that makes testing super quick since I don’t need another computer 
Yeah haha me too, but the path to the logs is a little long so sometimes I just go there directly using explorer.
I can confirm that I’m seeing these 2 exact messages to, so I assume my setup is working as intended.
Nice to know, I was curious about this because I noticed that with my local IP (192.168.1.xxx) I could connect but it won’t show the ping either.
Okay so this is something I forgot to mention in the DM i sent to you. I already did this because initially it didn’t work in localhost but I forgot, sorry about that. If I run:
AdminPowershell > CheckNetIsolation.exe LoopbackExempt -s
*some text in spanish since my windows is in spanish and i don't know what it says here in english*
[1] -----------------------------------------------------------------
Name: 001
SID: S-1-15-2-3624051433-2125758914-1423191267-1740899205-1073925389-3782572162-737981194-4256926629-1688279915-2739229046-3928706915
[2] -----------------------------------------------------------------
Name: AppContainer NOT FOUND
SID: S-1-15-2-3624051433-2125758914-1423191267-1740899205-1073925389-3782572162-737981194-4043415302-551583165-304772019-4009825106
[3] -----------------------------------------------------------------
Name: microsoft.minecraftuwp_8wekyb3d8bbwe
SID: S-1-15-2-1958404141-86561845-1752920682-3514627264-368642714-62675701-733520436
You can see I’ve already have added the Minecraft to the LoopbackExempt list thing
However, even doing all of this I still can’t connect, I don’t know why, even using the setup with one single identity you mentioned. I’m looking at Windows Firewall but I’m almost 100% sure that I’ve allowed the Minecraft app through it and I have opened the ports
This is what I see in my inbound rules
I’ve checked that it can be used in all domains of networks (private and public)
Then I’ve also added the .exe file
Also in my outbound rules there is Minecraft for Windows 10:
And I’ve allowed everything there too.
Sooooo I don’t know. For me it’s still a win because I can see in the logs that the traffic is being routed through the OpenZiti network, which is what I wanted in the end 
But thanks a lot for the extra help regarding this, I really appreciate it 
Btw as I was writing the above message, I even checked if the ziti networks were public or private. They were public so I changed them to private to test also that, still doesn’t work. I dunno man, I’m running out of ideas, this Windows of mine is trolling me >:(
That’s definitely a bummer. The last thing you can do to just prove once and for all that once windows (or minecraft) stops trolling you, you will be able to get things working…
I wrote a small powershell utility that will help prove/verify everything is working over UDP which is kinda hard to test via Windows… You can download the “udp-server.ps1” from this link if you are interested in proving that your UDP service is working and SEEING it work.
Usage is simple, you might (probably?) have to run this command to run it: set-executionpolicy remotesigned after you download it.
Here’s how you can try it out… Run all your OpenZiti overlay stuff as you have it. On the computer where you have the server and the identity both serving as the “client” and the “server” open two powershell windows and then:
-
download the script using:
iwr https://raw.githubusercontent.com/openziti/ziti/a335a9c762046c78c115d6be63705b6e016d06b5/etc/util/udp-server.ps1 -OutFile udp-server.ps1
-
source the script:
. .\udp-server.ps1
-
in the ‘server’ powershell window run:
Start-UDP-Server -Port 19132
-
in the ‘client’ powershell window run:
Test-UDP-Client -Server localhost -Port 19132
-
in the ‘client’ powershell type “make sure it works[enter]” and you’ll see it relayed to the ‘server’
-
stop the ‘client’ and now do the same thing but use the intercept address, in my case that’s “minecraft.bedrock.ziti” so I ran:
Test-UDP-Client -Server minecraft.bedrock.ziti -Port 19132
-
in the ‘client’ powershell type: “works over ziti” and you’ll see it relayed to the ‘server’
-
look in the logs and you’ll see those log statements if you want, but if you see it relayed we know it worked. Do note tha the logged port should change
Here’s a gif of me doing this if you need it:
unrelated - Java doesn’t use ‘ping’
I also noticed that on Java version, they don’t use ping for the ping time
Dangit, it’s not working, I thought it was the Minecraft server but yeah this doesn’t work either. You can check the gif below.
I’ve also tried from another computer (not in the gif) in my LAN using the local ip address and it worked, but using the OpenZiti tunnel for some reason it doesn’t work.
Is it possible for some reason for Windows to block incoming connections only coming from the OpenZiti tunnel? And only UDP? Seems super weird to me…
Question, is it needed to open the port 3022 on UDP apart from TCP in the Oracle Cloud Instance? I’ve only opened port 3022 on TCP, I’m not sure if that might affect or not…
Well that’s good that means there’s something else to check out / test. You don’t have any other VPN type software installed do you? Windows Defender/firewall does track UDP and TCP.
The clients will connect to the edge router over that port (3022) TCP. Even though we can intercept both tcp/udp, right now the transport is always over a persistent TCP stream. So if you intercept UDP, is still transits the OpenZiti overlay via TCP.
Can you shoot me over your tunneler logs from the Windows ZDEW (found at C:\Program Files (x86)\NetFoundry, Inc\Ziti Desktop Edge\logs\service\ziti-tunneler.log)?
You’ve proven that the port is open and that it would work when it’s not over ziti, there is likely something small wrong now.
It sounds like your firewall is not getting in the way although you can turn it off to be sure. Have you tried disabling your anti virus in case that is killing the connection?
This was a really difficult issue to track down. @jruiz94 and I were chatting back and forth in a Minecraft server I had stood up and sent him an identity for…
He eventually sent me a feedback log and I noticed in there one small line with server[udp:::1:19132] connected which I immediately wondered if IPv6 was getting in the way.
It seems that IPv6 was the problem! He was able to use the udp-server.ps1 to test UDP successfully by changing the host.v1 from using localhost to using either 127.0.0.1 or 192.168.x.x. I personally turn IPv6 support off on my Windows machine because I’ve had it cause issues like this in the past for me, which is why it worked for me…
Since he was able to test UDP successfully using that tool, I think he’ll be successful at using minecraft. We’ll wait for a follow up!
Hey! So yeah, as @TheLumberjack said, we were chatting in his Minecraft Server trying to sort this out… and after trying a couple things, lowering the log level to debug, getting killed by a Creeper and letting @TheLumberjack parse the advanced logs… we concluded it was the IPV6.
Since I set up the service config using localhost apparently it resolved using IPV6 and that didn’t quite work. I’ve been having these IPV6 problems because of my internet service provider. For some reason it defaults to IPV6 for some stuff, I also had tons of problems in the laptop i use for work because of this.
The fix was to change the service to use 127.0.0.1 instead (I think the LAN ip 192.168.x.x would also have worked) so IPV4 is forced.
I know I’ve already said this a gazillion times, but thanks for all the support. It’s been super fun and amazing to have this fixed with you guys I’ll see you around with, hopefully, “easier fixes” hahahah
Cheers!
P.S.: If you’re a visitor reading this forum post please give some stars to the OpenZiti Github repo, these nice folks totally deserve them and more!!
2 Likes