Hi, I have a few questions. First of all, I want to ask regarding the DNS name in the addresses. Is it can be a random name? For example, “webserver.ziti”. Or is it need the real DNS name?
Currently in the progress of doing the Totally Private Postgres. My environment is quite the same as in the Youtube video. The difference is that I’m using droplet (Digital Ocean Cloud), the VM1 is the platform (using the docker-compose setup), and the VM2 is just a simple web server using httpd (not inside a docker). After setting up everything, I still cannot access the web server from the client side.
When I try to curl the private.webserver.ziti, it will give the timeout.
Curl: (28) Failed to connect to private.webserver.ziti port 80 after 16670 ms: Timed out
Also, I’m curious about the ping IP, the reply from 100.64.0.4 and what it means by smartrouting in the terminator strategy.
Besides, do I need to configure routing in the router or anything related?
Thank you in advance again for sharing and helping me.
Yep. whatever name you want. Isn't that neat? It just needs to be 'valid'. So mostly ascii characters and a few other. Any valid DNS entry.
You can't use ping. Ping is a network level tool and uses ICMP. It's really only useful to query the IP associated with the DNS name. Curl should work though. If it's not working, and your setup is pretty much the same as the video, you should tail (or look at) the tunneler log, and you should look at the log on the router. If there are no helpful messages in there (there usually is) then you can look at the controller log. Finally, I will often use ziti edge policy-advisor identities -q (or policy-adisor services, shown below) to check my policies are correct. You might start with policy advisor first though. More specifically you can use a service name like this:
ziti edge policy-advisor services svc.device-virtual -q
OKAY : svc.device-virtual.identity (1) -> svc.device-virtual (1) Common Routers: (1/1) Dial: N Bind: Y
Notice here, I have a service that can be BOUND -- but no identities can Dial it (lol - not very useful!)
Now look at this example:
ziti edge policy-advisor services svc.core-data -q
OKAY : _cdaws_clint (1) -> svc.core-data (1) Common Routers: (1/1) Dial: Y Bind: N
OKAY : svc.core-data.identity (1) -> svc.core-data (1) Common Routers: (1/1) Dial: N Bind: Y
That service has an identity 'binding' it -- and an identity that can dial it. It's useful! That make sense?
The most common problem is forgetting one or the other service policy. I can tell that because your tunneler has the service, you have the dial properly set correctly. You might have a missing bind policy or the policy might be setup wrong. policy-advisor will show you if you have it wrong, but you’ll have to figure out ‘how’ you did it wrong by reviewing the policy you have in place.
Another common problem is that the router/identity offloading the traffic to the final destination doesn’t have access to the service you’re trying to access. Maybe a firewall is in the way, or the IP in the host.v1 config is wrong etc.
It’s almost certainly some small configuration issue, sometimes it’s hard to find them but I’m sure you’ll spot it!
I do understand that we need to bind the service representing the HTTP server. So for the identity server, do we just randomly eventhough it’s an id? I’m confused about this part.
Correct. In tunneler-based solutions, like that example, you need one identity that is doing the dialing (the client side) and you need one identity that's offloading from the overlay network (the far/server side, doing the 'bind'). You're missing the bind.
If I remember correctly, this is what will end up matching the bind service policy. I think that's what gets you the bind side.
Yep. That was my guess above. But you need to. You could run the same command just use 'private-web-servers' instead of postgres. Make sure you run both of those commands though. You need a bind service policy and you need identities that will match the identity-roles attribute you use in the service policy
Not sure exactly what you mean by randomly. Here you're explicitly telling the overlay that any identity with the matching attribute is expected to be able to bind the service. It's how you could provide redundancy/additional throughout to your back end. For test stuff, you rarely need that redundancy/throughout. It's just used to be instructive that it works this way.
Update:
I already add the bind services and the attribute
ziti edge policy-advisor services private.web -q
OKAY : web-client (2) -> private.web (5) Common Routers: (2/2) Dial: Y Bind: N
OKAY : ziti-edge-router (2) -> private.web (5) Common Routers: (2/2) Dial: N Bind: Y
Here is step by step
7. ziti edge create identity user web-client -a 'web-clients' -o web.client.jwt
//copy jwt file into any of root file
8. docker compose cp ziti-controller:/persistent/web.client.jwt /etc
9. Open winSCP to transfer the file from the droplet by using SFTP into the host machine as client side.
10. Open ZDEW, then 'Add Identity, choose http.client.jwt.
11. Automatically show up in the interface of ZDEW.
12. ziti edge list identities "limit none" | grep (id router)
//ip bla2 is the router name, the private-web-servers is attribute
extra step: ziti edge update identity (id router) -a "private-web-servers"
//pws=private web server
//This config is used instruct the server-side tunneler how to offload the traffic from the overlay, back to the underlay.
13. ziti edge create config pws.intercept.v1 intercept.v1 '{"protocols":["tcp"], "addresses":["web.ziti"], "portRanges":[{"low":80, "high":80}]}'
//Create a host.v1 config. This config is used instruct the server-side tunneler how to offload the traffic from the overlay, back to the underlay.
14. ziti edge create config pws.host.v1 host.v1 '{"protocol":"tcp", "address":"(private IP web server)", "port":80}'
//step 15 if want to update the addresses can use ziti edge update
15. ziti edge create service private.web --configs pws.intercept.v1,pws.host.v1
//private web dial =pwd
//Create a service-policy to authorize "Private Web Clients" to "dial" the service representing the HTTP server.
16. ziti edge create service-policy pwd Dial --service-roles "@private.web" --identity-roles '#web-clients'
//private web bind = pwb
//Create a service-policy to authorize the "Private Web Server" to "bind" the service representing the HTTP server.
//the private-web-servers is the attribute for the router
17. ziti edge create service-policy pwb Bind --service-roles '@private.web' --identity-roles "#private-web-servers"
Then i try to curl the web.ziti.
curl: (7) Failed to connect to web.ziti port 80: Connection refused
I already add the inbound rules of HTTP for the droplet. Hopefully, I can find what I miss in this mini-experiment. Thank you for the help. It helps me to understand each of the commands.
Yes that’s looking better. What do the logs in the local tunneler and the remote router show? Any hints in there? I’m expecting one or the other to have a helpful clue.
You’re sure that on the edge router, it can curl to (private IP web server):80, right? Can you verify that ziti-edge-router can do that? Let’s look at the last 40 ish lines of logs for both sides and verify the curl works.
If you don’t want to publish your logs, you can send them via email to clint at openziti.org
If you're running using systemd, that's totally normal. If you followed the quickstart, you'll have run a systemctl enable command (using systemd). In that case you need to look at the logs using journalctl: journalctl -u ziti-router "should" be the command you want. Can you check those? From your logs you emailed me (thank you) it looks like your controller is totally offline -- that'd cause problems too.
Your image is great. It's very possible to route like that from docker, sure. I would expect that it would work fine. You can "exec" into the docker container using something like docker exec -it ubuntu_ziti-controller_1 bash (replace ubuntu_ziti-controller_1 with whatever the name is of your docker container).
Once inside the container, you should be able to use a curl command to probe that endpoint.
[2023-01-11T05:20:55.210Z] WARN ziti-sdk:ziti.c:1458 api_session_cb() ztx[0] failed to get api session from ctrl[https://pssnet.ddns.net:1280] api_session_state[1] CONTROLLER_UNAVAILABLE[-15] software caused connection abort
[2023-01-11T05:21:00.210Z] INFO ziti-sdk:ziti.c:866 ziti_re_auth_with_cb() ztx[0] starting to re-auth with ctlr[https://pssnet.ddns.net:1280] api_session_status[0] api_session_expired[TRUE]
[2023-01-11T05:21:00.456Z] ERROR ziti-sdk:ziti_ctrl.c:154 ctrl_resp_cb() ctrl[pssnet.ddns.net] request failed: -4079(software caused connection abort)
[2023-01-11T05:21:00.456Z] WARN ziti-sdk:ziti.c:1458 api_session_cb() ztx[0] failed to get api session from ctrl[https://pssnet.ddns.net:1280] api_session_state[1] CONTROLLER_UNAVAILABLE[-15] software caused connection abort
[2023-01-11T05:21:05.314Z] INFO ziti-edge-tunnel:ziti-edge-tunnel.c:637 on_cmd() received cmd <{"Data":{"DumpPath":"C:\\Program Files (x86)\\NetFoundry, Inc\\Ziti Desktop Edge\\logs\\service"},"Command":"ZitiDump"}
>
[2023-01-11T05:21:05.314Z] INFO tunnel-cbs:ziti_tunnel_ctrl.c:274 process_cmd() ziti dump started
[2023-01-11T05:21:05.316Z] INFO tunnel-cbs:ziti_tunnel_ctrl.c:325 process_cmd() ziti dump finished
[2023-01-11T05:21:05.316Z] INFO ziti-edge-tunnel:ziti-edge-tunnel.c:223 on_command_resp() resp[1,len=25] = {"Success":true,"Code":0}
[2023-01-11T05:21:05.464Z] INFO ziti-sdk:ziti.c:866 ziti_re_auth_with_cb() ztx[0] starting to re-auth with ctlr[https://pssnet.ddns.net:1280] api_session_status[0] api_session_expired[TRUE]
This is quite interesting to me. Of course, the first thought when doing something like this “It’s going to work as I’m following it, just change the environment and use a different setup”. Yet, it didn’t work, and I learned many things while trying to configure how it went wrong or which part I forgot. If everything is work, not very thrilling in the research part, right? I am just taking my sweet time learning as much as possible about zero trust and experimenting few things like this before finishing my internship soon.
At this point I think I’m going to produce a video that shows how to make this work. I’ll probably use aws, not digital ocean, but it should not matter (other than the obvious differences in cloud providers). Then, you’ll be able to watch that step by step and we’ll see if that helps.
You can you check the controller logs too and see if there are any hints in there.
Ran the quickstart on one VM and verified it works as expected
Ran the docker container crccheck/hello-world and exposed it on port 80 (this is the web server that shows the docker whale) docker run --rm --name web-test -p 80:8000 crccheck/hello-world
Enable “private.web” access
obtain a docker env file and the simplified compose file in whatever folder you like:
update the .env file to look similar to this. Hopefully obvious, replace advertised.address with your externally accessible address. like ‘ec2-3-134-108-218.us-east-2.compute.amazonaws.com’ for example etc
# OpenZiti Variables
ZITI_IMAGE=openziti/quickstart
ZITI_VERSION=latest
# OpenZiti Variables
ZITI_IMAGE=openziti/quickstart
ZITI_VERSION=latest
# The duration of the enrollment period (in minutes), default if not set
# shown - 7days
ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=10080
ZITI_EDGE_ROUTER_ENROLLMENT_DURATION=10080
# controller address/port information
ZITI_CONTROLLER_RAWNAME=ziti-controller
ZITI_CONTROLLER_HOSTNAME=ec2-3-134-108-218.us-east-2.compute.amazonaws.com
ZITI_CTRL_PORT=8440
ZITI_EDGE_CONTROLLER_RAWNAME=ziti-edge-controller
ZITI_EDGE_CONTROLLER_HOSTNAME=ec2-3-134-108-218.us-east-2.compute.amazonaws.com
ZITI_EDGE_CONTROLLER_PORT=8441
#ZITI_EDGE_CONTROLLER_IP_OVERRIDE=172.17.0.1
# router address/port information
ZITI_EDGE_ROUTER_RAWNAME=ec2-3-134-108-218.us-east-2.compute.amazonaws.com
ZITI_EDGE_ROUTER_PORT=8442
#ZITI_EDGE_ROUTER_IP_OVERRIDE=172.17.0.1
start docker - IMPORTANT: notice I used -p openziti to start docker. That produces a predictable name used in step 5 below. Make sure you either understand what the -p flag is doing here, or just run it exactly as shown!
# this removes EVERYTHING:
docker-compose -p openziti down -v
# start docker here
docker-compose -p openziti up
optional: If needed, get the latest ziti binaries and have them added to the path for you if needed
setup a file to source to make it easy to login to the environment. This pulls information from the container and makes a “docker.env” file that you can source and use to ziti login
authorize the router identity to bind the service. Notice i used the name of MY edge router here (ec2-3-134-108-218.us-east-2.compute.amazonaws.com) - you have to replace with yours:
ziti edge update identity ec2-3-134-108-218.us-east-2.compute.amazonaws.com -a "${service}.binders"
Thank you for following up and providing your insight. Really appreciate your willingness to share your own experience and knowledge. Gotta try it later and can’t wait to see the outcome of my own through this setup.
so in this setup how would you access zac web ui?
I was able to setup the docker-compose like the video, from my home server behind my pfsense firewall.
I do not see where port 1280 is setup.
would I just need to add this in to the .env file?
Hi @willb0t, welcome to OpenZiti and to the community!
With this setup as shown, you should have pets 1408 and 8443 opened by ZAC. Zac is running inside that docker environment so you would need to allow ZAC through whatever web ACL you're using (security group if aws for example) then you'd go to https://ec2.whatever:8443/
For an example where one changes/customizes the install, I usually use ports 8440-8443 because that is what we use in the 'host it anywhere' example. So if you followed these instructions to the letter, you should have your controller running on port 8441. You can change ZITI_EDGE_CONTROLLER_PORT=8441 if you really wanna. You can refer back a few posts to the actual inputs I used to see what ports are set, and what env var to use.
Thanks! Oh No! Glad they helped! On the middle issue though, first-time user feedback is really important. If you would be so kind as so send me a DM here on discourse with your candid feedback as to what was confusing/hard/unclear, that'd be very helpful overall to understand what tripped you up. Believe it or not, I like hearing that sort of 'constructive criticism' to help us decide where to invest time to make things better!
Talk to you soon, cheers
I am just taking my sweet time learning as much as possible about zero trust and experimenting few things like this before finishing my internship soon.
