So, I heard about openZiti on a podcast. Came and started following the Quickstart-docker-compose. Got to the end of it, and then felt like I was thrown off a cliff and left to figure out the rest with no direction.
What I would like to see, is a follow on article(s) where a client is connected (to the web service as part of the quickstart). I understand this breaks out of the docker controlled environment and there is a zillion permutations on how this could connect.
Specifically what I am after
Creation of an account in the console
Creation of a device in the console
Creation of the āpipeworkā to join users to services
what is required from the outside of the docker environment to make it work, ie host file entries
basic walk through on the client and connecting from a client
At the moment I feel I end up with the infrastructure installed, but no idea how to connect a client. Might be on TV (no link to it), but I like guides when I can follow it through and not listen.
If you want to stay with pure OpenZiti constructs, we will follow up with tips and suggestions.
If you want to take a step back, and run through āhello worldā type experiences to get familiar with services and connections, then you can sign-up for a free NetFoundry account (choose āTeamsāā¦it is free forever for up to 10 nodes), and use one of the quickstarts listed here, e.g. you could choose the Oracle Cloud one, the AWS one, etcā¦depending on your preferences.
First, welcome! Thanks so much for your feedback. I can relate to the feeling that you're experiencing. It's certainly not what we want.
I just reviewed each of the quickstarts and yeah you're right. There's no real great "let's test it out" guide that really brings you through it step by step.
I'm going to write one of these up and publish it. What quickstart did you end up following? I will focus on making an interesting and relevant one for that quickstart first.
Thanks so much for taking the time to provide this thoughtful feedback, cheers
Correct. Docker-compose. Chose it because it had a web server in the fabric already, which I was hoping would follow it through. In my case, I am looking at the tunnel type of implementation (publish that web server to the fabric and hook up a client), due to not being a developer to embed to an application. So, using a āgenericā web server as per that docker-compose mimics what I am thinking I would go through.
If in the howto, you can reference command lines (ah the power), and also the same commands through the ZAC (for those point and click folks) that would be mint.
Yeah, Iāll work something up very soon. Iām the next couple days. I could write it up here but I want to actually publish it on the doc pages. Iāll follow up here when I have something for you to try out! Thanks again, will follow up soon, cheers
If youāre feeling up for it and would like to try it out, feedback from users like you is incredibly valuable. We would very much appreciate your input on the doc to see if itās āhelpful enoughā.
As for a couple of good YouTube video, Iād think the āTotally Private Postgresā video I published not long back would be better. There was other good discussion on that topic over on the related discourse here: Totally Private Postgres. Most notably are the diagrams. That video also has all the CLI commands in the video description I ran to setup the network to expose a postgres server as an example.
Another good video would be the one where I demonstrated wildcard DNS. It also has all the CLI commands used in the video description.
Weāre getting lots of videos on the site now and finding relevant ones is a bit tough now. Youāve inspired me to make a new title slide and a new playlist for ātutorialsā or āhow-tosā.
Thanks again for the feedback and taking the time to share your experience
Thanks for the quick response. I have attempted to go through the steps, but then fell over at the client side configuration, step 9. Does not matter if I copy the contents out of the file into a jwt file, copy the file out of the docker container and SCP it too the client, or download in a browser and do it, I end up with:
root@beatbuilt:~# ./ziti-edge-tunnel enroll --jwt /tmp/http.client.jwt --identity /tmp/http.client.jwt
[ 0.000] INFO ziti-sdk:ziti_enroll.c:94 ziti_enroll() Ziti C SDK version 0.27.7 @8de0a26(HEAD) starting enrollment at (2022-05-30T11:15:36.398)
[ 0.000] ERROR ziti-sdk:jwt.c:31 parse_jwt_content() jwt input lacks a dot
[ 0.000] ERROR ziti-sdk:ziti_enroll.c:131 ziti_enroll() /home/runner/work/ziti-tunnel-sdk-c/ziti-tunnel-sdk-c/build/_deps/ziti-sdk-c-src/library/ziti_enroll.c:110 - load_jwt(opts->jwt, ecfg, &ecfg->zejh, &ecfg->zej) => -4 (JWT has invalid format)
[ 0.000] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:1148 enroll_cb() enrollment failed: enroll failed(-4)
The jwt token I am getting is this, but dont know if this looks correct or not: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbSI6Im90dCIsImV4cCI6MTY1MzkwOTI3NCwiaXNzIjoiaHR0cHM6Ly96aXRpLWVkZ2UtY29udHJvbGxlcjoxMjgwIiwianRpIjoiZmJlYWFkNGUtMTRkMS00MDA0LTk4YjgtN2M4NTBjYzI2NTNiIiwic3ViIjoiM0d1NXhVYjlaeiJ9.k2WKWok-GwVZla4Q1DMLnPNW3zzGivsiGc_KLbPx3kS7zR0T_zaoXwkE1zYb6Gl5vXLLG4uMvoj1sJ5yUAscYZPZ7wwsKjBBjLsSb_bHAXiW_36YjuZHRIh0eDOpC5dGkEawg7_66nXFWXP4pcdTXM_am9PncAFOB95biWMZp0VjvvvzL9Ri_MHBXWxWlxGQ--Uc52IvHM2N8Ymh_6FgldY6wFeYtQmI9s2gsT6inrq1wqncL8d3ExOoa-R4gDinEWRy0ckSMXgDqtJqHjtgeLucOMu7PrS9CZ-KCzaX9qIL5y5RyRxpFmcwPKZPgfk-4AoetCC8f4yRq1r84o8h6dxjohRhskSSCiyV9IU0K3cmILfPsa0zy2Zj2IqC914wtqe0VMSIIGG7N_JQvyNd4peo0piVMtsX2iQuGUD1ejkILAjybWffQYuO22tQ767b3rkQq37O62OCzEj5PXxnmz2gwwXscMtHN9h5_5w1PHDTk4HYDuIxFMtD9AgkuY5fDkf14IusQdgiu0cSpEeSAOgxGsEMWbBwIxiguttktp1d1YZVBr61zYT14II4FuGmaXFpj5gij7vH8W-MqghSw3yA_ZMp4DW97sxeon9egawIYBFmmyufQaxqD00UVE4nBLO6Q1BQTA5751UoMHy4rMHFVTNDiwGviEKu87KX0BE
As for comments regarding the documentation, this is what I have so far. Take this with a grain of salt - ie decide if you want the changes or notā¦
In the diagram of the HTTP solution overview before Ziti - do you need to show the firewall client side? mostly this is implied. Also, maybe put 80/443 as port numbers going through the firewall.
After ziti picture - do we need two edge routers (client side and server side). Is the tunneller not just connecting to the server side edge router, so the client side router is redundant?
Configuring the overlay - overview. There are links for intercept.v1 and hosts.v1, which I would expect to chain off to a page telling me about what those options do, vs a page which doesnāt tell me anything about them. I would also want the other config options, like intercept.v2 and host.v2 detailed. How do I know what they are, what they do, and why one over the other?
Step one of the howto, is connecting to the ZAC controller. ZITI_EDGE_CONTROLLER and ZITI_PORT are not defined for me. It might be better to just say, connect to the controller - see the ZAC installation documention?
Maybe put a brief reason why we are creating user identities vs device identities?
Under HTTP Server Security - maybe detail that the ID that you want for docker-compose configuration is ziti-edge-router ID. You explain what you want to put there, but not the value. ie you have * Run ziti edge list identities to find the name of the identity associated to the router*. There are two edge routers in docker-compose, and either should work, so maybe state choose either edge-router?
Other notes
Tunnellers page (different document page) - Wonder if it is better for the download links based on platform to instead be a link to the platform anchor of the operating system section that then has a link to the download, vs what it is today. I click on Windows (for instance) expecting to jump to the windows section, and not the client download page.
On the client download, at least linux, you download the ziti-edge-tunnel app, but the docs refer to ziti-tunnel (old name?) commands
as an aside - and just writing here as in the flow,but should be separate tickets:
If the jwt has expired (ie not available as download in console) there is no way to regenerate it without deleting the user and starting again (AFAIK). Can I suggest that this be an enhancement?
Windows client - when will the transport mode be available. This will be where I would be using the client more than linux and cursory reading is using proxy mode which would be problematic for large scale deployment?
Any thoughts about making access to the ZAC have 2FA capabilities. Last thing you would want for the zero trust fabric being compromised by some administrator chose a weak password!
Excellent feedback. Thank you. fwiw, the JWT looks okay to me. It loaded fine in https://jwt.io, and I was able to load it into macOS ZDE without issue (though itās of course expired at this point)
Honestly - really appreciate the great, constructive feedback!
The jwt you posted looks fine for sure. Is that what you get when you cat /tmp/http.client.jwt? You usually only see the "lacks a dot" error when your jwt is munged and this one you posted looks fine. Makes me think maybe the file is a different name or lacks permissions (like you copied via root and didn't use sudo to read the file?) What that is saying is that the jwt didn't have a "." inside it (a period). But clearly what you posted had the correct number. There should be two periods in a valid jwt file - each period distinguishes a section: header, payload, signature. I'm thinking it was the root permissions thing?
do you need to show the firewall client side ... Also, maybe put 80/443 as port numbers going through the firewall
no not really - but I did it for emphasis. It's taken for granted that it's closed, I wanted to illustrate it's closed. The ports are "often" 80/443 but not always. It's a delicate balance to keep the diagram useful but not too busy. I'll see what happens with ports on there. I also forgot to call out the 'hole' specifically.
do we need two edge routers
Nope. Just did that to illustrate that OpenZiti allows you to form a mesh of routers. If I add one, someone will say "can't I have two or more?" Might be worth a footnote though. I didn't tailor this specifically for the quickstarts - I was hoping to get away with "one diagram to rule them all" but maybe it makes sense to have options. I'll mull that over.
There are links for intercept.v1 and hosts.v1, which I would expect to chain off to a page telling me about what those options do
Totally agree. It's on the todo list, just need to get it done. I wanted to get you 'something' to get you started. All the configs should be documented for sure.
ZITI_EDGE_CONTROLLER and ZITI_PORT are not defined for me. It might be better to just say, connect to the controller
This one is always tough to be honest. Perhaps I'll just use a dummy value like "your-controller-here:port". It just gets wordy when you try to explain the various permutations of accessing your controller - but I appreciate the comment.
Maybe put a brief reason why we are creating user identities vs device identities?
Or backlinks to the relevant doc - that would make sense. I like that idea.
maybe detail that the ID that you want for docker-compose
Oh yah. Since we're going after the web-test-blue that makes sense to use the private edge router in that network. I'll make that change for sure
Other notes - Tunneller page
YIKES this stuff is crazy out of date! That's going on the top of my "todo" list. I'll probably end up just stripping much/most of this doc and making this be a simple jumping point for the download of each.
aside - jwt expired - there is no way to regenerate it without deleting the user and starting again
Not yet but that's coming very, very soon. I believe that functionality was just enabled in the API, just need to get it into the UI/CLI. I'll check that
aside - Windows client - when will the transport mode be available
This doc is so out of date it made me physically sad today. It has done all this for probably a year... You can do it all with Windows right now. In fact the videos I referenced will show you that I personally use the Windows client daily. If you watch that "Totally Private Postgres" video, you'll see and hear me refer to the Windows client and WSL to provide my local machine access to the Postgres server running in AWS.
Any thoughts about making access to the ZAC have 2FA capabilities
Funny you mention this - another discourse user had a similar thought just the other day here Turning ZAC dark The difference was he was thinking about using Ziti itself to protect it. Technically what we'll want to add 2FA to is probably the API, but that would need to be surfaced via the UI too. If you did take the ZAC offline and used ziti to protect ZAC, you CAN add 2FA to Windows (MacOS/iOS coming very soon). I use that every day too. In fact here's what that looks like on my client when I need to enter the MFA digits:
#8. Start the server-side tunneller (unless using the docker-compose quickstart) with the HTTP server identity.
# [optional] if you don't use an edge-router as your tunneler, you will need to download and run the tunneler for your OS
# if you are using a ziti-router, skip to step 9 below
#
# This step is dependant on platform. For this demo we'll be using a virtual machine running linux and we'll be using the
# ziti-edge-tunnel binary. Copy the http.server.jwt from step 2 to the server machine. For the example we'll use /tmp/http.server.jwt
#
# enroll the server identity using ziti-edge-tunnel
I am using docker-compose example, so do I skip this step altogether - if I do, then the identity is not mapped to the service. Therefore, I assume that I need to do two things
a) copy the http.server.jwt to the edge router that I configured in step 7
b) jump into that container, and run `ziti-tunnel enroll path to file
Hopefully magic will just happen and I dont need to restart the edge router?
Problem I have is that the jwt expired (see another question) before I got to the server config, and as I cannot reset the jwt, I trash the container environment including docker volumes and start again as I think is easier and quicker.
Posting this, as wont get back to it until tomorrow and you will answer off the top of your head probably.
are you within the doccker environment? as in did you exec into it? the docker pages (should) both state that you will need to modify your hosts file if you wish to interact with ziti from outside the docker environment. I can try to make this "bolder" or more of a warning type of dialog. here's the link to the compose page but I see I need to add this to the "non docker compose page" too https://openziti.github.io/ziti/quickstarts/network/local-docker-compose.html#using-docker-locally
Actually, now when I read this back and seeing your comment, you could also skip step 2. You won't need the http.server.jwt either. I'll make these updates right now to that doc.
Yes - the magic will just happen. You won't need to restart the router at all unless your docker env is "older" (meaning like 2 weeks). There was a bug fixed around then to turn the "private" routers into "private edge routers". I think that was 0.25.9. Make sure you have at least that version.
A quick note. If you are not well-versed with Docker you might forget that exposing ports in Docker is one thing, but you'll also need to have a hosts entry for the containers you want to access from outside of of the Docker network. This quickstart will expect that you understand this and for every router you add you will want to make sure you add a host entry. In the examples above we are adding three entities: ziti-edge-controller, ziti-edge-router-1 and ziti-edge-router-2.
docker - with compose
A quick note. If you are not well-versed with Docker you might forget that exposing ports in Docker is one thing, but you'll also need to have a hosts entry for the containers you want to access from outside the Docker network. This quickstart will expect that you understand this and for every router you add you will want to make sure you add a host entry. In the docker-compose example you will want/need hosts entries for at least:
ziti-edge-controller,
ziti-edge-router
And if you want to expose any other routers - of course you'll need/want to have entries for those as well.
I added the note to the steps that you can skip step 2 if using the docker-compose:
Create an identity for the HTTP server if you are not using an edge-router with the tunneling option enabled (see below). Also note that if you are using the docker-compose quickstart or just plan to use an edge-router with tunneling enabled you can also skip this step.
With regard to step 1, I think there may be confusion still. Step one says to connect to the Ziti edge controller. The command is ziti loginā¦. That does not work for me, because I am not within the docker environment when I start this tutorial. My host environment knows nothing about the ziti command so I just get an error. So I think it would be nice to say that if using docker compose or docker example then exec into the controller container first.
I also just use zitiLogin instead of the command given there. The command you give is more universal so that ok. I havenāt tried that yet but will do.
You are correct about needing to expose the host names to the outside. Which I had done. Maybe put that under pre-requisites. Maybe also put about exec into docker environment as a prerequisite.
Last comment - since the lifetime of the jwt is 5 mins by default, we should call this out as well. The first time I went through the steps I was trying to understand what each line did. Since there is no way to refresh the token in the gui people will end with an expired token before they get to use it. If the ziti command line can recreate that would be worth commenting on.
To really make it easier ( and what length do you go ) you could include the docker command to copy the client jwt from the controller to the local host, or reference the gui to do this so it can be copied to client. Maybe even cat the file and copy. Not sure.
It still should work for you though if you have edited your hosts file properly. The problem is that ziti is not on your path. I can clarify this to be a prerequisite. You need the ziti cli on your path to use these commands.
I like this idea - letting people know the clock is ticking is helpful. i'll add some words around this.
One thing to note that I just realized - there is another bug in the docker-compose setup that doesn't establish a policy properly for what you need. I want to either document that for you or fix this bug. That's going to be the next issue you hit and I just want you to know it's gonna cause you a problem. I expect I'll fix/doc this in the next couple of hours...
Ok - we updated the docker containers to fix the next issue you were likely to hit around policiesā¦ Make sure you are running the latest 0.25.10 from docker hub by running docker-compose pull
Hello. I have done a docker-compose pull. However, when I run a ziti version I get this:
NAME VERSION
ziti v0.25.9
ziti-controller *********************************************************************************
An update with v0.25.10 is available for ziti-controller v0.25.9 from
https://github.com/openziti/ziti/releases/latest/
*********************************************************************************
v0.25.9
ziti-fabric not installed
ziti-prox-c not installed
ziti-router *********************************************************************************
An update with v0.25.10 is available for ziti-router v0.25.9 from
https://github.com/openziti/ziti/releases/latest/
*********************************************************************************
v0.25.9
ziti-tunnel *********************************************************************************
An update with v0.25.10 is available for ziti-tunnel v0.25.9 from
https://github.com/openziti/ziti/releases/latest/
*********************************************************************************
v0.25.9
ziti-edge-tunnel not installed
ziti@0bd5a37d1fb3:/openziti$
When I do a docker image I get this: openziti/quickstart latest 4fd93aed720c 10 hours ago 400MB
so it appears to have got the correct image.
So more document changes:
On the Ziti Admin installation page (Installing Ziti Administration Console | Ziti) under docker-compose commands, step 2, the two docker cp command appear like one big long line, when they are two commands. You might to fix that.
Also, the second command is wrong, and is the incorrect file. I think the command should be: docker cp docker_ziti-controller_1:/openziti/pki/ziti-edge-controller-intermediate/certs/ziti-edge-controller-server.chain.pem .
Note the change in path.
I am getting closer to the prize, but havenāt hit the home run yet
Next questionā¦
On the zero trust host access documentation, step 4, for the docker-compose script, should this be port 8000 and not port 80? When you run the connectivity test in the docker-compose installation guide, we test curl to port 8000.
I am about to start troubleshooting client connect issue, but from my windows client (network connected), I cannot connect to http.ziti, but I can ping it - or something that is responding to it (maybe the edge router) - it is responding with a 10.64.2.1 address.
I suspect this is a docker thang. I think your existing docker-compose environment is simply not getting upgraded by docker for a reason that's not quite clear to me. You could either issue a down -v and wipe everything, start a new compose environment up (both of those options don't sound great to me) or try to figure out how to shake docker loose and upgrade the container for you. I expect you have the right image now but somehow docker is being a bit of a stickler. I'm not exactly sure why yet. If I find out why I'll reply... If you do this just beware that it makes all your identities invalid as you'll be removing the full pki
0.25.10 is in the image ID you cited. it also has a new 'init' container - that's the thing you really need to update the two policies.
# Allow all identities to use any edge router with the "public" attribute
ziti edge create edge-router-policy all-endpoints-public-routers --edge-router-roles "#public" --identity-roles "#all"
# Allow all edge-routers to access all services
ziti edge create service-edge-router-policy all-routers-all-services --edge-router-roles "#all" --service-roles "#all"
Yikes yes! That's a typo indeed! Ugh I am sorry, thought I doublechecked all these commands. For example I just ran this for my step 4:
and cursory reading is using proxy mode which would be problematic for large scale deployment?
I'll see what happens with ports on there. I also forgot to call out the 'hole' specifically.
Might be worth a footnote though. I didn't tailor this specifically for the quickstarts - I was hoping to get away with "one diagram to rule them all" but maybe it makes sense to have options. I'll mull that over.
