Using https to access services over OpenZiti?

Support General Questions
1

(Apologies beforehand if I’m misusing some terminology)

I have deployed OpenZiti following these tutorial series, and I successfully tested it with a Minecraft Server (just like the examples)

This made me thought that it would be greatly useful for self hosting certain services, but then I remembered that some of them require HTTPS, with a valid certificate and so on, or otherwise they won’t work (for example, Vaultwarden for self hosting a Bitwarden instance).

My guess is that just by using OpenZiti this won’t work, just like with a regular vpn, so do I have to deploy a reverse proxy in front of my services, then add the device and ports 80/443 to my OpenZiti network, and then it will work using my.service.ziti url? Are there any tutorials on this?

Thanks in advance, this is truly a huge project and looks super promising!

1 Like
2

Hi @jruiz94, welcome to OpenZiti. Glad to hear you got your Minecraft server up and running! :slight_smile:

YES!! You can absolutely use OpenZiti to use a “valid certificate” (side note, that’s one heckova loaded question around what makes a certificate “valid”) and I think that would make for a great getting started guide. I even filed an issue for us to document that process just now add "how to HTTPS" getting started guide · Issue #251 · openziti/ziti-doc · GitHub

There are a few steps to using https. I’m going to provide you the short version here and hope that’s all you need, but if not, just ping back and I’ll see how much help I can provide. The flow would go like this:

Using HTTPS with OpenZiti

  • Certificate Related
    • register a domain name somewhere. It doesn’t matter where. It just matters that the domain name is able to be queried by a certificate provider like LetsEncrypt… Cloudflare, GoDaddy, namecheap, whatever. Get a domain name. For our purposes we’ll call it jruiz94.xyz (yah, that’s a legit top level domain)
    • Ok, now use LetsEncrypt to get yourself a cert using the DNS challenge type Horray, now you have a server certificate that’s valid for jruiz94.xyz. That’s the hardest bits…
  • OpenZiti Related
    • Make a new intercept.v1 config and use jruiz94.xyz as the intercept name and any port you want
    • Make a new host.v1 config on the far side on whatever server you want and offload to localhost:port or servername:port (however you chose to do it)
    • Make a service using the intercept and host configs you just made
    • authorize the hosting side and dialing side using service-policies

And that’s all there is to it. The magic here is using the DNS challenge instead of the HTTPS challenge to get that “valid” certificate.

Now on your client ends, when you authorize the identity for that service they’ll be able to resolve jruiz94.xyz just fine. That will translate into a local 100.64.x.x IP address, it’ll get intercepted and tunneled just fine. On the far side, OpenZiti will make a TCP connection to your server and the TLS exchange will be from the client to the server and it’ll JUST WORK!!! :slight_smile: (I mean, assuing you did everything right and all that, of course) hehehe.

Hope that gives you enough information to get you going. I think it does, but like I said if not, just ping back.

-Clint

1 Like
3

@jruiz94 in addition to the info provided by @TheLumberjack, you might take a peek at an article I recently published concerning How to Acquire Wildcard TLS Certificates from a Public CA. This article is part of a series related to our BrowZer initiative, which you might also find interesting.

4

Wow that was a fast response, thanks to both!! I’m going to check out all the things you mentioned when I get some time, and I’ll ping back if any questions arise :slight_smile:

5

Hello again! I managed to do a bunch of stuff but I didn’t quite make it. I did the following:

1- First, since I had a domain (for the sake of the example, let’s say that I really own jruiz94.xyz), I added an A record for wildcard DNS to point to the machine where I have OpenZiti (lets assume this machine’s IP is 1.2.3.4), like so: *.test.jruiz94.xyz A record points to 1.2.3.4

2- Then I followed this tutorial to get my cert using the DNS challenge type and I got it, to be precise, i got 2 pem files and they got stored on the server at 1.2.3.4, in some /etc/letsencrypt subfolder.

3- As I needed some kind of server as test field, I started an echo http server on my local machine (lets call my local machine ‘Desktop’). I changed the default ports so it listens http on port 57575 and https on 57577. Of course when connecting to https://localhost:57577 it complained that the cert was from unknown issuer, as expected.

4- Then on my OpenZiti server, I created 2 identities, one for my Desktop and another for a client (let’s call the client ‘Laptop’). Following the same steps as the Minecraft tutorial, I created hosts/intercepts/services etc so that:

  • http://echo.test.jruiz94.xyz got redirected to the desktop’s localhost and port 57575
  • https://echo.test.jruiz94.xyz got redirected to the desktop’s localhost and port 57577

5- I enrolled the identities on the desktop and the laptop

6- And from the laptop I connected to the desktop and, to my very surprise, worked both on http and https, and the redirections from port 80 to 57575 and from port 443 to 57577 worked like a charm at the first try! However, the https complained again that the certificate was from unknown issuer.

From what I can understand, I think I’m missing the step where either

  • OpenZiti picks up the LetsEncrypt cert
  • The echo http server picks up the LetsEncrypt cert

I’m unsure on which one of the two answers is the correct one. Also perhaps the issue is that I chose an echo http server that isn’t that good for this example (since it’s dockerized and may have its own cert settings? Im not fully sure)

My knowledge on how to use certs is pretty limited, can you point me on the right track?

Thanks a ton for your help, this tool is truly amazing, I still can’t believe I made the http redirection work on the first try hahah

1 Like
6

Hey @jruiz94 and congrats on the progress you’ve made so far! One thing you may be able to look into is Caddy which is supposed to provide valid SSL certs through letsencrypt by default: Automatic HTTPS — Caddy Documentation

If you click the lock on the browser to show cert info, is the webserver properly serving the cert that was issued through letsencrypt rather than like a self-signed? I’m also learning OpenZiti and working to get certs working :slight_smile:

7

Great job so far. Everything you described sure seems right.

When this happens, you should look to see what url is in the browser window. Lots of times a server will redirect you from one place that was correct, to a different place that was not. Like, that server, if you went to https://echo.test.jruiz94.xyz it might redirect you to https://localhost/ (because it didn't end with that slash: https://echo.test.jruiz94.xyz/) something silly like that...

If you could hit the url again, see what is shown in the browser url bar. If you get the https error (like i do here) click on the not secure (if you're in chrome) click on "Not secure" then click details...

image

click on "Certificate Subject Alternative Name" and make sure the cert you expect is shown. You can see here I have an entry for https://ec2-3-22-101-126.us-east-2.compute.amazonaws.com:8443/ shown, but i still get the error because my cert is NOT from LetsEncrypt, it's my own cert

image
image566×705 27.1 KB

Can you verify that you see the proper wildcard cert? For example https://netfoundry.io looks like:
image

If it worked one time, another option is to try using curl (assuming you're comfortable with command line tools) or openssl. I can show you how to test using openssl too:

openssl s_client -connect echo.test.jruiz94.xyz:443

should return a code of "Verify return code: 0 (ok)" like it does when trying against nfconsole.io:

openssl s_client -connect nfconsole.io:443

hope that helps - you're so close :slight_smile:

8

@ccravens I haven’t used Caddy before, but AFAIK that’s a reverse proxy right? I made some experiments with Traefik, Cloudflare and wildcard domain certs before and they worked, but it’s Traefik who did the magic (with the correct Cloudflare settings added) of getting the correct cert. Does this mean that I have to add a reverse proxy before the actual echo https endpoint?

@TheLumberjack The certificate that my browser gets is for the website my.example.com and also there’s an entry for example.com. I’m not sure from where is this being obtained, but I suspect it might be the echo http docker container, since there’s an option for setting a custom volume to change the certs used by the container, so maybe by default uses some other certs that it might have inside.

Perhaps the culprit here is the docker container i chose as a test? Is there another way to open some http and https endpoints in my localhost to test this? :thinking:

BTW I tried the openssl commands you mentioned but I got an error trying to get echo.test.jruiz94.xyz:443, but the browser request from my Firefox works just fine (except for the connection not secure popup)

Thanks again for all the help you’re giving me :people_hugging:

9

Well, that would definitely cause problems. The certificate returned 100% must match the DNS name you are using. So, if you are going to https://echo.test.jruiz94.xyz and your cert is being served as my.example.com. To me, that would indicate that the docker container web server is not configured properly. It sounds like it's not serving up your certificate properly.

You don't need to check that with ziti - you can just test that locally with "https://localhost:57577" and make sure that endpoint returns the cert for echo.test.jruiz94.xyz.

You could go "crazy" if you want, and update your hosts file and add an entry in there for echo.test.jruiz94.xyz. Then (again without ziti), you should be able to successfully access https://echo.test.jruiz94.xyz:57577... If you did that, and get THAT working (without ziti), you could then remove the hosts file entry and I expect you would be able to access that service over ziti like you planned without fail.

So - my advice - test it locally first and get that docker container serving up echo.test.jruiz94.xyz properly, modify your hosts file, and successfully (without scary error), access https://echo.test.jruiz94.xyz:57577... Once you do that - you'll be golden, I'm sure :slight_smile:

10

As I suspected, the issue was that the Docker container was using other certs. So I copied fullchain.pem and privkey.pem to my ‘Desktop’ machine where the Docker container was, and then i mapped some volumes so the D:\testdir\fullchain.pem was mapped to /app/fullchain.pem and D:\testdir\privkey.pem was mapped to /app/privkey.pem.

And then it worked like a charm, now from my ‘Laptop’ machine I navigate to echo.test.jruiz94.xyz and I get that the connection is secure with no warning whatsoever.

So if I have understood everything, once the certificate is generated, you need to bring that certificate to the “side” where the applications are actually running so when the request comes from OpenZiti tunnels it gets validated and the HTTPS works, right?

11

once the certificate is generated, you need to bring that certificate to the “side” where the applications are actually running

There's a bunch of doc online that would be more thorough than what I'm about to write, I'd suggest researching that if you are interested in more details. I won't be able to write enough words :slight_smile:

Buuuuuut... That said, it usually works like this (this not mTLS https):

  • "client" reaches out to "server" and says to server, "I want to use a secure protocol please"
  • "server" says, "hi client, here is my certificate. I claim to be echo.test.jruiz94.xyz and I was made by LetsEncrypt"
  • "client" goes, "echo.test.jruiz94.xyz eh? Ok, that checks out, the user typed in echo.test.jruiz94.xyz and that's the same place... Cool coo coo coo...." first check passed
  • "client" then says to self, "Ok, so this server says it's echo.test.jruiz94.xyz and that it's signed (made) by LetsEncrypt. Let me just check that.... Yep, that checks out, the Issuer of this certificate was indeed LetsEncrypt and I have the LetsEncrypt CA in my local operating system... Everything checks out..."

And at that point the client (in this case your browser) says, "everything is A-OK" and you don't get a scary warning that it's not 'valid'.

So -- yes. The certificate needs to be put on the 'server' side in this example. I dunno if that helps or not but I hope so

12

Oh and congrats on getting HTTPS working over ziti! You got up and running very quickly! :slight_smile: Tell all your friends and don’t forget to tell them to star the repo GitHub - openziti/ziti: The parent project for OpenZiti. Here you will find the executables for a fully zero trust, application embedded, programmable network repo. The star helps us get the word out to others! Nothing says “cool open source project” like GitHub stars, amirite? You can also go to the top of the page and click this thing too if you prefer :smiley:
image

Let us know what other interesting questions or successes you have! Happy Ziti-ing!

13

Yes I almost had that same "workflow" in my head, but since OpenZiti did such an incredible amount of magic for me (you can't figure how happy I was to see that my-server.minecraft.ziti resolved at the first try when I did the Minecraft tutorial, I tried Wireguard before but inputting the IPs is a bit ugly, and the DNS part of it drove me nuts), I thought that perhaps this was also handled by OpenZiti automatically too. But it makes more sense that it works like this, tbh.

Yeah that was the part that I was missing :slight_smile:

Thanks!! And don't worry, I starred the project long before posting here :stuck_out_tongue: As I said many times before, this is such a great project, so it deserves it!

Also going through the process of adding http and https helped me a lot understand better the services part, the bind/dial, etc. So it has been great! Now I have to re-think my whole self hosted infrastructure of services, cause it's going to have OpenZiti for sure!

Thanks again :slight_smile:

1 Like