Zrok self hosted instance

Hi there,

I am trying to setup a self hosted Zrok intance following this video guide: YouTube
I have successfully started and tested the non TLS variant in a couple of minutes. I have added my custom DNS and made a successful test on a remote machine by sharing the web server on port 80. Then i tried to setup a TLS variant using Caddy. Since i do not want to use plugin for Caddy since i do not have DNS_PLUGIN_TOKEN i use the caddy self generated certificate feature. I start the containers ok. I can even do zrok enable on a remote machine using https://zrok.my.dns.com. I did the zrok reserve public http://localhost:80 with correct reply: https://uvqc73ewh1sh.my.dns.com:443. The problem is when i do zrok share reserved --headless token i get this error: "[ERROR]: unable to create 'proxy' backend (error listening: failed to listen: no apiSession, authentication attempt failed: Post "https://ziti.my.dns.com:80/edge/client/v1/authenticate?method=cert": http: server gave HTTP response to HTTPS client)" i know that https and then port 80 is not valid. This is my .env: "# Required settings
ZROK_DNS_ZONE=my.dns.com
ZROK_USER_EMAIL=emial@something.me
ZROK_USER_PWD=zrokuserpw
ZITI_PWD=zitiadminpw
ZROK_ADMIN_TOKEN=zroktoken

Expose services only on localhost (default)

ZROK_INSECURE_INTERFACE=0.0.0.0

Service ports

ZROK_CTRL_PORT=18080
ZROK_FRONTEND_PORT=8080
ZROK_OAUTH_PORT=8081
ZITI_CTRL_ADVERTISED_PORT=80
ZITI_ROUTER_PORT=3022

CADDY_INTERFACE=0.0.0.0
CADDY_HTTPS_PORT=443

:globe_showing_europe_africa: ACME provider (Let's Encrypt default)

CADDY_ACME_API=https://acme-v02.api.letsencrypt.org/directory
" If i chnage the ZITI_CTRL_ADVERTISED_PORT to 443 containers wont start. Please i need some help with this since it seems it is some wrong config but i can not find it. I have used the compose.yaml from the link and also some custom CaddyFile which looks like this: {
email {$ZROK_USER_EMAIL}
acme_ca {$CADDY_ACME_API}
admin {$CADDY_INTERFACE}:2019
}

Ziti Admin Console

ziti.{$ZROK_DNS_ZONE} {
log {
output stdout
format console
level INFO
}

reverse_proxy ziti-quickstart:1280 {
   transport http {
        tls_insecure_skip_verify
    }
}

}

OAuth Frontend

oauth.{$ZROK_DNS_ZONE} {
log {
output stdout
format console
level INFO
}

reverse_proxy zrok-frontend:{$ZROK_OAUTH_PORT}

}

Zrok Controller

zrok.{$ZROK_DNS_ZONE} {
log {
output stdout
format console
level INFO
}

reverse_proxy zrok-controller:{$ZROK_CTRL_PORT}

}

Default frontend (used for tunneling or frontend UI)

frontend.{$ZROK_DNS_ZONE} {
log {
output stdout
format console
level INFO
}

reverse_proxy zrok-frontend:{$ZROK_FRONTEND_PORT} {
    header_up Host {http.request.host}
}

}

and compose.override.yml for Caddy: services:
caddy:
image: caddy:latest
restart: unless-stopped
environment:
ZROK_USER_EMAIL: ${ZROK_USER_EMAIL}
CADDY_ACME_API: ${CADDY_ACME_API:-https://acme-v02.api.letsencrypt.org/directory}
ZROK_DNS_ZONE: ${ZROK_DNS_ZONE}
ZROK_CTRL_PORT: ${ZROK_CTRL_PORT:-18080}
ZROK_FRONTEND_PORT: ${ZROK_FRONTEND_PORT:-8080}
ZROK_OAUTH_PORT: ${ZROK_OAUTH_PORT:-8081}
ZITI_CTRL_ADVERTISED_PORT: ${ZITI_CTRL_ADVERTISED_PORT}
CADDY_INTERFACE: ${CADDY_INTERFACE}
expose:
- ${CADDY_HTTPS_PORT:-443}/tcp
- ${CADDY_HTTPS_PORT:-443}/udp
- 2019/tcp
ports:
- ${CADDY_INTERFACE:-0.0.0.0}:${CADDY_HTTPS_PORT:-443}:${CADDY_HTTPS_PORT:-443}
- ${CADDY_INTERFACE:-0.0.0.0}:80:80
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
- caddy_config:/config
networks:
zrok-instance:

zrok-frontend:
environment:
ZROK_FRONTEND_SCHEME: https
ZROK_FRONTEND_PORT: ${CADDY_HTTPS_PORT:-443}

volumes:
caddy_data:
caddy_config:
Thanks a lot in advance!

hI @Tino, welcome to the comunity and to zrok (and OpenZiti)!

I see below you set ZITI_CTRL_ADVERTISED_PORT=80. Is this the actual port your OpenZiti controller is running on? It looks like it to me since that's where your authenticate call is going. Did you try to put the controller behind caddy? You can't do that if that's what you did?

It's hard to know exactly where it went wrong, but that's my best guess right now.

Hi @TheLumberjack Thank you for your reply! Let me check my config and will get back to you. My idea was: since it was pretty easy to start it without Caddy (no TLS) i got stuck to TLS version. I dont have valid DNS token nor certificate so wanted to use it with Caddy's own generated certificate (without using cloudflare plugin nor any other). If you have some guide for this would appreciate it. In fact i want to setup self host on my office ubuntu pc. Thanks again

Personally, I use cert-bot and docker and obtain a wildcard cert for my own zrok that way. Once you have a wildcard cert, you can use it for the OpenZiti overlay and also for zrok (or anything). I find that to be the simplest way, for me. There's no 'right' way, but if you want to see how I have done it in the past you can have a peek at this openziti-scripts/letsencrypt/get_cert.sh at main · dovholuknf/openziti-scripts · GitHub

I see that you're following the zrok self-hosting guide for Docker: Self-hosting guide for Docker | zrok

You must obtain trusted certs, e.g., from LetsEncrypt. Otherwise, you won't be able to enable your zrok account or create any shares, because zrok enforces certificate trust when communicating with the zrok controller.

A wildcard certificate obtained via the DNS-01 solver is recommended because it vastly simplifies satisfying this requirement. While it is technically possible obtain trusted certs for each domain name, including all zrok public shares you will create, without a wildcard certificate, which requires a DNS-01 solver and DNS provider token in Caddy, it is substantially more difficult and fragile.


You may choose any TCP ports you wish for your ziti and zrok ports. However, changing the ports will break the zrok environments you already enabled with your zrok account token. Though 80/TCP is not the default port for HTTPS, it is perfectly valid. :slightly_smiling_face: This is the port used in the zrok self-hosting guide for Docker for the ziti controller because it is usually allowed egress from networks where you might wish to create zrok shares.