Accessing remotely (Docker)

Support General Questions
1

Well, I've followed along with the Docker instructions and I've been able to establish a connection (no services yet).
However, When I create an identity, it contains this information:

{
  "iss": "https://ziti-edge-controller:1280",
  "sub": "Oyf2IVDKf",
  "aud": [
    ""
}

I edited my Windows hosts file and I was able to connect. Success. Now, I don't want to do that. I want to access it using my external hostname.

I've read this topic: Where do you specify the public dns name for the docker containers? - #9 by gberl002
And I ended up blasting everything and turning it up again with environment changes.
However, when I generated the identity it still contains the docker container name and not my external URL.

docker run \
  --name ziti-controller \
  -e ZITI_CTRL_ADVERTISED_ADDRESS=ziti-edge-controller \
  -e ZITI_ROUTER_ADVERTISED_HOST=me.org \
  -e ZITI_ROUTER_ADVERTISED_ADDRESS=me.org \
 ....

How do I make this work? Thanks!

1 Like
2

Oh boy! Am I to assume this is not as straight forward as I thought it was?

3

Hi @jagauthier, welcome to the community and to OpenZiti!

When running things in docker alone, routing can definitely become "messy" in a lot of ways. Unless you know how to configure you docker environment with some DNS server of your choice, editing the hosts file is often a fine way to start but not all that maintainable in the longer run, imo.

If you want to access it using your external hosts name, you just need to set a few more parameters as you've discovered. Let's clarify that you want to run docker (not compose) commands, right?

Let's also clarify that you're running docker from docker desktop in windows with a VM, and not running from WSL on its own? (that's how I run it and I don't use docker desktop myself). Can we clarify exactly what your setup looks like? It'll help me test/verify things with you better...

4

Absolutely. I've run everything with manual docker commands (docker run). I prefer this over compose, for reasons that I probably can't explain. I'm not running docker with Windows at all - it's 100% Linux. Windows is my target client, and then Android. So, the hosts file I've edited is on a remote Windows machine.

In my working (but not ideal scenario) I am running the controller like so:

docker run \
  --name ziti-controller \
  -e ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=ziti-edge-controller \
  --network plexnet \
  --network-alias ziti-controller \
  --network-alias ziti-edge-controller \
  -p 1280:1280 \
  -it \
  --rm \
  -v /data/containers/ziti:/persistent \
  openziti/quickstart \
  /var/openziti/scripts/run-controller.sh

And the edge router like so:

docker run \
  --name ziti-edge-router-1 \
  -e ZITI_ROUTER_NAME=ziti-edge-router-1 \
  -e ZITI_ROUTER_ADVERTISED_ADDRESS=ziti-edge-router-1 \
  -e ZITI_ROUTER_ROLES=public \
  --network plexnet \
  --network-alias ziti-edge-router-1 \
  -p 3022:3022 \
  -it \
  --rm \
  -v /data/containers/ziti:/persistent \
  openziti/quickstart \
  /var/openziti/scripts/run-router.sh edge

Following the instructions for Docker, I also initialize the controller (every time I delete the files) like this:

docker run \
  --network plexnet \
  --network-alias ziti-controller-init-container \
  --name ziti-controller_init \
  -it \
  --rm \
  -v /data/containers/ziti/:/persistent \
  openziti/quickstart \
  /var/openziti/scripts/run-with-ziti-cli.sh  /var/openziti/scripts/access-control.sh

I also have a container for ZAC, but that's not too releveant yet, so I will omit it.

If I create an identity, and add "ziti-edge-controller" to my remote Windows' host file I can connect with that identity.

So, my next step is to extend this to work externally without the host file addition.
From what I read, I thought it would be just something like:
ZITI_CTRL_ADVERTISED_ADDRESS=ziti.doimain.com

However, attempts to do that completely fail when it comes to certificate generation during the init phase.
I hope I've generated enough information that this makes sense.
I'm happy to elaborate. Thank you for your time.

5

Oh then this will make things easier for sure. :slight_smile: Let me find/create the relevant command so that it's usable from more than one machine...

bwahahaha -- that's fine. just wanted to level-set where we were at. thanks

1 Like
6

Alright here are the steps for a deployment that looks like this... I made two routers, one in the cloud and one at my house. Replace the variables as you want.

image
image1212×578 82.8 KB

start the controller

docker run \
  --name ziti-controller \
  -e ZITI_CTRL_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_ADVERTISED_PORT=8441 \
  -e ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_EDGE_ADVERTISED_PORT=8441 \
  -e ZITI_USER=admin \
  -e ZITI_PWD=myzitipasswordhere \
  -p 8441:8441 \
  -it \
  --rm \
  -v openziti-controller-files:/persistent \
  openziti/quickstart \
  /var/openziti/scripts/run-controller.sh

Add some default policies

docker run \
  -it \
  --rm \
  -e ZITI_CTRL_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_ADVERTISED_PORT=8441 \
  -e ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_EDGE_ADVERTISED_PORT=8441 \
  -e ZITI_USER=admin \
  -e ZITI_PWD=myzitipasswordhere \
  openziti/quickstart \
  /var/openziti/scripts/run-with-ziti-cli.sh  /var/openziti/scripts/access-control.sh

Start your first router

docker run \
  --name ziti-edge-router-1 \
  -e ZITI_CTRL_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_ADVERTISED_PORT=8441 \
  -e ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_EDGE_ADVERTISED_PORT=8441 \
  -e ZITI_USER=admin \
  -e ZITI_PWD=myzitipasswordhere \
  -e ZITI_ROUTER_NAME=ziti-edge-router-1 \
  -e ZITI_ROUTER_ADVERTISED_ADDRESS=ziti-edge-router-1 \
  -e ZITI_ROUTER_ROLES=public \
  -e ZITI_ROUTER_LISTENER_BIND_PORT=3022 \
  -p 3022:3022 \
  -it \
  --rm \
  -v openziti-router-1-files:/persistent \
  openziti/quickstart \
  /var/openziti/scripts/run-router.sh edge

Start another router

docker run \
  --name ziti-edge-router-2 \
  -e ZITI_CTRL_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_ADVERTISED_PORT=8441 \
  -e ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=ec2-3-134-108-218.us-east-2.compute.amazonaws.com \
  -e ZITI_CTRL_EDGE_ADVERTISED_PORT=8441 \
  -e ZITI_USER=admin \
  -e ZITI_PWD=myzitipasswordhere \
  -e ZITI_ROUTER_NAME=ziti-edge-router-2 \
  -e ZITI_ROUTER_ADVERTISED_ADDRESS=ec2-13-58-222-94.us-east-2.compute.amazonaws.com \
  -e ZITI_ROUTER_ROLES=public \
  -e ZITI_ROUTER_LISTENER_BIND_PORT=8477 \
  -p 8477:8477 \
  -it \
  --rm \
  -v openziti-router-2-files:/persistent \
  openziti/quickstart \
  /var/openziti/scripts/run-router.sh edge

two possibly interesting things to notice:

My "Router 1" is at my house and behind a firewall. My Router 2 cannot connect TO router 1 but router 1 CAN connect to router 2 so they can form a link. verified by running:

ziti fabric list links
╭────────────────────────┬────────────────────┬────────────────────┬─────────────┬─────────────┬─────────────┬───────────┬────────┬───────────╮
│ ID                     │ DIALER             │ ACCEPTOR           │ STATIC COST │ SRC LATENCY │ DST LATENCY │ STATE     │ STATUS │ FULL COST │
├────────────────────────┼────────────────────┼────────────────────┼─────────────┼─────────────┼─────────────┼───────────┼────────┼───────────┤
│ 5WRp7PgZwHcwy87uQN4iSD │ ziti-edge-router-1 │ ziti-edge-router-2 │           1 │      41.1ms │      46.0ms │ Connected │     up │        88 │
╰────────────────────────┴────────────────────┴────────────────────┴─────────────┴─────────────┴─────────────┴───────────┴────────┴───────────╯
results: 1-1 of 1

You should have two routers online then (obviously)

ziti edge list ers
╭────────────┬────────────────────┬────────┬───────────────┬──────┬────────────╮
│ ID         │ NAME               │ ONLINE │ ALLOW TRANSIT │ COST │ ATTRIBUTES │
├────────────┼────────────────────┼────────┼───────────────┼──────┼────────────┤
│ xRUdZnU59c │ ziti-edge-router-1 │ true   │ true          │    0 │ public     │
│ zCIdZnx5.c │ ziti-edge-router-2 │ true   │ true          │    0 │ public     │
╰────────────┴────────────────────┴────────┴───────────────┴──────┴────────────╯
results: 1-2 of 2

hope that helps

7

Thank you for the help. I mostly got this to work but had to make a couple changes based on my environment. Hopefully, you can help me understand those.

First one (no explanation needed) my home network is (obviously) behind a firewall/router. For some reason these containers (or docker in general) aren't respecting my hosts DNS. So, when it tries to resolve my domain this happened to be the external IP which creates all kinds of problems. I fixed this by adding --dns <IP> to the docker command line.

Secondly, once the edge router tried to run it attempted to connect to the controller on port 6262. I had to expose that port through the container because the communication is leaving the docker network.

What I don't quite gather is the port definitions. You've specified both ZITI_CTRL_ADVERTISED_PORT
and ZITI_CTRL_EDGE_ADVERTISED_PORT as the same 8441.

The docker compose file has these as 1280, and 6262, respectively. Are these the same service?

8

Did you set ZITI_CTRL_ADVERTISED_PORT=8441 ? I actually specifically used the same port (which is probably not obvious) because OpenZiti leverages something called ALPN and we support 1 port with 2 functions... I suspect you thought that was a typo and used a different port (or didn't set it/left it blank) which is why it used the default 6262.

What I don't quite gather is the port definitions. You've specified both ZITI_CTRL_ADVERTISED_PORT
and ZITI_CTRL_EDGE_ADVERTISED_PORT as the same 8441.

LOL -- i answered the question in the previous question. I generally answer them 'in-line' and moved on to your next point only to discover that is indeed the case! :slight_smile:

As far as ports go, 1280 is the default REST endpoint port and 6262 is the default "control plane" port. REST endpoint is used by ziti cli and SDK clients, the control plane is used by the overlay itself (router to controller comms).

As mentioned prior, ZITI_CTRL_ADVERTISED_PORT controls the control plane port, ZITI_CTRL_EDGE_ADVERTISED_PORT controls the rest port and they can both be set to the same value. (this is why i only had to expose a single port with the docker command -p 8441:8441)

I recommend you do that. It's easier only having one port open. :slight_smile:

hth

9

Perfect. Thank you for the explanation!

1 Like
10

@TheLumberjack
Today I added a new service. It seems that I'm running into the same type of complication.

As a quick refresher, I am now creating my router like so:

docker run --rm -it \
  --name ziti-edge-router-1 \
  -e ZITI_CTRL_ADVERTISED_ADDRESS=host.mydomain.com \
  -e ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=host.mydomain.com \
  -e ZITI_USER=admin \
  -e ZITI_PWD=234234234 \
  -e ZITI_CTRL_ADVERTISED_PORT=1280 \
  -e ZITI_CTRL_EDGE_ADVERTISED_PORT=1280 \
  -e ZITI_ROUTER_NAME=ziti-edge-router-1 \
  -e ZITI_ROUTER_ADVERTISED_ADDRESS=host.mydomain.com \
  -e ZITI_ROUTER_ROLES=public \
  --network plexnet \
  --network-alias ziti-edge-router-1 \
  -p 3022:3022 \
  --dns 192.168.80.1 \
  -v /data/containers/ziti:/persistent \
  openziti/quickstart \
  /var/openziti/scripts/run-router.sh edge

In my application tunnel logs (connecting from Windows) I see this:
DEBUG ziti-sdk:channel.c:704 reconnect_cb() ch[7] connecting to ziti-edge-router-1:3022

And if I add that to my hosts, sure enough - the service works.

11

My guess is that the ZITI_ROUTER_ADVERTISED is not set correctly or there's a missing env var. I should have verified the commands I shared work for devices and pass traffic -- I didn't do that... Let me go through the steps I provided and verify the routers advertise the proper address...

12

Hi @jagauthier. I've confirmed that data will pass traffic if you set the ZITI_ROUTER_ADVERTISED_ADDRESS. You might need to dump your persistent volume to pickup that change? Or you can just go into the container and edit the configuration files...

The key in the config file is the "listeners: edge: options: advertise" has to be set correctly. For example my local only router has (i've edited my hosts file so ziti-edge-rotuer-1 is 127.0.0.1):

listeners:
# bindings of edge and tunnel requires an "edge" section below
  - binding: edge
    address: tls:0.0.0.0:13022
    options:
      advertise: ziti-edge-router-1:13022

The AWS router looks like:

listeners:
# bindings of edge and tunnel requires an "edge" section below
  - binding: edge
    address: tls:0.0.0.0:8477
    options:
      advertise: ec2-13-58-222-94.us-east-2.compute.amazonaws.com:8477
      connectTimeoutMs: 5000
      getSessionTimeout: 60

The easiest thing imo, is to:

  • ziti edge delete your routers
  • docker volume rm your docker persistent volumes
  • start your routers back up (they'll recreate themselves in the controller) and you'll be good to go.

One further note, even though you are using the default ports for the routers, you MIGHT want to set both of these:

  -e ZITI_ROUTER_PORT=8477 \
  -e ZITI_ROUTER_LISTENER_BIND_PORT=8477 \

Then you'll only need a single port for your router to be open too (not 3022 and 10080).

hopefully that gets you sorted

13

Hi, thanks for that.

To simplify my life I did set both environment variables for the port to 8477 as recommended.
I had previously deleted my router and my config files, so I just fire it up and created it anew.

I noticed my tunnel client was still reporting:
connecting to ziti-edge-router-1:3022

And I thought.. that's strange. There is no reference to that port anymore.
I restarted my client tunnel and that solved it.

I may have very well fixed it before posting but didn't realize because the client tunnel needed to be restarted as well.

Thanks!

14

This is probably an interesting bug. I'm glad you're up and working and I'll point a few people to this thread to see if there's a bug in here... Feels like it.

Happy OpenZiti-ing! :slight_smile:

15

this indeed is a bug. The good news is was already fixed in ziti C-sdk and in the new release of ziti-edge-tunnel

16

Cool! How often do those changes get rolled into the Windows Desktop client?
Looks like that thing is pushing a year since it's last release.

I haven't looked heavily into building it. But I am going to assume (since it's Windows) it's non-trivial

17

I believe it is coming real soon :tm: now. Maybe, @TheLumberjack can provide an update

1 Like
18

Generally, we release more often than that but yea, it's been a very long while. A candidate release was built in December but the automatic upgrade functionality wasn't passing manual tests and we've not had the time to resolve the issue yet.

We'll definitely be releasing soon™, but i don't know exactly when it will be. After we get through this next release, we'll probably go back to releasing 'as needed' which is usually somewhat often.

1 Like
19

@TheLumberjack
Great, thanks!

When looking at your previous instruction, I did deviate in one area. My container storage for the controller and router are the same directory.

So, when the router starts up, it loads ziti.env.

But if I do something similar where I want to stick a private router on a different host, it stalls when it runs because it cannot actually find ziti.env.
So I copied it over, but then the same process that occurred with router-1 does not happen. ie: Certificate creation and enrollment.

My output after copying the ziti.env is this:

ZITI_ROUTER_NAME set to: ziti-edge-router-2
config has not been generated, generating config...
Untrusted certificate authority retrieved from server
Verified that server supplied certificates are trusted by server
Server supplied 5 certificates
Server certificate chain written to /home/ziti/.config/ziti/certs/host.mydomain.com
Token: 0d3d66c6-7b69-4f4d-a2ad-e3e89551bcfd
Saving identity 'default' to /home/ziti/.config/ziti/ziti-cli.json
----------  Creating edge-router ziti-edge-router-2....
CREATING EDGE ROUTER CONFIG: ziti-edge-router-2
public router configuration file written to: /persistent/ziti-edge-router-2.yaml
----------  Found existing edge-router ziti-edge-router-2....
[   0.001]   ERROR ziti/ziti/router.run: {configFile=[/persistent/ziti-edge-router-2.yaml] error=[unable to load identity (open /persistent/ziti-edge-router-2.key: no such file or directory)] arch=[amd64] build-date=[2024-02-10T05:53:17Z] revision=[7c53aa006529] version=[v0.32.2] go-version=[go1.21.6] os=[linux]} error loading ziti-router config
panic: unable to load identity (open /persistent/ziti-edge-router-2.key: no such file or directory)

(followed by and exception)
I must be missing something else file-wise?

20

I specifically ran on three separate machines, and remove the persistent storage to make sure this sort of thing wouldn't happen. I'm surprised that you're finding it to stall on that section. I expect it'll just try to source the file, fail, and continue on. I think it's maybe stalling elsewhere. OFTEN it's because the advertised address is not routable, so my guess/hunch is that the ziti.env you copied is probably using a value for the controller you don't want it to (like the docker hostname it came from)...

I think the only thing you really need from the .env file is ZITI_USER and ZITI_PWD anyway... Which I showed in the commans with -e...

I'd just recommend you make a ziti.env of your own (or a .env or however you want to do it) with the same -e variables as I showed before. Those are the minimum ones you need.