Ooooh i see, so maybe I don’t need to have all open ports since I’m already using 1280 and 6262, for example. I can close the 8441 and 8442 perhaps.
So the only one I’m missing is the 10080, that goes in the router right? Like so
ziti-controller:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
ports:
- ${ZITI_EDGE_CONTROLLER_PORT:-1280}:${ZITI_EDGE_CONTROLLER_PORT:-1280}
- ${ZITI_CTRL_PORT:-6262}:${ZITI_CTRL_PORT:-6262}
. . .
ziti-edge-router:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
ports:
- ${ZITI_EDGE_ROUTER_PORT:-3022}:${ZITI_EDGE_ROUTER_PORT:-3022}
- 10080:100080 # I don't know if this port has a special variable name in OpenZiti?
. . .
All right so I finally fixed it! It was the port 10080 that wasn't being mapped in the docker-compose.yml, such a dumb mistake but oh well, the thing is I registered successfully the router and it does appear indeed in the list!
However... I tried testing it doing what you mentioned here
My plan was to try to connect to the SSH server on the server machine behind my router from the Client computer, and I thought it'd use the private router that it's running on the same machine.
There's a docker container with the edge router and a docker container with ziti-host and a docker continer running sshd server
The ssh connection works but I think I only see the public router
Progress! My guess is that you have not authorized identities to use your new router and you need to add/update an “edge-router-policy”.
Refer back to:
Notice that the private router has an attribute of #private and the public router has an attribute of #public. Notice that “Computer n3” and “Computer n2” are identities with the #home attribute while the android/iOS phones are given #roaming attributes… That’s all for the specific purpose of being able to create edge-router-policies and authorizing which edge routers any given identity is authorized to connect to…
In that case, the person was using satellite internet and wanted to be more restrictive, only allowing the “local network clients” to connect to the #private router and then allowing identities with the #roaming attribute to connect to BOTH the public router and the private router…
You probably don’t need to be as restrictive, but you could if you choose to. You probably just need to add your new edge router to the erp/edge-router-policy
But still seeing the same in the ziti fabric list circuits perhaps i need to restart the edge router or something? This is what i have in ziti edge list identities btw
Also can you confirm the listeners.binding(edge).options.advertise of the private router config? It’s found in this section here:
listeners:
# bindings of edge and tunnel requires an "edge" section below
- binding: edge
address: tls:0.0.0.0:8442
options:
advertise: ec2-3-134-108-218.us-east-2.compute.amazonaws.com:8442
That advertise address is “some.resolvable.host:3022” (or whatever). Can you then probe that port from one of the client machines and make sure it can reach that configured host:port?
In my LAN, the privatemachine:3022 is not resolvable, so I assume I should change this to my local machine's IP which is something like 192.168.1.123 so the above should be this instead
Yes you are correct. That 'advertise' is what clients will try to connect to your edge router at. They need to be able to get to that host/port on the local/private network.
But, you'll need to delete the router now and re-enroll it. ALSO make sure you go down and change the "edge.csr.sans" section... that needs to match what you plan to advertise:
you can add more than one entry there or more than one IP. When you enroll the router, the router PKI will get generated and it will have these values in the certificate... So you'll need to re-enroll... Hopefully that makes sense?
But im still seeing the same in the ziti fabric list circuits, while I was typing this I noticed that this section migh have an error now
link:
dialers:
- binding: transport
listeners:
- binding: transport
bind: tls:0.0.0.0:10080
advertise: tls:192.168.1.123:10080 # <-- Here, shouldnt it be the public machine's address?
options:
outQueueSize: 4
Since I’m using the docker container quickstart, I think it’s getting the variables from the .env file and perhaps I’m misusing a variable. This is what I currently have
# OpenZiti Variables
ZITI_IMAGE=openziti/quickstart
ZITI_VERSION=latest
# The duration of the enrollment period (in minutes), default if not set
ZITI_EDGE_IDENTITY_ENROLLMENT_DURATION=
ZITI_EDGE_ROUTER_ENROLLMENT_DURATION=
# From gooselegs response
ZITI_EDGE_ROUTER_ROLES=private
# Credentials
ZITI_USER=admin
ZITI_PWD=mysupermegasecretpassword
# From clint's script
ZITI_EDGE_ROUTER_PORT=3022
ZITI_CONTROLLER_HOSTNAME=publicmachine.mydomain.com
ZITI_CTRL_EDGE_ADVERTISED_PORT=1280
ZITI_CTRL_LISTENER_PORT=6262
ZITI_CTRL_PORT=${ZITI_CTRL_LISTENER_PORT}
ZITI_EDGE_CONTROLLER_PORT=${ZITI_CTRL_EDGE_ADVERTISED_PORT}
ZITI_CTRL_ADVERTISED_ADDRESS=${ZITI_CONTROLLER_HOSTNAME}
ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=${ZITI_CONTROLLER_HOSTNAME}
# These ones were provided because the container complained
# These two were 'privatemachine' before so i changed it to the local IP 'fix' it
ZITI_EDGE_ROUTER_HOSTNAME=192.168.1.123
ZITI_EDGE_ROUTER_RAWNAME_OVERRIDE=192.168.1.123
I think I’m missing the variable to set the hostname of the public router because right now it seems to pick up the same variable as the private router
i am going to make a whole walkthrough video and a new gist… i found other things about the instructions/gist that I didn’t like and I want to make this simpler for you to follow. You are indeed very close, but let me try to make a new gist which will hopefully make this easier…
I followed your steps but in the middle I noticed a couple things:
1- The router config was almost exactly the same as the one I have, but the section link → dialers → listeners was commented out in your private router’s config and it was not commented out in the config I made
2- I searched through OpenZiti’s Ziti repository and I found the scriptrun-external-router.sh and I realized that there are 4 possible parameters to call the script with: edge, wss, fabric and, the one that’s probably most important, private
And I was running the script inside my docker-compose.yml using the edge parameter, not the private parameter, probably that’s why the config wasn’t generating properly and it wasn’t working
So without touching anything else from my .env file the final docker-compose.yml that I’m using is the following
docker-compose.yml
services:
ziti-edge-router:
image: "${ZITI_IMAGE}:${ZITI_VERSION}"
env_file:
- ./.env
ports:
- "3022:3022"
networks:
- ziti
volumes:
- ziti-fs:/persistent
entrypoint: /bin/bash
command: "/var/openziti/scripts/run-router-external.sh private" # <- This is the only thing I changed, changed edge to private
networks:
ziti:
volumes:
ziti-fs:
After running docker compose up (and after deleting old routers / volumes that weren’t working of course) and establishing a ssh connection, it seems to be working properly.
@TheLumberjack is this the expected result to the command you mentioned the other day?
ziti fabric list circuits
╭───────────┬───────────────────────────┬────────────────────┬────────────────────────┬─────────────────────────────────────────────────────────────────╮
│ ID │ CLIENT │ SERVICE │ TERMINATOR │ PATH
├───────────┼───────────────────────────┼────────────────────┼────────────────────────┼─────────────────────────────────────────────────────────────────┤
│ nc2F3RcKD │ clh7m9pd71exnc1mk4y0v8x7i │ test_ssh_service │ 2TnxdeJJ3kouNQGOE2MOgc │ r/public.mydomain.com |
│ uXou3mpKD │ clh7m9pd71exnc1mk4y0v8x7i │ test_ssh_service │ 2bGRWU70uvKvDH3Mz2SPnS │ r/192.168.1.123 -> l/IjKPVAFzneryWxu5322Mq -> r/public.mydomain.com │
╰───────────┴───────────────────────────┴────────────────────┴────────────────────────┴─────────────────────────────────────────────────────────────────╯
ziti edge list sessions 'id="clh7m9pd71exnc1mk4y0v8x7i"'
╭───────────────────────────┬───────────────────────────┬────────────────────┬──────╮
│ ID │ API SESSION ID │ SERVICE NAME │ TYPE │
├───────────────────────────┼───────────────────────────┼────────────────────┼──────┤
│ clh7m9pd71exnc1mk4y0v8x7i │ clh7hlzz818wac1mkbhjhj7j7 │ test_ssh_service │ Dial │
╰───────────────────────────┴───────────────────────────┴────────────────────┴──────╯
results: 1-1 of 1
ziti@6654d0aa7325:/persistent$ ziti edge list api-sessions 'id="clh7hlzz818wac1mkbhjhj7j7"'
╭───────────────────────────┬──────────────────────────────────────┬───────────────╮
│ ID │ TOKEN │ IDENTITY NAME │
├───────────────────────────┼──────────────────────────────────────┼───────────────┤
│ clh7hlzz818wac1mkbhjhj7j7 │ 0a1bcb0d-b516-4afe-aa91-ed2a1ac80007 │ my-client │
╰───────────────────────────┴──────────────────────────────────────┴───────────────╯
Hey great, glad you got something working! I need to brush up on my path reading.
I expected to only see r/192.168.1.123 on that path if you’re ssh’ing locally. What’s on both ends – is it a tunneler initiating ssh and a tunneler offloading toward sshd? Are you ssh’ing to something local or are you ssh’ing to something the public router can reach? I need a tiny bit better picture in my head to understand exactly. If you’re ssh’ing from local network to something ‘public router’ can reach that path can definitely make sense though. Can you give me a bit more information?
Still, that private router sure looks like it’s setup correctly now! I see the link formed and I see your client is connecting to that local router too!
Sure let me try to illustrate
I have my personal computer at home
I also have a computer that I use as a server connected to the same router as the PC
The personal computer is enrolled with a user identity, client
The private server is using the docker container ziti-host enrolled with a device identity privatemachine
In that same private server machine, I’ve also started another container, sshd server, which is listening on port 2222 on that local machine. I did this because apparently I couldn’t connect to that machine’s ssh on port 22 using OpenZiti, so I assume it’s some firewall stuff and that’s why I tried the sshd container.
So the client sends the connection request through OpenZiti (Ziti Desktop Edge) and it should exit through the Ziti Host container in the Server to reach the SSHD Server that’s also there.
I did it like this to test because it was the simplest setup I could think of that worked, but if you think that in order to be 100% sure that this works we should try another setup (with maybe another machine) I can also try that
I was reading the forums and correct me if I’m wrong but you can use the Private Router as a Tunneler just like the Ziti Host container? Perhaps my setup is a little bit wrong then…
Based on what you are showing, with the ziti-private-rotuer and ziti-host in the same docker ecosystem, the first thing that strikes me is that you don’t need the ziti-host. If it was me, I would probably just use the ziti-private-router for acessing sshd targets within docker.
Based on the nice diagram you provided, it makes sense that you wouldn’t expect to see traffic going to the public router when you were home. The traffic should flow from client to private router to sshd.
The first thing I would do, is remove that ziti host / identity. Then I would make sure the private edge router was “tunneler” enabled. Meaning -t was used when issuing ziti edge create edge-router. Then I would setup the “bind” config for sshd to be the identity of the private router.
As to ‘why’ it looks like you’re seeing the path go back through the public router, I still find that strange. I haven’t been able to try debugging the situation and the fella that is might be able to help has been under the weather. I’ll see how he’s feeling today and point him to his post and see if he has other thoughts I missed.
Yeah that's what I suspected, I'll have the ziti-host container removed and I'll have the router set in tunneler mode as you said, I'll try to figure out how to do that with the docker container or, if i can't, I'll use the github gist you provided.
Don't worry, in fact, I'm in no hurry right now and also starting tomorrow I'll be unavailable until sunday afternoon, so I won't be able to provide more details if needed.
I'll try to leave the final docker-compose.yml and .env files that I'm using for my public and private machines though, in case they help somehow.
You mean that I can remove the ziti-host docker container, alongside the privatemachine identity and if i put the same attributes sshd to the 192.168.1.123 Router, it will start working the same as the ziti-host? Or should I also check what you mentioned here?
I also think that I'll have to set the network type of the router container as "host" just like the ziti-host currently has, because currently the host config is mapped to 127.0.0.1, but that's a super easy fix., if you can confirm what I said on previous paragraph, I'll give it a go when I come back!
My guess is that you have not authorized identities to use your new router and you need to add/update an “edge-router-policy”.
