Hi @arslane, welcome to OpenZiti! You’re in the right place for secure communication!
Looks to me like you nailed it with your outline. Don’t feel overwhelmed we can take it one step at a time together and we’ll get you there.
Step 1 - provision an OpenZiti network. It doesn’t need to be a separate machine - all the devs run an OpenZiti network locally all day long. Having a separate machine is nice, particularly if it’s internet addressable (VPS) since it will allow all the other devices to connect into it.
Step 2 - Yes, you must have at least one edge router. You can make more but for starters, one will be plenty. The ‘host it anywhere’ quickstart will get you through step one and two (hopefully) quickly and easily. You can follow the guide and you can watch me do it on a video if you like
Step 3 - Yep you’ll need an identity on any and every device that will be part of an OpenZiti network. Starting with “your client” makes perfect sense, I do exactly the same thing after making my network.
Step 4 - Sounds like you’re going to be trying OpenZiti with ansible, nice! Yes, you’ll want an identity for the ansible controller. You’ll probably run ziti-edge-tunnel on that machine and you’ll use ziti to provide secure, dark access from other ziti-enabled machines to your ansible controller (and from the ansible controller to the other machines)
Step 5 - Create an identity for every machine - yep. You’ll transfer the .jwt to each machine, where you will then enroll. Do note that ziti-edge-tunnel can be used to enroll. that’s what I’d probably use: ./ziti-edge-tunnel enroll --jwt machine1.jwt --identity machine1.json (etc.) I would then start ziti-edge-tunnel and make sure that you have the process up and running and things are looking good.
Step 6-8 - You would want one intercept config, one host config, and one service for each service you wish to expose. Optionally you’ll need one ‘dial’ service policy and one ‘bind’ service policy per service- or you can configure them with attributes. Since that’s a bit ‘advanced’ for a starter, I’ll leave it up to you if you want to work through attribute based access control (ABAC) that comes with ziti. It’s easiest to just make a bind/dial policy for each service until you get your feet under you.
If you get stuck anywhere along the way, just post back. I’m sure we can make you successful. Looking forward to seeing what you do with OpenZiti!
Ziti Fabric Routers are the fundamental building blocks of the Ziti Network. These routers are responsible for securely and reliably delivering traffic from one Ziti Network node to the traffic’s destination.
Knowing that, shouldn't I setup Fabric routers on each of my servers ? Instead of edge router ?
i mean, Edge routers are used to expose the network to the internet isn't it ?
@arsene! Welcome to the discourse and thanks for checking out Ziti TV and your kind words!
What an amazing question! There's some nuance here, hopefully, I can sum it up. I would personally exclusively deploy edge routers, I would just configure them accordingly. Hopefully this all makes sense...
fabric routers
I personally would only use "fabric routers", or to put another way, routers without the "edge features" enabled, into spaces where I wanted to form more mesh network topology to give my network more throughout, redundancy, more paths for faster circuits, etc.
edge routers
Edge routers are also capable of being "fabric routers", but they are also capable of being edge routers, routers with the edge features enabled. The edge features you'll probably end up wanting, ones I'd recommend, is to run the router "as a tunneler". That gets you the ability to offload traffic from the router to other pieces in the network. We've been referring to that as "ZTNA" - zero trust network access. BUT, the OpenZiti is even more powerful when you use "ZTHA" or "zero trust host access". That's when you have a tunnelling app deployed on your hosts. If you have an edge router on your network, you edge clients could all connect to the 'local' edge router and get onto the ziti fabric in a single place.
Just because it's an edge router, doesn't mean it needs to have an inbound port enabled. It can sit on your local network, behind a firewall and simply not turn the "link listener" function on (done in the config file). Instead, you'd leave your "private" edge router with only a "link dialer" configured so that the router can dial OTHER routers and form the mesh network that way. Then I would deploy one ore more edge routers on the open internet and let them have the "link listener" enabled, so other routers would rendezvous to that router. The docker overview image shows this pretty well I think. Here the green circles are "public edge routers" and the blue/red circles are "private edge routers" and a single "ziti fabric router" (without ege deployed) is used to make a more redundant network.
But, when i run the express install, it gets stuck on
waiting for the controller to come online to allow the edge router to enroll
waiting for https://cryptoinsights-master01-83312:1280
waiting for https://cryptoinsights-master01-83312:1280
waiting for https://cryptoinsights-master01-83312:1280
...
waiting for https://cryptoinsights-master01-83312:1280
I’ve set EXTERNAL_DNS variable to the instance’s public IP as i couldn’t find any public IPV4 DNS on Oracle Cloud
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 8441 -j ACCEPT
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 6262 -j ACCEPT
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 8442 -j ACCEPT
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 8443 -j ACCEPT
sudo iptables -I INPUT 6 -m state --state NEW -p tcp --dport 10080 -j ACCEPT
sudo netfilter-persistent save
I tried to nc -zv 144.24.196.224 8441 from my personal computer while having nc -l 8441 on the remote machine and it works, so ports are correctly opened.
Unfortunately, it’s still stuck at this step :
******** Setting Up Controller ********
cat: /home/ubuntu/.ziti/quickstart/instance-20220619-1542/pki/instance-20220619-1542-intermediate/certs/instance-20220619-1542-server.chain.pem: No such file or directory
cat: /home/ubuntu/.ziti/quickstart/instance-20220619-1542/pki/instance-20220619-1542-signing-intermediate/certs/instance-20220619-1542-signing-intermediate.cert: No such file or directory
wrote CA file to: /home/ubuntu/.ziti/quickstart/instance-20220619-1542/pki/cas.pem
-bash: /home/ubuntu/.ziti/quickstart/instance-20220619-1542/ziti-bin/ziti-v0.25.13/ziti: cannot execute binary file: Exec format error
Controller configuration file written to: /home/ubuntu/.ziti/quickstart/instance-20220619-1542/instance-20220619-1542.yaml
ziti-controller initialized. see /home/ubuntu/.ziti/quickstart/instance-20220619-1542/instance-20220619-1542-init.log for details
ziti-controller started as process id: . log located at: /home/ubuntu/.ziti/quickstart/instance-20220619-1542/instance-20220619-1542.log
waiting for the controller to come online to allow the edge router to enroll
waiting for https://144.24.196.224:8441
waiting for https://144.24.196.224:8441
waiting for https://144.24.196.224:8441
waiting for https://144.24.196.224:8441
waiting for https://144.24.196.224:8441
waiting for https://144.24.196.224:8441
I even tried to install firewalld on my instance and open the ports with firewalld and not with iptales, without success
I apologize for the length of this answer and for flooding you with all those questions, I hope you that doesn't bother you. I'm just trying to really understand how Ziti works because I'm planning to integrate it in the Stack of the company I'm starting.
@TheLumberjack Looks like the ziti controller isn't starting.
The startZitiController function of the script ziti-cli-functions.sh should log the PID:
echo -e "ziti-controller started as process id: $pid. log located at: $(BLUE "${log_file}")"
But here's the log I'm getting :
ziti-controller started as process id: . log located at: /home/ubuntu/.ziti/quickstart/instance-20220619-1627/instance-20220619-1627.log
By checking the log file, the only logged line is :
bash: /home/ubuntu/.ziti/quickstart/instance-20220619-1627/ziti-bin/ziti-v0.25.13/ziti-controller: cannot execute binary file: Exec format error
Looks like the problem in in the ziti-cli-functions.sh script, check out these functions :
function verifyZitiVersionExists {
setupZitiHome
if ! setOs; then
return 1
fi
ZITI_ARCH="amd64"
if [[ "$(uname -a)" == *"arm"* ]] && [[ "${ZITI_OSTYPE}" == "darwin" ]]; then
echo -e "$(YELLOW "WARN: It has been detected that you are using an Apple computer with ARM architecture. Deployment of Apple ARM architecture distributions is currently unsupported through git, the installer will pull darwin amd distribution instead.")"
elif [[ "$(uname -a)" == *"arm"* ]]; then
ZITI_ARCH="arm"
fi
unset ZITI_BINARIES_VERSION
ziticurl="$(curl -s https://api.github.com/repos/openziti/ziti/releases/tags/"${ZITI_VERSION_OVERRIDE}")"
# shellcheck disable=SC2155
export ZITI_BINARIES_FILE=$(echo "${ziticurl}" | tr '\r\n' ' ' | jq -r '.assets[] | select(.name | startswith("'"ziti-${ZITI_OSTYPE}-${ZITI_ARCH}"'")) | .name')
# shellcheck disable=SC2155
export ZITI_BINARIES_VERSION=$(echo "${ziticurl}" | tr '\r\n' ' ' | jq -r '.tag_name')
# Check if there was an error while trying to get the requested version
if [[ "${ZITI_BINARIES_VERSION-}" == "null" ]]; then
return 1
fi
echo "ZITI_BINARIES_VERSION: ${ZITI_BINARIES_VERSION}"
}
function getLatestZitiVersion {
setupZitiHome
if ! setOs; then
return 1
fi
ZITI_ARCH="amd64"
if [[ "$(uname -a)" == *"arm"* ]] && [[ "${ZITI_OSTYPE}" == "darwin" ]]; then
echo -e "$(YELLOW "WARN: It has been detected that you are using an Apple computer with ARM architecture. Deployment of Apple ARM architecture distributions is currently unsupported through git, the installer will pull darwin amd distribution instead.")"
elif [[ "$(uname -a)" == *"arm"* ]]; then
ZITI_ARCH="arm"
fi
unset ZITI_BINARIES_VERSION
if [[ "${ZITI_BINARIES_VERSION-}" == "" ]]; then
zitilatest=$(curl -s https://api.github.com/repos/openziti/ziti/releases/latest)
# shellcheck disable=SC2155
export ZITI_BINARIES_FILE=$(echo "${zitilatest}" | tr '\r\n' ' ' | jq -r '.assets[] | select(.name | startswith("'"ziti-${ZITI_OSTYPE}-${ZITI_ARCH}"'")) | .name')
# shellcheck disable=SC2155
export ZITI_BINARIES_VERSION=$(echo "${zitilatest}" | tr '\r\n' ' ' | jq -r '.tag_name')
fi
echo "ZITI_BINARIES_VERSION: ${ZITI_BINARIES_VERSION}"
}
They use uname -a to detect wich architecture type is used, but I believe that it detects what type of OS architecture our hardware is capable of, not what is actually installed (i may be wrong tho, not a unix guru).
Also, when I run dpkg --print-architecture i get ARM64, but when i run uname -a, i get ARCH64.
I changed the script to use dpkg --print-architecture instead (only works for debian based OS, but Im on Ubuntu so), and now the express install works.
I don't know if this is a bug, or if it's only related to Oracle Cloud's instance. @TheLumberjack If you think that's a bug, I'd be glad to create an issue on ziti's github.
So, now I have a Ziti Controller and a Ziti Edge Router setup on the same server , right ? I have a few questions :
Is it possible to put my only edge router outside of the server where my controller lives ?
Is it possible to make the host where the controller lives not accessible from outside unless I'm authenticated with a ziti edge client ?
On the linux ziti-edge-tunnel, what's the difference between the commands run and run-host ?
When creating the host config, which IP address should i provide ? the public one or the private one ? I'm asking this question because on your video Totally Private Postgres, you provided the private IP address of the AWS EC2 instance 172.31.50.50
4.1 On the same video, when you configured Inbound Rules for Postgresql, SSH and HTTP, you've set the the traffic source to 172.31.0.0/16 was this arbitrary ? Or did you set this because all you setup, namely the VM hosting controller & edge router and the VM hosting the Postgresql container are on AWS and they share the same subnet mask ? If so, what should I do If all of my servers does not share the same Cloud provider, ?
Is it possible to make the ZAC reachable only if I'm authenticated with a Ziti Client ?
If i decide to setup a Fabric router on the same server or on another server, the traffic coming from outside the Ziti Network will pass from the edge router, then from the Fabric router, then to the controller and all the way back to the Ziti Edge Router ?
6.1 Are there other possible scenarios ?
About Fabric router, when you say to form more mesh network topology to give my network more throughout, redundancy, more paths for faster circuits, etc., does the Fabric Routers act like a kind of Load Balancer ?
It's a bug of some kind for sure. The quickstart was/is a "best effort" to get people going, *nix has so many variants that it's not hard to believe there's a bug here/there. It does surprise me that dpkg --print-architecture prints ARM64.
I just used these images in Oracle Cloud and had no issue with the quickstart. I hate to ask, but, you're sure? Maybe try with the image/shape I did? (I DID have to issue the iptables rules as you highlighted here)
Certainly. When I ran the quickstart, even with the IP address set in the EXTERNAL_DNS field, I still noticed my controller was incorrect. The config file located at $ZITI_HOME/$(hostname).yaml has the hostname in the edge.api.address and web.name.address fields. You need to change those to your IP address. You will need to make sure the cert presented by the controller has a SANS that matches the advertised address:
You'll also need to update the edge-router config. It's also pointing to the hostname for ctrl.endpoint. You'll want that to be the IP address. These seem like bugs that I'll ask @gberl002 to look into.
the "client" API, but always be accessible. The "management" API can absolutely be separated out. Take a look at this discourse post Making ZAC dark - #6 by TheLumberjack, I think it contains what you need. It's on my 'todo' list to document this, but we haven't gotten to it yet.
The "run-host" means, "do not intercept traffic, only 'host' it". You'd use this in a situation where you wanted to only offload from the zero trust overlay. It does not/shouldn't require sudo access as it doesn't need to make a TUN device. "Run" means, "allow traffic to offload, but ALSO intercept traffic" and requires systems to be present and to run with CAP_NET_ADMIN/sudo priv's to make the TUN device and run a couple other commands.
Up to you. Do you want the traffic to land at your "tunneler" and go back to the public IP? Probably not, I'd wager? You just need to configure the IP you want the traffic to go toward, relative from where it leaves ziti. So, I would guess you want the private IP as long as it's addressable from the tunneler that's hosting the service (via the host.v1 config).
Not arbitrary, no. But also not "really specific". I could have used 172.31.42.64/32 since that's the private IP address of my expressInstall/quickstart. I was just lazy and let Amazon use the full /16. 172.31.0.0/16 covers the 172.31.42.64 IP. At 3m25s you can see the IP reflected by the name of my router. It might be in other places in the video but that's the first time I noticed the actual IP.:
It sure is. The discourse post I mentioned earlier covers exactly that Making ZAC dark
"Maybe" is the only answer that's correct here. It all depends. Networks are fickle and sometimes it's faster to go through the 'fabric' router, sometimes it's faster to go around it. On top of that, without knowing exactly how you configure your network, this one's impossible to answer. If you want to dig in on this - would you mind making another post? this one is getting long in the tooth. If you could, make a diagram and show us what you're thinking for us to answer best. There's "lots" of scenarios. Far too many to enumerate, it's better to know where you're at, what you're doing and then we can answer better.
I understand, I don't really have a schema of my network in mind, I just know that I'll be using Kubernetes to deploy some Big Data tools such as Apache Kafka & Flink, and I want to have ZTHA all over my nodes.
I'm actually asking these question so i can think of my network configuration haha
I really appreciate you help, very clear answers. I'll just play with Ziti for a week or so now and probably make another thread with more relevant questions
Note that setting MFA does not lock you to a specific MAC address. There is a separate posture check for specified MAC addresses.
Some more details:
MFA is set up at the identity as a secondary authentication check.
Posture checks of type “MFA” can also be configured that require MFA to be enabled for an identity to access specific services, providing an additional authorization check.
MAC address checking (as well as OS version checking, and verifying specific applications are running and are appropriately signed) are also posture checks (separate and distinct from MFA and MFA posture checks).
we can take it one step at a time together and we’ll get you there.
