Assuming you have sourced the script, you will have an environment variable set named $ZITI_EDGE_CONTROLLER_API. After the controller has started, your controller should be listening at that hostname:port combination. You can see what your value is set to by running echo $ZITI_EDGE_CTRL_ADVERTISED. This variable defaults to: $(hostname):1280. Make sure the controller is on and listening and then start the edge router.
I have sourced the script (although it would be helpful to be more specific in this statement and say exactly which script we are talking about, I can only assume it's the install script used above?) and this doesn't work. I neither have that $ZITI_EDGE_CONTROLLER_API variable nor do I get any response when trying to echo $ZITI_EDGE_CTRL_ADVERTISED. Has something maybe changed that needs to be reflected in the quickstart documentation to match new variable names, or is this a problem where I haven't sourced something correctly?
My controller is called zctl. I do have ZITI_CTRL_EDGE_ADVERTISED_ADDRESS and ZITI_CTRL_ADVERTISED_PORT in my printenv output and the zctl.env file. There isn't a single variable with "API" in it when you grep through the zctl.env file. I get a blank line when I try to echo as the instructions suggest:
Yah, good point. I'll update the doc to spell it out clearly. It should say "sourced the .env file created by the quickstart. The .env file is generally located at $HOME/.ziti/quickstart/$(hostname)/$(hostname).env. That's what it should say...
There are healthchecks from the controller you can specify, but the quickstart doesn't enable them by default. Generally, I just check that the process is running and then curl to the API port to make sure it seems healthy. You can also portcheck using any port scanning tool. I will often use openssl s_client to just probe the port but anything should work. I've actually not configured any router healthchecks but they are there as well Router Configuration Reference | OpenZiti
Yeah, that is likely how I got it done but it would be helpful to have that sequentially up there as you suggested. Seems like the variables are not what is currently correct as well? I think that Verify the Controller is Running section needs correction or clarification overall.
After running expressInstall, you will have environment variables set named ZITI_CTRL_EDGE_ADVERTISED_ADDRESS and ZITI_CTRL_EDGE_ADVERTISED_PORT. After the controller has started, your controller should be listening at that address:port combination. (Note, if you do not have these environment variables, you’ve probably closed your shell and opened it up again. You can get the environment variables by sourcing the “.env” file. See the section at the bottom of the page for details)
@TheLumberjack Hi, sorry to still be struggling here but I need some more help or advice on whether I just start the quickstart all over again. When I left this build a few days ago, I had a controller and router running from my quickstart and I had successfully installed and configured ZAC with the ability to log into the web UI for ZAC. I thought everything was set for me to start configuring identities and services.
When I returned to this host today I did some basic maintenance updates, as I tend to do, and this included a kernel update which required a reboot. Upon reboot I of course tried to verify that the ziti controller and router and ZAC were all up and running. None of them appear to be, so I assumed that I hadn’t properly succeeded in getting the systemd stuff configured or perhaps was missing some shell or environment variables, etc. I’ve checked through all of these, and I also followed your advice and even added export PATH=$PATH:/home/jfj/.ziti/quickstart/zctl/ziti-bin/ziti-v0.28.4 and source $HOME/.ziti/quickstart/zctl/zctl.env to my .bashrc, and I can verify that they’re loading correctly:
jfj@zctl:~/.ziti/quickstart/zctl$ curl "${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS}:${ZITI_CTRL_EDGE_ADVERTISED_PORT}"
curl: (7) Failed to connect to zctl port 1280 after 0 ms: Connection refused
So apparently those variables are properly showing port 1280 but of course nothing is started.
What’s my best course of action here- should I follow the instructions to reset and start over?
I went to verify that there is indeed a ziti-controller.service and a ziti-router.service file in /etc/systemd/system. They are there and contents look correct.
Here is the output from checking status:
jfj@zctl:/etc/systemd/system$ sudo systemctl -q status ziti-controller --lines=0 --no-pager
● ziti-controller.service - Ziti Controller
Loaded: loaded (/etc/systemd/system/ziti-controller.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Wed 2023-07-05 02:12:41 UTC; 144ms ago
Process: 8150 ExecStart=/home/jfj/.ziti/quickstart/zctl/ziti-bin/ziti-v0.28.1/ziti controller run /home/jfj/.ziti/quickstart/zctl/zctl.yaml (code=exited, status=203/EXEC)
Main PID: 8150 (code=exited, status=203/EXEC)
CPU: 1ms
Yeah that's what I would expect. I can see you're just a bit confused with respect to systemd vs the helper scripts functions startController/Router. if you "enabled" systemd, then you should have had the services start on reboot and you check that using these commands
sudo systemctl -q status ziti-controller --lines=0 --no-pager
sudo systemctl -q status ziti-router --lines=0 --no-pager
Those should show you the process running. If not you'll want to look to the logs using journalctl for hints as to what happened...
I'll finish this reply because I see you found the status
You’ll want to look to the controller logs to see what’s happening. I definitely wouldn’t have expected an issue with a patch type of update. Let’s see what the logs show us
I’ll start digging- in the meantime, can you help me out with understanding the helper script functions which must not be correctly loaded for me? A way to troubleshoot or fix that?
Sure. I just had to explain it to someone else just tonight so I filled an issue to clarify.
These are functions you get when sourcing ziti-cli-functions. They simply run the binary directly. They are just functions that do exactly what the systemd unit files do but you're running them directly
jfj@zctl:/etc/systemd/system$ tail /var/log/syslog
Jul 5 02:19:10 zctl systemd[8533]: ziti-controller.service: Failed to locate executable /home/jfj/.ziti/quickstart/zctl/ziti-bin/ziti-v0.28.1/ziti: No such file or directory
Jul 5 02:19:10 zctl systemd[8533]: ziti-controller.service: Failed at step EXEC spawning /home/jfj/.ziti/quickstart/zctl/ziti-bin/ziti-v0.28.1/ziti: No such file or directory
Jul 5 02:19:10 zctl systemd[1]: Stopped Ziti Router for zctl-edge-router.
Jul 5 02:19:10 zctl systemd[1]: Started Ziti Router for zctl-edge-router.
Jul 5 02:19:10 zctl systemd[8534]: ziti-router.service: Failed to locate executable /home/jfj/.ziti/quickstart/zctl/ziti-bin/ziti-v0.28.1/ziti: No such file or directory
Jul 5 02:19:10 zctl systemd[8534]: ziti-router.service: Failed at step EXEC spawning /home/jfj/.ziti/quickstart/zctl/ziti-bin/ziti-v0.28.1/ziti: No such file or directory
Jul 5 02:19:10 zctl systemd[1]: ziti-controller.service: Main process exited, code=exited, status=203/EXEC
Jul 5 02:19:10 zctl systemd[1]: ziti-controller.service: Failed with result 'exit-code'.
Jul 5 02:19:10 zctl systemd[1]: ziti-router.service: Main process exited, code=exited, status=203/EXEC
Jul 5 02:19:10 zctl systemd[1]: ziti-router.service: Failed with result 'exit-code'.
I can immediately see the obvious problem, although I’m not sure how this occurred. Maybe I did a previous quickstart run with a different earlier version which wrote these .service files and they’re pointing to the wrong directory now. I’m at v0.28.4 and the executable directory structure reflects that. I can’t use the createControllerSystemdFile so something isn’t loaded here and if I can get the helper scripts working I should be able to fix this, but I don’t know how to do that other than start over?
Perfect, thanks for explaining that. The folder doesn’t exist:
jfj@zctl:~/.ziti/quickstart/zctl/ziti-bin$ ls -la
total 12
drwxrwxr-x 3 jfj jfj 4096 Jun 25 21:23 .
drwxrwxr-x 7 jfj jfj 4096 Jun 25 23:28 ..
drwxrwxr-x 2 jfj jfj 4096 Jun 25 21:23 ziti-v0.28.4
I’m definitely most likely to believe this was something I did that I’m not remembering- like running the script a second time days later and maybe the older version wrote those, but I’ll look at the script to make sure there isn’t some absolute reference to that other path leftover from somewhere.
You probably want to run ‘getZiti’. It’ll fetch the binaries for you… Or just download and put them into place and I expect it’ll start back up. (getZiti is another helper functions)
Technically you should probably download the file, read it and make sure it’s safe… Yeah.
I find once you get through this learning phase you’ll probably rarely/never need those functions again… I personally always source the .env file making logging in easier. But I nearly never need these functions and I just source them if I need to
