The next step from the Linux router deployment guide is to start the service.
Yes, I started the service using sudo systemctl enable --now ziti-router.service
Still I'm getting the same tls error. I'm wondering if I'm using the wrong port or if something else is off in my bootstrap.env file
ajay@ajay-ubuntu:~/ziti_codes$ cat /opt/openziti/etc/router/bootstrap.env
#
# this is the ziti-router.service bootstrapping inputs file where answers are recorded for generating a
# configuration
#
# the controller's DNS name (required)
ZITI_CTRL_ADVERTISED_ADDRESS='192.168.1.83'
# the controller's port (default: 1280)
ZITI_CTRL_ADVERTISED_PORT='1280'
# this router's DNS name or IP address (default: localhost)
ZITI_ROUTER_ADVERTISED_ADDRESS='ziti_edge_router_2'
# this router's port (default: 3022), if <= 1024, then grant the NET_BIND_SERVICE ambient capability in
# /etc/systemd/system/ziti-router.service.d/override.conf
ZITI_ROUTER_PORT='3022'
# token will be scrubbed from this file after enrollment
ZITI_ENROLL_TOKEN=''
# additional arguments to:
# ziti create config ${ZITI_ROUTER_TYPE:-edge} --tunnelerMode ${ZITI_ROUTER_MODE:-host}
ZITI_BOOTSTRAP_CONFIG_ARGS=''
ZITI_BOOTSTRAP_NOW='2024-10-30T15:42:22+00:00'
ZITI_HOME='/var/lib/private/ziti-router'
ZITI_ROUTER_BIND_ADDRESS='0.0.0.0'
ZITI_ROUTER_DNS_IP_RANGE='100.64.0.1/10'
ZITI_ROUTER_IDENTITY_CERT='router.cert'
ZITI_ROUTER_MODE='host'
ZITI_ROUTER_NAME='router'
ZITI_ROUTER_TPROXY_RESOLVER='udp://127.0.0.1:53'
ZITI_ROUTER_TYPE='edge'
Thanks,
Ajay
Thanks for confirming you started the router service after re-enrolling and that edge-router-2 is still offline. Please also confirm you completed all of these steps in this order:
-
re-enroll the router
ziti edge re-enroll edge-router "edge-router-2" -o edge-router-2.jwt -
disable, reset, and clean the router service state
sudo systemctl disable --now ziti-router.service sudo systemctl reset-failed ziti-router.service sudo systemctl clean --what=state ziti-router.service -
enroll and generate a router configuration
sudo /opt/openziti/etc/router/bootstrap.bash -
enable and start the service
sudo systemctl enable --now ziti-router.service
Hi, Thanks for your response. I'm attaching the log messages for all the 4 steps
ajay@ajay-ubuntu:~/ziti_codes$ ziti edge re-enroll edge-router "edge-router-2" -o edge-router-2.jwt
re-enroll edge-router with id Z7.Z9.pswh: OK
Enrollment expires at 2024-11-08T21:08:56.999Z
ajay@ajay-ubuntu:~/ziti_codes$
ajay@ajay-ubuntu:~/ziti_codes$ sudo systemctl disable --now ziti-router.service
ajay@ajay-ubuntu:~/ziti_codes$ sudo systemctl reset-failed ziti-router.service
ajay@ajay-ubuntu:~/ziti_codes$ sudo systemctl clean --what=state ziti-router.service
systemctl: unrecognized option '--what=state'
ajay@ajay-ubuntu:~/ziti_codes$ sudo /opt/openziti/etc/router/bootstrap.bash
INFO: bootstrap completed successfully and will not run again. Adjust /var/lib/private/ziti-router/config.yml to suit.
ajay@ajay-ubuntu:~/ziti_codes$ sudo systemctl enable --now ziti-router.service
Created symlink /etc/systemd/system/multi-user.target.wants/ziti-router.service → /lib/systemd/system/ziti-router.service.
ajay@ajay-ubuntu:~/ziti_codes$ sudo systemctl status ziti-router.service
● ziti-router.service - OpenZiti Router
Loaded: loaded (/lib/systemd/system/ziti-router.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/ziti-router.service.d
└─override.conf
Active: active (running) since Fri 2024-11-08 13:11:31 EST; 6s ago
Process: 5859 ExecStartPre=/opt/openziti/etc/router/entrypoint.bash check config.yml (code=exited, status=0/SUCCESS)
Main PID: 5868 (ziti)
Tasks: 14 (limit: 4915)
CGroup: /system.slice/ziti-router.service
└─5868 /opt/openziti/bin/ziti router run config.yml --extend
Nov 08 13:11:32 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:32 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:32 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:32 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:32 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:33 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:33 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:34 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:35 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
Nov 08 13:11:37 ajay-ubuntu ziti[5868]: {"endpoint":"tls:192.168.1.83:1280","error":"error connecting ctrl (tls: failed
ajay@ajay-ubuntu:~/ziti_codes$ ziti edge list edge-routers
╭────────────┬───────────────┬────────┬───────────────┬──────┬────────────╮
│ ID │ NAME │ ONLINE │ ALLOW TRANSIT │ COST │ ATTRIBUTES │
├────────────┼───────────────┼────────┼───────────────┼──────┼────────────┤
│ QHcUxCYpST │ edge-router-1 │ true │ true │ 0 │ │
│ Z7.Z9.pswh │ edge-router-2 │ false │ true │ 0 │ │
╰────────────┴───────────────┴────────┴───────────────┴──────┴────────────╯
results: 1-2 of 2
Thank you for your continued support! I really appreciate it.
Best,
Ajay
This means the clean command might not actually be cleaning the state directory. If not, then you will need to clean it manually. I'm guessing the --what=state option doesn't exist in your version of systemd.
sudo systemctl disable --now ziti-router.service
sudo systemctl reset-failed ziti-router.service
sudo systemctl clean --what=state ziti-router.service || sudo rm -rfv /var/lib/private/ziti-router
sudo ls -lR /var/lib/private/ziti-router
The final ls command verifies the state directory is empty or does not exist.
With a successfully cleaned state, you can proceed to enroll.
It works now! Thank you so much!! I was assuming it'd overwrite the existing files, my bad.
Now, I'm able verify that both the routers are online and are connected as a fabric
ajay@ajay-ubuntu:~/ziti_codes$ ziti edge list edge-routers
╭────────────┬───────────────┬────────┬───────────────┬──────┬────────────╮
│ ID │ NAME │ ONLINE │ ALLOW TRANSIT │ COST │ ATTRIBUTES │
├────────────┼───────────────┼────────┼───────────────┼──────┼────────────┤
│ QHcUxCYpST │ edge-router-1 │ true │ true │ 0 │ │
│ jtDaiw6-g │ edge-router-2 │ true │ true │ 0 │ │
╰────────────┴───────────────┴────────┴───────────────┴──────┴────────────╯
results: 1-2 of 2
ajay@ajay-ubuntu:~/ziti_codes$
ajay@ajay-ubuntu:~/ziti_codes$ ziti fabric list links
╭────────────────────────┬───────────────┬───────────────┬─────────────┬─────────────┬─────────────┬───────────┬────────┬───────────╮
│ ID │ DIALER │ ACCEPTOR │ STATIC COST │ SRC LATENCY │ DST LATENCY │ STATE │ STATUS │ FULL COST │
├────────────────────────┼───────────────┼───────────────┼─────────────┼─────────────┼─────────────┼───────────┼────────┼───────────┤
│ 5eMMkg5TAnln5yZMNF74lW │ edge-router-2 │ edge-router-1 │ 1 │ 3.5ms │ 4.4ms │ Connected │ up │ 8 │
╰────────────────────────┴───────────────┴───────────────┴─────────────┴─────────────┴─────────────┴───────────┴────────┴───────────╯
results: 1-1 of 1
Up next, I will connect another router and will try the switchover mechanism between routers to have a consistent data stream
Thanks again!
Best,
Ajay
1 Like
Hi,
I moved the controller to AWS EC2 instance and I'm trying to deploy an edge router on my local machine. Everything worked pretty well and the router status is 'online' but when I check the status of the systemctl service I get the 'bind: permission denied" error.
ubuntu@ubuntu-ATOPNUC-MA90:~/ziti_codes_backup$ sudo systemctl status ziti-router.service
● ziti-router.service - OpenZiti Router
Loaded: loaded (/lib/systemd/system/ziti-router.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/ziti-router.service.d
└─override.conf
Active: active (running) since Fri 2024-11-15 21:22:11 EST; 2s ago
Process: 82093 ExecStartPre=/opt/openziti/etc/router/entrypoint.bash check config.yml (code=exited, status=0/SUCCESS)
Main PID: 82099 (ziti)
Tasks: 8 (limit: 8776)
Memory: 17.8M
CPU: 265ms
CGroup: /system.slice/ziti-router.service
└─82099 /opt/openziti/bin/ziti router run config.yml --extend
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"file":"github.com/openziti/ziti/router/xgress_edge_tunnel/tunneler.go:71","func":"github.com/openziti/zi>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"file":"github.com/openziti/ziti/router/xgress_edge/certchecker.go:124","func":"github.com/openziti/ziti/>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"error":"exit status 1","file":"github.com/openziti/ziti/tunnel/dns/server.go:57","func":"github.com/open>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"file":"github.com/openziti/ziti/tunnel/dns/server.go:89","func":"github.com/openziti/ziti/tunnel/dns.New>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"error":"dns server failed to start: listen udp 127.0.0.1:53: bind: permission denied","file":"github.com>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"file":"github.com/openziti/ziti/tunnel/dns/dummy.go:37","func":"github.com/openziti/ziti/tunnel/dns.NewD>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"file":"github.com/openziti/ziti/tunnel/intercept/iputils.go:51","func":"github.com/openziti/ziti/tunnel/>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"channel":"ctrl","file":"github.com/openziti/ziti/router/handler_edge_ctrl/extendEnrollmentCerts.go:126",>
Nov 15 21:22:12 ubuntu-ATOPNUC-MA90 ziti[82099]: {"file":"github.com/openziti/ziti/router/xgress_edge/certchecker.go:124","func":"github.com/openziti/ziti/>
Nov 15 21:22:13 ubuntu-ATOPNUC-MA90 ziti[82099]: {"file":"github.com/openziti/ziti/router/state/apiSessionAdded.go:124","func":"github.com/openziti/ziti/ro>
ubuntu@ubuntu-ATOPNUC-MA90:~/ziti_codes_backup$
ubuntu@ubuntu-ATOPNUC-MA90:~/ziti_codes_backup$ cat /etc/hosts
127.0.0.1 localhost
#127.0.1.1 ubuntu-ATOPNUC-MA90
192.168.1.83 ziti_edge_router_1
3.141.153.46 ziti_controller
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
Previously, I got this error when I was trying to deploy an edge router in the same network but simply deleting and re-deploying the router fixed it. But now the issue seems to be persistent even after re-deploying. Any suggestions on how to resolve this issue?
Thanks,
Ajay
This problem was fixed in v1.2.0 of the ziti binary which is available but has not yet been promoted to "stable" status in the Linux package repository.
I think your best option is to be an early adopter and download v1.2.0 from GitHub.
(set -euxo pipefail
curl -sSLf https://github.com/openziti/ziti/releases/download/v1.2.0/ziti-linux-$(dpkg --print-architecture)-1.2.0.tar.gz | tar -zx \
&& sudo install --mode 0755 ./ziti /opt/openziti/bin/ziti \
&& ZITI_VERSION=$(ziti version) \
&& [[ "${ZITI_VERSION}" == v1.2.0 ]] \
&& sudo systemctl restart ziti-router.service \
|| echo "ERROR"
)
should print
+/usr/bin/zsh:32> tar -zx
+/usr/bin/zsh:32> dpkg --print-architecture
+/usr/bin/zsh:32> curl -sSLf https://github.com/openziti/ziti/releases/download/v1.2.0/ziti-linux-amd64-1.2.0.tar.gz
+/usr/bin/zsh:33> sudo install --mode 0755 ./ziti /opt/openziti/bin/ziti
+/usr/bin/zsh:34> ZITI_VERSION=+/usr/bin/zsh:34> ziti version
+/usr/bin/zsh:34> ZITI_VERSION=v1.2.0
+/usr/bin/zsh:35> [[ v1.2.0 == v1.2.0 ]]
+/usr/bin/zsh:36> sudo systemctl restart ziti-router.service
Hi @qrkourier ,
That worked great! I used the same steps to install two edge routers on two Linux devices connected locally with the controller on AWS EC2 instance.
Also, I have a python program that uses Flazk (with ziti sdk) to stream data running on the device with edge router 1 and a client program that uses requests (with ziti sdk) to receive the data. (figure below)
My goal is to ensure seamless data flow when an edge router fails. The ideal setup would have 2 edge routers on 2 different networks. So, if one network fails, the data would still flow via the other edge router.
To simulate this failover, I'm streaming the data from server and while the client is receiving the data, I will stop the edge_router_1 or edge_router_2 systemctl service and the data should flow via the active edge router because of smart-routing.
Am I doing this right? Also, how do I verify through which edge router the data is currently flowing? I tried using the 'stream circuits' but I got the below output
Thank you for you continued guidance and support!
-Ajay
1 Like
Hi,
I'm able to stream the data and test the smart-routing using ziti fabric stream events.
thivish@thivish-XPS-8960:~$ ziti fabric stream events
{"namespace":"edge.entityCounts","timestamp":"2024-12-10T17:41:55.035539324Z","counts":{"apiSessionCertificates":9,"apiSessions":14,"authPolicies":1,"authenticators":3,"cas":0,"configTypes":5,"configs":0,"controllers":0,"edgeRouterPolicies":3,"enrollments":0,"eventualEvents":0,"externalJwtSigners":0,"identities":5,"identityTypes":2,"mfas":0,"postureCheckTypes":5,"postureChecks":0,"revocations":0,"routers":2,"routers.edge":2,"serviceEdgeRouterPolicies":1,"servicePolicies":2,"services":1,"services.edge":1,"sessions":2,"terminators":2},"error":""}
{"namespace":"fabric.circuits","version":2,"event_type":"created","circuit_id":"Ww55V5vsU","timestamp":"2024-12-10T17:41:55.079348754Z","client_id":"cm4ir0n42vlwisvnvwd2kuog1","service_id":"6ONBj74fU5eZa9NaOLVMEx","terminator_id":"1qXGVk7WxCVAVrbYS20yMd","instance_id":"","creation_timespan":43187739,"path":{"nodes":["fWkCotzhIu"],"links":null,"ingress_id":"3WoY","egress_id":"9Lqy"},"link_count":0,"path_cost":262140,"tags":{"clientId":"3vbp6TFh-u","hostId":"FERG29Fl-","serviceId":"6ONBj74fU5eZa9NaOLVMEx"}}
{"namespace":"fabric.circuits","version":2,"event_type":"deleted","circuit_id":"Ww55V5vsU","timestamp":"2024-12-10T17:41:55.130997497Z","client_id":"cm4ir0n42vlwisvnvwd2kuog1","service_id":"6ONBj74fU5eZa9NaOLVMEx","terminator_id":"1qXGVk7WxCVAVrbYS20yMd","instance_id":"","path":{"nodes":["fWkCotzhIu"],"links":null,"ingress_id":"3WoY","egress_id":"9Lqy"},"link_count":0,"duration":51651917,"tags":{"clientId":"3vbp6TFh-u","hostId":"FERG29Fl-","serviceId":"6ONBj74fU5eZa9NaOLVMEx"}}
thivish@thivish-XPS-8960:~$ ziti fabric stream events
{"namespace":"edge.entityCounts","timestamp":"2024-12-10T17:42:11.035320379Z","counts":{"apiSessionCertificates":9,"apiSessions":14,"authPolicies":1,"authenticators":3,"cas":0,"configTypes":5,"configs":0,"controllers":0,"edgeRouterPolicies":3,"enrollments":0,"eventualEvents":0,"externalJwtSigners":0,"identities":5,"identityTypes":2,"mfas":0,"postureCheckTypes":5,"postureChecks":0,"revocations":0,"routers":2,"routers.edge":2,"serviceEdgeRouterPolicies":1,"servicePolicies":2,"services":1,"services.edge":1,"sessions":2,"terminators":2},"error":""}
{"namespace":"fabric.circuits","version":2,"event_type":"created","circuit_id":"N9jRVH3P2","timestamp":"2024-12-10T17:42:11.188383637Z","client_id":"cm4ir0n42vlwisvnvwd2kuog1","service_id":"6ONBj74fU5eZa9NaOLVMEx","terminator_id":"1tnMKYkbNMzcGqmggicudY","instance_id":"","creation_timespan":41423129,"path":{"nodes":[".4lz8tzl-u"],"links":null,"ingress_id":"3qXG","egress_id":"a5qP"},"link_count":0,"path_cost":262140,"tags":{"clientId":"3vbp6TFh-u","hostId":"FERG29Fl-","serviceId":"6ONBj74fU5eZa9NaOLVMEx"}}
{"namespace":"fabric.circuits","version":2,"event_type":"deleted","circuit_id":"N9jRVH3P2","timestamp":"2024-12-10T17:42:11.236519927Z","client_id":"cm4ir0n42vlwisvnvwd2kuog1","service_id":"6ONBj74fU5eZa9NaOLVMEx","terminator_id":"1tnMKYkbNMzcGqmggicudY","instance_id":"","path":{"nodes":[".4lz8tzl-u"],"links":null,"ingress_id":"3qXG","egress_id":"a5qP"},"link_count":0,"duration":48139679,"tags":{"clientId":"3vbp6TFh-u","hostId":"FERG29Fl-","serviceId":"6ONBj74fU5eZa9NaOLVMEx"}}
Currently, I'm checking the "nodes" to check the edge router ID through which the data is being passed. But, is there a better way to know which routers are in use for this stream of data?
Thank you,
Ajay
I think this would be better as a new top-level topic. Would you mind?
1 Like
Sure, I just created a new topic.
Thank you,
Ajay