Oh geeze, this is a bit nutty and I almost certainly did something wrong somewhere. These are longer, so I threw them on pastebin.
ziti-router log: ziti-router - Pastebin.com
ziti-controller log: ziti-controller - Pastebin.com
So I followed the log (journalctl -u ziti-router --no-pager -f) and noticed it was giving errors connecting to the controller (roughly at 10:36:17 in this log. So I shut down the router, shut down the controller, turned on the controller, turned on the router, and it looks working now.
10:36:17 in the router log:
May 01 10:36:17 ran ziti-router[27060]: {"endpoint":"tls:ziti.broken-mirror.net:6262","error":"error connecting ctrl (context deadline exceeded)","file":"github.com/openziti/fabric@v0.22.87/router/env/ctrls.go:125","func":"github.com/openziti/fabric/router/env.(*networkControllers).connectToControllerWithBackoff.func2","level":"error","msg":"unable to connect controller","time":"2023-05-01T10:36:17.266Z"}
May 01 10:36:19 ran ziti-router[27060]: {"endpoint":"tls:ziti.broken-mirror.net:6262","error":"error connecting ctrl (context deadline exceeded)","file":"github.com/openziti/fabric@v0.22.87/router/env/ctrls.go:125","func":"github.com/openziti/fabric/router/env.(*networkControllers).connectToControllerWithBackoff.func2","level":"error","msg":"unable to connect controller","time":"2023-05-01T10:36:19.155Z"}
May 01 10:36:21 ran ziti-router[27060]: {"endpoint":"tls:ziti.broken-mirror.net:6262","error":"error connecting ctrl (context deadline exceeded)","file":"github.com/openziti/fabric@v0.22.87/router/env/ctrls.go:125","func":"github.com/openziti/fabric/router/env.(*networkControllers).connectToControllerWithBackoff.func2","level":"error","msg":"unable to connect controller","time":"2023-05-01T10:36:21.053Z"}
May 01 10:36:24 ran ziti-router[27060]: {"endpoint":"tls:ziti.broken-mirror.net:6262","error":"error connecting ctrl (context deadline exceeded)","file":"github.com/openziti/fabric@v0.22.87/router/env/ctrls.go:125","func":"github.com/openziti/fabric/router/env.(*networkControllers).connectToControllerWithBackoff.func2","level":"error","msg":"unable to connect controller","time":"2023-05-01T10:36:24.129Z"}
At the same time in the controller:
May 01 10:36:17 ran ziti-controller[3327]: {"_context":"tls:0.0.0.0:6262","file":"github.com/openziti/channel/v2@v2.0.58/classic_listener.go:154","func":"github.com/openziti/channel/v2.(*classicListener).acceptConnection.func1","level":"error","msg":"error receiving hello from [tls:127.0.0.1:54728] (receive error (EOF))","time":"2023-05-01T10:36:17.076Z"}
May 01 10:36:18 ran ziti-controller[3327]: {"_context":"tls:0.0.0.0:6262","file":"github.com/openziti/channel/v2@v2.0.58/classic_listener.go:154","func":"github.com/openziti/channel/v2.(*classicListener).acceptConnection.func1","level":"error","msg":"error receiving hello from [tls:127.0.0.1:54736] (receive error (EOF))","time":"2023-05-01T10:36:18.325Z"}
May 01 10:36:20 ran ziti-controller[3327]: {"_context":"tls:0.0.0.0:6262","file":"github.com/openziti/channel/v2@v2.0.58/classic_listener.go:154","func":"github.com/openziti/channel/v2.(*classicListener).acceptConnection.func1","level":"error","msg":"error receiving hello from [tls:127.0.0.1:54752] (receive error (read tcp 127.0.1.1:6262-\u003e127.0.0.1:54752: read: connection reset by peer))","time":"2023-05-01T10:36:20.264Z"}
May 01 10:36:22 ran ziti-controller[3327]: {"_context":"tls:0.0.0.0:6262","file":"github.com/openziti/channel/v2@v2.0.58/classic_listener.go:154","func":"github.com/openziti/channel/v2.(*classicListener).acceptConnection.func1","level":"error","msg":"error receiving hello from [tls:127.0.0.1:54762] (receive error (read tcp 127.0.1.1:6262-\u003e127.0.0.1:54762: i/o timeout))","time":"2023-05-01T10:36:22.124Z"}
I didn’t scroll up far enough to see this when I just glanced at the log output on service ziti-controller status, so that’s on me for not reading enough.
The controller was running and didn’t have any recurring errors except for that small block when the router tried to do stuff, so I’m not sure if I configured something wrong there. The restart of both the router and controller appears to have fixed it for now though, which makes me a bit wary that it might come back.
EDIT: Additional info: The most surprising thing is that this happened past a restart of the host raspberry pi in between setting up the quickstart and currently, which is why I’m a bit weirded out that restarting both services appears to have fixed it for now.
