Update Controller - Terminators are terminated

Support
1

I feel like I have been through this a few times before... I am assuming 1,115 times to be exact, and I am sure its 93.7% my fault for not remembering what to do all 1,114 times I did it, but whenever I update my controller and router (the ziti.exe) I get a slew of "Unknown Terminator" errors and I cannot reach any of my services.

{"file":"github.com/openziti/ziti/controller/network/router_messaging.go:516","func":"github.com/openziti/ziti/controller/network.(*terminatorValidationRespReceived).DeleteInvalid","level":"info","msg":"queuing terminator for delete","reason":"UnknownTerminator","routerId":".s7pr4Isqy","terminatorId":"O3QPd1lRCDK2gis2qy0D9","time":"2024-11-27T17:34:21.555Z"}

They look like that....

If Skynet knew how easy it was to kill off all the Terminators the future would not seem too bleak at least.

Anywho, please remind me what I need to do if nothing works after an update so I can access all my servers again and don't have to use my super secret back door to access everything.

I thought it was just "restart all the tunnellers" but that did not work.

Thanks!

Terminator
Terminator1000×500 77.9 KB

2

I have never seen "UnknownTerminator" myself. I actually don't know what that means. Are all of your services encountering this issue or is it just a certain few? Does the tunneler that is supposed to be hosting the service have any relevant errors in the logs?

My guess is that there are errors in the log that might help us understand what's happening.

3

@jeremy.tellier and I DM'ed for a bit and we ended up in a video call. During the call, we stopped both routers and controller, and brought the controller online, verified it started cleanly, then the public router and connectivity was restored.

@jeremy.tellier - please restart your private router and verify things are working after the upgrade.

Cheers

4

Oh, just as an added note as I am going to come back here and see what we did next time... I also had some oddities around terminators that seemed like they went away when we shut down the hosting tunnellers during this reboot as well because I could swear I had rebooted those routers before, that said, I also found that one of my routers was sooooper old too.