Ziti fabric list reports an invalid circuit_id?

Ziti Overlay
1

After rebooting the ziti-controller I observe the following:

$ ziti fabric stream events --usage | grep 17pXhmLqsvzJz0vmOFkW8U | jq -c '[.timestamp,.circuit_id,.tags.serviceId]'
["2025-08-04T10:14:10.492986393Z","U1Fk47yTr","17pXhmLqsvzJz0vmOFkW8U"]
["2025-08-04T10:14:10.492990459Z","U1Fk47yTr","17pXhmLqsvzJz0vmOFkW8U"]


$ ziti fabric list circuits 'limit none' | grep U1Fk47yTr | wc -l
0

$ ziti fabric stream events --usage | grep 17pXhmLqsvzJz0vmOFkW8U | jq -c '[.timestamp,.circuit_id,.tags.serviceId]'
["2025-08-04T10:17:57.393085608Z","U1Fk47yTr","17pXhmLqsvzJz0vmOFkW8U"]
["2025-08-04T10:18:10.492855693Z","3T6ecUXOm","17pXhmLqsvzJz0vmOFkW8U"]



$ ziti fabric list circuits 'limit none' | grep U1Fk47yTr | wc -l
0

The service 17pXhmLqsvzJz0vmOFkW8U is in use, but ziti fabric list circuits does not report any circuits on this service.

Apparently the circuit U1Fk47yTr has been replaced by another one with a different circuit_id.

I have restarted the service. It looks correct now:

$ ziti fabric stream events --usage | grep 17pXhmLqsvzJz0vmOFkW8U | jq -c '[.timestamp,.circuit_id,.tags.serviceId]'
["2025-08-04T10:42:37.883018422Z","EfXEWEXRm","17pXhmLqsvzJz0vmOFkW8U"]
["2025-08-04T10:42:54.838297688Z","ljbJkE4Om","17pXhmLqsvzJz0vmOFkW8U"]


$ ziti fabric list circuits 'limit none' | grep EfXEWEXRm | wc -l
1
$ ziti fabric list circuits 'limit none' | grep ljbJkE4Om | wc -l
1
2

Are you running in an HA cluster? If so, list circuits will only show circuits on the controller you’re connected to. Circuit information is not distributed.

Paul

3

There is only 1 ziti-controller. Zrok does not support 2 controllers yet.

You are right I run a single node HA configuration.

4

This is a tricky issue for a number of reasons.

  1. Usage data is collected in intervals and sent out after the interval is over, so you can see it arrive after the circuit is gone.
  2. A circuit will stay alive in the routers, as long it has a valid path and the end points don’t close the connection. If the controller goes down it will lose the information about the circuit, but the circuit will continue until the endpoints close it or something disrupts the path.

If you wanted to try and track this down more, here are some steps you could take:

  1. Enable file logging for circuit and usage events. That way you can see if a circuit delete event is generated and share the ordering of events.
  2. You can run ziti fabric inspect circuit <circuit id> to get information from the routers. That will show you forwarding tables and endpoints for the circuit and will verify that circuit is still active.

Hope that’s helpful,

Paul