Hi,
we plan an application where user’s could dynamically request access to services.
For this concept we also plan to track usage and do some ‘expiry’ of the requested access. Are there any hints on this?
The ‘closest’ thing i’ve found is fabric list circuits
ziti@ziti-controller-5db6867b48-gnfvl:/persistent$ ziti fabric list circuits
╭───────────┬───────────────────────────┬───────────────────────────────┬──────────────────────────────────────┬────────────────────╮
│ ID │ CLIENT │ SERVICE │ TERMINATOR │ PATH │
├───────────┼───────────────────────────┼───────────────────────────────┼──────────────────────────────────────┼────────────────────┤
│ 0OqhJAzoU │ cle3xd0o210oi01a828fi2bpt │ central.jumpy-guaca.svc │ 1ea7c0c6-c65b-4a30-995e-d4283f6341dd │ r/core-router │
│ 9xlKKkroU │ cleyd9hzf4oym01a82lhrdfo0 │ central.default.svc │ 461bdecc-12fc-4d6e-87ad-70b5ec910ee4 │ r/core-router │
│ DHpoJJrx8 │ cleyd9gsv4oyj01a8zqm9gs9s │ central.jumpy-guaca.svc │ 886bff38-4bf9-4df9-81b1-3256463148c8 │ r/core-router │
From here it seems I could get the details. But (especially for HTTP Browser/Rest requests) it seems to be short-lived - so I have to pull and parse it quite often. Is there a way to get this streamed? Or a way to specify a time frame, i.e. give me all circuits that have been active during the last 5 mins?
I assume i also can get this via API. But it seems the fabric API is currently not published: Fabric APIs | OpenZiti - is there a way to get the OpenAPI schema i.e. directly from the controller?
Maybe there is even a better option 
Thanks & Bye,
Chris
1 Like
The logging is quite extensive, including utilization logging that includes the data volume per circuit (Individual data connection) and the information to ascribe this back to an identity. You'll want to configure the logging options on the controller, then you can post process at will. There is also a Prometheus endpoint available for collection.
Look at the events section of the controller config documentation for more information on the logging configuration.
An example logging config that we use:
metrics:
reportInterval: 1m
events:
jsonLogger:
subscriptions:
- type: metrics
metricFilter: "link.latency."
- type: metrics
sourceFilter: .
metricFilter: "(.session.create.)|(bolt.)"
- type: metrics
sourceFilter: .
metricFilter: "/edge*"
- type: metrics
metricFilter: "ctrl.*"
handler:
type: file
format: json
path: /var/log/ziti/utilization-metrics.log
maxsizemb: 1024
maxbackups: 1
edgeSessionLogger:
subscriptions:
- type: edge.sessions
include:
- created
handler:
type: file
format: json
path: /var/log/ziti/utilization-edgesessions.log
maxsizemb: 1024
maxbackups: 1
utilizationLogger:
subscriptions:
- type: fabric.circuits
include:
- created
- failed
- type: fabric.usage
include:
- usage.ingress.tx
- usage.egress.tx
- usage.ingress.rx
- usage.egress.rx
handler:
type: file
format: json
path: /var/log/ziti/utilization-usage.log
maxsizemb: 1024
maxbackups: 10
eventsLogger:
subscriptions:
- type: edge.apiSessions
- type: fabric.routers
- type: services
- type: edge.entityCounts
- type: fabric.terminators
- type: fabric.links
handler:
type: file
format: json
path: /var/log/ziti/utilization-events.log
maxsizemb: 1024
maxbackups: 1
To follow up on what @mike.gorman wrote, the events are generally streamed to disk, where a separate process can do whatever needs to be done. They can also be streamed over websocket, however the websocket endpoint isn’t very useful because there’s no persistence for events so you can’t guarantee that you won’t miss events. It can be helpful for debugging, though. See ziti fabric stream events for more details.
We do have plans to add a plugin system for events (see Add event plugin system · Issue #561 · openziti/fabric · GitHub), which will hopefully allow for more flexibility in the future.
Cheers,
Paul
Hi. Thanks for the feedback - That is what I need!
Also the websocket endpoint sounds quite interesting. We could implement a small process connecting to it and forwarding the events to a message queue like kafka. So the event plugin’s would be a great thing
Is there any docu on the websocket interface? Or maybe just a quick pointer to the implementation in ziti console, I could extract it from there
Thanks a lot!
Chris
The CLI client code is here: ziti/stream_events.go at release-next · openziti/ziti · GitHub
I think it would be safer to have a process reading the events file and forwarding to Kafka, just because there are fewer opportunities for race conditions. Longer term, we’re hoping the event plugins system will let us do things like feed data to Kafka directly.
Cheers,
Paul
Is there a way for Ziti to perform a resolution before logging?
Similar to auditd's log_format=enriched, I am looking for a way to be able to get human readable information in the logs.
This would be especially useful for the fields identity_id and service_id.
The specific usecase for us is to log to our SIEM and detect e.g. possible violations for service use abuse and also create proof to log access to services.
There isn’t a current way to enrich the logging within OpenZiti. The information is all available via API, so an enrichment pipeline could be written. What SIEM are you using? Does it have enrichment pipelines or a processor of some sort that could add data gathered via the API to add to the stream?
1 Like
Thanks, yes, that would work. I didn’t know the API was this thorough. Which API should I use here?
/edge/client/v1/services ?
A fetch from the SIEM to the controller every e.g. 1 hour would be totally sufficient by using the Edge Router’s tunneler.
### Edit: ###
Found it, sorry about that: Session Types | OpenZiti
API Sessions are represented by opaque strings and are provided in the HTTP header zt-session and in Edge Router connection requests initiated by Ziti SDKs. API Sessions remain valid as long they have not timed out.
###
I'm just right now trying this out. How do I state the established session while making requests to the client API?
I was able to successfully authenticate to the API like here: Authentication | OpenZiti but I couldn't find where I'll have put the token in the upcoming requests. I assume it's going to be an HTTP header?
1 Like
You put the token in a ‘zt-session’ header. That;'s what that blurb is saying, perhaps just not clearly enough.
BTW, I posted some Grafana work at Using Grafana to view OpenZiti API's | OpenZiti It may move, so if that link doesn’t work, just search for Grafana.