OpenZiti config schema and REST newb questions

Support General Questions
1

Hello, a few more OpenZiti newb questions :slight_smile:

  • I see references to host.v1, host.v2, and intercept.v1 for service configuration. I haven’t looked at the controller REST API swagger definition mentioned here yet, but do the v1 and v2 here denote use of a REST API version where the schema of these config service definitions match up with a corresponding REST version functionality which consumes the config?

  • Will there be an intercept.v2 coming soon, similar to thehost.v2?

  • Where is the best place to find documentation/information for the [host|intercept].v[1|2] schemas? Is just looking directly at the code definitions for the schema currently the best option? If so, in searching for refs for the config schema for these, I see related code in several different repos and languages. Is there one place in particular which should be considered authoritative or source of truth that I should be looking at to try and get a good handle on all the config options?

  • In searching the OpenZiti repos on these types of questions I see an issue which looks relevant to this sort of info I’m trying to find, here.

Thanks!
John

2

Hi @johnalotoski , we love newb questions! :slight_smile:

  • Documenting those configs is absolutely “on our list” of things to do. The v1/v2 of the APIs is probably more “in case we need it” than anything else. They were all mashed together in the past and when we did the work to separate them, I think the idea was along the lines of: “Hey, why don’t we just set ourselves up for change if need be”. Really you probably only want to grab the schema for the edge/v1 and management/v1 APIs and those correspond to the docs endpoint.

  • I don’t expect an intercept.v2 - but who knows! Really these are the configs that OpenZiti itself needs/users. You are very much encouraged to make your own once you start getting into the application embedded zero trust space. Our “tunnelers” use those configs, they’ve been pretty stable for a long time so unless something big we missed comes around, probably not. host.v1 is identical to host.v2 in practice. it’s just that host.v2 is an array of ‘host.v1’ structures. So it’s just a flexibility thing. There’s no great place for information other than reading the schema and code :confused: so imo - you’re best off asking questions until you grok what’s going on out there… Actually - maybe I’ll do a Ziti TV on exactly these structures tomorrow! It’ll be late since I have a meeting, but I think a Ziti TV covering these make a lot of sense. thanks for that idea!!! :slight_smile:

It really makes a lot more sense to just see an example or two. let me dig up a couple, and then i’ll post back.

1 Like
3

Ok. Here is where you can find the schemas that drive intercept.v1 and host.v1/host.v2:

Host.v1 / Host.v2

There are jsut a few key concepts you need to know about… When using one of these config types, you can choose to “forward” the protocol/address/port or not. Those do exactly what they sound like they do. Usually forwarding the port/protocol makes sense “always” and fwd’ing the address is something I find I do infrequently myself. Usually I’ll be intercepting something like “my.service.ziti” and sending it over to an IP on the far side (127.0.0.1 for example). But when you start doing IP intercepts or “wildcard domain” intercepts, then you’ll be forwarding the address too… I think that makes sense, but if not follow up.

here’s a host.v1 config, which has a few examples

{
  "forwardProtocol": true,
  "allowedProtocols": [
    "tcp",
    "udp"
  ],
  "forwardAddress": true,
  "allowedAddresses": [
    "172.20.0.0/16"
  ],
  "forwardPort": true,
  "allowedPortRanges": [
    {
      "low": 1,
      "high": 32768
    }
  ]
}

Here’s a host.v2 example:

{
    "terminators" : [
      {
        "forwardProtocol":true, 
        "allowedProtocols":["tcp","udp"], 
        "forwardAddress":true, 
        "allowedAddresses":["*.bluesite","192.168.0.0/24"], 
        "forwardPort":true, 
        "allowedPortRanges":[ {"low":80, "high":80},{"low":8000,"high":8000}]
      },
      {
        "forwardProtocol":true, 
        "allowedProtocols":["tcp","udp"], 
        "forwardAddress":true, 
        "allowedAddresses":["*.redsite","10.0.0.0/16"], 
        "forwardPort":true, 
        "allowedPortRanges":[ {"low":80, "high":80},{"low":8000,"high":8000}]
      }      
    ]
}

Intercept.v1

Here’s an intercept.v1 config example I used in the prometheus series of articles… The important bit here that’s “non obvious” is the dialOptions bit. I’d say leave that out for now and don’t even use it yet. That’s a “more advanced” feature which allows you to specify the actual identity you want to dial. SUPER cool stuff - but it’s a lot easier imo to just stay with “underlay”-related stuff when starting out. When you’re ready to learn about ‘addressable terminators’ though - we’ll be here waiting for that newb question too! And who knows - perhaps we’ll document it before you get to ask teh question! :slight_smile:

{
  "protocols": [
    "tcp"
  ],
  "addresses": [
    "kubeA.reflect.scrape.svc.ziti"
  ],
  "portRanges": [
    {
      "low": 80,
      "high": 80
    }
  ],
  "dialOptions": {
    "identity": "kubeA.reflect.id"
  }
}
1 Like
4

Oh, and if you’re interested - here’s a video where I show people how to use wildcard DNS

The video’s description has all the CLI commands I ran, including an intercept.v1 and host.v1 config

1 Like
5

Awesome, thank you! I’ll get a chance to watch the windows wildcard video probably tonight. I’ll be sure to check out the upcoming Youtube Ziti TV video once it’s available as well.

Appreciate the schema links and examples – the distinction between v1/v2 on hosts schema is very clear now too. My primary interest in seeing the schemas themselves was to review if there were other options available I hadn’t seen in the examples which might be of interest, so that I had the full picture, so to speak.

In the github documentation issue linked above, it mentions 5 standard config types. If I assume the intercept and host config count as 2, are the other 3 types the controller, edge router and fabric router configs?

If so, it looks like the best reference for those additional 2 config types are already provided in the OpenZiti docs as:

https://openziti.github.io/ziti/manage/sample-controller-config.yaml
https://openziti.github.io/ziti/manage/sample-edge-router-config.yaml

I’m not immediately finding the link for the fabric router example in the web docs, although I could just be overlooking it, but looks like I can just edit the url to follow the naming pattern and there it is as well:

https://openziti.github.io/ziti/manage/sample-fabric-router-config.yaml

Onward with the learning!
:slight_smile:
John

6

host.v1, host.v2, intercept.v1 == 3. then there are the older/legacy ones: ziti-tunneler-client.v1 and ziti-tunneler-server.v1 round out the 5. You can see those listed in the ZAC when you make a new service:
image

For example configurations, I’d really recommend you fire up the docker compose example and peek at those. There you’ll find a “private” router example, a “transit” router example and a “public” router example. The differences being if parts are “edge” enabled or listening on links… It occurred to me that I can just grab them for you though so I did that…

I started up the docker-compose example and copied them out of the container for you… Once started I ran:

docker cp docker_ziti-controller_1:/openziti/ziti-edge-router.yaml /mnt/v/temp
docker cp docker_ziti-controller_1:/openziti/ziti-fabric-router-br.yaml /mnt/v/temp
docker cp docker_ziti-controller_1:/openziti/ziti-private-blue.yaml /mnt/v/temp

These configs are for the “topmost” routers show on: the docker compose image

hpoe that helps?

ziti-edge-router.yaml

v: 3

identity:
  cert:                 "/openziti/pki/routers/ziti-edge-router/client.cert"
  server_cert:          "/openziti/pki/routers/ziti-edge-router/server.cert"
  key:                  "/openziti/pki/routers/ziti-edge-router/server.key"
  ca:                   "/openziti/pki/routers/ziti-edge-router/cas.cert"

ctrl:
  endpoint:             tls:ziti-controller:6262

link:
  dialers:
    - binding: transport
  listeners:
    - binding:          transport
      bind:             tls:0.0.0.0:10080
      advertise:        tls:ziti-edge-router:10080
      options:
        outQueueSize:   4

listeners:
# bindings of edge and tunnel requires an "edge" section below
  - binding: edge
    address: tls:0.0.0.0:3022
    options:
      advertise: ziti-edge-router:3022
      connectTimeoutMs: 1000
      getSessionTimeout: 60s
  - binding: tunnel
    options:
      mode: host #tproxy|host


edge:
  csr:
    country: US
    province: NC
    locality: Charlotte
    organization: NetFoundry
    organizationalUnit: Ziti
    sans:
      dns:
        - ziti-edge-router
        - localhost
      ip:
        - "127.0.0.1"

#transport:
#  ws:
#    writeTimeout: 10
#    readTimeout: 5
#    idleTimeout: 5
#    pongTimeout: 60
#    pingInterval: 54
#    handshakeTimeout: 10
#    readBufferSize: 4096
#    writeBufferSize: 4096
#    enableCompression: true
#    server_cert: /openziti/pki/routers/ziti-edge-router/server.cert
#    key: /openziti/pki/routers/ziti-edge-router/server.key

forwarder:
  latencyProbeInterval: 10
  xgressDialQueueLength: 1000
  xgressDialWorkerCount: 128
  linkDialQueueLength: 1000
  linkDialWorkerCount: 32

ziti-fabric-router-br.yaml

v: 3

identity:
  cert:                 "/openziti/pki/routers/ziti-fabric-router-br/client.cert"
  server_cert:          "/openziti/pki/routers/ziti-fabric-router-br/server.cert"
  key:                  "/openziti/pki/routers/ziti-fabric-router-br/server.key"
  ca:                   "/openziti/pki/routers/ziti-fabric-router-br/cas.cert"

ctrl:
  endpoint:             tls:ziti-controller:6262

link:
  dialers:
    - binding: transport
  listeners:
    - binding:          transport
      bind:             tls:0.0.0.0:10080
      advertise:        tls:ziti-fabric-router-br:10080
      options:
        outQueueSize:   4

#listeners:
# bindings of edge and tunnel requires an "edge" section below
#  - binding: edge
#    address: tls:0.0.0.0:3022
#    options:
#      advertise: ziti-fabric-router-br:3022
#      connectTimeoutMs: 1000
#      getSessionTimeout: 60s
#  - binding: tunnel
#    options:
#      mode: host #tproxy|host


csr:
  country: US
  province: NC
  locality: Charlotte
  organization: NetFoundry
  organizationalUnit: Ziti
  sans:
    dns:
      - ziti-fabric-router-br
      - localhost
    ip:
      - "127.0.0.1"

#transport:
#  ws:
#    writeTimeout: 10
#    readTimeout: 5
#    idleTimeout: 5
#    pongTimeout: 60
#    pingInterval: 54
#    handshakeTimeout: 10
#    readBufferSize: 4096
#    writeBufferSize: 4096
#    enableCompression: true
#    server_cert: /openziti/pki/routers/ziti-fabric-router-br/server.cert
#    key: /openziti/pki/routers/ziti-fabric-router-br/server.key

forwarder:
  latencyProbeInterval: 10
  xgressDialQueueLength: 1000
  xgressDialWorkerCount: 128
  linkDialQueueLength: 1000
  linkDialWorkerCount: 32

ziti-private-blue.yaml

v: 3

identity:
  cert:                 "/openziti/pki/routers/ziti-private-blue/client.cert"
  server_cert:          "/openziti/pki/routers/ziti-private-blue/server.cert"
  key:                  "/openziti/pki/routers/ziti-private-blue/server.key"
  ca:                   "/openziti/pki/routers/ziti-private-blue/cas.cert"

ctrl:
  endpoint:             tls:ziti-controller:6262

link:
  dialers:
    - binding: transport
#  listeners:
#    - binding:          transport
#      bind:             tls:0.0.0.0:10080
#      advertise:        tls:ziti-private-blue:10080
#      options:
#        outQueueSize:   4

listeners:
# bindings of edge and tunnel requires an "edge" section below
  - binding: edge
    address: tls:0.0.0.0:3022
    options:
      advertise: ziti-private-blue:3022
      connectTimeoutMs: 1000
      getSessionTimeout: 60s
  - binding: tunnel
    options:
      mode: host #tproxy|host


edge:
  csr:
    country: US
    province: NC
    locality: Charlotte
    organization: NetFoundry
    organizationalUnit: Ziti
    sans:
      dns:
        - ziti-private-blue
        - localhost
      ip:
        - "127.0.0.1"

#transport:
#  ws:
#    writeTimeout: 10
#    readTimeout: 5
#    idleTimeout: 5
#    pongTimeout: 60
#    pingInterval: 54
#    handshakeTimeout: 10
#    readBufferSize: 4096
#    writeBufferSize: 4096
#    enableCompression: true
#    server_cert: /openziti/pki/routers/ziti-private-blue/server.cert
#    key: /openziti/pki/routers/ziti-private-blue/server.key

forwarder:
  latencyProbeInterval: 10
  xgressDialQueueLength: 1000
  xgressDialWorkerCount: 128
  linkDialQueueLength: 1000
  linkDialWorkerCount: 32
1 Like
7

Yes, all very helpful! I’m good to chew on all this for awhile. Thanks!

8

Corresponding Ziti TV video for this thread: Ziti TV Oct 07 - Config Types - YouTube

Learned several more things. Thanks!

1 Like
9

And the example configs as emitted from the ziti edge show config ${id} command:

ziti-tunneler-server.v1

{
    "hostname": "localhost",
    "port": 54321,
    "protocol": "tcp"
}

ziti-tunneler-client.v1

{
    "hostname": "ken.nc.ziti",
    "port": 12345
}

host.v1

{
    "address": "localhost",
    "port": 54321,
    "protocol": "tcp"
}

intercept.v1

{
    "addresses": [
        "ken.nc.intercept.v1a",
        "ken.nc.intercept.v1",
        "ken.netcat.intercept",
        "fee.fi.fo",
        "10.10.10.10/32",
        "192.168.100.0/24"
    ],
    "dialOptions": {
        "connectTimeoutSeconds": 30,
        "identity": ""
    },
    "portRanges": [
        {
            "high": 1000,
            "low": 100
        },
        {
            "high": 55000,
            "low": 3000
        }
    ],
    "protocols": [
        "tcp"
    ],
    "sourceIp": ""
}
10

What does .dialOptions.identity do in an intercept.v1 config?

11

it allows you to very specifically control what identity will get dialed instead of letting the controller figure it out for you. You can indicate that you specifically want to dial this identity. You must use it with “listenOptions.bindUsingEdgeIdentity” or “listenOptions.identity” and while it works with tunnelers, I’d say it’s probably more useful for SDK solutions where you’re the one writing the client/server code.

You can absolutely make it work with tunnelers, but from what I’ve seen, it’s not all that useful (happy to be corrected about that too from anyone else).

zssh makes use of dialOptions.identity:

Once you understand that - the zssh cheatsheet will make more sense when you see how it makes the configs:

ziti edge create config "${service_name}"-host.v1 host.v1 '{"protocol":"tcp", "address":"localhost","port":'"${the_port}"', "listenOptions": {"bindUsingEdgeIdentity":true}}'

# intercept is not needed for zscp/zssh but make it for testing if you like
ziti edge create config "${service_name}"-client-config intercept.v1 '{"protocols":["tcp"],"addresses":["'"${service_name}.ziti"'"], "portRanges":[{"low":'"${the_port}"', "high":'"${the_port}"'}]}'
12

I think this is pretty useful, as I demonstrated on ZitiTV with the multi-region AKS API access.

"dialOptions": {
            "identity": "$dst_hostname"
        }
...
"listenOptions": {
            "identity": "$tunneler_id.name"
        },
13

I still need to watch that. I keep forgetting about the variables! For those interested here’s the video

14

Hello, I’m working through the controller configuration for an initial set up and see some discrepancies between the sample controller config given in the docs (link here), and the quickstart controller config.

The quickstart controller config (file: ziti-edge-controller.yaml) appears significantly more comprehensive, but there are also sections where the mentioned required config arguments don’t match up. For example, in the doc linked config, for edge → api config section, we have the following with two required configuration parameters (removing extra config comments from both to keep it short):

edge:
  api:
    # (required) The interface and port that the Edge API should be served on.
    listener:  127.0.0.1:1280
    # (required) The host/port combination that is reported as publicly accessible for the Edge API
    advertise: localhost:1280

But in the equivalent quickstart config, the edge → api section has neither of the required parameters mentioned above; and conversely the sample doc config doesn’t have the required address parameter shown here:

edge:
  api:
    sessionTimeout: 30m
    # address - required
    # The default address (host:port) to use for enrollment for the Client API. This value must match one of the addresses
    # defined in this Controller.WebListener.'s bindPoints.
    address: ziti-edge-controller:1280

It looks to me like maybe the document linked config is outdated and the newer config example is found in the quickstart? @TheLumberjack, I know you mentioned above to take a look at the quickstart config which I’m doing, but I would also assume the other OpenZiti doc linked config example is still valid config too since people are being pointed to it, right?

In examining only one each of these config documents, one would only know that only postgres exclusively or file based boltDb exclusively (if I’m interpreting what I’m seeing correctly) is supported for the controller storage backend, which is why I like to look at a single source of truth of latest documented config schemas, or alternatively, raw schemas to see all the options available before investing time trying to configure something that I might have done differently if I knew all the currently available options at the start, or might have configured incorrectly if I happened to find and use an invalid/out-of-date config without realizing it.

Similar to the questions in the thread above, I’ll ask and answer what I think are correct answers to some similar questions. Please correct me where I’m wrong! :slight_smile:

❯ fd options fabric/controller
fabric/controller/api_impl/query_options.go
fabric/controller/network/options.go

Oh, it actually looks like postgres config options were removed from at least some of the components a couple years ago back around Ziti release 0.15.0, and boltDb has been default since release 0.20.10 or so? If so, then the linked controller sample document is quite outdated and would no longer work at this point, right?

15

for starters, those example configs are being removed “very soon” (see update/remove example configs · Issue #152 · openziti/ziti-doc · GitHub also). Keeping them updated has been hard. We need to rethink how we document it, and for now they are doing more harm than good being out of date. I hope nobody pointed you to the example other than ‘finding it in the doc’… The docs are all getting updated right now so this issue has been just waiting to be fixed. Sorry about that :frowning: I had pointed you at the quickstart because I know that it’s more up to date, and also now able to be kept up to date when/if a change occurs. I just removed those nodes right now from the main doc site. They’ll be removed soon when the actions run through and the site redeploys.

  1. Yes, I consdier “the source of truth” the quickstarts.

  2. The code is meant to be that place. You can find them for the controller and routers in the code. They are what the quickstart uses to generate its output

    –EDIT-- hit ‘post’ too early… :expressionless:

  3. Is there raw or code based config schema available for the controller config? No there is no schema for the config. It’s processed by our own parser.

1 Like
16

No worries! The only thing pointing me to the outdated config was the github openziti docs which provided a handy link to it in the left sidebar menu somewhere in the “Manage > Controller” section. Looks like it’s already received your update to the newer configs! :slight_smile:

That config template is perfect for what I’m looking for, thanks!

John