# Declarative configuration with files

**URL:** <https://openziti.discourse.group/t/declarative-configuration-with-files/3393>\
**Category:** Ziti Overlay\
**Created:** [November 7, 2024, 12:40pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393 "2024-11-07T12:40:30Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 7, 2024, 12:40pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/1 "2024-11-07T12:40:30Z")

</div>

Hello everyone,  
is it possible to configure services, identities, policies, ... in files and the controller then configures “itself” based on configuration files?  
This would have several advantages:

1. deleting configuration files, would automatically remove the entities in the controller =\> declarative approach, no drift
2. the configuration directory e.g. `/config` could be passed read-only to the controller, allowing only changes to the files themselves

An example for better understanding would be if a “Simple Service”, as in ZAC, could only be defined via a YAML file and an associated identity could be created directly:

```yaml
# Config all-in-one
services:
  my-service.ziti:
    grant_access_to:
      - my-service.ziti
    intercept:
      hostname: my-service.ziti
      ports:
        - 80
        - 443
      protocols:
        - tcp
        - udp
    bind:
      to: @my-service-host
      hostname: traefik
      ports:
        - 80
        - 443
      protocol: tcp

identities:
  DemoLaptop:
# type: Default
# isAdmin: false
    roleAttributes:
      - my-service.ziti
    authPolicyId: default

```

In the best case, the configuration could then be split into several files and put flexible together using a template function:

```yaml
# Main-Config-file
services:
{{- range $file := glob "/config/*" }}
{{- with (index (fileContent $file | fromYaml) "services") }}
{{- . | toYaml | expandenv | nindent 2 }}
{{- end }}
{{- end }}

identities:
  DemoLaptop:
# type: Default
# isAdmin: false
    roleAttributes:
      - my-service.ziti
    authPolicyId: default

```

```yaml
# Example of a configuration file, e.g. for each host that hosts/provides services
services:
  my-service.ziti:
    grant_access_to:
      - my-service.ziti
    intercept:
      hostname: my-service.ziti
      ports:
        - 80
        - 443
      protocols:
        - tcp
        - udp
    bind:
      to: @my-service-host
      hostname: traefik
      ports:
        - 80
        - 443
      protocol: tcp

```

What do you think? Is that already possible? I'm looking forward to your opinions 🙂

---

<div class="post-metadata">

**Author:** ![jens](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/jens/32/507_2.png) [@jens](https://openziti.discourse.group/u/jens)\
**Post date:** [November 7, 2024, 1:11pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/2 "2024-11-07T13:11:13Z")

</div>

HI ZzenID,

I'm currently working on something very similar to what you are requesting. It is still a rough draft and i expect it will take a few more weeks before it's ready though.  
What I'm adding specifically is a sub command in the cli that lets you both export and import entities, either an entire network or just selected parts.

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 7, 2024, 1:31pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/3 "2024-11-07T13:31:28Z")

</div>

That sounds exciting. It would be cool if the configuration files could be written in YAML and the file changes will then be automatically mirrored into the controller.

This would also enable fully GitOps-based operation without external tools. And by the way, a config rollback to the previous (Git)version can be performed at any time 🙂

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [November 7, 2024, 2:56pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/4 "2024-11-07T14:56:32Z")

</div>

I want this, too, if declarative is defined as "an approach to managing configuration where the necessary operations to reconcile the desired state with the actual state are performed automatically."

Here are a few examples of declarative configuration management:

- Kubernetes "resources" are reconciled by a "controller"
- Terraform "resources" are reconcile by a "provider"
- Ansible "tasks" and Salt Stack "states" express a state that is reconciled on the target host with a "module"

Here's an overview of what's available and what's planned in the Ziti space, as far as I know.

Available now as a community contributed experiment:

- [Terraform provider](https://registry.terraform.io/providers/qrkourier/restapi/latest/docs) - a thin restapi wrapper around Ziti's mgmt API - declare a Ziti entity's desired state with an HCL representation of its respective mgmt API object and reconciling operations are performed synchronously when the calling module is applied.

Planned soon:

- CLI cloning tool helps create a new Ziti network that is a clone of an existing network by importing a complete set of entities (not for managing an existing network declaratively)

Planned eventually:

- Kubernetes Operator - manage Ziti entities as CRDs declaratively, asynchronously (this is the way, enables GitOps, etc.)

Nice to have, but no owner or plan:

- Collection of Ansible modules for Ziti mgmt

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 7, 2024, 5:35pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/5 "2024-11-07T17:35:39Z")

</div>

That sounds good. I see the advantage of pure config files in a directory that the controller reads and implements in the fact that no third-party provider or additional tool (Terraform, Ansible, ...) is required to implement the configuration.

OpenZiti can therefore be used without dependencies on third-party providers, only with standard tools such as cp, vi, ... can be managed.

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 13, 2024, 12:36pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/6 "2024-11-13T12:36:24Z")

</div>

I have now started to implement this myself and will share my code here as soon as it is running 🙂

But I still have two problems/questions:

1. How can I log in to the controller with curl and the admin user so that I get a valid `zt-session`-token?

2. Assuming I have created a new identity called “test-identity”, how can I read its `id` with curl?

Thanks for your help

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [November 13, 2024, 12:58pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/7 "2024-11-13T12:58:27Z")

</div>

are you looking to use curl exclusively or are you looking to leverage our golang code/sdks?

If you want to use curl, you can gert the zt-session a few ways... On this thread [I made a comment about making//using a new identity](https://openziti.discourse.group/t/q-generating-certificates-for-zac-without-using-docker-quickstart/3339/2). you can also use `ziti edge login` and then get the token:

```auto
ziti edge login "${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS}:${ZITI_CTRL_EDGE_ADVERTISED_PORT}" -u "${ZITI_USER-}" -p "${ZITI_PWD}" -y
Token: d579cae7-042e-4edf-b491-2f0c513a3e79
Saving identity 'default' to /home/ubuntu/.config/ziti/ziti-cli.json

```

If you're looking to use golang code, I'd have a look at how [I did `ziti ops verify-traffic`](https://github.com/openziti/ziti/blob/main/ziti/cmd/verify/ops_verify_traffic.go)

That give you enough to go on?

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 13, 2024, 1:37pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/8 "2024-11-13T13:37:02Z")

</div>

Thanks for the quick answer, I would like to log in with curl and username password (without certificates) and have a session\_token in the response. Is that possible?

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [November 13, 2024, 1:43pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/9 "2024-11-13T13:43:57Z")

</div>

sure...

```auto
export zt_session=$(curl -sk -H "Content-Type: application/json" \
    https://${ZITI_EDGE_CTRL_ADVERTISED}/authenticate?method=password \
    -d "{\"username\":\"${ZITI_USER}\",\"password\":\"${ZITI_PWD}\"}" \
    | jq -j .data.token)

```

Referenced from this thread (with a video if interested) from a few days ago: [Issues with enrolling on windows desktop edge && 3rd Party CA issues - #2 by TheLumberjack](https://openziti.discourse.group/t/issues-with-enrolling-on-windows-desktop-edge-3rd-party-ca-issues/3403/2)

Then you use it how you wish, like listing /cas as in that post:

```auto
curl -sk \
    -H "Content-Type: application/json" \
    -H "zt-session: ${zt_session}" \
    "https://${ZITI_EDGE_CTRL_ADVERTISED}/edge/management/v1/cas" 

```

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [November 13, 2024, 1:56pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/10 "2024-11-13T13:56:19Z")

</div>

i really should have shown you how to fetch the cacert bundle for trust too so that you can avoid using the -k in curl... here's a full example that fetches the bundle, outputs it to a file and then uses it later on with curl so that you can properly trust the controller

```auto
curl -sk https://"${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS}:${ZITI_CTRL_EDGE_ADVERTISED_PORT}/.well-known/est/cacerts" \
	| openssl base64 -d \
	| openssl pkcs7 -inform DER -outform PEM -print_certs -out "/tmp/${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS}.cacerts.pem"

curl -s -H "Content-Type: application/json" \
	--cacert "/tmp/${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS}.cacerts.pem" \
    "https://${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS}:${ZITI_CTRL_EDGE_ADVERTISED_PORT}/authenticate?method=password" \
    -d "{\"username\":\"${ZITI_USER}\",\"password\":\"${ZITI_PWD}\"}" |
	jq -j .data.token

```

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 18, 2024, 2:48pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/11 "2024-11-18T14:48:26Z")

</div>

Thank you, that worked. I am now quite far with the declarative automation. But I still have one question: How can I delete a service policy via curl, for example?

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [November 18, 2024, 2:53pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/12 "2024-11-18T14:53:45Z")

</div>

Ah, sorry we neglected to send you the management API reference!

> **[Edge Management API Reference | OpenZiti](https://openziti.io/docs/reference/developer/api/edge-management-reference/)**
>
> open source zero trust

With this, you can compose a REST request for any operation, including deleting a service policy: [Edge Management API Reference | OpenZiti](https://openziti.io/docs/reference/developer/api/edge-management-reference/#tag/Service-Policy/operation/deleteServicePolicy)

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 21, 2024, 2:01pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/13 "2024-11-21T14:01:20Z")

</div>

Hello everyone,  
here is the first draft for a declarative configuration.

The setup is as follows:

1. Create a folder named “config” in $ZITI\_HOME (in my container that is /ziti-controller)
2. Store any yaml files in this folder that are to be configured in the controller. You can simply convert and insert the JSON code from ZAC into YAML or create it manually. A possible configuration file could look like this:

```yaml
---
configs:
  - name: my-service.ziti-host-config
    configTypeId: host.v1
    data:
      protocol: tcp
      address: traefik
      forwardPort: true
      allowedPortRanges:
        - high: 80
          low: 80
        - high: 443
          low: 443
  - name: my-service.ziti-intercept-config
    configTypeId: intercept.v1
    data:
      portRanges:
        - high: 80
          low: 80
        - high: 443
          low: 443
      addresses:
        - my-service.ziti
      protocols:
        - tcp

services:
  - name: my-service.ziti
    configs:
      - my-service.ziti-host-config
      - my-service.ziti-intercept-config

service-policies:
  - name: my-service.ziti-bind-policy
    type: Bind
    serviceRoles:
      - "@my-service.ziti"
    identityRoles:
      - "@your-host"
  - name: my-service.ziti-dial-policy
    type: Dial
    serviceRoles:
      - "@my-service.ziti"
    identityRoles:
      - "#my-service.ziti"

```

It is important to note that the entries can be sorted into different files as required, only the keywords “identities”, “service-policies”, ... may only appear once per file.

1. Place the following Python script named `config-sync.py` in the $ZITI\_HOME or any other directory:

```python
import yaml
import json
import subprocess
import os
import sys
import urllib.parse
import time
from watchdog.observers import Observer
from watchdog.events import FileSystemEventHandler
import threading

ALL_ROOT_KEYS = ["service-policies", "identities"]
ATTRIBUTE_API_MAPPING = yaml.safe_load("""
configs: configs
serviceRoles: services
identityRoles: identities
configTypeId: config-types
""")

class YamlHandler(FileSystemEventHandler):
    def on_closed(self, event):
        if event.is_directory:
            return
        if event.src_path.endswith(".yaml") or event.src_path.endswith(".yml"):
            process_yaml_file(event.src_path)

def run_cleanup_periodically(interval):
    while True:
        cleanup()
        time.sleep(900) # Run every hour

# Convert ies to y, e.g. service-policies -> service-policy
def singular(word):
    if word.endswith('ies'):
        return word[:-3] + 'y'
    # Prevents modification of words such as “process”
    elif word.endswith('s') and not word.endswith('ss'):
        return word[:-1]
    return word

# Interact with file
def handle_file(file_path, mode, file_type, save_data=None):
    if mode == 'r':
        if not os.path.exists(file_path) or os.path.getsize(file_path) == 0:
            print(f"INFO file {file_path} does not exist or is empty, returning empty dict")
            return {}

    try:
        with open(file_path, mode) as file:
            if mode == 'r':
                if file_type == 'yaml':
                    result = yaml.safe_load(file)
                elif file_type == 'json':
                    result = json.load(file)
                else:
                    raise ValueError(f"Unsupported file type: {file_type}")
                print(f"INFO file {file_path} loaded successfull")
                return result
            elif mode == 'w':
                if file_type == "json":
                    json.dump(save_data, file, indent=2)
                    print(f"INFO file {file_path} saved successfull")
                else:
                    file.write(save_data)
    except (yaml.YAMLError, json.JSONDecodeError) as e:
        print(f"ERROR file {file_path} cannot be loaded, reason: invalid {file_type}")
    except IOError as e:
        print(f"ERROR file {file_path} cannot be {'loaded' if mode == 'r' else 'saved'}, reason: {e}")

# Login to Controller
def login(address, username, password):
    login_command = [
        'curl', '-s', '-k',
        '-H', f'content-type: application/json',
        urllib.parse.quote(f'https://{address}/authenticate?method=password', safe=':/?="'),
        '-d', json.dumps({"username": username, "password": password})
    ]
    try:
        result = subprocess.run(login_command, capture_output=True, text=True, check=True)
        session_token = json.loads(result.stdout).get('data', [{}]).get('token')
        print(f"INFO login to {address} as '{username}' successfull")
        return session_token
    except (subprocess.CalledProcessError, json.JSONDecodeError, IndexError) as e:
        print(f"ERROR login to {address} as '{username}' failed, unexpected error: {e}")
        sys.exit(1)

def update_known_ids(action, name, id, root_key):
    if root_key not in known_ids:
        known_ids[root_key] = []

    if action == "add":
        existing_item = next((item for item in known_ids[root_key] if item['id'] == id), None)
        if not existing_item:
            known_ids[root_key].append({"name": name, "id": id})
    elif action == "remove":
        existing_item = next((item for item in known_ids[root_key] if item['id'] == id), None)
        if existing_item:
            known_ids[root_key].remove(existing_item)

    # Remove the root_key if the list is empty
    if not known_ids[root_key]:
        del known_ids[root_key]

def evaluate_response(response, action, name, root_key):
    if 'data' in response:
        if response['data'] == {}:
            print(f"INFO {singular(root_key)} '{name}' {action}")
        elif 'id' in response['data']:
            print(f"INFO {singular(root_key)} '{name}' created, id: {response['data']['id']}")
            update_known_ids("add", name, response['data']['id'], root_key)

    elif 'error' in response:
        match response['error']['code']:
            # 400
            case "COULD_NOT_VALIDATE":
                print(f"ERROR {singular(root_key)} '{name}' cannot {action}, reason: syntax error in config-file")
            # 401
            case "UNAUTHORIZED":
                print(f"ERROR {singular(root_key)} '{name}' cannot {action}, reason: {response['error']['message']}")
            # 409
            case "CONFLICT_CANNOT_MODIFY_REFERENCED":
                print(f"ERROR {singular(root_key)} '{name}' cannot {action}, reason: {response['error']['causeMessage']}")
            # 429
            case "RATE_LIMITED":
                print(f"ERROR {singular(root_key)} '{name}' cannot {action}, reason: {response['error']['causeMessage']}")
    else:
        print(f"ERROR {singular(root_key)} '{name}' cannot {action}, unexpected response: {response}")

# Matches config files with existing configuration in the controller
def cleanup():
    print(f"INFO cleanup started")
    for root_key in ALL_ROOT_KEYS:
        root_key_command = [
            'curl', '-sk',
            '-H', f'content-type: application/json',
            '-H', f'zt-session: {session_token}',
            urllib.parse.quote(f'https://{ZITI_CTRL_ADVERTISED_ADDRESS}/edge/management/v1/{root_key}', safe=':/?="')
        ]
        try:
            result = subprocess.run(root_key_command, capture_output=True, text=True, check=True)
            data = json.loads(result.stdout).get("data", [])
            returned_ids = {item["id"]: item["name"] for item in data}
            known_ids_set = {item['id'] for item in known_ids.get(root_key, [])}

            # Remove IDs from the controller that are not in the known_ids
            for returned_id, returned_name in returned_ids.items():
                if returned_id not in known_ids_set:
                    delete_command = [
                        'curl', '-k',
                        '-X', "DELETE",
                        '-H', f'content-type: application/json',
                        '-H', f'zt-session: {session_token}',
                        urllib.parse.quote(f'https://{ZITI_CTRL_ADVERTISED_ADDRESS}/edge/management/v1/{root_key}/{returned_id}', safe=':/?="')
                    ]
                    try:
                        result = subprocess.run(delete_command, check=True, capture_output=True, text=True)
                        response = json.loads(result.stdout)
                        evaluate_response(response, "deleted", returned_name, root_key)
                    except:
                      print(f"ERROR {singular(root_key)} '{returned_name}' cannot deleted, unexpected error: {e}")

            # Remove IDs from known_ids that are not returned by the curl command
            for known_id in known_ids_set:
                if known_id not in returned_ids:
                    update_known_ids("remove", returned_name, known_id, root_key)

        except (subprocess.CalledProcessError, json.JSONDecodeError, IndexError) as e:
            print(f"ERROR some {root_key} cannot deleted, unexpected error: {e}")
    print(f"INFO cleanup finished")

# Query id for entities
def get_id(root_key, name):
    check_command = [
        'curl', '-sk',
        '-H', f'content-type: application/json',
        '-H', f'zt-session: {session_token}',
        urllib.parse.quote(f'https://{ZITI_CTRL_ADVERTISED_ADDRESS}/edge/management/v1/{root_key}?filter=name="{name}"', safe=':/?="')
    ]
    try:
        result = subprocess.run(check_command, capture_output=True, text=True, check=True)
        return json.loads(result.stdout).get('data', [{}])[0].get('id')
    except (subprocess.CalledProcessError, json.JSONDecodeError, IndexError):
        return None

# Function for executing the curl command
def execute_curl(root_key, item_data):
    existing_id = get_id(root_key, item_data['name'])
    if existing_id:
        REQUEST_METHOD = 'PATCH'
        url = f'https://{ZITI_CTRL_ADVERTISED_ADDRESS}/edge/management/v1/{root_key}/{existing_id}'
        action = "updated"
        update_known_ids("add", item_data['name'], existing_id, root_key)
    else:
        REQUEST_METHOD = 'POST'
        url = f'https://{ZITI_CTRL_ADVERTISED_ADDRESS}/edge/management/v1/{root_key}'
        action = "created"

    # Applying the default values
    if root_key in default_values:
        for key, value in default_values[root_key].items():
            if key not in item_data:
                item_data[key] = value

    # Replace each name with its openziti-id, based on the ATTRIBUTE_API_MAPPING
    for key, value in item_data.items():
        if key in ATTRIBUTE_API_MAPPING:
            api_endpoint = ATTRIBUTE_API_MAPPING[key]
            if isinstance(value, list):
                # If the value is a list, we process each element
                 item_data[key] = []
                 for name in value:
                     if name.startswith('#'):
                         # If the name starts with '#', we add it unchanged
                         item_data[key].append(name)
                     elif name.startswith('@'):
                         # If the name starts with '@', we remove the '@' and call get_id
                         item_data[key].append(f"@{get_id(api_endpoint, name[1:])}")
                     else:
                         # For all other cases, we call get_id with the full name
                         item_data[key].append(f"{get_id(api_endpoint, name)}")

            elif isinstance(value, str):
                # If the value is a string, we process it directly
                if value.startswith('#'):
                    item_data[key] = value
                elif value.startswith('@'):
                    item_data[key] = f"@{get_id(api_endpoint, value[1:])}"
                else:
                    item_data[key] = f"{get_id(api_endpoint, value)}"

    curl_command = [
        'curl', '-k', url,
        '-X', REQUEST_METHOD,
        '-H', f'accept: application/json',
        '-H', f'content-type: application/json',
        '-H', f'zt-session: {session_token}',
        '-d', json.dumps(item_data)
    ]
    try:
        result = subprocess.run(curl_command, check=True, capture_output=True, text=True)
        response = json.loads(result.stdout)
        evaluate_response(response, action, item_data['name'], root_key)

    except subprocess.CalledProcessError as e:
        print(f"ERROR {singular(root_key)} '{item_data['name']}' cannot {action}, unexpected error: {e.stdout}")

# Process a yaml file
def process_yaml_file(file_path):
    data = handle_file(f"{file_path}", "r", "yaml")
    for root_key, value in data.items():
        for item in value:
            execute_curl(root_key, item)

if __name__ == " __main__":
    print(f"INFO initializing config-sync")
    for var_name in ["ZITI_CTRL_ADVERTISED_ADDRESS", "ZITI_HOME", "ZITI_USER", "ZITI_PWD"]:
       var_value = os.getenv(var_name)
       if var_value is not None:
           globals()[var_name] = var_value
           print(f"INFO set {var_name}='{var_value}'")
       else:
           print(f"ERROR {var_name} not found, please run 'export {var_name}=<YOUR-VALUE>' first")
           sys.exit(1)

    default_values = handle_file(f"{ZITI_HOME}/config/defaults.yml", "r", "yaml")
    known_ids = handle_file(f"{ZITI_HOME}/statefile.json", "r", "json")
    session_token = login(ZITI_CTRL_ADVERTISED_ADDRESS, ZITI_USER, ZITI_PWD)

    for filename in os.listdir(f"{ZITI_HOME}/config"):
        if (filename.endswith('.yml') or filename.endswith('.yaml')) and not filename.endswith('defaults.yml'):
            file_path = f"{ZITI_HOME}/config/{filename}"
            process_yaml_file(file_path)

    handle_file(f"{ZITI_HOME}/statefile.json", "w", "json", known_ids)

    # Monitoring the config directory for changes
    observer = Observer()
    observer.schedule(YamlHandler(), f"{ZITI_HOME}/config", recursive=False)
    observer.start()

    # Start the cleanup thread periodically (3600 seconds = 1 hour)
    cleanup_thread = threading.Thread(target=run_cleanup_periodically, args=(3600,))
    cleanup_thread.daemon = True
    cleanup_thread.start()

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        observer.stop()
    observer.join()

```

1. Start the script in the container with `python config-sync.py`. The following happens:

- The script logs on to the controller
- Configures all entities from the configuration files on the controller
- Creates an entry for each in $HOME\_ZITI/statefile.json (is created and maintained automatically, therefore no manual intervention necessary)
- Recognizes if something has been configured on the controller that is not in the configuration files -\> deletes the configuration from the controller
- Monitors the files for changes and synchronizes them with the controller
- Regularly (every 15 minutes) synchronizes the controller configuration and the configuration files

1. Optionally create the file `$ZITI_HOME/config/defaults.yml`. All values defined here do not have to be entered in the configuration files each time, but are added dynamically. For example, my `defaults.yml` looks like this:

```yaml
---
services:
  semantic: AnyOf
  encryptionRequired: true
service-policies:
  semantic: AnyOf
identities:
  type: Default
  isAdmin: false
edge-router-policies:
  semantic: AnyOf

```

If we want to integrate this script into the container, the following python libraries must be added to the controller container:

- urllib.parse
- watchdog

That's it, what do you think? 🙂

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [November 21, 2024, 2:54pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/14 "2024-11-21T14:54:31Z")

</div>

I have two additional questions:

1. Is it possible to get from the API a list of all available endpoints ("identities", "service-policies", ...) back?
2. I find the commands `ziti edge policy-advisor services` and `ziti edge policy-advisor identities` very good for checking the configuration. Is it possible to perform these queries via API as well? I would then run these function checks after the initial setup and every cleanup.

---

<div class="post-metadata">

**Author:** ![jens](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/jens/32/507_2.png) [@jens](https://openziti.discourse.group/u/jens)\
**Post date:** [November 21, 2024, 4:25pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/15 "2024-11-21T16:25:48Z")

</div>

The structure you have is very similar to what I've done in the cli as well. Unfortunately still not ready but it hopefully will be soon.  
What I'm working on is not quite the same however, it's more import/export rather than monitoring for changes.

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [April 29, 2025, 6:35am UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/16 "2025-04-29T06:35:20Z")

</div>

This post was written a while ago, but is there now such a possibility via the API?  
Unfortunately, I haven't found anything in the documentation.

---

<div class="post-metadata">

**Author:** ![plorenz](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/plorenz/32/54_2.png) [@plorenz](https://openziti.discourse.group/u/plorenz)\
**Post date:** [April 29, 2025, 11:43am UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/17 "2025-04-29T11:43:23Z")

</div>

Sorry, I missed your last post. The CLI policy advisor uses the policy-advice endpoint, defined here: [edge-api/source/management/identities.yml at main · openziti/edge-api · GitHub](https://github.com/openziti/edge-api/blob/main/source/management/identities.yml#L354)

The policy advice CLI uses the REST API for all the information it pulls, you can see how it interacts with the API here: [ziti/ziti/cmd/edge/policy\_advisor.go at main · openziti/ziti · GitHub](https://github.com/openziti/ziti/blob/main/ziti/cmd/edge/policy_advisor.go#L333)

Hope that's helpful  
Paul

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [April 29, 2025, 7:34pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/18 "2025-04-29T19:34:07Z")

</div>

> [@ZzenlD](#):
>
> Is it possible to get from the API a list of all available endpoints ("identities", "service-policies", ...) back?

Each identity that wants to use services must query the client API for available routers ([list this client's authorized routers](https://openziti.io/docs/reference/developer/api/edge-client-reference#tag/Edge-Router/operation/getCurrentIdentityEdgeRouters)) and services ([list this client's authorized services](https://openziti.io/docs/reference/developer/api/edge-client-reference/#tag/Service/operation/listServices)).

> [@ZzenlD](#):
>
> I find the commands `ziti edge policy-advisor services` and `ziti edge policy-advisor identities` very good for checking the configuration. Is it possible to perform these queries via API as well? I would then run these function checks after the initial setup and every cleanup.

Yes, to supplement Paul's share about the specification sources, you can discover policy-advisor operations in the management API reference ([Edge Management API Reference | OpenZiti](https://openziti.io/docs/reference/developer/api/edge-management-reference#tag/Identity/operation/getIdentityPolicyAdvice)).

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [April 29, 2025, 7:40pm UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/19 "2025-04-29T19:40:28Z")

</div>

@ZzenlD Also, an OpenZiti Terraform Provider was released after you opened this topic to inquire about declarative configuration management.

> **[Terraform Registry](https://registry.terraform.io/providers/netfoundry/ziti/latest)**

---

<div class="post-metadata">

**Author:** ![ZzenlD](https://avatars.discourse-cdn.com/v4/letter/z/7feea3/32.png) [@ZzenlD](https://openziti.discourse.group/u/ZzenlD)\
**Post date:** [May 9, 2025, 9:30am UTC](https://openziti.discourse.group/t/declarative-configuration-with-files/3393/20 "2025-05-09T09:30:02Z")

</div>

Thanks, i will give it a try 🙂
