Add identity to new conatiner in docker-compose quickstart

Hi there,

im trying to understand the concept of creating and using an identity for another container that I have added in the docker-compose quickstart as outlined at:

If I want to add an ubuntu server container to the docker-compose quickstart by e.g. adding below to docker-compose.yml file:

ubuntu-server-blue:
image: ubuntu:latest
privileged: true
networks:
zitiblue:
aliases:
- ziti-blue-ubuntu
command: sleep infinity

Do I then need to add another container using "image: openziti/ziti-host" etc to host an identity for this if I want to allow it to access e.g. the blue webpage etc ?

Of can I just create the tunneler on my new ubuntu container ?

I may not be explaining this correctly as im not understanding the process on the helppage above

Thanks,

Col

Let's say you've got a compose service like this with a web server on 1080/tcp inside the compose network, and you want it to be the target of a Ziti service so you can reach it from anywhere with Ziti.

ubuntu-server-blue:
    image: ubuntu:latest
    privileged: true
    networks:
        zitiblue:
            aliases:
              - ziti-blue-ubuntu
    expose:
      - 1080
    command: python -m http.server --bind 0.0.0.0 1080

ziti-host:
    image: openziti/ziti-host
    networks:
        zitiblue:
    environment:
      - ZITI_IDENTITY_JSON

The env var is the JSON you get from enrolling the identity, not a path to the file, but there's a way in that article you linked to mount a file instead if you prefer that.

With the new ziti-host running you can create a Ziti service that targets your ubuntu server. They're on the same "bridge" network, so the target address of the server is ubuntu-server-blue:1080 (or ziti-blue-ubuntu:1080). That's what you put in your Ziti service config (where to send the service's traffic).

Thaks you,

Do you mean:

I add below to the .env file from the quickstart docker compose and add the output of the string value of the identity.jwt that I created to that :

ZITI_IDENTITY_JSON=jljfofjifuwiodfjwlf etc

Does the jwt file have to be put somehwere like with a regular linux server in the container - ```
/opt/openziti/etc/identities




Then just create this ziti-host container and bring the all up and it should enroll : 
ziti-host:
    image: openziti/ziti-host
    networks:
        zitiblue:
    environment:
      - ZITI_IDENTITY_JSON

is the ziti-host container an example or required for all containers as a way of setting this up as im still unclear after reading the web page documentation 

Thanks

Col

I hear you. You're starting with a JWT. I assumed incorrectly you already used the JWT to get an identity JSON.

That's OK because the ziti-host container can do it for you if you mount a volume like this.

ziti-host:
    image: openziti/ziti-host
    networks:
        zitiblue:
    volumes:
      - ziti-host-id:/ziti-edge-tunnel
    environment:
      - ZITI_ENROLL_TOKEN="eyJhbGciOiJSUzI1NiIsImtpZCI6ImRkNDRkMzYwM2Y0YmIyMjFiYmE0OTVjM2RiNjJkMjBlMmUyNjI2NzIiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2VkZ2UuYmluZ25ldC5jbG91ZDo1MyIsInN1YiI6InpVUEc5N2o2ViIsImF1ZCI6WyIiXSwiZXhwIjoxNzIwNjQ1MDU0LCJqdGkiOiI0N2JhNjhlMC0xYTc1LTQ0YWYtOWE4Ni1hODFlODM4Y2IyNzIiLCJlbSI6Im90dCIsImN0cmxzIjpudWxsfQ.awYECW3bs2fiYtowZi5FQV5CBgLY7ZJWhyPYGWRUrLrBPGKjzgGylmbPXKZye01tdddIKL5Wgp0itmUguB89UpqC00qSwk3QWqB7pRbg323wGiw8vW-qY7IT0biDND_JcnJt_QpKPbfBIxlSaRTpzoScrLJN7NRiuJ2auCBBU4YxOsLWacb_exLIOGoNqUPE6hsCQsUhDPcZy7_ZrYPJ0_lUfRp-XZBlMWYky6pfK1OYsGqki5d4Km8VorcC9VWewiJa5QvQhPavZrDvXcTzI7iT5E0DvWhVjNg9lp7ygTDD1YRFSzy05x30Pfjsj2Wt_TZyo0IlLT4WcCdR4fMEwbBC2HaiJXdkZzbO70d7iv38usI_Pk4CVnM2Mk5wheVNMAWBPG8Rlzf8mnjduACb0SnsMztrNpYFgsw46GyYKkhyXqieAVqskNQi0mvwBDuV2-BRY-KTranHGwEMQbU8NI8xWS3PLQkpSh-7PLQqBHyY88EA6CajX1reI_G_nFzyUChcVdZ03AGFdQXpWVxoZjfuxjVP7lXQdwNFyJGSVNdiR8WMrB9ZTp5Yw7zFxzC7wvugyJvvzsjdSsTGhKzDWs3emFHIU7K8YvHLM0ttJwV3xIAuNBVDDctMMzsCKlXFGK-d0hBQR57dqvD77iRYCwuInu3qtW_zQaFHOA2UM-Q"

volumes:
    ziti-host-id:

On first run, ziti-host will get the JSON and save it in /ziti-edge-tunnel and start up immediately. On the next run, it will find and load the identity from the same place.

Im still getting errors when I add above to my docker-compose.yaml file (using the jwt I generated from the console) and it will not enroll :

col@Ubu4:~/Desktop/docker/docker2$ docker compose logs --tail 100 ziti-host
ziti-host-1 | WARN: clobbering non-empty Ziti enrollment token file /ziti-edge-tunnel/ziti_id.jwt with contents of env var ZITI_ENROLL_TOKEN
ziti-host-1 | DEBUG: waiting 3s for /ziti-edge-tunnel/ziti_id.json (or token) to appear
ziti-host-1 | DEBUG: identity file /ziti-edge-tunnel/ziti_id.json not found
ziti-host-1 | DEBUG: /var/run/secrets/netfoundry.io/enrollment-token/ziti_id.jwt not found
ziti-host-1 | DEBUG: /enrollment-token/ziti_id.jwt not found
ziti-host-1 | INFO: enrolling /ziti-edge-tunnel/ziti_id.jwt
ziti-host-1 | (7)[ 0.000] INFO ziti-sdk:utils.c:201 ziti_log_set_level() set log level: root=3/INFO
ziti-host-1 | (7)[ 0.000] INFO ziti-sdk:utils.c:172 ziti_log_init() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting at (2024-07-13T16:13:18.469)
ziti-host-1 | (7)[ 0.000] INFO ziti-sdk:ziti_enroll.c:90 ziti_enroll() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting enrollment at (2024-07-13T16:13:18.480)
ziti-host-1 | (7)[ 0.000] ERROR ziti-sdk:ziti_enroll.c:123 ziti_enroll() /__w/ziti-tunnel-sdk-c/ziti-tunnel-sdk-c/build/_deps/ziti-sdk-c-src/library/ziti_enroll.c:105 - load_jwt(opts->jwt, ecfg, &ecfg->zejh, &ecfg->zej) => -4 (JWT has invalid format)
ziti-host-1 | (7)[ 0.000] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:2221 enroll_cb() enrollment failed: enroll failed(-4)
ziti-host-1 | ERROR: failed to enroll with token from /ziti-edge-tunnel/ziti_id.jwt (1001B)

im using the docker compose quickstart from here below with just a few other services added which start up ok (I have not done anything with openziti identities/enrollments etc on these yet though):

Ive deleted and recreated the jwt token and its the same problem - just shows as unregistered in the ziti console :

It looks like you're doing everything right, but there's a problem with the format of staffubu's enrollment one-time token. Let's narrow possible causes for this enrollment error from ziti-edge-tunnel enroll command that was run by the entrypoint script.

  1. The error may have disclosed the precise problem, a malformed token (JWT). You can verify that it is roughly the correct format just by glancing as the long, literal string. It should be three base64 encodings on a single line separated by stop characters (., a period). If that much looks correct, then you can paste the token in jwt.io to ensure it can be parsed. Finally, you can load the token as a string or as a file in this Python script to analyze it offline in greater detail. Common problems include getting tokens mixed up and loading an expired token or a token of the wrong type that was meant for a router, for example.

  2. Next, verify that staffubu's device can reach the client API. The URL is in the token as claim "iss" (issuer). That's where staffubu calls out to enroll, and uses that API's server certificate to verify the token signature, so it's essential that the client API is not behind a TLS proxy.

  1. Ive created the JWT Token inside the ZAC gui if that makes any difference and looks ok from what I can see - ive done this a few times now - although it has "invalid signature" at bottom if thats the problem - see screenshot below

2. Ive created an ubuntu server in my docker compose file like so if thats what you mean? :

staffubu:
image: ubuntu:latest
privileged: true
networks:
zitiblue:
aliases:
- ziti-blue-ubuntu
command: sleep infinity

The jwt file I have is:

Ive logged into the staffubu server and was able to download the webpage after using --no-check-certificate option:

root@4fd46c71cacf:/# wget https://ziti-edge-controller:1280
--2024-07-14 23:36:27-- https://ziti-edge-controller:1280/
Resolving ziti-edge-controller (ziti-edge-controller)... 172.19.0.8
Connecting to ziti-edge-controller (ziti-edge-controller)|172.19.0.8|:1280... connected.
ERROR: cannot verify ziti-edge-controller's certificate, issued by 'CN=ziti-edge-controller-intermediate,OU=ADV-DEV,O=NetFoundry,L=Charlotte,C=US':
Self-signed certificate encountered.
To connect to ziti-edge-controller insecurely, use `--no-check-certificate'.
root@4fd46c71cacf:/# ls
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root@4fd46c71cacf:/# wget https://ziti-edge-controller:1280 --no-check-certificate
--2024-07-14 23:36:58-- https://ziti-edge-controller:1280/
Resolving ziti-edge-controller (ziti-edge-controller)... 172.19.0.8
Connecting to ziti-edge-controller (ziti-edge-controller)|172.19.0.8|:1280... connected.
WARNING: cannot verify ziti-edge-controller's certificate, issued by 'CN=ziti-edge-controller-intermediate,OU=ADV-DEV,O=NetFoundry,L=Charlotte,C=US':
Self-signed certificate encountered.
HTTP request sent, awaiting response... 200 OK
Length: 513 [application/json]
Saving to: 'index.html'

index.html 100%[===============================================================================================================>] 513 --.-KB/s in 0s

2024-07-14 23:36:58 (100 MB/s) - 'index.html' saved [513/513]

If I might interject here, can we go back to the start on this for a moment? I'm not exactly sure what you're trying to do. Back on the OP you wrote:

Can you help me understand what it is you're trying to do? Are you trying to allow access TO your ubuntu-server-blue container from outside the blue network, or are you trying to access a resource elsewhere on the OpenZiti overlay FROM the ubuntu-server-blue container?

The linked post you shared makes me think the latter, but I'd like to make sure I actually understand. Would you be able to mock up the diagram and just illustrate what you're trying to do?

Hi, thank you both for the help

Basically if you take the diagram below that you published:

dockerdiagram

In the Blue Network, I want to add in another web server so there are two web servers in there. Im also adding in two ubuntu servers so that one will have access to one web server only and the other will have access to the other webserver only - all within the Blue network.

Im trying to simulate an environment where say there is an "admin" department (the blue network) and certain users will have access to certain services in that admin department - but not necessarily everything in the admin department

Then the "Staff" department (Red Network) would have its own services on a different network - again with different levels of access e.g. sales users with access to sales servers, marketing with access to marketing servers etc ..

Going back to the Blue network and trying to follow the docker-compose documentation, I assumed I would need to have a tunneler on the ubuntu servers I set up for them to
access the individual web servers in the blue network that I want to create?

I hope I have explained that a bit better for you in what Im trying to achieve? Or have I completely misunderstood how this works in this context?

Apologies in advance for how long this will be... :confused:

Ok. Let's start here. Given the way docker networking works, I don't think this is actually possible. Docker is powerful, but it does complicate things from time to time, particularly with docker networking. I've found that it cannot truly emulate segregated environments. It's usually good enough and I think you can do what you want but since those servers are all in the blue network, any ports you expose are going to be accessible from any other server in the blue network. So I don't think you'll want them in the blue network. Are you sure you want them in one network?

If we take a step back instead and just focus on the goal:

You can accomplish this with the existing docker compose environment, just by having more than one identity. I think that's how I'd recommend you start out. After you get that working, the next stop imo will be to remove your containers from the blue network and segregate them from one another, but that'll be after you get your feet under you OpenZiti-wise...

So here's what I recommend:

  • update your docker compose file with the "extra servers" in both the red and blue networks
  • create new identities in the ZAC or using the ziti CLI
  • authorize the blue router (ziti-private-blue) to offload ALL "admin" services
  • authorize the red router (ziti-private-red) to offload "Staff" services
  • authorize whichever users you want to access whichever services you want

So in practicality that would be something like this - i hope this makes sense (after updating your compose file):

  • deploy your web servers in the 'admin' network and for every 'admin' server you deploy, create an OpenZIti service and assign that service an attribute of "admin-service"
  • deploy your web servers in the 'staff' network and for every 'staff' server you deploy, create an OpenZIti service and assign that service an attribute of "staff-service"

For example, with the ziti cli that would look something like this. Step 1 make services with the "admin" or "staff" attributes....

(These are not full and complete samples... If you get lost and feel like you want/need it, I could make a video that does more educating around this topic... )

ziti edge create service admin-service-1 --configs "admin.intercept.config-1,admin.host.config-1" -a "admin-service"
ziti edge create service staff-service-1 --configs "staff.intercept.config-1,staff.host.confg-1" -a "staff-service"

Now you have services (and presumably configs), now you authorize those services. Give the routers access to 'bind' (or host) the services:

ziti edge create service-policy admin-bind-policy Bind --identity-roles '@ziti-private-blue' --service-roles "#admin-service"
ziti edge create service-policy staff-bind-policy Bind --identity-roles '@ziti-private-red' --service-roles "#staff-service"

Now, authorize the specific users to access specific services:

ziti edge create service-policy admin-1-dial Dial --identity-roles '@some-admin-user' --service-roles "@admin-service-1"
ziti edge create service-policy staff-1-dial Dial --identity-roles '@some-staff-user' --service-roles "@staff-service-1"

I hope all that helps, I hope it all makes sense. If it's too long or too dense, let us know and we can try to clear anything up.

Thanks for that - its a big help and I have a better understanding now of how to approach this. I will spend a dew days learning and playing about with what you have shown here and come back if I get stuck for advice.

How is the docker tunneler used then as I seem to be completely off point on that ? - can you give me an example of how it could work to explain it ?

You're not completely off point, but in this situation, since there's ALREADY a tunneling-type device on the blue network (that router), adding another tunneler (ziti-host) won't do much good (well i mean, there's redundancy-related reasons but i'm leaving that off the table right now).

This is where things get more complex, really quickly... Recently, (a few months ish ago) @qrkourier made an excellent discovery with respect to docker. He discovered that one can basically assign a docker container as the "network" for another docker container. When you do this, it effectively melds the two containers into one bigger container. If you're familiar with kubernetes even a little, this is very similar to the "pod" idea where basically two or more containers can all be aggregated together.

This was really exciting to me, because in general when you're using docker, we had no great way to accomplish this sort of design. If you use ziti-edge-tunnel and want to have it intercept packets it works by making a TUN device but that needs to happen on the actual OS itself. That means everything on the machine will have access to that TUN and that's not what we wanted...

Instead, ziti-router has a mode called tproxy that allows one to shuttle bytes to a process without the need for a TUN which means now we can use ziti-router within docker for strategic intercept-type modes but we could also use something like ziti-edge-tunnel in host mode (where it only offloads data) to offload data specifically for one (or more containers)...

I'll pause here, as I'm getting deep into the weeds now. Here's an attempt to demonstrate that visually though... Here's the example I'm recommending you start with.You'll see the "new-ubuntu-svr" is in the blue network and accessed from the ziti-private-blue router:
image

But, with the container as another container's network approach, it MIGHT look more like this... Notice the the ziti-private-blue was removed and replaced with a ziti-edge-tunnel in host mode (which is basically what ziti-host is) and each "network" has two containers, the ziti-host 'network' and then the 'server' container that uses that ziti-host network... Hopefully this all makes sense, like i said, it can get complex quickly ! :slight_smile:

1 Like

Could you give an example of a docker compose yaml file demonstrating this? I have a controller deployed in AWS and I want to set up a docker compose on my local machine with one container providing the network tunnel to another container running the open-webui for ollama. Following the docker tunneler docs, I can't get the ziti-host container to work at all.

You want to publish open-webui with Ziti, and for the open-webui container to talk to the ollama API via Ziti, right?

Oh, and it's your first time posting so welcome to the community!

Thanks!

In my current testing setup, open Ziti aside, I am running open-webui and the ollama server on the same machine. The open-webui is running in a docker container and the ollama server is not, but they are able to talk to each other. For now, you can ignore the ollama server part, although at some point, what you are suggesting would be cool to get working too, in case I want to run the ollama server somewhere other than where I host open-webui or encrypt traffic between the two.

What I am trying to do is hide the open-webui page within a Ziti network so that you need an Ziti tunnel in order to access the ui. I think this falls under the use case of using Ziti as an alternative to a VPN where you can log into a VPN and there's an ip address on the network serving the UI.

Since I first posted, I have been trying more things out and I discovered that I cannot even register an identity with my macOS tunneler to my controller, so I think my controller configuration might be off? I followed the host anywhere docs to set up the controller on AWS, so not sure what is wrong. I copy/pasted the config stuff right out of the docs.

I have the ZAC working, which is what I have been using to generate the jwt's. My mac Ziti Desktop Edge client just says CONTROLLER_UNAVAILABLE whenever I try to enroll.

1 Like

Yep. I've configured my open-webui container to talk to ollama API via Ziti in the past. It works and is useful for the same reason you mentioned. Currently I'm running both in containers in the same Docker network that is already isolated at the network layer.

Perfect. I'll focus on publishing open-webui with Ziti.

That's how my lab is set up. It's working well for chat/instruct cases, especially when I'm mobile and using the Ziti Mobile Edge tunneling app to access open-webui in a web browser.

I've also used an Ollama or OpenAI-compatible mobile app to use the Ollama API directly (or its OpenAI-compatible /v1/chat/completions endpoint) via Ziti.

When uploading tens or hundreds of thousands of open-webui "knowledge" fragments for RAG, I needed to interact with open-webui on the same system where it's running to reduce the latency. Otherwise, it would take weeks to load that much knowledge. This will be less of an issue if/when open-webui releases an API client library or CLI to load knowledge from the local disk. I think it's presently necessary to load it via the web UI.


To publish a server with Ziti, you can embed a Ziti SDK in the app you want to publish or run a Ziti tunneler nearby as a proxy. The tunneler only needs network access to the private port.

Here's an overview of the steps you'll take, summarized from Your First Service | OpenZiti.

  1. Create the Ziti identities for both ends of the Ziti service: a client identity to which you'll give "dial" permission, e.g. "bivalve.felt.laptop" to install on the device where you will access open-webui in a browser, and a hosting identity, e.g. "open-webui-host," to which you'll give "bind" permission and install on a new Ziti container inside the Docker network where you're running the open-webui server.
  2. Create Ziti configs that map the Ziti service to network addresses: an "intercept" address for open-webui clients, e.g., open-webui.ziti.internal:80, and a server address for the hosting identity which will be the name of the open-webui container, e.g., open-webui:8080.
  3. Create a Ziti service with those Ziti configs.
  4. Create a Ziti Dial Service Policy authorizing "bivalve.felt.laptop" to dial the open-webui service.
  5. Create a Ziti Bind Service Policy authorizing "open-webui-host" to bind the open-webui service.
  6. Install the identities on their respective devices. If "bivalve.felt.laptop" is a MacBook, install "Ziti Desktop Edge" from the store (Downloads | OpenZiti). For "open-webui-host," add a container to the open-webui Docker bridge (Containers | OpenZiti).

For the last step, here's what I would do in the open-webui compose project. You can keep the Ziti stuff in a separate YAML file if you want to avoid changing the one from GitHub. Just change your docker compose up command to specify each YAML file. Creating the Ziti identity "open-webui-host" produces an enrollment token file like "open-webui-host.jwt." Save the contents of that file in your compose project's .env file like this:

ZITI_ENROLL_TOKEN=eyJhbGciOiJSUzI1NiIsImtpZCI6IjczNjJjZDQ2MjVkZjQ2ZjgzY2E5MjY4NjgzNDNiOTYxMmQzZWU1YTEiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL21pbml6aXRpLWNvbnRyb2xsZXIuMTkyLjE2OC40OS4yLnNzbGlwLmlvOjQ0MyIsInN1YiI6IlNZcWxvZ0JHODAiLCJhdWQiOlsiIl0sImV4cCI6MTczMjY3NDUwOSwianRpIjoiOWY0YzhhZDktNDVkOC00MGVmLTg2MzEtZWQwOWNkMWZlZDdjIiwiZW0iOiJvdHQiLCJjdHJscyI6bnVsbH0.BwFAyR2uM73zApGDhnGd4w4iZsl-_HtYdt62ZWcf3nX6JIqh9Kgy8v5vmacicRWe7kWbXwcmpCghLrVbEgrF2EzNSqZhwJmXSffEJs9ieHsDW4oiL6kRCEnb24psoltB-K82Ym6GFtugFGkf5x5me1BR_xdMBXr10Mh4T7rkJBpXdSYHakaucAb7Dah4cxR8t6Zd15mBwcCBFDQUTFJ1z2Xx0kF8dZT_kLxAs5VLOtBRj-RZ8QLADs2kKMLTbs3-M5Yrc-z_s8SKPrW2hWHC92z4sLLlxAxrWRUyg0GjsfVbaQl5_nB5qqSEDLEgBOK-FKwoKMksX2i9OJG18KEG9_HoBNDjx36LvBs4Jo9j-MT4aFRNwSFL8vTrYKXUaP-Iv9jDVHta64H_bHgO4BriIVvGxm1UCWa8dlJ6aBXw8o84_1a7gKyuAICV6LZSm6W163OfEJf4T_ldxAjJUi8rZTljhtxITsmqC0D8-ByIe4cBdplq8Yq5lF30j87rmAvnIKutrJN-8sIsC7MeR8VMcUcNoP6MT5hLwiuHBFUGMW8HwLsVYPxImF-YutM61V7pLWi3PqXQ2sVNJy-SddiDeeJ6jcJsGazR915Hbi_9HUfKStQ_Ncf3levtgzwKQHTsQdCAIF-Mv039qtXMG-n4eR6TUfWFi5uIYgCcuXdpzDY

Then add a "ziti-host" service to the project like this. You can use this verbatim. No need to customize unless you want to.

services:
    ziti-host:
        image: docker.io/openziti/ziti-host
        volumes:
            - ziti-host:/ziti-edge-tunnel
        environment:
            - ZITI_ENROLL_TOKEN
volumes:
    ziti-host:

That will put ziti-host on auto-pilot. You can control it with Ziti policies from here on out.

Thanks for this! Do I need to do anything more to connect the ziti-host service to the open-webui service? This part of the docs alludes to something along these lines, I think although I don't understand exactly what sort of variable the docs are referring to and where I would set it.

Also, once everything is connected, can I just request the ziti address somehow in my browser? What address would that be?

The variables mentioned in that doc are unique to the expressInstall quickstart instructions which are intended for a temporary, local network. You can think of them as placeholders in an example, not assignments you need to make or locate.

This is step 2 from my overview that talks about creating the Ziti "configs." The configs will map both ends of your Ziti service to a network address. The intercept config tells your client tunneler which address to use for the Ziti service, i.e., the address you type in your web browser's navbar, and the host config tells the ziti-host container where to send traffic that arrives from your Ziti service, i.e., the open-webui's {container name}:{exposed port}.

Ah okay. This took a minute to sink in. I will try this. Thanks!

1 Like

My ziti-host container is complaining that I do not have a Ziti Edge Router...

 WARN ziti-sdk:bind.c:226 session_cb() server[0.0] failed to get session for service[ollama.svc]: -17/NO_EDGE_ROUTERS_AVAILABLE

Am I supposed to have one after going through the Host Ziti Anywhere quickstart?

Edit: It appears I should have an edge router set up after going through the Host Ziti Anywhere quickstart. Inspecting my AWS image running the controller and router, there is an edge router service running. It is not showing up in my ZAC though...