Problems accessing services after Quickstart

Support General Questions
1

Hey everyone!

I just got into OpenZiti as it's an interesting concept for me.

I used the quickstart Docker Compose file with a few changes that I need to run it in Mac's Docker Desktop engine. Nothing major though. Dotenv file is the same as in the example except I set an admin password for convenience.

Everything starts and works flawlessly, I can bash into the controller and list everything, create and enroll identities, access my website etc. Problems start when I try to connect to my network.

Using Ziti Desktop Edge - creating an identity yields a docker network hostname instead of localhost (e.g. ziti-edge-controller:1280, which is inaccessible). When changing ZITI_CTRL_EDGE_ADVERTISED_ADDRESS to localhost, controllers stop connecting, so that's not a way to solve it. I know that advertised address should not be localhost in production environment, but I can't even connect to test it.

Ok, I can work around that by creating a custom JWT and signing it using the same certificate. Now Desktop Edge can see the controller, but says that CONTROLLER_UNAVAILABLE. Upon further research, I find that it's not the controller - it gets the request, but container logs say that tls handshake failed, and desktop logs say that jwt signature is invalid, even though it easily valides using a simple Go scratch or a 3rd-party site.

Maybe I don't have the certificates in my local Mac machine? Got everything from the container, installed it on my Mac, trusted everything by hand, rebooted, tried again - nope.

So desktop app is not working in my particular case - let's try step 2, this might be easier.

Using Go SDK - we create and enroll an identity, feed the resulting json to the example code, input our service name and everything should just connect - but no. I'm stuck with the same problem again - an instance of *rest_model.SessionDetail has a list of edge routers, but their addresses are again using docker network aliases, so I can't connect to it at all.

What do I need to do? I believe the simplest way to test the network is to create a VM, set up a reverse proxy to controller, ZAC and main edge router (that's in purple, for Docker Compose example) and use domains as advertised addresses for almost every container in the network (so that it stops communicating inside docker network and has to go outside at least to DNS to resolve advertised domains, and that's a bummer).

What I want from OpenZiti as a result? I have 15 to 25 VMs in my home server, and now I use the reverse proxy, but it would be much safer and easier to communicate using ZTN for all the clients. I want to spin up another VM to host a docker-compose for the network controller and place a router inside every service and connect it to the main, so that I can use ZAC to create identities and policies (which identity has access to which service) and I'd have only two ports forwarded on my router (for the controller and main ziti router) :slight_smile:

Much thanks! Hope I solve this mystery.

2

Hi @milkywayfarer, welcome to the community and to OpenZiti!

This is indeed a common problem and the exact issue was also asked relatively recently in the forum and prompted me to do a Ziti TV on the topic :slight_smile: You can see this thread here: Ziti TV Apr 26 2024 - Building an overlay with Docker at 11 AM ET/1500UTC for some hints as to how to setup your overlay network so that it's accessible from outside the docker environment. You can watch the replay on YouTube linked in that thread too.

At the end of the day, you'll need to change the .env file as you discovered but using "localhost" won't work. Every device needs to be able to get to the controller and to the router. When you use localhost, you'll be able to get to the controller from the docker host but that's about it. Instead you really need to use an address that all the devices will be able to resolve.

I think if you look at that thread and skim (or watch intently) the video, it should answer the questions you have. If you have a look at those and still are stuck let's follow up?

1 Like
3

Thanks for the response.

I ended up spinning up the compose on a VM with domain tied to it, and everything runs and connects perfectly, except one thing - services list is empty and nothing is accessible :frowning:

List of outputs from several commands that I ran:

  • ziti edge list configs
╭────────────────────────┬──────────────────────────┬──────────────╮
│ ID                     │ NAME                     │ CONFIG TYPE  │
├────────────────────────┼──────────────────────────┼──────────────┤
│ 4TAHq5EVhDlD7sFEk5fs9O │ profile-intercept-config │ intercept.v1 │
│ HDpqDu4qZQ1Oly2MVI8Wi  │ profile-host-config      │ host.v1      │
╰────────────────────────┴──────────────────────────┴──────────────╯
  • ziti edge list services
╭────────────────────────┬─────────┬────────────┬─────────────────────┬────────────╮
│ ID                     │ NAME    │ ENCRYPTION │ TERMINATOR STRATEGY │ ATTRIBUTES │
│                        │         │  REQUIRED  │                     │            │
├────────────────────────┼─────────┼────────────┼─────────────────────┼────────────┤
│ 5MxxVOxBnBbtxdxKCaVI5y │ profile │ true       │ smartrouting        │ default    │
╰────────────────────────┴─────────┴────────────┴─────────────────────┴────────────╯
  • ziti edge list identities
╭────────────┬───────────────────────┬─────────┬────────────┬─────────────╮
│ ID         │ NAME                  │ TYPE    │ ATTRIBUTES │ AUTH-POLICY │
├────────────┼───────────────────────┼─────────┼────────────┼─────────────┤
│ 1i0tMUtJhY │ ziti-private-red      │ Router  │            │ Default     │
│ 22gQ2UQJUY │ ziti-edge-router-wss  │ Router  │            │ Default     │
│ QMgt2hQJU  │ ziti-private-blue     │ Router  │            │ Default     │
│ WdgQMUQlU  │ ziti-fabric-router-br │ Router  │            │ Default     │
│ eN0Q2UQJUY │ ziti-edge-router      │ Router  │            │ Default     │
│ gpJl4KrhK  │ Default Admin         │ Default │            │ Default     │
│ yf13BhQJUY │ mway                  │ Default │            │ Default     │
╰────────────┴───────────────────────┴─────────┴────────────┴─────────────╯
  • ziti edge list service configs profile
╭────────────────────────┬──────────────────────────┬──────────────╮
│ ID                     │ NAME                     │ CONFIG TYPE  │
├────────────────────────┼──────────────────────────┼──────────────┤
│ 4TAHq5EVhDlD7sFEk5fs9O │ profile-intercept-config │ intercept.v1 │
│ HDpqDu4qZQ1Oly2MVI8Wi  │ profile-host-config      │ host.v1      │
╰────────────────────────┴──────────────────────────┴──────────────╯

Steps were:

  1. Create 2 default configs for host and interceptor (specifying host, port and protocol for host config; specifying protocol, address and port range for interceptor config)
  2. Create service and bing said configs
  3. ...
  4. Profit?

What did I forget?

4

YIKES -- that's a pretty big thing! I can see you're using the complex "docker compose" example based on your identities. What client are you trying to use to connect? Have you looked into those logs to see if there's any indication of the problem? A very common issue when using docker is that the advertised addresses are not set properly. That would be my guess as to what went wrong but it also might be policy related too. You could run ziti edge policy-advisor identities -q to see if your mway identity has access to any services.

Are you trying to access the test container deployed? Or something else?