# What would openziti arch for this look like

**URL:** <https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616>\
**Category:** Ziti Overlay\
**Created:** [May 30, 2024, 4:00am UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616 "2024-05-30T04:00:45Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 1:21pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/22 "2024-06-04T13:21:19Z")

</div>

Thanks for that information. That's the correct value for the controller's address. Let's also double-check the router configuration. I think you were able to enroll your tunneler successfully, and have only one Ziti router that was also successfully enrolled. Now your tunneler is failing to reach the edge listener provided by your router.

The edge listener is provided by the router on the router's advertised address, so the tunneler must be able to look up the router's address (FQDN) through the host's DNS resolver to find its current IP address. It must have an IP route to that address. Please ensure the router's address and port are not blocked by a firewall.

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 1:34pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/23 "2024-06-04T13:34:00Z")

</div>

Checked , for simplicity I'm allowing All traffic .

Note : There is only one private router here , The IP address given to router here is the internal private IP of the machine residing in private subnet .

Just checking if we can still reach with just one private router

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 1:47pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/24 "2024-06-04T13:47:59Z")

</div>

You can undoubtedly use Ziti with only private IP addresses. It will work if the tunneler, router, and controller can communicate.

Sometimes, we say "private router and "public" router. It's not a precise meaning, though. Typically, a "private" router has a private IP address, so it does not provide links to other routers, which it moot with only one router.

It will work if the tunneler can connect to the router's advertised address and the router and tunneler can connect to the controller's advertised address.

EDIT: say "tunneler" instead of "identity" to clarify

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 3:44pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/25 "2024-06-04T15:44:06Z")

</div>

I'm also getting error

```auto
{"_context":"ch{edge}-\u003eu{classic}-\u003ei{zO50}","chSeq":2,"connId":0,"edgeSeq":0,"error":"service 5BxjZOIl576wSWUcl0PsM3 has no terminators","file":"github.com/openziti/ziti/router/xgress_edge/listener.go:199","func":"github.com/openziti/ziti/router/xgress_edge.(*edgeClientConn).processConnect","level":"warning","msg":"failed to dial fabric","time":"2024-06-04T15:41:51.136Z","token":"c0c6497b-afdd-476f-8b54-22674b793594","type":"EdgeConnectType"}

```

following the guide of k3ds deployment , last time did not see such errors

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 3:52pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/26 "2024-06-04T15:52:14Z")

</div>

I even get

```auto
OKAY : router1 (1) -> hello-service (1) Common Routers: (1/1) Dial: N Bind: Y 

OKAY : hello-client (1) -> hello-service (1) Common Routers: (1/1) Dial: Y Bind: N

```

for the service and router

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 3:55pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/27 "2024-06-04T15:55:27Z")

</div>

The error message indicates there are zero terminators for the dialed Ziti service.

> error: service 5BxjZOIl576wSWUcl0PsM3 has no terminators

Let's check your Ziti service's policies.

```bash
ziti edge policy-advisor services -q 5BxjZOIl576wSWUcl0PsM3

```

The expected output is two lines beginning with `OKAY`. There should be an authorization each for bind and dial. It seems there is not yet an authorization for bind, which must be granted with a bind service policy.

You can create the bind service policy to grant permission for some identity to host (reverse proxy) the Ziti service.

For example, I will use your service's ID because it's available, but you may use the friendly name instead.

```bash
ziti edge create service-policy "my-service-bind-policy" Bind \
    --service-roles '@5BxjZOIl576wSWUcl0PsM3' --identity-roles '#my-service-hosts'

```

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 4:00pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/28 "2024-06-04T16:00:02Z")

</div>

Thank you for that extra info. The router identity named "router1" has permission to host/bind the service.

I believe you have no terminators.

```bash
ziti edge list terminators

```

Let's also check your router policies. You can try relaxing the router policies if they do not already permit `#all` identities and `#all` services.

```bash
ziti edge list edge-router-policies;
ziti edge list service-edge-router-policies

```

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 4:03pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/29 "2024-06-04T16:03:30Z")

</div>

![Screenshot 2024-06-04 at 9.33.15 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/9/9fd73faa3ae27f1db9fa587b5bac950ad033f388.png)  
have it like this

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 4:05pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/30 "2024-06-04T16:05:06Z")

</div>

I actually followed this [Kubernetes Service | OpenZiti](https://openziti.io/docs/learn/quickstarts/services/kubernetes-service/) .

I believe its referring to target service as terminator , I have deployed hello application as target along with ziti-service pointing to it

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 4:06pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/31 "2024-06-04T16:06:45Z")

</div>

Is there an error message in the log for the router1 process deployed in your cluster? For some reason, it's failing to create the terminator.

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 4:08pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/32 "2024-06-04T16:08:40Z")

</div>

![Screenshot 2024-06-04 at 9.38.10 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/6/6aaf6d55fa2f1a023de07cea4148bec2ef804e36.jpeg)  
Logs of the router1 pod

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 4:32pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/33 "2024-06-04T16:32:10Z")

</div>

I have just followed the steps in the blog and first service tutorials and couldn't trigger the same problem.

There are four authorizations a Ziti service needs to function:

1. service (dial) policy allows a client identity (tunneler proxy or SDK)
2. service (bind) policy allows a hosting identity (tunneler reverse proxy or SDK)
3. service edge router policy allows the service to use one of the online, in-common routers
4. edge router policy allows the client and hosting identities to use one of the online, in-common routers

We've established these authorizations were granted, so it must be a connectivity or configuration problem for the router or controller, or both.

Let's check the controller log, too, for error messages, and dump the controller and router configuration YAML.

Will you post these as text instead of screenshots, if possible?

```bash
kubectl --namespace ziti logs --selector app.kubernetes.io/component=ziti-controller --tail=-1

```

```bash
kubectl --namespace ziti logs --selector app.kubernetes.io/component=ziti-router --tail=-1

```

```bash
kubectl --namespace ziti get configmap ziti-controller-config -o go-template='{{index .data "ziti-controller.yaml" }}'

```

```bash
kubectl --namespace ziti get configmap ziti-router-config -o go-template='{{index .data "ziti-router.yaml" }}'

```

You may find it helpful to filter the logs by level. Here's how I found only error level logs from the controller container. This is a common error to show the expected format. You need the `jq` command if not already installed.

```bash
kubectl logs \
--namespace=ziti \
--selector=app.kubernetes.io/component=ziti-controller \
--tail=-1 \
--container=ziti-controller \
| jq 'select(.level == "error")'

```

```json
{
  "_context": "tls:0.0.0.0:1280",
  "error": "remote error: tls: bad certificate",
  "file": "github.com/openziti/transport/v2@v2.0.131/tls/listener.go:257",
  "func": "github.com/openziti/transport/v2/tls.(*sharedListener).processConn",
  "level": "error",
  "msg": "handshake failed",
  "remote": "10.42.0.1:36096",
  "time": "2024-06-04T16:14:56.700Z"
}

```

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 6:28pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/34 "2024-06-04T18:28:01Z")

</div>

![Screenshot 2024-06-04 at 9.38.10 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/6/6aaf6d55fa2f1a023de07cea4148bec2ef804e36.jpeg)  
 ![Screenshot 2024-06-04 at 11.53.57 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/3/3d4b09d87341ff2081c97b7e09b8af16a38b919b.jpeg)  
 ![Screenshot 2024-06-04 at 11.54.49 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/6/6894b16b56cc532dd7c75d4467f24a45f6b2df5b.jpeg)  
 ![Screenshot 2024-06-04 at 11.55.27 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/c/cbfbee705de777dfdfd585d93e1ccd1022ec6145.png)  
 ![Screenshot 2024-06-04 at 11.55.44 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/c/cb5ebeed0f86e1d9d8279bf6858eb388c5e2d9f3.png)

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 6:28pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/35 "2024-06-04T18:28:43Z")

</div>

![Screenshot 2024-06-04 at 11.57.03 PM](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/d/d6b3f95b2b114d376606b7944216450cfef288e6.png)

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 6:29pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/36 "2024-06-04T18:29:12Z")

</div>

Let me know if I can be more clear thanks

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 6:43pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/37 "2024-06-04T18:43:08Z")

</div>

I wish there are was more observability on the mesh to pin point errors

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 6:47pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/38 "2024-06-04T18:47:49Z")

</div>

@sadath-12 It would help me a lot to have these as text, not images, especially since I need to parse the JSON to find interesting messages.

I suppose the terminal app might not be cooperating, making it difficult to capture the terminal scrollback on the clipboard.

When you run a command, you can redirect the output to a file with `| tee ./log.txt` or `> ./config.yml` , then you can paste the file's contents into the forum.

Here's one example.

```bash
kubectl get configmap ziti-router-config \
--output=go-template='{{index .data "ziti-router.yaml" }}' \
--namespace=ziti \
| tee ./ziti-router.yaml

```

This will create the YAML file you can then paste inside ```yaml fences in Discourse.

Regarding observability, I hope to find a clear error message. Did you find any interesting error level messages when you parsed the logs for the router and controller?

The handshake errors in the screenshot are common and can usually be ignored.

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 6:49pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/39 "2024-06-04T18:49:07Z")

</div>

ok then let me try and find specific error messages and post it here

---

<div class="post-metadata">

**Author:** ![sadath-12](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/sadath-12/32/2043_2.png) [@sadath-12](https://openziti.discourse.group/u/sadath-12)\
**Post date:** [June 4, 2024, 6:55pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/40 "2024-06-04T18:55:40Z")

</div>

Note: All things are happening in same cluster right now which is k3d (just to figure out the issue)

---

<div class="post-metadata">

**Author:** ![qrkourier](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/qrkourier/32/52_2.png) [@qrkourier](https://openziti.discourse.group/u/qrkourier)\
**Post date:** [June 4, 2024, 6:56pm UTC](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616/41 "2024-06-04T18:56:52Z")

</div>

From my computer, I can see the controller and router are responding on the expected ports with the correct protocols. This reduces the chance of a connectivity problem.

[Previous page](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616.md?page=1)

[Next page](https://openziti.discourse.group/t/what-would-openziti-arch-for-this-look-like/2616.md?page=3)