TCP-Proxying services into Openziti without DNS Intercept?

Ziti Overlay
1

I am doing my first steps with Openziti and I believe I have understood most of the basics. Actually, I have one usecase I would like to share my thoughts about with you and discuss what you think about that.

I want to use Openziti (between other things) for multicluster - intercluster Kubernetes communication. I have checked the different methods the ziti-tunneler is being able to run and found out that there is (except using the SDKs) no way to get into Openziti but to use the ziti-tunneler in the “run” mode which means pretty high privileges are needed (tun devices, dns intercept) which I am not too happy about.

I would wish to use openziti in a way to have an ingress controller (basically, a reverse proxy) on one cluster together with the services hosted by this ingress controller and one ziti-tunneler in “host” mode, and an openziti client on another cluster which participates in the Overlay network but exposes two TCP ports (could be 8443 and 8000, i.e.) and everything that arrives at these ports is sent to a (configured) service inside Openziti. This way, this client would be able to run without high privileges and had to do less fancy network stuff. In order to make the services hosted on the internal ingresscontroller available from the outside cluster, CoreDNS would be extended with some *.central.myfancynetwork.com custom DNS zone pointing everything in this domain to the cluster-internal address of the tunneler-proxy client.

As of what I understood, this kind of setup would be possible employing a haproxy or similar with a ziti-tunneler as a sidecar, but this sidecar would then still work based on DNS intercept to figure out into what service to push data in, and would therefore need the rather high privileges. Another approach that I came up with would be a relatively simple Go or Python application employing the SDK to achieve exactly that simple usecase.

Is a setup like the one described by me something that you have ever received a request upon? Is there a way to employ ziti-edge-tunnel in that way which I haven’t seen yet?

Looking forward for your opinions.

Christian

2

Hi @ChristianAnton, welcome to the community and OpenZiti!

I think I’m gonna tap @dariuszSki for this one. He’s more kubernetes savvy than I am and he’s also a better network engineer than me and he might have a better answer than I come up with! :slight_smile: He usually has great ideas and he might have done what you’re looking for, or maybe he’ll know. I’ll also poke @scareything and @qrkourier too, they might have some extra thoughts as well.

My initial reaction is that I’m “pretty sure” you can do this. Robert made a great video where he demonstrates what I think is this setup (well, again, not a network engineer, but I think it’s the same?) where he sets OpenZiti up as a LAN gateway. Maybe that video will show you something interesting…

You can find that here:

Sorry I can’t be more help, but I’ll follow this thread with interest!

3

Hi Christian. The router can be configured with a proxy mode that binds a specified Ziti service by name to a specified local TCP port. This is a raw, opaque, TCP proxy as a point of ingress from the normal network to the Ziti overlay. All packets arriving on that TCP port have their payloads forwarded to the Ziti service.

There’s been some recent work that might be helpful if you’re looking to deploy the router in Kubernetes, a Helm chart for ziti router. A community member contributed charts for the Ziti controller and router. They’re baking in this draft pull request.

I can imagine a new chart like ziti-router-ingress that’s configured with a list of triplets like cluster service:port:ziti service name:cluster ingress port. Then, the chart would map each cluster ingress to a particular Ziti service. Is that what you’re looking for?

4

Hi @ChristianAnton,

If I understand correctly, you are trying to figure out a way to deploy ziti client/server apps (i.e. ZETs, possible ERs as mentioned by @qrkourier ) in two clusters without any escalated privileges. In Cluster 1 (i.e. client side cluster ), this ziti app would run as a forward proxy. In Cluster 2 (i.e. server side cluster ), it would run as a reverse proxy. Thus, forwarding direction is unidirectional (C1 —> C2). The requirement is to proxy 2 ports. Does that summarize what you are trying to accomplish?

Thank you,
Dariusz

5

Hey @qrkourier. What you write about the router and the ability for it to configure it with a proxy mode sounds exactly like what I am looking for. I will deep-dive into this configuration and the linked documentation and let you know whether it works as I expected. Actually and indeed, it kind of logically makes much more sense to have this functionality in the “router” instead of in the “edge” component. Thanks a lot, it’s really amazing how quick one gets replies here and how many skilled people are to be found here. It’s notable the dedication you work on the project with.
Regarding the Kubernetes helm charts for controller, router etc. I am actually working together with “that community member” and I am using his helm charts. In case this proxy feature is easily addable to the charts in the works, there will be a pull request from me, otherwise maybe a new chart.

6

@ChristianAnton That’s great to hear, Christian. My short-term goal for the charts branch I linked is an umbrella chart that deploys a controller and router with ingresses. Another user is working on adding Istio ingresses, possibly as conditional resources in the chart.

I like the idea of keeping the controller and router charts sufficiently generic that they can be used separately or together with the umbrella chart. I expect plenty of time to work on it with you all shortly. It might be nice to have a utility container too that’s loaded up with admin and troubleshooting tools.

I’m looking forward to having a powerful and flexible way to declare a Ziti stack in K8s. All input and PRs welcome. There’s a monthly forum where we can discuss any steering decisions inclusive of the community (like a SIG), but as of this moment that PR seems like the best focal point for discussing the direction of this first crop of charts.

7

Hi @qrkourier

Do you have by instance an example on how to configure the Router with this proxy mode? I tried to understand the documentation, but from my point of view it's not yet 100% clear.

I suppose I have to configure the listen part of the Router accordingly and use the binding set to proxy if I want to just open a TCP port that connects me to a service directly. The documentation for the proxy xgress component says:

proxy - allows ingress TCP connections to connect directly to a service defined in options

How do I set this service in the options?

Something like this?

listeners:
  - binding: proxy
    address: tcp:0.0.0.0:10080
    options:
      advertise: my-cool-router.company.com:10080
      service: my-ziti-service.svc

Also, what do I have to configure in Openziti configuration? Do I still need an "intercept" config? I suppose no, right?

And can I use this listeners config for proxying on any existing (maybe the only one) Router I have? And if I want to "grab" proxied services from somewhere else, just deploy another router there and connect it with any of the others reachable to him?

Sorry for probably stupid question. Learning curve is quite steep at the beginning.

Thanks

Christian

8

Hi again! Here’s an example:

listeners:
  - binding: tunnel
    options:
      mode: proxy
      services: 
        - zedsDemoHttpHttpbin:8080

This example will configure the router to bind its local port 8080/tcp for Ziti service named “zedsDemoHttpHttpbin”. Any traffic arriving on that port will have its payload delivered to the Ziti service’s server socket, and the server’s response will be symmetrical, egressing from the same port.

The listeners array is a top-level property in the router config. listeners is a list of bindings, and each binding is for some “xgress” component. In this case, the xgress component being bound is tunnel.

There won’t be any intercept configuration per se for this particular tunnel binding because it is configured to opaquely, not transparently, proxy traffic for a particular Ziti service:TCP port pair.

You’re doing really great I have to admit. Please keep asking and we’ll keep answering! :smiling_face:

2 Likes
9

Thanks for the clarification. That looks good...

Well, you told me to do so, so I just continue asking. Simple minded as I am I just have added the following to the (only) router's config inside the listeners section:

      - binding: tunnel
        options:
          mode: proxy
          services:
            - whoami.svc:80

"whoami.svc" is a service that I already have configured, which ends up showing a simple web site.

Unfortunately, the router doesn't want to connect to the controller anymore. Seems that the router needs some kind of role when registering it?

{"ctrlId":"ziti-controller-server","error":"tunneling not enabled","file":"github.com/openziti/edge@v0.24.95/router/xgress_edge_tunnel/fabric.go:175","func":"github.com/openziti/edge/router/xgress_edge_tunnel.(*fabricProvider).authenticate.func1","level":"error","msg":"failed to authenticate","time":"2023-02-03T10:46:51.912Z"}
{"error":"tunneling not enabled","file":"github.com/openziti/edge@v0.24.95/router/xgress_edge_tunnel/servicepoll.go:103","func":"github.com/openziti/edge/router/xgress_edge_tunnel.(*servicePoller).pollServices","level":"fatal","msg":"xgress_edge_tunnel unable to authenticate to controller. ensure tunneler mode is enabled for this router or disable tunnel listener. exiting ","time":"2023-02-03T10:46:51.912Z"}

Would be great if you would push me to the right direction.

1 Like
10

Okaaay, I have meanwhile found the following command to enable the tunneler on the router:

ziti edge update edge-router core-router -t

With this set, my router connects to the controller again. Also, I noticed that the config snippet above did not expose a tcp port, because address was missing. Done so, with this config snippet that’s fine also:

  - binding: tunnel
    address: tcp:0.0.0.0:10080
    options:
      mode: proxy
      services:
        - whoami.svc:80

Now I see a port open. Looks good but doesn’t work yet: when I try to contact that port from another pod under the router’s IP, that happens…

curl http://10.42.0.116:10080
curl: (52) Empty reply from server

and router says this:

{"_context":"tls:0.0.0.0:10080","file":"github.com/openziti/channel/v2@v2.0.25/classic_listener.go:158","func":"github.com/openziti/channel/v2.(*classicListener).acceptConnection.func1","level":"error","msg":"error receiving hello from [tls:10.42.0.118:49964] (receive error (tls: first record does not look like a TLS handshake))","time":"2023-02-03T13:24:17.564Z"}
{"file":"github.com/openziti/edge@v0.24.95/router/xgress_edge/fabric.go:74","func":"github.com/openziti/edge/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","terminatorId":"1OnLX3g5e8e4mJ1xGpgCsq","time":"2023-02-03T13:24:41.435Z","token":"95036a6c-749e-478b-b652-2eef0ed3a643"}
{"file":"github.com/openziti/edge@v0.24.95/router/xgress_edge/fabric.go:74","func":"github.com/openziti/edge/router/xgress_edge.(*edgeTerminator).close","level":"info","msg":"removing terminator on controller","terminatorId":"fyi5EVBlvhvtvxd2SVNzr","time":"2023-02-03T13:24:41.437Z","token":"ba6100a9-64f4-4a4a-a994-fe859b9721e3"}
{"_context":"ch{edge}-\u003eu{classic}-\u003ei{8ymk}","file":"github.com/openziti/edge@v0.24.95/router/xgress_edge/listener.go:114","func":"github.com/openziti/edge/router/xgress_edge.(*edgeClientConn).HandleClose","level":"warning","msg":"failed to remove terminator for session on channel close","terminatorId":"1OnLX3g5e8e4mJ1xGpgCsq","time":"2023-02-03T13:24:41.440Z","token":"95036a6c-749e-478b-b652-2eef0ed3a643"}
{"_context":"ch{edge}-\u003eu{classic}-\u003ei{8ymk}","file":"github.com/openziti/edge@v0.24.95/router/xgress_edge/listener.go:114","func":"github.com/openziti/edge/router/xgress_edge.(*edgeClientConn).HandleClose","level":"warning","msg":"failed to remove terminator for session on channel close","terminatorId":"fyi5EVBlvhvtvxd2SVNzr","time":"2023-02-03T13:24:41.441Z","token":"ba6100a9-64f4-4a4a-a994-fe859b9721e3"}
{"_context":"ch{edge}-\u003eu{classic}-\u003ei{91xG}","chSeq":1,"connId":1,"edgeSeq":0,"file":"github.com/openziti/edge@v0.24.95/router/xgress_edge/listener.go:302","func":"github.com/openziti/edge/router/xgress_edge.(*edgeClientConn).processBind","level":"info","msg":"created terminator","routerId":"pSr.piroh7","sessionId":"990d0cf3-4480-4b96-ba05-e4453f2138bd","terminatorId":"","time":"2023-02-03T13:24:41.557Z","type":"EdgeBindType"}
{"_context":"ch{edge}-\u003eu{classic}-\u003ei{91xG}","chSeq":2,"connId":0,"edgeSeq":0,"file":"github.com/openziti/edge@v0.24.95/router/xgress_edge/listener.go:302","func":"github.com/openziti/edge/router/xgress_edge.(*edgeClientConn).processBind","level":"info","msg":"created terminator","routerId":"pSr.piroh7","sessionId":"550d7853-e42a-4d58-b3a6-d785432eeec1","terminatorId":"","time":"2023-02-03T13:24:41.558Z","type":"EdgeBindType"}
{"_context":"ch{edge}-\u003eu{classic}-\u003ei{93lR}","chSeq":1,"connId":1,"edgeSeq":0,"file":"github.com/openziti/edge@v0.24.95/router/xgress_edge/listener.go:302","func":"github.com/openziti/edge/router/xgress_edge.(*edgeClientConn).processBind","level":"info","msg":"created terminator","routerId":"pSr.piroh7","sessionId":"89760906-957f-49d4-ae53-4005f161d74d","terminatorId":"","time":"2023-02-03T13:24:46.680Z","type":"EdgeBindType"}

I suppose this is now just a matter of not yet knowing how to configure the service in Ziti console?

11

You can eliminate this portion from your tunnel binding. There's no address option here because the router's local port to bind is per-ziti-service, not per-binding. Hence, the TCP port to bind is in the list of services, e.g., 80.

12

Got it! So, to make that clear, the port in

services:
   whoami.svc:8000

is both the port where the connection is directed to and also the one the router will open to listen on for connections? In that case, I need to understand better how to actually configure the service for such a setup to work. And I would like to know if it is possible to have the router, say, opeining port 8000 (high port, not running as root) but the service connecting to something on port 80).

Thanks… I feel we are getting there.

13

OK, got it working. Thanks a lot for all the help guys! Last piece missing was to assign the router identity the role policy so it was allowed to be an “initiator” for this service.

Awsome stuff. Thanks

1 Like
14

That’s exciting to hear, nice job getting it working.

15

That’s right, the router must be created with isTunnelEnabled: true for it to later be configured with bindings of type tunnel. Additionally, the tunnel “identity” that is automatically created must be granted access to each Ziti service in the options of the tunnel binding. It sounds like you figured out that’s accomplished by matching role attributes on a Dial Service Policy like this:

Dial Service Policy

identities services
#router-ingress-proxies #example-services

In this example, the router’s tunnel has role attribute #router-ingress-proxies, and the Ziti service named “whoami.svc” has role attribute #example-services. The dial service policy allows the tunnel to access the service as a client on behalf of proxy viewers.