Why does the tunneler not support the proxy tunnel mode?

Ziti Overlay
1

Especially for using it as a sidecar for Kubernetes pods I would like to use the proxy tunnel mode, meaning opening specific ports which redirect traffic sent to them to specific Ziti services. The reason: I don’t want to or in many cases can’t (Openshift, Azure Container Services) run containers with elevated privileges (NET_ADMIN). I just want to tunnel one or two ports which are what the main container wants to talk with, and have a specific Endpoint for it in ziti. Deploying a Router as a sidecar container, even a private one, feels just wrong in such a use case… is it planned for the tunneler in non-privileged mode to support this, the same way as the router does?

Of course, I am dreaming of more than that… some kind of magically “just working” sidecar proxies as istio has them, maybe even with some kind of injecting controller that would even work across clusters and still in a secure way… but with the benefits Openziti brings in, such as establishing a real connection between routers instead of just relying on some vpn tunnel to do that, being independent from ip addresses and having the tunneler on client side. But the above would help me already…

Looking forward to hear your opinions.

Christian

2

As for “why”… We moved from a “go-based” tunneler to a “c-based” tunneler a while back. At that time proxy use was not high, so it wasn’t ported over to the new “ziti-edge-tunnel”.

There’s an issue to add it back to ziti-edge-tunnel: support proxy mode · Issue #22 · openziti/ziti-tunnel-sdk-c · GitHub

@qrkourier probably has more thoughts on this since he deals with this request more than myself.

3

Hey again @ChristianAnton. I used an enrolled identity from ZEDS, to take advantage of the built-in HTTPBin testing service, and the ziti tunnel proxy testing command for this example. This container image runs ziti tunnel ${@} and a complicated entrypoint script for handling several potential enrollment vectors.

$ docker run \
    --detach --rm \
    --name ziti-proxy \
    --publish 8080:80 \
    --env ZITI_IDENTITY_JSON="$(</tmp/zeds-test-id.json)" \
    --workdir /netfoundry \
    openziti/ziti-tunnel \
        proxy zedsDemoHttpHttpbin:80

$ curl -sfw '%{http_code}' -o/dev/null localhost:8080/status/202
202%

This proxy command was built for testing and is deprecated by ziti-edge-tunnel for all intents and purposes, though you’re correct. It doesn’t do this one thing yet! I encourage you to voice your support for re-adding opaque proxy support in the preferred tunneler, e.g., ziti-edge-tunnel run-proxy in the GitHub issue @TheLumberjack linked earlier.

I noticed a problem with container image while I was writing this. It doesn’t have the /netfoundry dir so I sent the --workdir option to create it as a workaround. The next release will be fixed by the patch I just sent.

BTW, the same ziti tunnel CLI sub-command has an IPTables-based ziti tunnel tproxy mode too. It was used in this sidecar guide back when it was invoked as ziti-tunnel tproxy. I seem to recall that when I tried out that guide I had some trouble with DNS. I’ll be really interested to hear how it goes for you.

4

Ok, do I get it right then that ziti-edge-tunnel is the new and written in C version of ziti-tunnel, which is written Go, has more features but is deprecated in favor of the new one?

5

That’s right: ziti-edge-tunnel uses the Ziti Edge SDK for C, and ziti tunnel sub-command of the CLI uses the Ziti Edge SDK for Go.

The features of ziti tunnel were subsumed into ziti router. Since you found a good use for the deprecated proxy command I wouldn’t be surprised to see it re-appear in ziti-edge-tunnel. Let me check with my team about that.

6

Ok thanks a lot for this explanation. So instead of using ziti-edge-tunnel I could actually use ziti-tunnel for my use case. Also nice to know that this indeed is just the ziti command line that is also used for administering ziti, right?

Allow me another question: how exactly does the tproxy differ from the “normal” DNS based tunneling ziti-edge-tunnel uses by default? I have tried understanding the docs, but no luck so I tried to use it with the router. Unfortunately as I am using the experimental (and already patched by me) helm charts of marvkis, I couldn’t get it to work because it uses the “quickstart” container image which doesn’t have iptables installed and runs as non-root.

7

tproxy mode in the go tunneler uses the linux tproxy feature, see docs here: Transparent proxy support — The Linux Kernel documentation

ziti-edge-tunnel use tun, see docs here: Universal TUN/TAP device driver — The Linux Kernel documentation

At a high level, tun is generally supported across many OSes, but requires you to implement your own TCP stack, since it only deals with IP packets. Tproxy, on the other hand, is only support on linux, but abstracts all that away and lets you deal with connections using TCP or UDP directly. (Other devs, feel free to correct me where I’m wrong, and/or missed an important nuance).

Cheers,
Paul

1 Like
8

Right @ChristianAnton, it’s a sub-command of the main ziti CLI that was preserved for testing when the features were subsumed in ziti router. You can use the openziti/ziti-tunnel image I mentioned if you want to run tproxy with IPtables.

$ docker run \
    --detach --rm \
    --name ziti-tproxy \
    --cap-add=NET_ADMIN \
    --dns 127.1.2.3 --dns 1.1.1.1 \
    --network host \
    --env ZITI_IDENTITY_JSON="$(</tmp/zeds-test-id.json)" \
    --env PFXLOG_NO_JSON=true \
    --workdir /netfoundry \
    openziti/ziti-tunnel \
        tproxy --resolver udp://127.1.2.3:53 --dnsSvcIpRange 100.80.0.0/12

$ dig +noall +answer @127.1.2.3 httpbin.ziti
httpbin.ziti.           60      IN      A       100.80.0.5

$ curl -sfw '%{http_code}' -o/dev/null 100.80.0.5/status/202
202

Here’s an inventory of the available container images.

  • openziti/ziti-cli: runs ziti as “nobody”
  • openziti/ziti-controller runs ziti controller run as “nobody”
  • openziti/ziti-router runs ziti router run as “nobody”
  • openziti/ziti-tunnel runs a wrapper script that runs ziti tunnel run (implies tproxy) as “root”
  • openziti/ziti-edge-tunnel runs a wrapper script that runs ziti-edge-tunnel run as root
  • openziti/ziti-host runs a wrapper script that runs ziti-edge-tunnel run-host as “nobody”

You can override the defaults for these when you run the container. For example, a different run-as user, working dir, exposed ports, command, args.

9

Hi!

I am continuing to understand the different binaries and tools there are.

I have one or two additional doubts:

If I download the “ziti relase” tgz from github, I get some binaries.

~/Downloads/ziti/ ls -1
ziti
ziti-controller
ziti-router
ziti-tunnel

Now, with these I can start ziti controller run but I also have a ziti-controller, for example Same for ziti-router and also a ziti-tunnel… Are these this the same thing?

Especially regarding the ziti-tunnel delivered here: is this the Go-based one or the C-based? As both are supporting only “host” and “proxy” mode, I suppose these are some kind of different packaging of the same Go-based tunneler subcommand of the “ziti” command?

Now, if these both were the Go-based version, why none of them supports the “tproxy” mode? I haven’t been able to find this mode using their command line help functions. When the openziti/ziti-tunnel docker container image is running “ziti tunnel”, why does it support the tproxy mode and the downloadable binary does not?

And finally a question on tproxy: shouldn’t it be possible to run the tproxy mode in a sidecar container without network administration capabilties if the necessary iptables rules were set by another (init) container which actually has the NET_ADMIN capabilties, but doesn’t run with them forever? Isn’t that the exact way the envoy proxy in Istio works?

Sorry if I may be asking stupid questions, but especially the “which is what” part is very confusing still.

Thanks a lot in advance!

Christian

10

The ziti-controller functionality was moved into the ziti binary in the subcommand ziti controller so they are identical. We plan to stop building ziti-controller in the near future. ziti controller is what you want to use. (ziti router is also equivalent to ziti-router and ziti-router is also planned to not be built soon)

ziti-tunnel == go-based. It's deprecated but available in the ziti cli for now as an unlisted ziti tunnel. It is not the C-based tunneler. We plan to implement a 'proxy mode' for the C-based ziti-edge-tunnel very soon. It's being worked on. The intention is for the ziti-edge-tunnel to replace the ziti-tunnel and ziti tunnel entirely.

It's just historical. For a long time there was no demand for a straightforward proxy, but we're adding it to ziti-edge-tunnel to accomodate the need.

Seems reasonable? I'm not an expert on how Istio/envoy proxy work, someone else might comment on that.

1 Like
11

Here’s the tproxy sidecar guide: Kubernetes Sidecar Proxy | OpenZiti

12

Here’s the issue tracking progress on adding a TCP proxy mode to zitu-edge-tunnel (the C tunneler).

At this time, the only way to transparently tunnel outgoing connections at the pod level is the IPTables TPROXY approach used in that guide I linked which runs ziti tunnel tproxy (the Go tunneler).

13

Thank you all for the very helpful explanations and clarifications. That all makes things quite a lot more clear to me.

I have tried the Kubernetes Sidecar proxy setup as explained in the docs and unfortunately also stumbled a bit. There is a problem with newer versions of alpine that makes the concept not to work anymore as intended. I have opened a new thread on that, but for completeness, here’s what i found: Tproxy mode as sidecar container not working with alpine >=3.13.

Cheers

Christian

1 Like