Ziti tunnel - reduce traffic consumption

Support General Questions
1

Hi all,

We want to use openziti tunnel in our infrastructure, all settings are done and it works fine. But we want to use lte in our devices and faced the problem that only when ziti-tunnel is running, we get eth traffic consumption of about 20 megabytes per hour.

I measure with this command and look at the file size:
tcpdump -i wan0 -w out.pcap &

When I perform
/etc/init.d/ziti-tunnel stop

then the size of the out.pcap file is about 10 kbytes per hour.

Please help me, what settings I need to make in order to reduce traffic consumption, but at the same time not greatly degrade the quality of the network.

Thanks!

2

Hi @ivan.lukianenko, welcome to the community and to OpenZiti!

20mb an hour, wow that seems like “a lot” for sure. Is that what you observe even when no network changes happen and with no data going over the overlay? Right now, I think the default is for ziti-edge-tunnel to check back in with the controller every15 seconds. So it should be making about 240 requests an hour. Even if the response from the controller was 1k (it should be much, much smaller), that should “only” be 240k.

Are you able to narrow down with the pcap what IP gets all that data? that seems like a lot!

3

Hi @TheLumberjack ,

Thanks for your reply.

According to the logs, the exchange is mainly between the device and our server.
At the same time, all processes that use ziti dns are stopped and only ziti tunnel works.

Is there any way to log SSL keys to decrypt the ziti traffic?

4

Do you mean "the OpenZiti controller" server is where all the traffic is going to? @ekoby can you comment on this? That seems quite high.

No there's really no way to do that.

5

Hi @ivan.lukianenko ,
Can you confirm that you’re using ziti-tunnel and not ziti-edge-tunnel? If you are, would you mind giving ziti-edge-tunnel a try? There shouldn’t be a noticeable difference in the amount of traffic, but ziti-edge-tunnel has had a lot more testing on smaller devices.

Thank you,
Paul

6

I use “ziti-edge-tunnel-0.31.0” with default build settings:

define Build/Compile
	(cd $(PKG_BUILD_DIR); ./scripts/openwrt-build.sh -s $(TOPDIR) -t mipsel_24kc)
endef

and running by command:

PROG="/usr/bin/ziti-edge-tunnel run  --identity-dir=/etc/ziti/"
7

Yes, I mean “the OpenZiti controller” as server part.
I also performed an experiment with only such services in the system

# service | grep running
/etc/init.d/cron                   enabled         running
/etc/init.d/dnsmasq                enabled         running
/etc/init.d/dropbear               enabled         running
/etc/init.d/network                enabled         running
/etc/init.d/odhcpd                 enabled         running
/etc/init.d/ziti_tunnel            enabled         running

And run command:
tcpdump -i eth0.2 -w out.pcap &

The size of out.pcap is 4,492,851 bytes after 1 hour. There are many TLSv1.3 connections in logs with application data (length is 1395, 390, 930 and others).

If I stop ziti-edge-tunnel then the size of out.pcap is 10,000 bytes after 1 hour.

There is a feeling that all the main traffic goes to the ziti tunnel, but I can’t understand what exactly is going on, because it’s impossible to decrypt.

8

If you feel ok sharing the pcap, I could take a look. Since it's impossible to decrypt, maybe you'll feel ok with sharing? :slight_smile:

You can definitely set it up that way, but by default, it does not do this. OpenZiti is exclusively "split tunnel" in that way. It'll only intercept the traffic you tell it to intercept. You can tell the difference by looking at what ports data is being sent to.

If you used the "host it anywhere" quickstart, data will be sent to port 8441 for the controller, and port 8442 for "data". If you see all the traffic going to 8442, that's definitely data being sent to the router. If you see port 8441, that's traffic going toward the controller.

Another area to inspect is the ziti-edge-tunnel logs. It will clearly show you at INFO level if anything is being intercepted. It'll look like this:

starting intercepting for service[HelloWorld]

(more detailed, FULL log example here)

ziti-tunneler.log.202307130000.log:[2023-07-13T11:16:54.079Z]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:686 on_service() starting intercepting for service[docker.whale]

I/we can perform the same experiment to see if we can replicate. I'll try to set that up.

9

Ok, I setup a test. I installed a brand new overlay network, I created and enrolled an identity with access to no services, and I started the tunneler and let it run for three minutes. In three minutes, with the default polling interval of 10s, we are sending 80k of data. Every 10s we send/rec ~4k. You can see this in my gif below.

Here’s the way I started the tcpdump:

sudo tcpdump -i any dst 13.58.222.94 -w tcp.test.pcap

Here’s the ‘watch’ I used to see the data consumed

watch -n1 du -hs tcp.test.pcap

And here’s how I started the tunneler:

sudo ./ziti-edge-tunnel run -i ./tcpmon.json

tcpdump

Extrapolating that over an hour, it should be about 1.6M, not 20M. That’s still more than we’d probably want. There is a --refresh option you can set to change the polling interval. I tested it out but it had a bug that I think was fixed recently by @ekoby. You could use that parameter to greatly reduce the polling interval, and thus the data sent.

Hope that helps

2 Likes
10

Hi @TheLumberjack,

Thanks a lot for the extensive answer. I will try to repeat your test and collect the traffic in a file.