Hi @yqzhu. Welcome to OpenZiti and welcome to the community! I found your post fascinating since I wasn't familiar with ab yet, but I wanted to replicate your tests. So I spent most of today running a bunch of tests using ab!
TLDR; OpenZiti is slower than direct connect to the internet by about ~30% ish but not orders of magnitude in my testing.
On Testing
I'll explain my testing below, but first I want to address your testing.
Multiple Machines
The consensus among all the OpenZiti devs is that running "everything" on one box is not ideal for this type of performance testing. We would recommend you have at least two different machines and separate them by the actual internet. One will have the OpenZiti controller/router/nginx and the other will run ab. We really recommend you let your traffic traverse the network. I know it sounds hard to believe, but it really does have the opportunity to make a difference.
CPU Never Hits 100%
You also want to make sure you're never hitting 100% CPU usage. If you do, you need to back it down to where it's running at 80%.
Network Stack Matters
Lastly, trying to simulate multiple clients, is shockingly difficult work. In "the real world" each machine will have it's own network stack, it'll be coming from a different IP address, there's a lot of variables. Getting an accurate, representative test is really hard even with 10's of machines operating.
My Testing
Ok. Here is what I did for testing... I made one C4.XLarge (16 cpu/30Gib ram) in AWS US West and I made another in AWS US East. On the east coast machine, I ran the quickstart to setup the OpenZiti overlay. That gives me a controller and an edge router to use. I then installed NGinx and docker. In docker I ran a simple web server and I used NGinx to provide a TLS endpoint and offload TLS to the web container... On the west coast, I installed ab, the ziti-edge-tunnel and ziti and ran tests there. Overall, the setup looks like this:
I then tried your same sort of tests you ran and I didn't get the same sort of results you did.
No OpenZiti
From west --> east, directly with ab --> nginx --> docker, and running:
ab -c 300 -n 30000 -k https://ec2-18-222-189-207.us-east-2.compute.amazonaws.com:8448/
| Time taken for tests | Requests per second |
|---|---|
| 8.447 seconds | 3551.60 [#/sec] (mean) |
| 10.455 seconds | 2869.39 [#/sec] (mean) |
| 8.470 seconds | 3542.04 [#/sec] (mean) |
| 8.451 seconds | 3549.68 [#/sec] (mean) |
| 8.435 seconds | 3556.65 [#/sec] (mean) |
OpenZiti - ziti-edge-tunnel
From west --> east, directly with ab --> nginx --> docker, and running:
ab -c 300 -n 30000 -k https://throughputtestzet.ziti:443/
tunnel command:
sudo ./ziti-edge-tunnel run -i ./throughputtest.client.new.json
| Time taken for tests | Requests per second |
|---|---|
| 10.343 seconds | 2900.49 [#/sec] (mean) |
| 10.487 seconds | 2860.77 [#/sec] (mean) |
| 11.444 seconds | 2621.45 [#/sec] (mean) |
| 10.832 seconds | 2769.62 [#/sec] (mean) |
| 11.432 seconds | 2624.15 [#/sec] (mean) |
OpenZiti - ziti tunnel
From west --> east, directly with ab --> nginx --> docker, and running:
ab -c 300 -n 30000 -k https://throughputtestzet.ziti:443/
tunnel command:
sudo ziti tunnel run -i ./throughputtest.client.new.json
| Time taken for tests | Requests per second |
|---|---|
| 11.794 seconds | 2543.74 [#/sec] (mean) |
| 13.926 seconds | 2154.17 [#/sec] (mean) |
| 11.375 seconds | 2637.28 [#/sec] (mean) |
| 13.297 seconds | 2256.12 [#/sec] (mean) |
| 12.214 seconds | 2456.13 [#/sec] (mean) |
So although it's slower, it's not the dramatic difference that you're seeing. I also discovered that -c 300 seemed to be the magic concurrency number that gave me the best throughput overal.
Addendum - OpenZiti CLI commands
Here's how I created my OpenZiti test service in case you're interested:
ziti edge create identity user throughputtest.client -o throughputtest.client.jwt
ziti edge create identity user throughputtest.host -o throughputtest.host.jwt
ziti edge create config throughputtest.hostv1 host.v1 '{"protocol":"tcp", "address":"localhost", "port":8000}'
ziti edge create config throughputtest.intv1 intercept.v1 '{"protocols":["tcp"],"addresses":["throughputtest.ziti"], "portRanges":[{"low":443, "high":443}]}'
ziti edge create service throughputtest --configs throughputtest.hostv1,throughputtest.intv1
ziti edge create service-policy throughputtest.bind Bind --service-roles '@throughputtest' --identity-roles '@ip-172-31-44-171-edge-router'
ziti edge create service-policy throughputtest.dial Dial --service-roles '@throughputtest' --identity-roles '@throughputtest.client'
ziti edge create config throughputtestzet.hostv1 host.v1 '{"protocol":"tcp", "address":"localhost", "port":8448}'
ziti edge create config throughputtestzet.intv1 intercept.v1 '{"protocols":["tcp"],"addresses":["throughputtestzet.ziti"], "portRanges":[{"low":443, "high":443}]}'
ziti edge create service throughputtestzet --configs throughputtestzet.hostv1,throughputtestzet.intv1
ziti edge create service-policy throughputtestzet.bind Bind --service-roles '@throughputtestzet' --identity-roles '@ip-172-31-44-171-edge-router'
ziti edge create service-policy throughputtestzet.dial Dial --service-roles '@throughputtestzet' --identity-roles '@throughputtest.client'
Addendum - Docker test
I used this docker command to run the docker web test:
docker run -d -p 8000:8000 crccheck/hello-world