I’ve been using Slack’s Nebula to provide an overlay network to my Kubernetes cluster for the past 6 months and it’s doing a great job.
I’ve discovered OpenZiti few days ago, went trough OpenZiti documentation and some of your videos on YouTube, and I can’t really find a reason to move to OpenZiti.
I find Nebula much more scalable, it supports multiple lighthouses (ziti controllers) and we can load balance traffic on those lighthouses, while OpenZiti only supports a single controller for now. Also, Nebula doesn’t need alot of resources, lighthouses can run on a 1GB RAM 1vCPU VPS, what about Ziti ?
OpenZiti can win the duel with its SDKs, but do people really want to re-develop parts of their applications instead of just putting them inside the overlay ? Isn’t it simpler to deploy a Zero Trust Host Access network ?
Are there reasonable reasons to move from Nebula and use OpenZiti ?
Nebula is a nice overlay solution, whereas OpenZiti provides zero trust overlays. You are correct - you can use the SDKs to embed the OpenZiti zero trust overlays directly inside the process space of our apps (so that your overlays extend everywhere your apps go…without you needing to deploy separate agents or gateways…for both cloud side and user/admin side). You can also use OpenZiti endpoints to gate your overlay if that is a better fit for your need.
As an example of the zero trust networking, companies like Ozone use OpenZiti to make their Kubernetes clusters fully private - unreachable from the networks with all inbound firewall ports closed, without needing to permit certain IPs (simplifying IP address, firewall and ACL management).
How does Ziti currently scales ? Does a single controller running on a low spec server (like 1 gig of RAM and 1 vCPU) would be enough to handle 100+ connections per second ?
I just saw that OpenZiti uses x.509 certs, doesn’t this increase the size and response time of the handshakes ?
Also, I’ve saw that OpenZiti uses ABAC to authorize identities to use service, which is kinda expected, because it is x.509, is it possible to use HashiCorp Vault with OpenZiti in order to store certificates and informations about the stored certificates ?
OpenZiti uses bbolt etcd key value store for identities, services, etc, right ? What happens if I have multiple Controllers on my network ? I mean they’d have two separate databases, which means that all my controllers cannot authenticate all my clients, unless I sync all those bbol k/v stores ?
I’m aware that OpenZiti is in it’s premises, but I kinda want to move away from Nebula for the reasons you mentioned above (no ACL management, firewall, etc), but I’m not really sure that it’s mature enough atm.
I went trough some of open issues on your github, and I think that Ziti can be a good solution when it reaches the V1. Do you guys have a “goal” date to release that V1 ?
For horizontal scale, look to the Edge Routers over the Controllers. This is where the data plane flows, and you can add additional Edge Routers as needed. You can also dedicate Edge Routers to specific sets of services - see Ziti Services | Ziti for some info on the availability and scaling of Edge Routers
For low-end device resources, we focused on the Endpoints, e.g., embedding Ziti connectivity in low-resource IoT devices. Memory resources are measured in the low MBs, a single CPU is common.
Re: TLS handshakes, OpenZiti Endpoints create persistent connections to Edge Routers. So the TLS overhead generally happens at startup rather than during usage of the overlay. End-to-end encryption happens over these established links. OpenZiti Superpower - the Fabric - YouTube includes an overview of how the connections are set up, and OpenZiti Superpower - e2e Encryption - YouTube overviews end-to-end encryption.
Re: “Ziti 1.0” - it’s a hot topic, as Ziti is already in significant production use. “Distributed Control” is a feature under heavy active development. It includes distributing the control functionality among the edge routers. We’ve been holding off moving Ziti to version 1.0 until the initial version of what will be a phased set of releases. We expect that to happen relatively soon - with high confidence in this calendar year.
As @smilindave26 says and I will build upon, we built OpenZiti with zero trust and high-performance design principles.
This is why we utilise a smart routing mesh network as part of the data plane. This allows us to be outbound only at source and destination, rather than relying
on hole punching and a firewall. This data plane is already built for HA/HS.
Distributed control will allow bbolt etcd key-value store and other aspects of the control plane to disseminate data to all group members - see Ziti Kitchen - Mar 30 2022 - YouTube for more details on HA Ziti Fabric.
We are currently actively working on alternatives for our embedded x509. We started with SPIFEE - Apr 22 - OpenZiti + SPIFFE == SO GOOD™ - YouTube - and there is no reason this couldn’t be extended out based on community needs (fancy doing some development and a PR )
Happy to share further references to people who use OpenZiti in production to see though we have not given it 1.0 yet, it is used extensively by organisations with stringent production requirements.
)