How do I create a service that connects to a Subnet via a router instead of just a single host?

A few thoughts to add here.

I often have a similar usecase to the OP. I have propriety windows desktop software that I need to use to connect to remote hosts in customer private networks which don’t use traditional ports/protocols. While it’s great to have services for every remote endpoint it’s extremely impractical (100s-1000s-10,000s in a customer environment). I can think of two ways to solve this with tunnelers in this case.

  1. Create subnet services…issue here is we deal with subnet collisions and I don’t have a simple way to enable/disable services on the client side or up a 1:1 NAT for each remote network.
  2. Zitify a VPN tunnel like Wireguard and use a service to govern access to the remote network. That would allow me to connect to a remote network via the WG client and use my own proprietary software.
    Make sense?

In our case I don’t think Proxyjump wouldn’t really solve our problem since our devices don’t have ssh on board.

To be more clear as to what we are trying to do, below is the configuration we use currently and which we would like to replace with a Ziti zero trust network:

We have proprietary software on our Admin teams PC’s which we are unable to change/Zitify, which needs to have TCP communication on a single port XXX with each device addressing them on the 192.168.1.0/24 network. All of our devices generally always keep the same IPs.

Any suggestions of what we could do?

Sounds very much like our situation.

I would suggest the following, and perhaps the Ziti team has some better ideas.

  1. If your target ports are well known, your target devices don’t move (IE they use static IPs, reserved DHCP, or DNS to be reached), and there’s a manageable and predictable number of them (you know exactly where to reach each device and there aren’t 1000s of them and growing), then the best approach is to make a service for each target device and host the services from the tunneler endpoint on prem. This is the best right way and the MOST zero trust way without embedding Ziti onto the IoT devices directly. NetFoundry makes this easy. It also eliminates your need for TeamViewer.
  2. If your target ports and devices are unmanageable (like in my world, different ports for different OEMs, many 1000s of devices, etc) then you can use Ziti to jump to your on prem windows box (RDP) and use your software there. Not great but I’m sure that it’s basically what you’ve been doing and it gets rid of TeamViewer.
  3. You might follow my line of thinking and work towards a Zitified VPN connection. Wireguard probably.
    Would love feedback from the community here as well!

FL

3 Likes

We have 1000s of sites and growing each with over 10 devices and it not possible to run the software on the remote PC since it doens't have the resources to do so. So is quite unmanageable to make a service for each target.

Something along the lines of a Zitified wireguard connection would be more along the lines of what we need to do.

thanks for the suggestions

For Zitified WG, here’s what I’ve been thinking:

  1. The Wireguard “server” side would be on the remote site. The client (your users) will initiate the connection to the WG peer. This would be an issue without Ziti since you’d need customers to have open firewall ports and port forwarding to your WG server on the remote site. Ziti to the rescue!
  2. You need a Wireguard app running on the remote host alongside the Ziti app.
  3. You need a Ziti service that allows the client (your users) to reach the WG port on the remote WG system (at your customer sites)…so the client would attempt to connect to “wg.customer1.private” on port 51820, and that would terminate on the remote WG system to 127.0.0.1:51820. You’ll also need to configure the allowed IPs and such. The wireguard docs should help.
  4. From there on it should be a simple as creating the WG client configs for each client (your support team’s devices). These can be exported & imported as bundles and shared between techs. They’ll also need the ZDE on each machine (since without it they can’t reach the WG service on the remote hosts). This still allows you to maintain some level of least-privilege on who can connect to what sites.

Hope that helps!

While not currently available in production, we are developing a solution for our windows edge clients that intercept wildcard dns services. By using a wildcard dns service such as *.site1.com with our Ziti Desktop Edge client loaded onto the admin pc and each site PC you could connect to any device with a certain host name with any port. Service definition could be *.site1.com with 1-65535 or whatever ports. The load on the PC would be no more or less than what is currently running…likely less. Obviously a single admin host connecting to thousands of sites could be problematic. Is it bi-directional or only admin pc to IOT devices via site pc?

I'm really enjoying this discussion - thanks to you both for bringing the conversation to the community!

Quick Clarification

I want to clarify this statement:

I think we're all on the same page here but that's not relevant to overall discussion any more right? The number of services for each target is "the problem" now because at this time that feels unmaintainable due to the sheer number of services? I think that's the case and I can understand that feeling since you need 10x+ the number of services for every site. I won't lie, I've not seen 100,000 services in use anywhere yet - but I've seen 5000+ on some 'bigger' networks. Think developers: one service for RDP, one for SQL Server access, one for file shares, one for the web server (etc) Those are often on the same machine but still - the idea is the same. If you're going that route you CLEARLY would want some kind of automation in place to be able to use the APIs and discover/add the services. I wouldn't want to do that manually at all :slight_smile:

One possible solution "right now"

@skipb wildcard DNS should work right now. Also funny we had the same thought!

I don't know if you could consider this but I was talking to the team and we came up with what might be a pretty good solution for now but it has one vital aspect - you're willing to modify the hosts file on that windows machine at the sites and add entries for your iot devices. If that is a possibility for you - I have what I think - is a pretty cool option for you... It'll be a pain to setup the first time but it's "a solution".. It goes like this...

  1. Add "domain names" for all your iot devices on the remote site machine in the hosts file
  2. Make a wildcard DNS service for each site
  3. Access those sites from a tunneller in windows

I have a short proof of concept/demo video below:

2 Likes

Thanks Clint for the brilliant solution. :grinning:

Not sure if it will work in our case as both the proprietary software and the IOT devices use only IPs (the proprietary software loads the IP configuration configured directly onto the device) and there is no possibility of using domain names. Will however give it a try and let you know.

Oh no - I think I understand what @skipb was saying now. I’m running a beta build of the Ziti Desktop Edge for Windows which has wildcard DNS enabled on it. I thought he was referring to the other parts of the ziti overlay. If you try this right now - I think you’ll hit a problem intercepting the wildcard dns. Might be worthwhile to wait for the next ZDEW release which will have that support.

We’ve been talking about enabling this exact use case a lot internally lately but I don’t think my demo video will work since the next release is still being worked on. Sorry about that :frowning:

I’ll try to put my thinking cap on and see if we have another solution, but right now it might be best to park this wildcard dns approach. Sorry about that

Hey all!

I really do think Ziti could solve this easily in 1 of 2 ways:

  1. When defining a service that uses a CIDR range, allow a 1:1 NAT :slight_smile: intercept 10.11.0.0/16 and map to 192.168.0.0/16 on host A. All we need to do is maintain a mapping table.
  2. Allow MutEx / selectable services on radio buttons in the client. If the CIDR is intercepting and passing through 192.168.0.0/16, then allow that to be mutually exclusive with any conflicting services…or at least manually selectable as an option in the client. Then the user can opt-in/out of that service from the client.
    Both of these things allow us to not need something like Wireguard underneath.
2 Likes

I am not sure about how the inner workings of Ziti function however in terms of logic we would need to be able to choose

  1. For a given service be able to choose which endpoint to connect to
  2. When connecting to a given endpoint be able to select to activate that service

The only other way you might do this is to allow a special wildcard DNS match. The service would intercept anything at *.client.zt and pass JUST the * portion along for local lookup. Importantly you’d need to look up local IPs or DNS entries this way. Ie. 192.168.3.5.client.zt or hostname.client.zt or some.fqdn.com.client.zt.

That would allow one service to reach any destination service without needing to intercept CIDR ranges.

@TheLumberjack thoughts? It’s really close to your suggestion but with the critical element of not needing to edit host records on endpoints. Just use the service host endpoint to perform the lookup or route.

1 Like

It's funny you mention that because we've been noodling on this exact issue since it cropped up and that exact sort of idea was considered. There's "work" that would need to be done to support this sort of thing. We also need to really mull over it to think about all the good things we could do (and any bad things that might come from doing that)... We've been considering all sorts of alternatives... What if the "configs" supported the notion of "find/replace", what if the config supported the basic notion of one-way NAT... etc. I've considered a purpose-built client or 'host' that isn't the 'standard' type of tunneling app for this sort of thing etc... Lots to consider to do what's best for the project on the whole.

We don't know exactly what we'll do to solve this issue but it's probably one that's worth solving. Particularly since two people in the community seem to have similar, if not entirely the same needs.

I think we'll come back to this soon it's just not at the top of our queue just yet. With two people in the community asking for it though - it personally ranks higher up my list "things that are next" list.

Thanks for the idea and thanks for sticking with it!

1 Like

Thanks for the dedicated interest in this issue. This is definitely not a simple issue to solve and always best to think things through properly.

I just wanted to highlight that in our case, the device configuration program only goes by IP address and it will not accept DNS records unfortunately. The devices are initially found via MAC address and then an IP gets assigned internally to the device and is used for all further communication.

DNS wildcards are an elegant way of solving such issues but only if they are supported by the programs being used.

Yeah, @Danieleb you might have a bigger issue on your hand…are you able to select a subnet to discover devices in (layer 3) or are you restricted to only being able to use broadcast / arp discovery (layer 2)? The former is do-able on Ziti, but the latter…you might be better off with ZeroTier or another L2 bridging-style technology. Another thing…if your application requires ICMP you’ll be a bit out of luck here too.

For us, we just opted to deploy a Windows VM on demand for these use cases.

And thanks @TheLumberjack for the feedback!

Normally we configure all our device beforehand, so we don’t really need arp discovery / icmp. What we do need however is

  1. address the devices locally with their actual IP (remote side) even if it means adding a route rule locally to direct it through the right interface.
  2. Access only one TCP port on the remote device

thanks for the suggestions

1 Like

I have been successfully using ziti-edge-tunnel to connect from our Admin machine to one our iot sites in order to test reliability and would really like to extend using ziti overlay to all of our sites.

I have been thinking about it for some time and while all of our client sites could potentially connect to the ziti network using the regular “ziti-edge-tunnel” to expose their defined services (I assume that even if all sites have same IP ranges this would not cause any issues). For our Admin nodes however connecting to the iot sites individually via IP addressing is problematic since all sites have the same IP ranges.

In order to solve a situation of this nature would need the redirection of intercepted IP traffic from an admin node to be directed exclusively to a specific iot site. ie be able to select which entity an admin node wants to connect to.

I am not sure if this is doable however

The simplest and cleanest solution I can think of would be when running an Admin tunneler would be to have a commandline override option to allow the possibility of directing any intercepted IP traffic (192.168.1.0/24) towards a specific allowed service provided by single site (site1.com)

.\ziti-edge-tunnel run -i .\AdminSvcClient.json -endpoint Site1SvcServer

OR possibly

.\ziti-edge-tunnel run -i .\AdminSvcClient.json -wildcard site1.com

in the case of no endpoint being specified, the tunneler would function as before. This would allow flexibility in deciding if an Admin node connects to the entire network using the tunneler or if only to a specific site.

1 Like

This is a really interesting problem that I am yet to tackle… and definitely keen to learn what @TheLumberjack recommends.

From what I know, this would be my first attempt

#1 integrate the ziti sdk into the iot app for native E2EE

#2. enroll an identity on each device to act as the local server/host

#3. create a service and intercept per device, proving a separate ziti private dns per device

#4 bind the server identity on each device to its corresponding service / intercept

#5 create a client identity and configure it so that it can call each of the services to consume the data on each iot device

#6 create a zitified reverse proxy, enroll the client identity on this server, and program the reverse proxy to dial each service separately in sequence

#7 run the reverse proxy on a scheduled basis that calls each intercept separately to consume the data and store it into a centralised database

PS… if you have lots of devices, you can script steps #2 to #5 to make it light work :slight_smile:

I dont know if this will work… but conceptually… I think it makes sense.

I have found building zitified reverse proxy servers to be very flexible in re-routing traffic in and out of the ziti overlay… especially using golang… which makes it quite simple.

Something to consider…and modify to suit your situation.

Yeah, you basically want some kind of "pinned ip" mode which allows you to specifically pin intercepted IP traffic and directs it to a specific, addressable terminator/endpoint to offload from.

When thinking about it internally, we had been thinking about some kind of simple subnet math to help here. For example if there were some way to apply a transform on the other end on say one or two of the IP octets... Giving you the ability to do something similar to NAT'ing... Where 100.64.1.x would be a /24 to one site, 100.64.2.x would be another /24 to another site...

Right now I can think of two ways you can do this today. Both have drawbacks...

Many Identities -- 'n' Identities per Admin

You could do this today with one identity per site. Since you're proposing to stop/start the tunneller - if you had one identity per site you could create a single service which is defined for your CIDR block you need, grant all your 'site tunnellers' the ability to bind that service, and then start the tunneller on the 'admin' machine using the proper identity. I don't know if that's tenable or not tbh. Maybe? I'm guessing probably not. The obvious drawback here is you need 'n' identities (one for each site) for every 'admin' you want to enable. That seems "not so great"...

Script Access as Needed

You could also do this by automating/scripting your target identity. Here you'll be able to make 1 identity per 'admin', but you'll want to enable that admin to be able to control what services they can dial. Here you would have 1 'client' identity per admin, 1 'site' identity per site but every time you want to access a site you need to update the policy that grants the client access... So to keep it simple what I would do here is simply have 1 dial policy for each site and then update the 'admin' identity with the proper site. So basically it'd be something like:

# remove access from site 1 while granting access to site 2 (presumably the admin had an attribute of "site1" before updating to "site2" 
ziti edge update identity ${admin.identity.name} -a "#site2"

That would work if you had a 'site2' service policy in place.. The downside to this is each admin basically needs to be able to add/remove themselves from this policy - they need ziti CLI access/creds...

If you want to see a demo of what I'm talking about, like via video let me know. Maybe that made sense, maybe you want to see it in action as a demonstration? I'll make a video if you think it'd help.

I'm going to surface this problem back to the team and see if your post or my follow-up jar any thoughts.