Configuring Service to access a Web Interface

Support General Questions
1

Hi all. Apologies if this question has been answered somewhere else in the forum.

I have just started diving in to OpenZiti and am struggling with knowing what I am doing with this service creation and the docs don't seem to cover this. So far, the controller (with console) is deployed and two edge routers are deployed as well and are all working.

To figure out the services, I have a VM that's hosting an app with a web interface and a desktop that I'm treating like a client device. Both have the Ziti Edge client installed. (The VM is rocky linux and the ziti-edge-tunnel is installed correctly according to the Ziti docs. And, the desktop is Windows and installed successfuly.) Both are registered in the Ziti Admin Console and have reported back in with their client and OS info. So, as a baseline, everything seems pretty normal and correct so far.

To create the service, I used the simple service creation wizard in the Ziti admin console. I selected the identity for desktop for what device that can access the service and provided an address (portal.domain.com for example) and port 443. Then, for the hosting configuration, I selected the identity of the VM that hosts the web app I wanted to access, left the protocol on TCP, entered 127.0.0.1 for the address, and used 443 for the port. Saving the service worked and all the necessary configs and policies were created successfully.

Now, I'll admit that I don't fully grasp the hosting configuration. I understand the accessing configuration: what devices are going to be trying to reach Thing A at what address and over what protocol. But the hosting configuration leaves me with more questions and I'm relying on some online guides from OpenZiti to try getting it running. I understand that the identity selected is what device will be hosting the application we're trying to get to (as far as OpenZiti cares: what's at the other end of the tunnel) but I don't quite understand what I've read about the address to be entered here. I initially tried using the actual DNS name of the VM (name.domain.com for example) but, when that didn't work, I edited the service to use the loopback address instead (127.0.0.1). The port was left at 443 because that's where this particular application's web portal is presenting on the hosting VM. I would assume that this port would change to reflect the specific port used by an application's web portal if it were different (1280 for example if I wanted to try putting the Ziti Admin console at the service end of a tunnel).

With this configuration, I would assume that I should be able to go into a web browser and navigate to https://portal.domain.com (as used in the example in the config above) and reach that web portal so long as I am accessing that from the desktop that has the Ziti edge client installed.

Thus far, please feel free to correct me if this is failing because my understanding of either these configs or that expectation of how the connection should work is faulty and the root of this issue.

For further context, both of these devices are on the same network. And, I was already able to access https://portal.domain.com normally in my browser. But, so far, in the Ziti Admin Console, I cannot tell whether this service has been used to know whether this configuration is correct. I expected that the intercept config applied on the desktop would capture that traffic and send it over the tunnel Ziti established with that service. Basically, that Ziti would take precedence as the 'primary' access method. And my hope there was that I would be able to configure services first, confirm access through the service, then block the traffic on the underlay network to get to real ZTNA.

So, this is the 3rd area where I may also be confused or may have not found the correct information in the docs.

Any help would be greatly appreciated. The app seems amazing and I'd love to really get a grasp of it. And it feels like I'm nearly at a solid initial footing.

[Unrelated, a GUI for the Linux edge client (tunneler) like there is for Windows would be amazing for desktop Linux installs with a desktop environment since Microsoft has murdered Windows and Linux on the desktop has stepped into the gap.]

2

Hi @bvh welcome to the community and to OpenZiti!

Thanks for all these details... You have a lot going on in this one post and I'll do my best to be brief, yet complete... The example you describe sounds to me like you are describing what is documented at Your First Service | NetFoundry Documentation I'll leave that link here in case you didn't find it yet.

understanding host configuration

When using the simple service wizard, the "hosting configuration" will simply indicate the location of the service once it exits (or egresses) the OpenZiti overlay. You understand how it enters (or ingresses) the overlay, but for the example you are describing you need to exit the overlay and go back to plain old IP underlay. MOST people will start either with "zero trust network access" or "zero trust host access".

The difference between ZTNA (network) and ZTHA (host) really just comes down to how much "trusted zone" is between the tunneler and the target service. If you are traversing more than just the host network and traffic is sent outside the host across the remote network (all by classic IP underlay), well that would be over the network or "ZTNA".

If you are targeting an application that is on the same machine as the tunneler, that traffic stays on that host. That is considered to be "ZTHA".

When you set up the 'hosting configuration', you specify the "identities that can host the service". This is one or more identities. You can choose the identities to use by using the @ symbol to specify one or more identities. OpenZiti has other mechanisms to specify identities as well but for now, let's leave those off the table. Since this is your FIRST service, it's easier to just use direct @-based identities. So yes, you pick which identity will "host" the service. This is where traffic exits the overlay back to the IP-based underlay.

The key to this section of the config is the 'address' fields. This value is relative to wherever that "hosting tunneler" is. That make sense?

both machines on the same network

Having both devices on the same network sure sounds like a plan at first but it can lead to misleading results. It can also lead to the confusion you're indicating:

I cannot tell whether this service has been used to know whether this configuration is correct

If you want to do that, it's fine, but I would make sure the hosting device has the firewall block all inbound connections. This, in effect, emulates being in some other network. Once you do that, you will know when OpenZiti is up and running for real because it will be the only way to access your target app. That also makes sense?

linux UI

It's been in the works for several years. We have recently started to hear more and more rumblings in the forums about having such a UI. It's planned, but we don't have it staffed at this time.

I have a few YouTube videos I've made through the years that cover the topic...

Hopefully all this helps?

3

Thanks @TheLumberjack !

I had been following that 'Your First Service' guide. And, it was honestly really helpful for understanding the underlying configurations that are created by the Simple Service creation wizard in the Ziti Admin Console. What I wasn't entirely sure of when following that guide is what, necessarily, would translate to entries required in that setup wizard from the admin console. I can absolutely follow 1:1 that script to everything shown as created in that completion window. I just seem to be missing the simple stupid step of understanding what values are intended for that exit / hosting configuration. :sweat_smile:

If I understand correctly (and the topology diagram supports this understanding), what I am attempting to create is ZTHA. I'll have client devices initiating a connection which the Ziti Edge Client intercepts with that config, sends over the Ziti overlay, and then exits onto the host where that application resides (obviously having the ziti-edge-tunnel installed on it as well for that to work).

I selected the identity @VMname that's for the VM hosting the application. That much made sense. (And I think you're referring to the attribute values that can be used there in place of identities. Amazing feature that I hope to take advantage of soon once I can sort this baseline out.)

Your explanation there of "The key to this section of the config is the 'address' fields. This value is relative to wherever that "hosting tunneler" is." makes a lot of sense and I think is finally giving me the right frame of reference for knowing what should be going in as the address. Because the traffic is set to exit the Ziti overlay at the tunneler on the identity @VMname, the address specified then needs to be the respective address to that exit point (on @VMname) where the application can be located (sort of like defining a next hop). In this case, since the VM with the ziti-edge-tunnel installed is also the VM hosting the application (because this is ZTHA I am trying to accomplish), the address specified should be the loopback address: so that it points straight back to itself to direct that traffic. Is that the correct understanding and application? And then the port selection for the hosting configuration portion of the service is just the port which that application is configured to be exposed on. (However, by comparison, if this were a network entry point for applications elsewhere on the subnet as shown in the other topology where the host running the ziti-edge-tunnel is acting more like a traditional VPN endpoint, the address information relative to that Identity would be the actual IPv4 address of that other VM on the network with the desired application.)

If that all is the right way to look at it and configure it, then I think this service may actually be almost entirely correct at the moment.

An interesting question this leaves me with then is: Is Ziti allowing us to do some fun things with port mismatches for intercepting traffic on any port we want and then 'depositing' it to the correct port on the exit side? Totally beside the core question. But I like the fun this opens up if that understanding is correct.

As for testing on the same network, makes perfect sense. I was thinking of going one step further toward the end state and only allowing 3022 to that vm from the ziti edge routers IP addresses on the underlay, really force myself to make the service work one way or another. :joy:

4

you got it. if you're doing ZTHA the address is always some local address to the tunneler, be it 127.0.0.1, ::1, localhost or some other local IP. take special note of localhost. On some machines in the past we have seen the OS favor ipv6 for localhost (::1) when the app was bound to IPv4 only (127.0.0.1). This was very difficul to discover so I tend to favor using 127.0.0.1 to eliminate any odd confusions like that.

It surely can be used to translate a port that way, yes. You can intercept first.service.intercept:443 and send that to 127.0.0.1:4443, intercept second.service.intercept:443 and send it to 127.0.0.1:5443 and so on... There are other interesting (and advanced) and fun ways to use OpenZiti as well such as 'addressable terminators' for services like ssh or rdp when you want to send traffic to specific identity while only having one service declared/defined...

Sounds like you're on the right path now and it's clicking for you. Have fun :slight_smile:

5

Thanks for taking the time to help and explain. I had stumbled into a working config but now understand why and how. That tip that the address in the hosting config is where the host with the tunnel's exit needs to send that traffic next was the key to really understanding it.

Out of curiosity, is there a session log in the Ziti Admin Console I could look at to see evidence of sessions initiating and all?

6

yes/no. It's not captured in the ZAC but you can use the ziti CLI to stream events:

ziti fabric stream events --circuits
{"namespace":"circuit","event_src_id":"NetFoundry Inc. Server s0GbhVWDq","timestamp":"2026-04-22T16:30:32.977941675Z","version":2,"event_type":"created","circuit_id":"4WnSHPAaNQz5LA55JsY5qK","client_id":"cmoa9r6qn2wpy93qkey078vow","service_id":"2IKzw4GBM5G9t1oJqWldtt","terminator_id":"5rr4YsGgsxXybe0s2WeX0b","instance_id":"","creation_timespan":2079737,"path":{"nodes":["eM0NBWcsdI"],"links":null,"ingress_id":"4XZVSgW90CX1OiSICpw0ua","egress_id":"5aqwNxjbht7HmOv3ZdsMYL"},"link_count":0,"path_cost":1073741823,"tags":{"clientId":"NQUpmBRI5","hostId":"yYvpmBYI5","serviceId":"2IKzw4GBM5G9t1oJqWldtt"}}
{"namespace":"circuit","event_src_id":"NetFoundry Inc. Server s0GbhVWDq","timestamp":"2026-04-22T16:30:34.001059476Z","version":2,"event_type":"deleted","circuit_id":"4WnSHPAaNQz5LA55JsY5qK","client_id":"cmoa9r6qn2wpy93qkey078vow","service_id":"2IKzw4GBM5G9t1oJqWldtt","terminator_id":"5rr4YsGgsxXybe0s2WeX0b","instance_id":"","path":{"nodes":["eM0NBWcsdI"],"links":null,"ingress_id":"4XZVSgW90CX1OiSICpw0ua","egress_id":"5aqwNxjbht7HmOv3ZdsMYL"},"link_count":0,"duration":1023120521,"tags":{"clientId":"NQUpmBRI5","hostId":"yYvpmBYI5","serviceId":"2IKzw4GBM5G9t1oJqWldtt"}}

You can reproduce this example by using a separate ziti cli command ziti ops verify traffic which is how i made these events:

ziti ops verify traffic
WARNING no prefix and mode [] is not 'both'. default prefix of 2026-04-22-1634 will be used
INFO    generating P-384 EC key
INFO    generating P-384 EC key
INFO    waiting 10s for terminator for service: 2026-04-22-1634.traffic
INFO    successfully bound service: 2026-04-22-1634.traffic.

INFO    Server is listening for a connection and will exit when one is received.
INFO    found terminator for service: 2026-04-22-1634.traffic
INFO    found service named: 2026-04-22-1634.traffic
INFO    Server has accepted a connection and will exit soon.
INFO    successfully dialed service: 2026-04-22-1634.traffic.
INFO    traffic test successfully detected
INFO    client complete
INFO    Server complete. exiting
7

Still fighting this one a little bit and wanted to check in in order to rule this one out.

For the intercept half of the service that is set up: I understand that the address is what, when navigated to, Ziti is going to intercept and pull into the Ziti fabric and route according to this service's rules. However, I want to confirm, does the address need http:// or https:// ahead of it like https://app.domain.com or is just app.domain.com fine? Technically, defining port 443 tells us that it is https:// but I don't remember the guide specifying a syntax for that field.

Oh, and I guess I should ask in a tangent to that, will Ziti treat an fqdn as a different address than the underlying IP address, thereby requiring separate services to account for people navigating using both methods? (ex. going to app1.domain.com which is 10.1.1.11)

8

This is entirely up to you and whatever is on the other end of the connection. It is perfectly valid (albeit strange) to use http with port 443. For example NetFoundry will often use https on port 80 because ports 80 and 443 are often allowed through firewalls without much hassle... It just depends on the service you are sending data to and whether or not it expects http or https. Modern browsers and tooling is often friendly, adding :443 or :80 accordingly when a user leaves the port off, but it's entirely up to you and how you deploy the service....

Nuanced question/answer here... First, know that it is always safest and easiest when you are learning to just keep the DNS entires separate. I won't ever have a first-time/new user 'shadow' a domain unless they understand ziti and know what they are doing and how to debug because it causes confusion -- just like this...

Continuing.... It will depend most notably on DNS caching. Let's take a REAL example... NetFoundry uses Mattermost for our instant messaging needs. We self-host mattermost. For example, I can resolve mattermost.tools.netfoundry.io:

$ Resolve-DnsName mattermost.tools.netfoundry.io

Name                                           Type   TTL   Section    IPAddress
----                                           ----   ---   -------    ---------
mattermost.tools.netfoundry.io                 A      60    Answer     100.64.0.3

However, if YOU tried to resolve that (which you can, it's a valid FQDN) you will see:

Resolve-DnsName mattermost.tools.netfoundry.io

Name                                           Type   TTL   Section    IPAddress
----                                           ----   ---   -------    ---------
mattermost.tools.netfoundry.io                 A      300   Answer     15.197.139.43
mattermost.tools.netfoundry.io                 A      300   Answer     3.33.149.198

First you can see that the TTL on the live one is 600 seconds. If YOUR ttl is much longer then when an OpenZiti client starts, well it might have that 'old' (wrong) IP for a long time. That is no good - that's why when our tunnelers start you might see a log like:

[2026-04-22T00:33:46.925Z]    INFO ziti-edge-tunnel:tun.c:165 flush_dns() DnsFlushResolverCache succeeded

We need to make sure the FQDN when it gets requested has an OpenZiti delivered IP...

Hopefully that all makes sense... It seemed like an 'easy' question but it's pretty nuanced and that's why I do not recommend shadowing a FQDN until you understand OpenZiti... Totally doable once you know how it all works but until then, can be confusing.

I recommend adding .ziti to all your intercepts and then it's super obvious! :slight_smile: my.cool.service.ziti....

9

I had someone point out to me that you might me in your intercept config -- for that the answer is definitely "no". the address is just the address of the local / remote machine. the http/https stuff you add to the browser's address accordingly. So for the intercept.v1 config -- no http no https.

10

Hey @TheLumberjack

Sorry for the continued questions. I am still struggling with this much more than expected and have been checking through other discussions on the forum for nuggets of info in the hope that I can get to the root of why traffic doesn't seem to be making it through the Ziti overlay for these sample services.

At this point, I think the former assumption that the Ziti overlay infrastructure was healthy and functioning correctly was incorrect. Revisiting the current deployment: I have 1 RHEL VM hosting the controller and ZAC, and 2 RHEL VMs each hosting a Ziti router. These were all deployed according to the guides and did not use docker, they are 'thick' installs.

Both Ziti Edge Routers were enrolled with their JWTs and are showing up in the admin console as registered and connected (if I am reading the interface correctly)

However, to try to dig into why these services are not behaving, I ran """ziti ops verify traffic""" in the CLI on the controller VM. It prompted for auth so I authenticated as the Ziti default admin account and this is the output I get, which seems to suggest that the Ziti Edge Routers are not communicating with the Controller correctly.

image
image1588×1193 490 KB

These VMs are all on the same network (same subnet, no vLANs), do not have any traffic restrictions set between them on the network, and have the inbound port allowances set for firewalld as the guides indicated (1280 on the controller and 3022 on the routers).

Am I interpreting this command output incorrectly? What would lead to this discrepancy between the ZAC and the command output if these edge routers are somehow knackered and not communicating with the controller? And then, the stupid but important question: what is missing that would get them communicating and reporting back properly (presumably also passing traffic sent into the Ziti fabric properly then as well)?

I would have assumed that successful enrollment of the Ziti Router to its identity with the jwt would have indicated that everything was happy between controller and router.

11

Here's how they show up in the Ziti Admin Console

image
image1553×415 18.4 KB

12

Yes you are interpreting it properly. What you're being told is there are no edge routers availalbe to connect to. This is often one of two problems. One - the advertised addresses of the controller is somehow incorrect and the routers are not showin up as 'online' or that you have no policies allowing the router to be used by your client. I'm going to guess it's the latter - your policies are incorrect.

OpenZiti requires everything to be authorized, including what routers an identity can access. As you have two routers installed, I'm going to assume you have haven't given these routers a policy that would allow identities to use them yet.

Run:

ziti edge policy-advisor identities -q

I would expect you to see something like

OKAY : clint (1) -> cdaws-controller (1) Common Routers: (1/1) Dial: Y Bind: N

(note the Common Routers: (1/1) part)

But I'm going to bet you don't have any and it'll show 0/2.

Either the routers are not online because they cannot find the controller, or there's no policy allowing identities to use the routers. Since you show them as online, I expect it's the policy

13

Thanks for that clarification. Now that you've pointed it out, that makes sense - part of zero trust and all. :sweat_smile:

So, I need to specifically authorize my admin user to reach the Ziti Routers and then that should start to behave. I'll dig into that and see what those logs can show me.

Zooming out from the command interface for a second for a sanity check:
The Ziti fabric should (traffic wise) 'just work' so long as

-the controller has 1280 open

-the routers have 3022 open and are registered in the controller

-and the client devices (with the ziti clients installed for fabric entry/exit) can dns resolve the routers and controller and can be reached on the network by the controller and routers?

Given the weird difficulty getting the service working, I'm trying to throw out my assumptions about the fabric I set up and see if there's anything that I missed while following the docs and then following the simple service creation wizard in the Ziti Admin Console:

https://netfoundry.io/docs/openziti/guides/deployments/linux/controller/deploy

https://netfoundry.io/docs/openziti/guides/deployments/linux/console

https://netfoundry.io/docs/openziti/guides/deployments/linux/router/deploy

https://netfoundry.io/docs/openziti/reference/tunnelers/linux/redhat-package

Do the Ziti Routers and the Controller need anything more than the routers being enrolled with the controller in order for the fabric to be prepared to work and host service traffic?

14

For some added background, maybe some that points toward the issue. All of my host identities are showing up as 'Offline' (gray instead of green) after being enrolled successfully. Is that normal or indicative of something being broken?

image
image1902×565 36.8 KB

I also looked at the Visualizer (really cool feature to stumble across!!). At least Ziti knows that both services aren't working and shows an error on the connection lines for the services I've been testing with (simple navigation to the web portal for those VMs).

15

Yes but you also need to authorize all the identities to use routers. I generally make a #public/#all attribute for my edge router policies (any router with #public can be used by #all identities) along with a #all/#all service-edge-router-policy (allowing #all services access to #all routers).

Could you please re-run:

ziti ops verify traffic

I'd like to confirm that it's working - you have a clean run? Please show me?

and also run:

ziti edge policy-advisor identities -q

I'd like to see the policy advisor output please, to verify policy-advisor is happy.

You need that service-edge-router-policy too I referenced above. The identities for each router need to be authorized to 'bind' services too.

The next thing to do is to look at the logs at the tunnelers. The logs are quite good and should give you more ideas as to what might be wrong.

16

Hey @TheLumberjack ! Thank you! Your explanation finally gave me another lightbulb moment. Using the Simple Service creation wizard created the Service Policies I wanted for dialing and binding. But, it didn't do anything with the Router Policies and Service Edge Router Policies (which is just Service Router Policies in the tab text (in case that's an error)). Those are things which, presumably, would have been setup by the admin (me) during the initial setup of the whole Ziti network. So, effectively, there was a final setup step after deploying the controller, console, routers and then enrolling the routers: I needed to create those baseline policies to allow those routers to be used for anything (identity or service).

image
image1494×346 26.1 KB

I'll go work on all that for a bit and be back shortly with that ziti cli output to hopefully confirm it all behaves now.

17

Some progress and more questions. I think I either have the admin policy setup incorrectly or as the wrong policy type. As is, I created a Router Policy to allow my default admin (the identity attribute) to access the 2 Ziti Routers. But, when I ran the ziti ops verify traffic command, it once again tells me I don't have access and the proceeds to create 2 new identities and associated service policies (shown below).

image
image1412×350 29 KB

But, some progress has been made!! Based on your response, I've created a Service Edge Router Policy to allow all my services to run through both Ziti Routers. And I have split the Router Policy setup in two - one policy to allow workstations to hit the Ziti Routers and one policy to allow the VMs (hosts) to hit the Ziti Routers. And now, at least the lone Windows client is showing up as online and the visualizer is showing life between it and the Ziti Routers!!

image
image1411×588 70.7 KB

Maybe in the realm of a feature request but this Visualizer is amazingly helpful and I would argue it deserves more visible access (either as its own tab in the Nav menu under the Core Components header or baked into a rich info interface that opens when you click on any given identity or service).

18

Hi @TheLumberjack .

Good news! After fighting it a little bit longer, I realized that the VM hosting the web service was taken offline a few days ago (wasn't offline originally when things were broken but was offline at the time of creating those Router Policies). But, it is now online and behaving as expected. I had taken the entire homelab offline over the weekend to reconfigure the hosts and add some capacity in order to host Ziti and some more applications in perpetuity. And, by the time I had sat down with Ziti again with the network admin hat back on, that upgrade was a distant memory. You know, I'd fire the infra guy but I'm sort of stuck with him. :joy:

In this interim, I also dug into the documentation and some other forum posts. And, those raised a couple of questions, mostly around DNS resolution for the Ziti fabric.

First, and most broadly, do you need to actively mess with the DNS configuration if all devices on the underlay network have the same DNS server configured? Simple example, but everything here (being on the same internal network) would be pointing to the network's gateway (ex. 192.168.1.1) for DNS resolution which, independent of Ziti, works and resolves everything normally. The documentation seems like it's assuming more DNS complexity than that.

Second, mostly an extension from question 1, since systemd-resolved was not running by default on the RHEL hosts for the devices running the ziti-edge-tunnel, did I inadvertently dodge the ziti tunnel from configuring and using the default 100.64.0.2 nameserver and 'force' it to use the same DNS server as defined for the host?

image
image1318×731 124 KB

(Small aside, love to see the CG-NAT ranges being used.)

As always, I super appreciate the input. You have already filled in a couple key gaps in my understanding of Ziti.

19

The infra guy should not need to touch DNS imo. We knew he was trouble from the start, didn't we! :slight_smile: Seriously though, generally speaking, you are not expected to need to modify DNS in order to use OpenZiti. It's possible on some linux variants that you might need to but when you're using ziti-edge-tunnel (as the doc you're showing) you really should not need to modify anything. I would encourage you to leave it alone in general. The doc is perhaps confusing, it's trying to be helpful, not mandate that you modify DNS. I'll consider revising it.

As for "systemd-resolved was not running by default" - the ziti-edge-tunnel should:

  1. try systemd-resolved via libsystemd (D-Bus directly)
  2. fall back to systemd-resolved via the CLI tools (resolvectl, or older systemd-resolve)
  3. fall back to resolvconf
  4. fall back to editing /etc/resolv.conf directly (and warn -- this is the degraded mode)
  5. refuse to start if /etc/resolv.conf is a symlink into systemd-resolved but none of the systemd paths
    above worked (rather than fight a losing battle and break your DNS)

So on a stock system without systemd-resolved running, you'd usually land on step 3 or 4. You shouldn't need to configure anything yourself -- the tunneler picks the path that fits your box. Or tries to anyway...

If you force the system away from the tunneler's DNS then you won't get intercepted names at all, for sure since the IP's that are returned won't be in the 100.x range...

Yah, it's an area we have always wanted to invest in but haven't been able to justify it too much just yet. It got a mild revamp in 4.2.0 of the zac so if you aren't on the latest you might try it.

hth

20

Thanks. Yeah, I never want to touch DNS configs anywhere other than at the authoritative DNS server for the network. Everything else had better be able to hang out at stock and behave. Haha

It definitely sounds like my situation here (systemd-resolved not running out of the box on RHEL distros) cascaded down to #3 in your example and just referenced resolveconf which is all the 'stock' DNS config the VM should and does have. Makes sense and I'm glad it's basically something that should work without requiring any thought.

Thanks again @TheLumberjack for answering so many questions. I am curious about how best to manage DHCP and DNS traffic in a fully Ziti-'d network. But, that's a whole different thread that may already be somewhere on the forum so I'll start digging there instead of tangent-ing this thread.