Basic installation trouble

Support General Questions
1

Hi all. I have come over from Reddit based on some interaction with a couple of contributors.

I first saw this project a few months back and took a stab at it, to basically get nothing accomplished by it, so put it on the shelf for later.

I’ve seen a couple of people actively promoting this project as a solution to the old “How do I access my home lab resources without normal VPN (CGNAT) or relying on those other providers?”

So I wanted to express my experience, and issues I see.

I had a couple of hours last night to make an attempt at getting this solution configured, so I thought I would document that process and relay what worked vs what didn’t.

Debian 12 VPS - I’ve labeled it VPS-DAL-1
(2) CPU/ 4 GB RAM/ 60 GB SSD

I have this VPS (as all of my remote and local hosts are) configured to work with ZeroTier, Tailscale, and my own Wireguard tunnel.

So I started with the “Host Anywhere” documentation and followed it step by step.

It seemed to be working fine, until it got to a point of “waiting for https://vps-dal-1.<mytld>:8441”, and that continued to spam the screen forever.

Eventually I guessed that something had crashed and went and looked at the log file:
/root/.ziti/quickstart/hostnameofserver/hostnameoofserver.log

Within that log I see:

[   1.196]    INFO edge/controller/server.NewController: edge controller instance id: cll4fxiiy0000hhn6nis1blr1
[   1.196]    INFO edge/controller/server.NewController: edge controller instance id: cll4fxiiy0000hhn6nis1blr1
[   1.196]    INFO edge/controller/server.(*Controller).Initialize: initializing edge
[   1.196]    INFO edge/controller/server.(*Controller).Initialize: initializing edge
[   1.210]    INFO storage/boltz.(*migrationManager).Migrate.func1: edge datastore is up to date at version 32
[   1.210]    INFO storage/boltz.(*migrationManager).Migrate.func1: edge datastore is up to date at version 32
[   1.214]    INFO edge/controller/internal/policy.NewSessionEnforcer: {sessionTimeout=[30m0s] frequency=[5s]} session enforcer configured
[   1.214]    INFO edge/controller/internal/policy.NewSessionEnforcer: {sessionTimeout=[30m0s] frequency=[5s]} session enforcer configured
[   1.218]    INFO edge/controller/server.(*Controller).Run: starting edge
[   1.218]    INFO edge/controller/server.(*Controller).Run: starting edge
[   1.223]    INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {idleTime=[10s] maxQueueSize=[1] minWorkers=[1] maxWorkers=[16] poolType=[pool.listener.ctrl]} starting goroutine pool
[   1.223]    INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {idleTime=[10s] maxQueueSize=[1] minWorkers=[1] maxWorkers=[16] poolType=[pool.listener.ctrl]} starting goroutine pool
panic: error parsing web configuration [web] at index [0]: error loading identity: open /root/.ziti/quickstart/vps-dal-1/pki/vps-dal-1-edge-controller-intermediate/keys/vps-dal-1-server.key: no such file or directory

goroutine 1 [running]:
github.com/openziti/fabric/controller.(*Controller).Runpanic: error parsing web configuration [web] at index [0]: error loading identity: open /root/.ziti/quickstart/vps-dal-1/pki/vps-dal-1-edge-controller-intermediate/keys/vps-dal-1-server.key: no such file or directory

goroutine 1 [running]:
github.com/openziti/fabric/controller.(*Controller).Run(0xc000b0e540)
        github.com/openziti/fabric@v0.23.45/controller/controller.go:273 +0xb45
github.com/openziti/ziti/ziti/controller.run(0xc0004df200?, {0xc000ad0330, 0x1, 0x1?})
        github.com/openziti/ziti/ziti/controller/run.go:94 +0xaa5
(0xc000b0e540)
        github.com/openziti/fabric@v0.23.45/controller/controller.go:273 +0xb45
github.com/openziti/ziti/ziti/controller.run(0xc0004df200?, {0xc000ad0330, 0x1, 0x1?})
        github.com/openziti/ziti/ziti/controller/run.go:94 +0xaa5
github.com/spf13/cobra.(*Command).execute(0xc0004df200, github.com/spf13/cobra.(*Command).execute(0xc0004df200, {0xc000ad0300, 0x1, 0x1})
{0xc000ad0300, 0x1, 0x1})
        github.com/spf13/cobra@v1.7.0/command.go:944 +  github.com/spf13/cobra@v1.7.0/command.go:944 +0x847
github.com/spf13/cobra.(*Command).ExecuteC(0x847
github.com/spf13/cobra.(*Command).ExecuteC(0x50b9f00)
        github.com/spf13/cobra@v1.7.0/command.go:0x50b9f00)
        github.com/spf13/cobra@v1.7.0/command.go:1068 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/cobra@v1.7.0/command.go:992
1068 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
        github.com/spf13/cobra@v1.7.0/command.go:992
github.com/openziti/ziti/ziti/cmd.Execute()
        github.com/openziti/ziti/ziti/cmd/cmd.go:80 +0x38
github.com/openziti/ziti/ziti/cmd.Execute()
        github.com/openziti/ziti/ziti/cmd/cmd.go:80 +0x38
main.main()
        github.com/openziti/ziti/ziti/main.go:51 +0x17
main.main()
        github.com/openziti/ziti/ziti/main.go:51 +0x17

So, OK. Let’s delete the directory and try that again.

Same issue.

I can’t seem to find any reference to this error anywhere else, so where does one go? Support is not really documented in this regard. It’s just kind of a “well it broke, I guess it doesn’t work” thing. Maybe a message to alert people where to get assistance with this kind of failure?

So, I thought, Let’s try this on a different VPS:
(2) CPU/ 1GB RAM/ 25 GB SSD
Debian 12 as well.

This time it worked fine. AWESOME. Now following the documentation I see, in the next steps:

OK. So I go off and start reading.

Next step: STARTING WITH SERVICES. Well, I’m not a developer (I may write code but I’m no developer), so I want Zero Trust Host Access.

So let’s see what this quickstart can do for me!

Wow. OK. There’s a lot of information that seems irrelevant in there for someone trying to just get a network overlay up and running…but…wait, the prerequisite says " You will need an OpenZiti overlay network in place before you can complete this guide. If you do not have an OpenZiti overlay network provisioned yet, follow a quickstart and get a network up and running."

So wait, is my overlay network already running and working now? How do I test? You know what, let me go back and install the Admin Console (Web GUI) so I can visualize what’s going on…

So after that painful process (I didn’t want nodejs installed on that VPS, but whatever), I’m back to “What now?”

So back to that Quickstart services…

Still trying to translate that information to what I really want…just a “vpn like network overlay, like I have with those other solutions”. Oh, at the bottom it says: " Optionally, you may install the ZAC to manage your network with a UI." OK cool, I can use the web GUI to accomplish what I need.

Log into the admin console and I’m right back at “So what now?”

Well, I know I need some identities, and I know I need some kind of tunneller out of my network at home, right?

Cool. Let’s enroll my phone app and get a tunneller stood up on an Ubuntu host running on a NUC.

Done.

Now what? Is my tunneller configured correctly?

I’ve tried to create some services, but don’t fully understand what it is I need to just expose a simple subnet.

I’ve tried service policies? to allow my identities to access those services?

I’m stuck.

For all the push that I see on Reddit for people to use this project, it doesn’t seem very straight forward.

So what am I missing? How is this pushed as an alternative to those other solutions so heavily when none of it is really user-friendly?

Please don’t mistake this as a dig at anyone or the project in general, as I am evaluating all zero trust options for my organization to move to in the future…I just feel like I am missing some important step and understanding. Either the documentation is too good and I can’t follow it, or it’s incomplete. I can’t determine which is which. This isn’t something I come across too often. I have been working with servers and networks for over 23 years. I can’t imagine someone with little experience in networking or server management would be able to get this up and running without issues. So is there a “newbie guide”?

2

The hostname shown here shoud be the IP/Hostname you've set in env var EXTERNAL_DNS.
It needs to be publicly addressable. I think you've put your machine's hostname in there, you should try to put your VPS's IP.

3

@flamingm0e welcome to the discourse forum, thank you for the detailed walkthrough of your experience. The documentation is definitely being worked on, there are some gaps that we just haven’t gotten to yet. The fact that you’re getting stuck is a key indicator to me that we’re definitely lacking somewhere.

Regarding your first VPS issue, as @arslane mentioned, it appears your quickstart was set up using the hostname as the DNS. However, since you do have it running on a different VPS, we can work from there to get you where you need to be. That might help us figure out the gaps that we have in our doc when you hopefully reach the aha moment, we can pinpoint why that aha moment wasn’t reached with doc alone.

For starters, the best way to ensure your network is up and running is to try and ping the controller API. Can you try running curl -sk "${ZITI_CTRL_EDGE_ADVERTISED_ADDRESS}:${ZITI_CTRL_EDGE_ADVERTISED_PORT}" where, on a hosted network with the standard install, your ZITI_CTRL_EDGE_ADVERTISED_ADDRESS should be the DNS and the ZITI_CTRL_EDGE_ADVERTISED_PORT should be 8441. You could also echo these environment variables from the VPS where you ran the quickstart, or you can check your .env file which is located at ${ZITI_HOME}/quickstart/${hostname}/${hostname}.env

I might suggest starting with this question; what service or resource do you want to expose over your ziti network? Are you trying to access an API, a web portal, etc?

4

I apologize if I didn't make that part clear in my OP.

I put the correct EXTERNAL_DNS entry in. My hostname is ALSO the same as one of my external DNS entries, but I did fill in the DNS that I wanted to use in the EXTERNAL_DNS env variable. I configured both VPS the same with appropriate DNS entries.

I do know my network is up and running. I have connected a tunneller to it, as well as the ZME on my phone.

I was hoping to expose one of my VLANs on my home network over OpenZiti, to function similarly to Twingate, Tailscale, Zerotier, etc, which I also have working everywhere. I wasn't really wanting to do just a single service/api/etc. at this time. Based on the Reddit posts, where this project is being advertised, I wanted to be able to use this as an alternative to those other solutions. While I knew it was going to be more effort on my part, I didn't expect to have to post in the forums asking for help. I usually don't have a problem figuring these things out.

5

I think there’s a higher level of questioning here than just how to make it work. I’ll get into the nuts and bolts that I think will get you further along, but first some thoughts on why openziti might feel foreign, especially if you have a strong IP networking background.

The openziti tunnelers a just a part of what openziti is. They are applications that proxy tcp/ip & udp over ziti connections. The other openziti constructs - controller, router, etc are not aware of IP addresses or anything related to the underlay. That’s the tunnelers job is to deal with the underlay. The big picture is that you could use openziti SDKs to write your own apps, with no tunnelers involved.

So if you’re looking to use openziti as a VPN, the tunnelers can do that.

“service” is just the name that we use for something that is shared. It could be a single application that uses the openziti SDKs to communicate without tunnelers at all. It could be tunnelers that are configured to intercept specific IPs, hostnames and ports. The tunnelers can also intercept subnets and port ranges, and I think this capability is what you’re looking for?

Here are the basics:

First an intercept configuration that captures your VLAN’s subnet. For example:

ziti edge create config vlan1.cfg.intercept intercept.v1 '{
    "addresses": ["10.99.1.0/24"],
    "protocols": ["tcp","udp"],
    "portRanges": [ {"low":1024,"high":32768} ]
}'

And a host configuration that basically forwards the connection that was intercepted by the initiating/client tunneler:

ziti edge create config vlan1.cfg.host host.v1 '{
    "forwardProtocol": true,
    "allowedProtocols": ["tcp","udp"],
    "forwardAddress": true,
    "allowedAddresses": ["10.99.1.0/24"],
    "forwardPort": true,
    "allowedPortRanges": [{"low":1024","high":32768}]
}'

From here you create the “service” that refers to the configurations above and specifies some “roles” that define which identities will be able to access the service:

ziti edge create service vlan1 \
    --configs vlan1.cfg.intercept,vlan1.cfg.host \
    --role-attributes vlan1.clients,vlan1.servers

Now that the service exists we can define what those roles actually mean, with service policies:

ziti edge create service-policy vlan1.dial Dial --identity-roles "#vlan1.clients" --service-roles "@vlan1"
ziti edge create service-policy vlan1.bind Bind --identity-roles "#vlan1.servers" --service-roles "@vlan1"

So #vlan1.clients will be able to dial (initiate connections) to the service, and #vlan1.servers will be able to bind (provide the back-end).

Now you can update the roles of any identities that should have access to the service accordingly:

ziti edge update identity whatever_you_named_your_zme_identity --role-attributes whatever_role_attributes_existed_before,vlan1.clients

ziti edge update identity whatever_you_named_your_linux_identity --role-attributes whatever_role_attributes_existed_before,vlan1.servers

Unfortunately the CLI can’t currently add a role to identities, so you need to provide the entire list of roles when you do the update.

I’m not sure if this is what you’re looking for, or if it gets you unstuck. Let us know and we’ll be glad yo help if we can.

1 Like
6

I had kind of gathered that from what I was standing up, but couldn't figure out the relation.

It sounds like this is what I am looking for.

Do I run these commands on the VPS I got it installed on? Should all commands be run there?

When I try to run them (on the same machine), I get:

 error: error listing https://ziti.mydomain:8441/edge/management/v1/configs in Ziti Edge Controller. Status code: 401 Unauthorized, Server returned: {
    "error": {
        "code": "UNAUTHORIZED",
        "message": "The request could not be completed. The session is not authorized or the credentials are invalid",
        "requestId": "DvcCNjn2a"
    },
    "meta": {
        "apiEnrollmentVersion": "0.0.1",
        "apiVersion": "0.0.1"
    }
}

So that indicates something went wrong with my install...OK. Guess I'll try all that again. I would rather have it on my other VPS anyway, so perhaps wiping it all and trying again (4th time) will help me out.

Ran through the quick setup on the other server and everything worked...dunno what happened the first couple of tries?

I think so. Thank you. I'll give it a try.

After running through everything again on the original VPS I was trying to configure, and following your steps, it is indeed running as I envisioned. I will say this is a lot more complex than I think the Reddit "evangelists" let on.

Thanks for your assistance. I'm going to look over this configuration and try to let it all soak in.

2 Likes