Hi,
Following quickstart, I have the controller up and running on a host. I also setup an edge router on a different host following quickstart. What do I need to do in order to have the router to talk to the controller on the first host? I can imagine that I need to somehow “merge” the pki of the two hosts and modify the edge router yaml file. A checklist would be greatly helpful.
Thanks.
Yeah, we need to make that guide. We keep making the doc better but it's a process that just takes a while. In the meantime, I'll make that punchlist of things to do and follow up in a bit.
Awesome! I’ll be waiting for it.
Here’s a video outlining the process. We are close to having a helper function that does this, it’s just not released yet. Should be soon. Till then, here you go. Hope that helps
get the external address of the controller from the config file
grep ZITI_EDGE_CTRL_ADVERTISED_HOST_PORT $ZITI_HOME/$(hostname).env
export ZITI_EDGE_CTRL_ADVERTISED_HOST_PORT="ec2-3-134-108-218.us-east-2.compute.amazonaws.com:8441"
make sure the address is addressable from edge router 2 (ER2 from hereon). Make sure curl returns some json.
curl -sk https://ec2-3-134-108-218.us-east-2.compute.amazonaws.com:8441
determine the control plane port:
grep ZITI_CTRL_PORT $ZITI_HOME/$(hostname).env
export ZITI_CTRL_PORT="8440"
make sure ER2 can see the link plane port
curl -vsk -m 5 https://ec2-3-134-108-218.us-east-2.compute.amazonaws.com:8441
verify you don't see: "* Connection timed out after 5000 milliseconds"
make sure the controller and edge-router-1 are running
curl -vsk -m 5 https://ec2-3-134-108-218.us-east-2.compute.amazonaws.com:8440
verify you don't see: "* Connection timed out after 5000 milliseconds"
use getZiti to get the binaries (pass ‘yes’ to add ziti to your path)
source /dev/stdin <<< "$(wget -qO- https://get.openziti.io/quick/ziti-cli-functions.sh)"; getZiti yes
login to the controller. easier if you do this from ER2, but not necessary (use the value from above)
ziti edge login ec2-3-134-108-218.us-east-2.compute.amazonaws.com:8441
or `zitiLogin` if you're on the controller
create the ER2 in the controller.
ziti edge create edge-router -t ER2 -o $ZITI_HOME/ER2.jwt
set some variables, then decide if this is a “private” edge router or public (see below)
export ROUTER_NAME=ER2
export ZITI_CTRL_ADVERTISED_ADDRESS="ec2-3-134-108-218.us-east-2.compute.amazonaws.com"
export ZITI_CTRL_PORT=8440
# optional - if you want to allow addressing by some specific IP
export ZITI_EDGE_ROUTER_IP_OVERRIDE="177.77.77.77"
create a config file. this makes the router ‘private’, meaning it won’t advertise that routers should link to it.
ziti create config router edge --routerName $ROUTER_NAME \
--output $ZITI_HOME/$ROUTER_NAME.yaml \
--tunnelerMode host \
--private
enroll the router (does not matter which config you use, i chose the private one here):
ziti-router enroll $ZITI_HOME/$ROUTER_NAME.yaml --jwt $ZITI_HOME/ER2.jwt
run the public or private ER2 config:
ziti-router run $ZITI_HOME/$ROUTER_NAME.yaml
make sure ER2 shows online
ziti edge list edge-routers
make sure it linked:
ziti fabric list links
make systemd unit file:
createRouterSystemdFile $ROUTER_NAME
systemd file stuff (optional)
sudo cp /home/ubuntu/.ziti/quickstart/$ROUTER_NAME.service /etc/systemd/system/ziti-router.service
sudo systemctl daemon-reload
sudo systemctl enable --now ziti-router
thanks so much! will give it a try
Almost there. It failed at step 10:
mango@kyoto:~/tools$ ziti create config router edge --routerName er2 --output $ZITI_HOME/er2.yaml --tunnelerMode host
2023/01/17 11:18:34 if ZITI_EDGE_ROUTER_ADVERTISED_HOST[kyoto] is supplied, it *MUST* match the ZITI_EDGE_ROUTER_IP_OVERRIDE[] or resolved hostname[kyoto-edge-router]
panic: if ZITI_EDGE_ROUTER_ADVERTISED_HOST[kyoto] is supplied, it *MUST* match the ZITI_EDGE_ROUTER_IP_OVERRIDE[] or resolved hostname[kyoto-edge-router]
goroutine 1 [running]:
log.Panicf({0x2ff9ab2?, 0x11?}, {0xc00098fcc8?, 0xc000b08600?, 0x0?})
log/log.go:395 +0x67
github.com/openziti/ziti/ziti/cmd.SetZitiRouterIdentity(0x496bdc0, {0x7fff8d721af0, 0x3})
github.com/openziti/ziti/ziti/cmd/create_config_router_common.go:55 +0x408
github.com/openziti/ziti/ziti/cmd.NewCmdCreateConfigRouter.func1(0xc0009b1200?, {0x2f0b59d?, 0x6?, 0x6?})
github.com/openziti/ziti/ziti/cmd/create_config_router.go:72 +0x105
github.com/spf13/cobra.(*Command).execute(0xc0009b1200, {0xc000af6a20, 0x6, 0x6})
github.com/spf13/cobra@v1.6.1/command.go:896 +0x711
github.com/spf13/cobra.(*Command).ExecuteC(0x480aca0)
github.com/spf13/cobra@v1.6.1/command.go:1044 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/cobra@v1.6.1/command.go:968
github.com/openziti/ziti/ziti/cmd.Execute()
github.com/openziti/ziti/ziti/cmd/cmd.go:81 +0x38
main.main()
github.com/openziti/ziti/ziti/main.go:51 +0x17
Man, that’s one of the reasons it took me a while, and why I recorded the video, so that I knew it worked. 
Did you perhaps run this on the machine where the quickstart was run, or did you source the .env file from the output of the quickstart? I didn’t do it that way when I tested (I’ll try now), I suspect that might be why you hit this. If not, looks like you might be hitting a bug. I am hitting that same bug with a different discourse post.
I’ll work up alternate instructions by emitting the config and editing it.
oh -- that's my fault! I've updated step 10. This can't be run. You can fix that by running unset ZITI_EDGE_ROUTER_ADVERTISED_HOST
Then the other steps should work
One step closer. Step 12:
mango@kyoto:~/tools$ ziti-router run $ZITI_HOME/er2.yaml
[ 0.776] INFO ziti/ziti/router.run: {routerId=[wCyoZGsOKT] revision=[7de87562b20d] version=[v0.27.2] go-version=[go1.19.4] os=[linux] build-date=[2023-01-04T20:13:07Z] arch=[amd64] configFile=[/home/mango/.ziti/quickstart/kyoto/er2.yaml]} starting ziti-router
[ 0.777] INFO fabric/router/forwarder.(*Faulter).run: started
[ 0.777] INFO fabric/router/forwarder.(*Scanner).run: started
[ 0.778] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {poolType=[pool.link.dialer] minWorkers=[0] maxWorkers=[32] idleTime=[30s] maxQueueSize=[1000]} starting goroutine pool
[ 0.778] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {maxWorkers=[128] idleTime=[30s] poolType=[pool.route.handler] maxQueueSize=[1000] minWorkers=[0]} starting goroutine pool
[ 0.778] WARNING edge/router/internal/edgerouter.(*Config).LoadConfigFromMap: Invalid heartbeat interval [0] (min: 60, max: 10), setting to default [60]
[ 0.779] INFO fabric/router.(*Router).showOptions: ctrl = {"OutQueueSize":4,"MaxQueuedConnects":1,"MaxOutstandingConnects":16,"ConnectTimeout":1000000000,"DelayRxStart":false,"WriteTimeout":0}
[ 0.779] INFO fabric/router.(*Router).showOptions: metrics = {"ReportInterval":60000000000,"MessageQueueSize":10}
[ 0.779] INFO fabric/router.(*Router).initializeHealthChecks: starting health check with ctrl ping initially after 15s, then every 30s, timing out after 15s
[ 0.779] INFO fabric/router.(*Router).startXlinkDialers: started Xlink dialer with binding [transport]
[ 0.779] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {idleTime=[10s] maxQueueSize=[1] minWorkers=[1] poolType=[pool.listener.link] maxWorkers=[16]} starting goroutine pool
[ 0.779] INFO fabric/router.(*Router).startXlinkListeners: started Xlink listener with binding [transport] advertising [tls:kyoto-edge-router:10080]
[ 0.779] INFO edge/router/xgress_edge.(*listener).Listen: {address=[tls:0.0.0.0:3022]} starting channel listener
[ 0.780] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {minWorkers=[1] maxWorkers=[16] maxQueueSize=[1] poolType=[pool.listener.xgress_edge] idleTime=[10s]} starting goroutine pool
[ 0.780] INFO fabric/router.(*Router).startXgressListeners: created xgress listener [edge] at [tls:0.0.0.0:3022]
[ 0.780] INFO fabric/router.(*Router).startXgressListeners: created xgress listener [tunnel] at []
[ 0.780] INFO edge/router/xgress_edge.(*Acceptor).Run: starting
[ 0.780] FATAL ziti/ziti/router.run: {error=[error connecting ctrl (dial tcp 127.0.1.1:6262: connect: connection refused)]} error starting
I guess it’s trying to connect to the controller on the localhost while the actual controller is on a different host.
It is, yes. For this, it will probably be easier to just edit the config file directly. If you open $ZITI_HOME/er2.yaml it should have a section like this:
ctrl:
endpoint: tls:ip-172-31-45-134:6262
You want that endpoint to be the host and port of your controller. If you compare the edge router the quickstart created it should be really clear. For example, my “host it anywhere” router config looks like this:
ctrl:
endpoint: tls:ec2-3-134-108-218.us-east-2.compute.amazonaws.com:8440
worked! This is on top of an existing quickstart installation. I’ll try a blank host and see if I can repeat the success.
Great to hear! Let me/us know on the next one. Like anything, it gets easier each time you do it.