Creating a second public edge router

You need to change this on both routers. That's the address that OTHER routers will connect to your router over. You'll then subsequently have to open that port in your firewall for where you setup the quickstart. The advertise setting is what you want/need to change. make it:

tls:${external_address_of_link_listener_router}:10080

(or whatever port you choose)

1 Like

Want to expand on this a bit. As @TheLumberjack said, if you're app embedded, there's no need for the router-embedded tunneler because you don't need a tunneler at all. However, if you're going to run a tunneler, there are some advantages to using the router-embedded tunneler, specifically on the hosting side. The benefits are mostly around simplifying deployments.

You generally want/need one or more routers in your hosting site. If you use the embedded tunneling functionality, you don't need an extra tunneler running externally. You also bring the fabric one hop closer to the hosting server. Because of that, it's also likely to be the most performant option (though as with anything, testing to get actual performance numbers is a good idea).

In terms of functionality on the hosting side, anything you should be able to do with one, you should be able to do with the other.

For client side, ziti-edge-tunnel (or any of the desktop edge or mobile edge components) is recommended because the router is a relatively heavy component. On the hosting side it's not an issue because you will generally have routers sitting next to your server components anyway.

1 Like

Thanks for the further details.. this is very helpful

PS>> I have it now working with a few more tweaks

opc 87899 1 0 Jun05 ? 00:00:28 /home/opc/.ziti/quickstart/instance-20220416-1603/ziti-bin/ziti-v0.25.4/ziti-router run /home/opc/ziti/quickstart/instance-20220416-1603 /instance-20220416-1603-edge-router.yaml

opc 607469 606163 4 01:39 pts/1 00:00:02 /home/opc/.ziti/quickstart/instance-20220416-1603/ziti-bin/ziti-v0.25.4/ziti-router run /home/opc/ziti/quickstart/instance-20220416-1603/ instance-20220416-1603-2nd-edge-router.yaml

I needed to change the port for bot the link and edge binding and advertise sections..

link:
dialers:
- binding: transport
listeners:
- binding: transport
bind: tls:0.0.0.0:10081
advertise: tls:instance-20220416-1603:10081
options:
outQueueSize: 4

listeners:

bindings of edge and tunnel requires an "edge" section below

  • binding: edge
    address: tls:0.0.0.0:8443
    options:
    advertise: IP address of the server:8443
    connectTimeoutMs: 1000
    getSessionTimeout: 60s
  • binding: tunnel
    options:
    mode: host #tproxy|host

This was very helpful but unfortunately I’m getting an error while enrolling the JWT I’ve created in ZAC. I’ve created a default config using:

./ziti create config router edge --routerName zt-router-1 -o /home/ziti/zt-edge-router.yaml

What does erott mean? Is there anything else I need to keep in mind while enrolling another router?

ziti@zt-router-1:~$ ./ziti-bin/ziti-v0.29.0/ziti edge enroll /home/ziti/zt-router-1.jwt
INFO    generating 4096 bit RSA key
Usage:
  ziti edge enroll path/to/jwt [flags]

Flags:
      --ca string         Additional trusted certificates
  -c, --cert string       The certificate to present when establishing a connection.
  -h, --help              help for enroll
  -n, --idname string     Names the identity. Ignored if not 3rd party auto enrollment
  -j, --jwt string        Enrollment token (JWT file). Required
  -k, --key string        The key to use with the certificate. Optionally specify the engine to use. supported engines: [parsec]
  -a, --keyAlg RSA|EC     Crypto algorithm to use when generating private key (default RSA)
  -o, --out string        Output configuration file.
  -p, --password string   Password for updb enrollment, prompted if not provided and necessary
      --rm                Remove the JWT on success
  -u, --username string   Username for updb enrollment, prompted if not provided and necessary
  -v, --verbose           Enable verbose logging


failed to enroll: enrollment method 'erott' is not supported

@dmuensterer there are some minor mistakes in your command, mainly that ziti edge enroll is used for enrolling an identity, you would want ziti router enroll. But, aside from that, as mentioned in the beginning, this isn’t well documented, and since quickstart was recently refactored, it’d be good to come up with a set of steps for this until we get it officially in the doc.

Assuming this is on a different VPS/machine and you’re using the latest ziti (which you are), I would do the following.

EDIT < I should mention, you’ll want to run ziti edge delete edge-router zt-router-1 so the following doesn’t collide with the router you already tried creating > END EDIT

  1. Set the following variables appropriately based on your network, referring to the .env file of your network would be useful for any non-router values. I filled some of them in for you
export ZITI_HOME=/home/ziti/
export ZITI_ROUTER_NAME=zt-router-1
export ZITI_ROUTER_ADVERTISED_HOST=
export ZITI_BIN_DIR=./ziti-bin/ziti-v0.29.0
export ZITI_USER=admin
export ZITI_PWD=
export ZITI_CTRL_EDGE_ADVERTISED_ADDRESS=
export ZITI_CTRL_EDGE_ADVERTISED_PORT=
export ZITI_CTRL_ADVERTISED_ADDRESS=
export ZITI_CTRL_ADVERTISED_PORT=
  1. Run the ziti-cli-functions.sh’s Add Router function. Source the CLI script if you haven’t already.
source /dev/stdin <<< "$(wget -qO- https://get.openziti.io/quick/ziti-cli-functions.sh)"

This example will create an edge router (public) with the attribute public, the attribute is the fourth param so you can remove/edit it how you want. It will create the config, enroll the router, etc and place all of the files and logs in ZITI_HOME as set above.

addRouter "${ZITI_ROUTER_NAME}" public public
  1. Start up your router, also using a provided function from ziti-cli-functions.sh
startRouter

That should be everything, double check using ziti edge list edge-routers to be sure it’s showing “online”.

Let me know how this goes.

1 Like

Thanks for the help! Unfortunately I couldn’t do the addRouter part because the management API isn’t publicly accessible…
Instead what I did is

ziti@zt-router-1:~/ziti-bin/ziti-v0.29.0$ ./ziti router enroll /home/ziti/zt-edge-router.yaml --jwt /home/ziti/zt-router-1.jwt
[   1.442]    INFO edge/router/enroll.(*RestEnroller).Enroll: registration complete
ziti@zt-router-1:~/ziti-bin/ziti-v0.29.0$ startRouter
[1] 1987
Express Edge Router started as process id: 1987. log located at: /home/ziti/zt-router-1.log

which worked…

However, I didn’t get positive feedback while trying to list the edge-routers:

ziti@zt-router-1:~/ziti-bin/ziti-v0.29.0$ ./ziti edge list edge-routers
error: no identity 'default' found in cli config /home/ziti/.config/ziti/ziti-cli.json
[1]+  Exit 2                  "${ZITI_BIN_DIR}/ziti" router run "${ZITI_HOME}/${ZITI_ROUTER_NAME}.yaml" > "${log_file}" 2>&1

Also, the certificates are obviously not there yet… I assume they were somehow generated automatically while enrolling the JWT? But where?

ziti@zt-router-1:~$ cat zt-edge-router.yaml
v: 3

identity:
  cert:             "/home/ziti/ziti-bin/ziti-v0.29.0/zt-router-1.cert"
  server_cert:      "/home/ziti/ziti-bin/ziti-v0.29.0/zt-router-1.server.chain.cert"
  key:              "/home/ziti/ziti-bin/ziti-v0.29.0/zt-router-1.key"
  ca:               "/home/ziti/ziti-bin/ziti-v0.29.0/zt-router-1.cas"
  #alt_server_certs:
  #  - server_cert:  ""
  #    server_key:   ""

etc...

Sadly you cut off "the most important" part of your router config... the very next line is what I wanted to see.

ctrl:
  endpoint:             tls:ip-172-31-47-200:8440

I expect that endpoint is either "wrong" (wrong address) or the webacl doesn't allow port 8440 through.

Since you are more secure, and you have taken the management API offline you should follow a slightly different set of steps. You should:

  • create the router using the ziti CLI and output the .jwt file.
  • transfer the *.jwt file to the "external/other/second" router
  • run ziti router enroll on the router

The certificates will be generated when you successfully enroll. So my guess is you enrolled the router on your controller?

1 Like

Aha, it works! - I put the router hostname in the ctrl section by mistake!

I really appreciate the help throghout the last couple of days regarding all the different issues we had! :slight_smile:

Just to confirm:
Are the certificates put automatically to the filepaths which are defined in the YAML file while enrolling? They are there, I just want to understand what the direction is here:

  1. The YAML file defines where the cert files are being put while enrolling?
    OR
  2. While enrolling, the cert files are stored in the current dir and referenced in the YAML afterwards?

Could you please quickly confirm what the different ports are doing and do they have to be open?

ctrl:
  endpoint:             tls:zt.mydomain..com:6262

link:
  dialers:
    - binding: transport
  listeners:
    - binding:          transport
      bind:             tls:0.0.0.0:10080
      advertise:        tls:zt-router-1.mydomain.com:10080
      options:
        outQueueSize:   4

6262 is probably the communication between router and controller?
10080 says transport… do the ziti-edge-tunnelers have to communicate to port 10080? I assume not, but why?

I hope I didn’t miss any docs where this is explained but couldn’t find one … :slight_smile:

Yes. The act of enrolling writes those files. You specify where you want them before enrolling in the router config, then enroll and they get written out.

Also documented here (but I see I want to update that info a wee bit): Host OpenZiti Anywhere | OpenZiti

  • 8440/tcp: Edge Controller providing router control plane
  • 8441/tcp: Edge Controller providing client sessions
  • 8442/tcp: Edge Router providing client connections
  • 8443/tcp: Ziti Admin Console (ZAC) [optional]

And here: Router Configuration Reference | OpenZiti

ctrl.endpoint

ctrl.endpoint is the address of the control plane. This points to your controller's port. If you followed the host it anywhere quickstart, 6262 ==> 8440 on the controller (or in the controller config file the ctrl.listener value. mine right now is set to tls:0.0.0.0:8440 for example)

link.listeners.bind

this section is for router to router configuration. with this configuration you are stating "hello OpenZiti overlay. I am an edge router that other edge routers should connect to". If you intend this to be a "public" edge router -- one that other routers should link to. Then this is fine and you will want to have port 10080 open on this router.

However, if this is a 'private' edge router (one that you're using to provide reach into a private VPC for example) you would turn that whole section off by commenting it out/removing it. In fact if you run that ziti create config command with --private you'll see what i mean. A "private" edge router is one that will only dial outbound links to other routers, but won't advertise a 'link listener':

ziti create config router edge --private --routerName test | grep -A10 "link\:"
link:
  dialers:
    - binding: transport
#  listeners:
#    - binding:          transport
#      bind:             tls:0.0.0.0:10080
#      advertise:        tls:sg3u22:10080
#      options:
#        outQueueSize:   4

Hope that helps

1 Like

It absolutely did make things very clear. Thank you!

1 Like