Ziti fabric list links not showing other connections

I installed openziti using openziti_manual_pki(openziti_manual_pki/BOOTSTRAP_PKI.md at main · nenkoru/openziti_manual_pki · GitHub). Over that i have 2 different routers running on 2 different host devices. One router is running fine, even able to access services using 1st router but not able to access services using the 2nd router. The 2nd router is getting connected to the controller. The 'ziti fabric list links' shows only one connection to be connected which is between the test-edge-router and the 1st router. On the ZAC console, both the routers show connected to the edge router present on the controller. what could be the issue?
The log file for 2nd router shows tls : failed to verify certificate: x509: certificate is valid for 127.0.0.1, not IP. But it works perfectly fine for 1st router.

This section of the router config was likely unchanged.

When you enroll the router, it will create it's own certiifcate per that CSR section:

edge:
  csr:
    country: US
    province: LA
    locality: Los Angeles
    organization: MyBestOrg
    organizationalUnit: Openziti Public Edge Router
    sans:
      dns:
        - localhost

      ip:
        - "127.0.0.1"

I expect you'll want to add a DNS entry or a correct IP in there

Both the routers are getting connected to each other instead of individual routers getting connected to test-edge-router. What changes needs to be made in the yaml file of the router such that it connects to the edge-router instead of other 2 routers?

I'm sorry, I thought from your original post you had ONLY two routers but now it sounds like you have more than 2. No problem, could you provide a small diagram or describe the overall setup? It'll help me answer you better...

If you'd rather just hear "the answer", and are happy to go read/explore/understand more on your own, you want to look at the "link" and "listeners" section.

A router that you want OTHER routers to connect TO should have a link listener section declared:

link:
  listeners:
    - binding:          transport
      bind:             tls:0.0.0.0:10080
      advertise:        tls:ec2-3-18-113-172.us-east-2.compute.amazonaws.com:10080
      options:
        outQueueSize:   4

A router that you want to dial other routers shouldh ave a link dialer section declared:

link:
  dialers:
    - binding: transport

A router you want to do both tasks - should have both sections declared.

A router you want to have SDKs connect to, should have an "edge" binding listener declared:

listeners:
# bindings of edge and tunnel requires an "edge" section below
  - binding: edge
    address: tls:0.0.0.0:8442
    options:
      advertise: ec2-3-18-113-172.us-east-2.compute.amazonaws.com:8442
      connectTimeoutMs: 5000
      getSessionTimeout: 60
  - binding: tunnel
    options:
      mode: host #tproxy|host

Maybe that's enough details? :slight_smile: If not, let me know what sort of topology you have/want and we can iterate until we get it right.


This is the required topology that i want.


But this is what is happening

Ah thank you for the diagram! Router2 connecting to Router1 or vice versa means one of the two routers is 'advertising' a link.

In this scenario, you'll want "edge_router" to advertise a link and dial other "listening" routers:

link:
  dialers:
    - binding: transport
  listeners:
    - binding:          transport
      bind:             tls:0.0.0.0:10080
      advertise:        tls:ec2-3-18-113-172.us-east-2.compute.amazonaws.com:10080
      options:
        outQueueSize:   4

And you'll want Router1 and Router2 to ONLY have link dialers:

link:
  dialers:
    - binding: transport

This will achieve that topology

When i am making the changes as suggested, I am not able to establish link with the edge router. ziti fabric link lists command shows no entries. Also getting this error as tls: failed to verify certificate: x509: certificate is valid for 127.0.0.1, not IP whereas when I use openssl to verify the certificate and check the SANS field, i am getting IP address mentioned in the list. How to solve it?

For the topology i am able to setup,


this is the link setup I am getting. Using this I am able to access the services properly.

To be entirely transparent, it's hard to know how to help you since I cannot probe the network from my side to see anything. I'm not sure why/how you're getting the error, but the the error is indicating the router connecting to another router has an invalid PKI for a yet to be determined reason. Are you well-versed with PKI/openssl commands? The router trying to connect has an identity block at the top of it which contains the ca's that are used when connecting and the client cert/key that are used. You could plug all those into openssl and see what it reports. That might help?

It looks like one of the "private" routers is working as expected. Maybe you can look at the two configs and the two pki's to see why/where the difference is?

I'm just not sure I can help you with the information I have so far other than "the PKI is wrong somehow". :frowning:

I have deployed controller and test-edge-router using openziti_manual_pki/BOOTSTRAP_PKI.md at main · nenkoru/openziti_manual_pki · GitHub and deployed router 1 and 2 using normal ziti commands, where create jwt file, enroll it and then run on the respective systems. While router1 is able to connect properly to the edge-router but the router2 is getting connected to the router1.
Controller is running on v1.1.4 whereas router1 & 2 are v1.1.7 & v1.1.8 resp.
As u suggested that it might be some pki issues, I used openssl verify command to check the server certs on respective hosts. All the status is coming OK. What could be the other steps to debug the issue?

For the edge-router, how many routers can be connected to a single edge-router port that is mentioned in the link listeners section ?