Multiple links between two routers

Ziti Overlay
1

I’m trying to establish multiple links between two routers and would appreciate some clarification on the expected behavior.

From what I understand, it is possible to configure multiple listeners on a router by simply adding additional bindings to the listeners list (e.g., different ports).

On the other side, I would like the second router to connect to these multiple ports on the first router. My goal is to route traffic differently depending on the port — for example, sending traffic over different network interfaces or tunnels, potentially via different Tier-1 operators for redundancy or performance.

However, when I add a second binding to the dialers list on the connecting router, it does not result in multiple links. There is always only a single link established between the two routers.

2

Hi @Rantanplan , currently links are disambiguated by a tuple of (destination router, protocol, source binding, destination binding), where the source and destination bindings indicate binding to an interface.

So just creating extra listeners won't result in extra links unless the extra listeners are using a different protocol or are bound to a different network interface.

Cheers,
Paul

3

Could you please provide an example of how to achieve multiple links between two routers on different hosts (A and B)?

I attempted a workaround by creating multiple routers (RtA1, RtA2) on the same host (A), but this approach does not work well in practice:

  1. zrok attaches the service to only one router
  2. local data transfer between routers (over the loopback interface) is very slow

In this setup, the remote router (RtB) requesting the service would need to traverse something like:
RtB → RtA1 → RtA2

Due to the poor performance over the loopback interface(huge flow of small packets), this design introduces significant overhead and is not viable for my use case — the performance is very poor.

4

If you have multiple nics, you can take advantage of them as follows:

  link:
    listeners:                                                                                                                                                                           
      # Listener on eth0
      - binding:       transport
        bind:          tls:0.0.0.0:6004
        advertise:     tls:10.0.1.50:6004
        bindInterface: eth0                                                                                                                                                              
   
      # Listener on eth1                                                                                                                                                                 
      - binding:       transport
        bind:          tls:0.0.0.0:6005
        advertise:     tls:172.16.0.50:6005
        bindInterface: eth1                                                                                                                                                              
   
    dialers:                                                                                                                                                                             
      # Dial outbound over eth0
      - binding: transport
        bind:    eth0

      # Dial outbound over eth1
      - binding: transport
        bind:    eth1
5

Note that regular links already create multiple connections for each links. You adjust that on the link dialer as follows:

  link:                                                                                                                                                                                  
    dialers:
      - binding:               transport                                                                                                                                                 
        maxDefaultConnections: 5
        maxAckConnections:     2
6

What is the larger design goal and real set of constraints? A lot of network operators use clusters of routers which will form link meshes and load balance. Placing routers into various providers can help steer traffic, especially when dealing with larger networks. So there are a lot of options, some of which might work better that what you are attempting here.

7

The objective is to establish multiple links between two hosts (A and B) via different upstream providers.

In my setup, host A is connected to multiple upstream providers (for example: STO, AMS, FRA). These are geographically distinct locations where I have BGP announcements for my IPv6 prefixes. Each upstream delivers traffic to host A via separate tunnels, so effectively I have many network interfaces, all IPv6-based.

Host B should then establish multiple Ziti links to host A, one per upstream path:

  • B → STO → A
  • B → FRA → A
  • B → AMS → A

To model this, I created three routers on host A. On host B, the router attempts to connect to all of these endpoints, while each A_XXX router only connects to B via its specific interface (STO/FRA/AMS respectively).

B ──> STO ──> A_STO
 │
 ├──> AMS ──> A_AMS
 │
 └──> FRA ──> A_FRA

I was unable to make this work using only a single router on host A. However, this may actually be beneficial, as a terminator’s dynamic host is determined for a specific router. Since the service has terminators on all three routers, the Ziti controller should be able to select the optimal path based on the dialer’s location.