Hi folks,
I am scratching my head here and hope that you can give me a push into the right direction.
I have a newly set up ziti network right now only consisting in a router and a controller. I have been using the helm charts to install everything. All my stuff is on version 0.27.5. Control plane, Client API and my routerโs edge API are all available publicly, using Ingress objects and proper TLS passthrough on the ingress controller. I can successfully dial into the openziti network with my laptop using the desktop edge and communicate to services inside the network.
I am now failing to add a simple additional private router. I did simply use the cli on the controller to issue ziti edge create edge-router secadm-int-router -o /tmp/my-private-router.jwt -t --no-traversal, then copy this jwt file to my laptop and issue the installation of the router in another Kubernetes cluster like this: helm install private-router -f private-router_values.yaml --set-file enrollmentJwt=my-private-router.jwt openziti/ziti-router.
My ziti-router.yml which is created by the helm chart looks like this:
v: 3
identity:
cert: ${ZITI_ROUTER_IDENTITY_DIR}/client.crt
server_cert: ${ZITI_ROUTER_IDENTITY_DIR}/tls.crt
key: ${ZITI_ROUTER_IDENTITY_DIR}/tls.key
ca: ${ZITI_ROUTER_IDENTITY_DIR}/ca.crt
ctrl:
endpoint: tls:ctrlplane.sdn.my.org:443
link:
dialers:
- binding: transport
listeners:
- binding: edge
address: tls:0.0.0.0:3022
options:
advertise: ziti-router.kube.my.org:443
connectTimeoutMs: 1000
getSessionTimeout: 60
- binding: tunnel
options:
mode: host
edge:
csr:
sans:
dns:
- localhost
- ziti-router.kube.my.org
ip:
- 127.0.0.1
forwarder:
latencyProbeInterval: 10
xgressDialQueueLength: 1000
xgressDialWorkerCount: 128
linkDialQueueLength: 1000
linkDialWorkerCount: 32
The router starts up and at the first look it looks like itโs working. In Ziti Console I see two green dots in front of this new routerโs identity.
BUT: I see this when checking the fabric links from the controller:
ziti fabric list links
โญโโโโโโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโฌโโโโโโโโโโโโโโฌโโโโโโโโโฌโโโโโโโโโฌโโโโโโโโโโโโฎ
โ ID โ DIALER โ ACCEPTOR โ STATIC COST โ SRC LATENCY โ DST LATENCY โ STATE โ STATUS โ FULL COST โ
โโโโโโโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโโโโโโโผโโโโโโโโโโโโโโผโโโโโโโโโโโโโโผโโโโโโโโโโโโโโผโโโโโโโโโโโโโโผโโโโโโโโโผโโโโโโโโโผโโโโโโโโโโโโค
โ 2a1pwELa3rbUQ1y6zpZwsb โ private-router โ core-router โ 1 โ 65000.0ms โ 65000.0ms โ Failed โ up โ 130001 โ
โฐโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโโโโโโโดโโโโโโโโโดโโโโโโโโโดโโโโโโโโโโโโฏ
results: 1-1 of 1
The private router logs this every minute:
[5048.000] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {routerVersion=[v0.27.5] linkId=[4b5Td6QI1TujofphEhhhyC] routerId=[Ff8oRvyqtj] address=[tls:router-edge.sdn.my.org:443] linkProtocol=[tls]} dialing link
[5048.068] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {routerId=[Ff8oRvyqtj] routerVersion=[v0.27.5] linkId=[4b5Td6QI1TujofphEhhhyC]} link destination support heartbeats
[5048.068] INFO fabric/router/handler_link.(*closeHandler).HandleClose [ch{l/4b5Td6QI1TujofphEhhhyC}->u{classic}->i{a2EE}]: {linkId=[4b5Td6QI1TujofphEhhhyC] routerId=[Ff8oRvyqtj]} link closed
[5048.130] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {linkId=[4b5Td6QI1TujofphEhhhyC] routerId=[Ff8oRvyqtj] routerVersion=[v0.27.5]} link destination support heartbeats
[5048.130] INFO fabric/router.(*xlinkAccepter).Accept: accepted new link [l/4b5Td6QI1TujofphEhhhyC]
[5048.130] INFO fabric/router.(*linkRegistryImpl).applyLink: {linkProtocol=[tls] newLinkId=[4b5Td6QI1TujofphEhhhyC] dest=[Ff8oRvyqtj]} link being registered, but is already closed, skipping registration
[5048.130] INFO fabric/router/handler_link.(*closeHandler).HandleClose [ch{l/4b5Td6QI1TujofphEhhhyC}->u{classic}->i{XppP}]: {routerId=[Ff8oRvyqtj] linkId=[4b5Td6QI1TujofphEhhhyC]} link closed
And the other router, which should receive this connection says that:
[5491.461] ERROR edge/router/xgress_edge.(*sessionConnectionHandler).HandleClose: {id=[4b5Td6QI1TujofphEhhhyC]} session connection handler encountered a HandleClose that did not have a SessionTokenHeader
[5491.461] ERROR channel/v2.AcceptNextChannel.func1: {error=[no token attribute provided]} failure accepting channel edge with underlay u{classic}->i{a2EE}
[5491.524] ERROR edge/router/xgress_edge.(*sessionConnectionHandler).HandleClose: {id=[4b5Td6QI1TujofphEhhhyC]} session connection handler encountered a HandleClose that did not have a SessionTokenHeader
[5491.524] ERROR channel/v2.AcceptNextChannel.func1: {error=[no token attribute provided]} failure accepting channel edge with underlay u{classic}->i{XppP}
I donโt see what I might have done wrong and what could be different from other setups except the fact that my controller and โcore-routerโ are made public via a ingress and ingress controller, which I cannot yet see any error with.
Any hints that these log message might give you?
Thanks a lot in advance.
Christian
It seems like you probably know this but it doesnโt hurt to check even though I know your edge clients can connect to the edge listener.