Tunneling into private router in k8s

yemaney · June 13, 2024, 7:09pm

Created router via terraform helm provider.

resource "helm_release" "router0001" {
  name       = "private-router123"
  repository = "https://openziti.github.io/helm-charts/"
  chart      = "ziti-router"
  version    = "1.0.4"
  set {
    name  = "enrollmentJwt"
    value = file("router0002.jwt")
  }

  set {
    name  = "advertisedHost"
    value = "router0002-edge.ziti.svc.cluster.local"
  }

  set {
    name  = "ctrl.endpoint"
    value = "ec2-x-x-x-x.x-x-1.compute.amazonaws.com:6262"
  }
}

Ran the following commands to run ziti tunnel service in a linux machine

ziti edge login ec2-x-x-x-x.x-x-1.compute.amazonaws.com:1280 --yes --username "admin" --password $pass
ziti edge create edge-router router0002   --role-attributes default --tunneler-enabled --jwt-output-file router0002.jwt


ziti edge update identity "router0002" \
    --role-attributes trino-0002

ziti edge create identity "trino-client-0002" \
    --role-attributes trino-clients-0002 \
    --jwt-output-file trino-client-0002.jwt

ziti edge create config "trino-intercept-config-0002" intercept.v1 \
    '{"protocols":["tcp"],"addresses":["trino-0002.ziti.internal"], "portRanges":[{"low":8080, "high":8080}]}'

ziti edge create config "trino-host-config-0002" host.v1 \
    '{"protocol":"tcp", "address":"trino.default.svc","port":8080}'

ziti edge create service "trino-service-0002" \
    --configs trino-intercept-config-0002,trino-host-config-0002

ziti edge create edge-router-policy "default" \
    --edge-router-roles '#all' --identity-roles '#all'

ziti edge create service-edge-router-policy "default" \
    --edge-router-roles '#all' --service-roles '#all'

ziti edge create service-policy "trino-dial-policy-0002" Dial \
    --service-roles '@trino-service-0002' --identity-roles '#trino-clients-0002'

ziti edge create service-policy "trino-bind-policy-0002" Bind \
    --service-roles '@trino-service-0002' --identity-roles '#trino-0002'


sudo systemctl enable --now ziti-edge-tunnel.service
sudo ziti-edge-tunnel add --jwt "$(< ./trino-client-0002.jwt)" --identity trino-client-0002

sudo chown -cR :ziti        /opt/openziti/etc/identities
sudo chmod -cR ug=rwX,o-rwx /opt/openziti/etc/identities

# package users can restart with systemd
sudo systemctl restart ziti-edge-tunnel.service

Errors

Any ideas on what the issue is?

TheLumberjack · June 13, 2024, 7:15pm

Hi @yemaney, welcome to the community and to OpenZiti!

What is in the tunneller logs? Please put log snippets into text blocks so we can copy/paste/read them better. I can't read your router logs.

Logs should be pasted into code blocks like:

```
Logs should be inside triple ticks like this
and on new lines
```

If you can get the logs from the router and tunneler into a text block so I could read them, I can try to help more.

yemaney · June 13, 2024, 7:21pm

Here are some logs from router

{"error":"error dialing outgoing link [l/1GhtC320lAy0NWjrrlxmb0@1]: error dialing payload channel for [l/1GhtC320lAy0NWjrrlxmb0]: dial tcp 10.0.157.16:3031: i/o timeout","file":"github.com/openziti/ziti/router/link/link_registry.go:478","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":1,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"error","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"error dialing link","time":"2024-06-13T18:59:42.349Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":1,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"status updated","newState":"dialFailed","oldState":"dialing","time":"2024-06-13T18:59:42.349Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":1,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"status updated","newState":"dialing","oldState":"dialFailed","time":"2024-06-13T18:59:51.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:463","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState","iteration":2,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"queuing link to dial","time":"2024-06-13T18:59:51.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":1,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"status updated","newState":"dialing","oldState":"dialFailed","time":"2024-06-13T18:59:51.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:463","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState","iteration":2,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"queuing link to dial","time":"2024-06-13T18:59:51.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:475","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":2,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing link","time":"2024-06-13T18:59:51.060Z"}
{"connId":"e626e2d1-1b00-4ebb-be6c-1892339b0952","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:100","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing link with split payload/ack channels","time":"2024-06-13T18:59:51.060Z"}
{"connId":"e626e2d1-1b00-4ebb-be6c-1892339b0952","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:113","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing payload channel","time":"2024-06-13T18:59:51.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:475","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":2,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"dialing link","time":"2024-06-13T18:59:51.060Z"}
{"connId":"d1825b84-fe05-4338-ab5a-00249fc936ba","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:100","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"dialing link with split payload/ack channels","time":"2024-06-13T18:59:51.060Z"}
{"connId":"d1825b84-fe05-4338-ab5a-00249fc936ba","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:113","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"dialing payload channel","time":"2024-06-13T18:59:51.060Z"}
{"error":"error dialing outgoing link [l/1wErXEm7M7UZ4GAZaYzhQi@2]: error dialing payload channel for [l/1wErXEm7M7UZ4GAZaYzhQi]: dial tcp 10.0.157.16:3029: i/o timeout","file":"github.com/openziti/ziti/router/link/link_registry.go:478","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":2,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"error","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"error dialing link","time":"2024-06-13T18:59:56.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":2,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"status updated","newState":"dialFailed","oldState":"dialing","time":"2024-06-13T18:59:56.060Z"}
{"error":"error dialing outgoing link [l/1GhtC320lAy0NWjrrlxmb0@2]: error dialing payload channel for [l/1GhtC320lAy0NWjrrlxmb0]: dial tcp 10.0.157.16:3031: i/o timeout","file":"github.com/openziti/ziti/router/link/link_registry.go:478","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":2,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"error","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"error dialing link","time":"2024-06-13T18:59:56.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":2,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"status updated","newState":"dialFailed","oldState":"dialing","time":"2024-06-13T18:59:56.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":2,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"status updated","newState":"dialing","oldState":"dialFailed","time":"2024-06-13T19:00:06.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:463","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState","iteration":3,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"queuing link to dial","time":"2024-06-13T19:00:06.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":2,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"status updated","newState":"dialing","oldState":"dialFailed","time":"2024-06-13T19:00:06.061Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:463","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState","iteration":3,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"queuing link to dial","time":"2024-06-13T19:00:06.061Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:475","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":3,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"dialing link","time":"2024-06-13T19:00:06.061Z"}
{"connId":"233633e5-f721-4b09-84d3-0afffcf703a9","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:100","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"dialing link with split payload/ack channels","time":"2024-06-13T19:00:06.061Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:475","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":3,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing link","time":"2024-06-13T19:00:06.061Z"}
{"connId":"301bb7a5-2376-48b4-b63f-1ad921526ee7","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:100","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing link with split payload/ack channels","time":"2024-06-13T19:00:06.061Z"}
{"connId":"301bb7a5-2376-48b4-b63f-1ad921526ee7","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:113","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing payload channel","time":"2024-06-13T19:00:06.061Z"}
{"connId":"233633e5-f721-4b09-84d3-0afffcf703a9","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:113","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"dialing payload channel","time":"2024-06-13T19:00:06.061Z"}
{"error":"error dialing outgoing link [l/1wErXEm7M7UZ4GAZaYzhQi@3]: error dialing payload channel for [l/1wErXEm7M7UZ4GAZaYzhQi]: dial tcp 10.0.157.16:3029: i/o timeout","file":"github.com/openziti/ziti/router/link/link_registry.go:478","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":3,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"error","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"error dialing link","time":"2024-06-13T19:00:11.061Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":3,"key":"default-\u003etls:aNDzWc81O-\u003edefault","level":"info","linkId":"1wErXEm7M7UZ4GAZaYzhQi","msg":"status updated","newState":"dialFailed","oldState":"dialing","time":"2024-06-13T19:00:11.061Z"}
{"error":"error dialing outgoing link [l/1GhtC320lAy0NWjrrlxmb0@3]: error dialing payload channel for [l/1GhtC320lAy0NWjrrlxmb0]: dial tcp 10.0.157.16:3031: i/o timeout","file":"github.com/openziti/ziti/router/link/link_registry.go:478","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":3,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"error","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"error dialing link","time":"2024-06-13T19:00:11.061Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":3,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"status updated","newState":"dialFailed","oldState":"dialing","time":"2024-06-13T19:00:11.061Z"}
{"file":"github.com/openziti/ziti/router/link/link_state.go:97","func":"github.com/openziti/ziti/router/link.(*linkState).updateStatus","iteration":3,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"status updated","newState":"dialing","oldState":"dialFailed","time":"2024-06-13T19:00:21.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:463","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState","iteration":4,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"queuing link to dial","time":"2024-06-13T19:00:21.060Z"}
{"file":"github.com/openziti/ziti/router/link/link_registry.go:475","func":"github.com/openziti/ziti/router/link.(*linkRegistryImpl).evaluateLinkState.func1","iteration":4,"key":"default-\u003etls:NxWYfPm0eH-\u003edefault","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing link","time":"2024-06-13T19:00:21.060Z"}
{"connId":"612dfb99-69dc-490a-b2cd-4cd467015b0e","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:100","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing link with split payload/ack channels","time":"2024-06-13T19:00:21.060Z"}
{"connId":"612dfb99-69dc-490a-b2cd-4cd467015b0e","file":"github.com/openziti/ziti/router/xlink_transport/dialer.go:113","func":"github.com/openziti/ziti/router/xlink_transport.(*dialer).dialSplit","level":"info","linkId":"1GhtC320lAy0NWjrrlxmb0","msg":"dialing payload channel","time":"2024-06-13T19:00:21.060Z"}

Here is systemctl status of tunnerl. Not sure if there is anothe rcommand to get its logs.

ubuntu@ip-10-0-10-230:~$ sudo systemctl status ziti-edge-tunnel.service
● ziti-edge-tunnel.service - Ziti Edge Tunnel
     Loaded: loaded (/usr/lib/systemd/system/ziti-edge-tunnel.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-06-13 18:30:11 UTC; 51min ago
    Process: 191939 ExecStartPre=/opt/openziti/bin/ziti-edge-tunnel.sh (code=exited, status=0/SUCCESS)
   Main PID: 191943 (ziti-edge-tunne)
      Tasks: 6 (limit: 1078)
     Memory: 5.2M (peak: 5.7M)
        CPU: 8.647s
     CGroup: /system.slice/ziti-edge-tunnel.service
             └─191943 /opt/openziti/bin/ziti-edge-tunnel run --verbose=2 --dns-ip-range=100.64.0.1/10 --identity-dir=/opt/openziti/etc/identities

Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: Starting ziti-edge-tunnel.service - Ziti Edge Tunnel...
Jun 13 18:30:11 ip-10-0-10-230 ziti-edge-tunnel.sh[191939]: NOTICE: no new JWT files in /opt/openziti/etc/identities/*.jwt
Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: Started ziti-edge-tunnel.service - Ziti Edge Tunnel.
Jun 13 18:30:11 ip-10-0-10-230 ziti-edge-tunnel[191943]: (191943)[        0.000]    INFO ziti-sdk:utils.c:201 ziti_log_set_level() set log level: root=3/INFO
Jun 13 18:30:11 ip-10-0-10-230 ziti-edge-tunnel[191943]: (191943)[        0.000]    INFO ziti-sdk:utils.c:170 ziti_log_init() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting at (2024-06-13T18:30:11.>

TheLumberjack · June 13, 2024, 7:26pm

Thanks for using the code block! much appreciated...

journalctl -u ziti-edge-tunnel for tunneler logs.

Is your local tunneler using the local router for onboarding? It might be necessary to make a small network diagram of "what is where" to help me out too.

In the router i see "error dialing link". That makes me think the router cannot make a link to another router and that's why I think it might be necessary to have a network diagram to help.

Let's see the tunneler's logs too though and see if any errors are in there

yemaney · June 13, 2024, 7:51pm

logs from journalctl -u ziti-edge-tunnel for tunneler logsI did do asudo systemctl restart ziti-edge-tunnel.service` command at 18:05 which seems to have affected these logs.

Jun 13 18:02:42 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42638.029]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[1] failed to connect to ER[router0002] [-3001/temporary>
Jun 13 18:02:59 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42655.000]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[4] failed to connect to ER[router10] [-103/software cau>
Jun 13 18:03:40 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42695.759]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[4] failed to connect to ER[router10] [-103/software cau>
Jun 13 18:03:54 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42709.617]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[0] failed to connect to ER[router5] [-3001/temporary fa>
Jun 13 18:04:10 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42725.451]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[1] failed to connect to ER[router0002] [-3001/temporary>
Jun 13 18:04:13 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42729.201]   ERROR ziti-sdk:ziti_enroll.c:234 enroll_cb() failed to enroll with controller: https://ec2-13-60-60-200.eu-north-1.co>
Jun 13 18:04:13 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42729.201]   ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:319 tunnel_enroll_cb() enrollment failed: INVALID_ENROLLMENT_TOKEN(-3)
Jun 13 18:04:13 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42729.202]    WARN ziti-edge-tunnel:ziti-edge-tunnel.c:647 on_cmd() received from client - EOF. Closing connection.
Jun 13 18:04:13 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42729.202]    WARN ziti-edge-tunnel:ziti-edge-tunnel.c:659 on_cmd() IPC client connection closed, count: 0
Jun 13 18:04:33 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42748.583]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[3] failed to connect to ER[router12] [-111/connection r>
Jun 13 18:05:00 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42775.701]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[4] failed to connect to ER[router10] [-103/software cau>
Jun 13 18:05:07 ip-10-0-10-230 ziti-edge-tunnel[163201]: (163201)[    42782.460]   ERROR ziti-sdk:channel.c:903 on_channel_connect_internal() ch[1] failed to connect to ER[router0002] [-3001/temporary>
Jun 13 18:05:14 ip-10-0-10-230 systemd[1]: Stopping ziti-edge-tunnel.service - Ziti Edge Tunnel...
Jun 13 18:05:14 ip-10-0-10-230 systemd[1]: ziti-edge-tunnel.service: Deactivated successfully.
Jun 13 18:05:14 ip-10-0-10-230 systemd[1]: Stopped ziti-edge-tunnel.service - Ziti Edge Tunnel.
Jun 13 18:05:14 ip-10-0-10-230 systemd[1]: ziti-edge-tunnel.service: Consumed 3min 24.140s CPU time, 6.9M memory peak, 0B memory swap peak.
Jun 13 18:05:14 ip-10-0-10-230 systemd[1]: Starting ziti-edge-tunnel.service - Ziti Edge Tunnel...
Jun 13 18:05:14 ip-10-0-10-230 ziti-edge-tunnel.sh[179635]: NOTICE: no new JWT files in /opt/openziti/etc/identities/*.jwt
Jun 13 18:05:14 ip-10-0-10-230 systemd[1]: Started ziti-edge-tunnel.service - Ziti Edge Tunnel.
Jun 13 18:05:14 ip-10-0-10-230 ziti-edge-tunnel[179639]: (179639)[        0.000]    INFO ziti-sdk:utils.c:201 ziti_log_set_level() set log level: root=3/INFO
Jun 13 18:05:14 ip-10-0-10-230 ziti-edge-tunnel[179639]: (179639)[        0.000]    INFO ziti-sdk:utils.c:170 ziti_log_init() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting at (2024-06-13T18:05:14.>
Jun 13 18:14:02 ip-10-0-10-230 systemd[1]: Stopping ziti-edge-tunnel.service - Ziti Edge Tunnel...
Jun 13 18:14:02 ip-10-0-10-230 systemd[1]: ziti-edge-tunnel.service: Deactivated successfully.
Jun 13 18:14:02 ip-10-0-10-230 systemd[1]: Stopped ziti-edge-tunnel.service - Ziti Edge Tunnel.
Jun 13 18:14:02 ip-10-0-10-230 systemd[1]: ziti-edge-tunnel.service: Consumed 1.496s CPU time, 6.0M memory peak, 0B memory swap peak.
Jun 13 18:14:02 ip-10-0-10-230 systemd[1]: Starting ziti-edge-tunnel.service - Ziti Edge Tunnel...
Jun 13 18:14:02 ip-10-0-10-230 ziti-edge-tunnel.sh[185330]: NOTICE: no new JWT files in /opt/openziti/etc/identities/*.jwt
Jun 13 18:14:02 ip-10-0-10-230 systemd[1]: Started ziti-edge-tunnel.service - Ziti Edge Tunnel.
Jun 13 18:14:02 ip-10-0-10-230 ziti-edge-tunnel[185336]: (185336)[        0.000]    INFO ziti-sdk:utils.c:201 ziti_log_set_level() set log level: root=3/INFO
Jun 13 18:14:02 ip-10-0-10-230 ziti-edge-tunnel[185336]: (185336)[        0.000]    INFO ziti-sdk:utils.c:170 ziti_log_init() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting at (2024-06-13T18:14:02.>
Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: Stopping ziti-edge-tunnel.service - Ziti Edge Tunnel...
Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: ziti-edge-tunnel.service: Deactivated successfully.
Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: Stopped ziti-edge-tunnel.service - Ziti Edge Tunnel.
Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: ziti-edge-tunnel.service: Consumed 2.725s CPU time, 7.5M memory peak, 0B memory swap peak.
Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: Starting ziti-edge-tunnel.service - Ziti Edge Tunnel...
Jun 13 18:30:11 ip-10-0-10-230 ziti-edge-tunnel.sh[191939]: NOTICE: no new JWT files in /opt/openziti/etc/identities/*.jwt
Jun 13 18:30:11 ip-10-0-10-230 systemd[1]: Started ziti-edge-tunnel.service - Ziti Edge Tunnel.
Jun 13 18:30:11 ip-10-0-10-230 ziti-edge-tunnel[191943]: (191943)[        0.000]    INFO ziti-sdk:utils.c:201 ziti_log_set_level() set log level: root=3/INFO
Jun 13 18:30:11 ip-10-0-10-230 ziti-edge-tunnel[191943]: (191943)[        0.000]    INFO ziti-sdk:utils.c:170 ziti_log_init() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting at (2024-06-13T18:30:11.>

Here is a simplate diagram

running the controller in blue instance (k3)
logging into the controller and running ziti commands and deploying tunnel service in red instance from red instance
router deployed in green (k8s)

TheLumberjack · June 13, 2024, 8:06pm

Appreciate the diagram. I think it verifies what I thought was the topology but i don't see routers in the red and blue zones, i assume routers are in each? I'm assuming the red zone is 'public' and the blue and green zones are 'private' and each have routers...

It looks like your tunneler can't connect to any of your routers and I see enrollment failed in your tunneler too. There seems to be a fair number of issues logged from the tunneler as well.

There are numerous things to verify:

the app can connect to the green ziti-router edge listener?
the green router is able to connect to the red router's advertised link port?
the blue router is able to connect to the red router's advertised link port?
the router in the blue zone can connect to whatever the target is?
the green app can connect to the blue ziti controller's edge api port?

yemaney · June 13, 2024, 8:37pm

The only router I've currently deployed is the one in green.
Is a router required in each network?

I assumed a tunnel service in red alone would be good enough to reach the green router

TheLumberjack · June 13, 2024, 9:05pm

Only one really? Strange since it looks like there are at least three from the log? do any of these seem familiar?

failed to connect to ER[router0002] 
failed to connect to ER[router10] 
failed to connect to ER[router5]

Are all three colors networks? If they are all networks, are all of them public or are all of them private? One router is all you need as long as there's a ziti-edge-tunnel in the red network and a ziti-edge-tunnel in the blue network. Do you have those setup?

Where is the server the green app is trying to get to? (intercept trino-0002.ziti.internal:8080) Maybe expand on the diagram a bit more to help me understand the layout a bit more?

yemaney · June 20, 2024, 8:34pm

Hi, Iended up resolving this by including the option --tunneler-enabled in the command for creating the edge-router identity. That identity files could then be used to deploy a router in k8s cluster successfully.

ziti edge create edge-router $ROUTER_NAME \
  --tunneler-enabled 
  --jwt-output-file $ROUTER_NAME.jwt

Topic		Replies	Views
Adding new router to ziti network fails Ziti Overlay	13	338	April 2, 2023
How to access private eks cluster apiserver via openziti Ziti Overlay	2	28	July 7, 2024
Deploying router on k8s via helm gives identity secret mount error	29	498	March 18, 2024
Not able to install router via helm chart to connect to ziti controller at port 443 but able to connect via router script Ziti Overlay	18	78	July 18, 2024
Implementing an overlay network Ziti Overlay	10	260	May 1, 2023

Tunneling into private router in k8s

Created router via terraform helm provider.

Ran the following commands to run ziti tunnel service in a linux machine

Errors

Related topics