Issues enrolling Docker ziti-edge-tunnel

Support
1

Greetings,
I have been trying to test using docker to run my routers. Currently, I am running them under Linux, and they work well, but I would like to run them under docker. I haven't really used Docker much, so this might be my fault, but when I try to run a container, I get the following: (hostnames changed for security reasons)

[root@storziti03 routertest]# docker run --name ziti-tun --network mynet --ip "my_ip" --dns "my_dns" --privileged --volume ${PWD}:/ziti-edge-tunnel/ --volume "/var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket" --device "/dev/net/tun:/dev/net/tun" --env ZITI_IDENTITY_BASENAME="my_router_name" openziti/ziti-edge-tunnel
DEBUG: waiting 3s for /ziti-edge-tunnel/my_router_name.json (or token) to appear
DEBUG: identity file /ziti-edge-tunnel/my_router_name.json not found
DEBUG: /var/run/secrets/netfoundry.io/enrollment-token/my_router_name.jwt not found
DEBUG: /enrollment-token/my_router_name.jwt not found
INFO: enrolling /ziti-edge-tunnel/my_router_name.jwt
(7)[ 0.000] INFO ziti-sdk:utils.c:201 ziti_log_set_level() set log level: root=3/INFO
(7)[ 0.000] INFO ziti-sdk:utils.c:172 ziti_log_init() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting at (2024-06-17T17:43:15.020)
(7)[ 0.000] INFO ziti-sdk:ziti_enroll.c:90 ziti_enroll() Ziti C SDK version 1.0.4 @g1ef8211(HEAD) starting enrollment at (2024-06-17T17:43:15.020)
(7)[ 0.000] WARN ziti-sdk:ziti_ctrl.c:302 ctrl_body_cb() could not process controller response: 15 bytes not allocated for body
(7)[ 0.000] ERROR ziti-sdk:ziti_ctrl.c:164 ctrl_resp_cb() ctrl[openziti-controller.athenz-experiments.aws.oath.cloud] request failed: -4095(end of file)
(7)[ 0.000] ERROR ziti-sdk:ziti_enroll.c:235 enroll_cb() failed to enroll with controller: https://openziti-controller.athenz-experiments.aws.oath.cloud:4443 CONTROLLER_UNAVAILABLE (end of file)
(7)[ 0.000] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:2221 enroll_cb() enrollment failed: CONTROLLER_UNAVAILABLE(-3)
ERROR: failed to enroll with token from /ziti-edge-tunnel/my_router_name.jwt (705B)

I thought it was going to be something dumb, like lack of permissions to read my jwt file off of the local filesystem, so I used chmod 777 on it, and ruled that out. Then I thought it might be network related, but I used tcpdump on the parent interface of the mynet network, sniffing for traffic to and from the openziti controller, and there is a back and forth exchange that happens there. One variable of which I am unsure of consequences is that I believe our controller is running v0.30.3, and I am running the latest ziti-edge-tunnel container. A different team here runs the controller, so I don't directly have a say over the version, but I may be able to have it upgraded if that is determined to be an issue. Although with Linux routers running in AWS, I have been successfully enrolling new routers, so I don't believe that to be the issue.

My network in Docker looks like this:
[root@storziti03 routertest]# docker network inspect mynet
[
{
"Name": "mynet",
"Id": "5692c715fb91951ad1dfb828fef1321759c59fd2d1edf82cc75bbce8681b105a",
"Created": "2024-06-14T21:24:37.392940633Z",
"Scope": "local",
"Driver": "macvlan",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "10.10.10.128/25",
"IPRange": "10.10.10.192/31",
"Gateway": "10.10.10.129"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {
"parent": "enp4s0f0"
},
"Labels": {}
}
]

If anyone has any insight into this, I would really appreciate it!
Thanks

2

Hi @greggw01,

You're trying to run ziit-edge-tunnel from within docker? Is there a reason you'd like to do that? If this were me, I would run ziti-edge-tunnel outside of docker, and connect to the routers running in docker from outside of docker, which I expect you're doing either to test the routers are setup correctly or so that you can host things entirely in docker and not have to expose any ports to your underlay, right?

It might be best/easiest if you start with one thing at a time and just move your router into docker first.

The logs you posted are indicating that the jwt enrollment failed which can happen for a myriad of reasons.

It shouldn't be a problem with newer edge not working with older controller, but i suppose it's possible. I just wouldn't immediately expect it since it was working outside of docker (right)?

One final question, the title of the post states: "Issues enrolling Docker ziti-edge-tunnel " and then in the first paragraph you write "trying to test using docker to run my routers". Is it possible you're running the wrong docker image?

I guess I'm a little confused as to what you're tyring to do now... Maybe another post to try to describe the situation would help me help you better.

3

Yes, I too see the log indicated some bytes were received from the controller's client API, so I didn't expect the CONTROLLER_UNAVAILABLE message. It makes me wonder if the enrollment token was for a Ziti router, not a Ziti identity. The token type for a router is erott (edge router one time token) and the type of an identity is ott. You can parse the JWT with jwt.io or this Python script.

Here are the correct Docker images for:

  • router: openziti/ziti-router - link to doc
  • identity: openziti/ziti-host (hosting proxy only) and openziti/ziti-edge-tunnel (bi-directional proxy) - link to doc
4

Yeah, that was my issue. I was using the wrong container for what I was trying to do.

2 Likes