Debugging the Ziti Mobile Edge tunneller

#situation

  1. I have installed the Ziti Mobile Edge tunneller on my phone :ok_hand:

  2. I created the client jwt file :ok_hand:

  3. I copied hte jwt file to the mobile :ok_hand:

  4. I added the identity using jwt file :fire:

I was looking into the logs on at the application level in the app.. this is what it said

ERROR ziti-sdk:ziti_ctrl.c:152 ctrl_resp_cb() ctrl[instance-20220317-1005] unavailable: -3008(unknown node or service)
... : 200 on_enroll() unable to decode context
... deleteCertificate().. the specified item could not be found in the keychain

I also noticed that the dns ip address was different to the ip address of the remote server

do you need to update these settings to use the Ziti Mobile Edge tunneller?

Does your mobile app show you that your identity has any services? This error on its own doesn’t provide quite enough context as to what “the problem” might be (or if there is even ‘a problem’).

Don’t get hung up on the reported dns ip vs the remote server. With OpenZiti I would say generally speaking, I never end up really caring about ip addresse in use. Usually when something goes wrong it’s because I’ve misconfigured a service/policy/identity or my hosting identity is offline.

1 Like

ok… unfortunately… the mobile app does not show much.

What happens is that it starts… then it goes into a “spinning wheels”… I have waited 5 min then went backwards…

Then… when I try to restart… the jwt has expired.

I thought to look at the logs on the server… but not sure where too look

My understanding is that I dont need to have the http server running to enroll a client

These are all of the commands that I have run.

Let me know if you have any debug tips to further investigate

create new service and bind & dial policies

ziti edge create config “${service_name}”-host.v1 host.v1 ‘{“protocol”:“tcp”, “address”:“localhost”,“port”:’"${the_port}"’, “listenOptions”: {“bindUsingEdgeIdentity”:true}}’

ziti edge create config “${service_name}”-client-config intercept.v1 ‘{“protocols”:[“tcp”],“addresses”:["’"${service_name}.ziti"’"], “portRanges”:[{“low”:’"${the_port}"’, “high”:’"${the_port}"’}]}’

ziti edge create service “${service_name}” --configs “${service_name}”-client-config,"${service_name}"-host.v1

ziti edge create service-policy “${service_name}”-binding Bind --service-roles ‘@’"${service_name}" --identity-roles ‘#’"${service_name}"‘ServerEndpoints’
ziti edge create service-policy “${service_name}”-dialing Dial --service-roles ‘@’"${service_name}" --identity-roles ‘#’"${service_name}"‘ClientEndpoints’

create and enroll server identities

ziti edge create identity device “${server_identity}” -a "${service_name}"ServerEndpoints -o “${server_identity}”.jwt

ziti edge enroll “${server_identity}”.jwt

You absolutely must have the controller running, and it must be contactable to enroll a client. The excellent 5-part (very detailed) series on bootstrapping trust gets into that.

At the end of the day your phone will make a CSR (certificate signing request) and send it to the controller. That needs to succeed. If your controller is down (or if it's misconfigured perhaps advertising the wrong ports) then that would be why it failed. After that jwt expired - take it to jwt.io and paste it into their jwt debugger... see what the "iss" (shown below) shows you. Verify that is the external address of YOUR controller (you can see mine shown, on port 8441) and verify your phone can reach it.

1 Like

ok.. the controller is running as I have enrolled the server jwt.

what I meant was the http server..

.. nice tool are jwt.io.. not used that before.. lots of debugging tools to learn :slight_smile:

I guess the question I have is how to confirm that the controller is accessible.. this could be the problem as I changed some of the policies.. from all.. to be service specific

I will return back to the reflect example to see if I can get this to work

Other than that.. is there any tips on how to check if you have access via curl or something like that ? I am on a long walkabout... and way up the creek with no water..

I personally nearly always start with a tunneling app on the client - and a tunneling app on the host side. Those both a UI or have logs that show me that they are either dialing a service/hosting a service pretty clearly.

If the ‘host’ tunneller says “hosting new service” like this:

[        0.178]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:592 on_service() hosting server_address[tcp:httpbin.org:80] service[httpbin]

you know you have the ‘host’ side correct…

if the client side shows you an intercept like my ZDEW shows here:
image

Well then you are pretty sure you’ve configured things correctly… At that point pretty much the only things that go wrong are “anti-virus/firewalls” blocking traffic at the client - or the “host” config sending traffic to the wrong port/ip/dns name…

ok.. I think I know the source of the problem... but dont know how to fix it.

This is the summary from an expired jwt

"em": "ott",
"exp": 1649246270,
"iss": "https://instance-20220317-1005:1280",
"jti": "5ae674e0-2e3c-4064-988d-277820609dfb",
"sub": "PCmL.adN5"

The problem is.. instance-20220317-1005 is the name of the machine.. not the DNS

Is there where the dig tool comes in handy.. as I dont really know the full path.

When I do a dig on the IP address... I get a longer pathname..

So.. my assumption is to change the EXTERNAL_DNS variable to this longer version..

though wouldn't it still work if I used the IP name instead of the DNS.. or is there something special that goes on behind the scenes that would cause this to fail

I’d recommend you follow that “fourth” quickstart and do the “host it anywhere”. Your PKI is probably all setup for “local only” not “anywhere” (which is what you really want).

for anyone coming to this thread - EXTERNAL_DNS Is referenced in that quickstart and over here: Error creating identities instance in Ziti Edge Controller - #5 by dovholuknf

EXTERNAL_DNS is only really used to setup those other variables as shown:
image

Those environment variables are then used to bootstrap all your config files. Once the config files are generated, once the PKI generated, these environment variables aren’t the right thing to focus on…

1 Like

Making progress... though I am still having some problems.

The main question I want to check is the port required to enroll the jwt from a ZitiMobileEdge app.

In the example below.. its using port 1280.. which is what I have configured for the controller

"iss": "https://instance-20220317-1005.subnet03170958.vcn03170958.oraclevcn.com:1280",

I have setup the port for the ZITI_EDGE_ROUTER to be 1281.

What happened in some earlier tests this morning is that I had ZITI_EDGE_ROUTER set to 6262..

which was creating a conflict with the following that is defined in the QuickStart env file

ZITI_CTRL_PORT="6262".

I then tried port 1280.. but then it was already being used by this variable

export ZITI_EDGE_CONTROLLER_PORT="1280"

the key learning here is that you need to have three ports open to run the QuickStart on a remote server..

However.. I am still getting some type of configuration issue..

I am trying to do this the hard way so that I can understand all of the config issues..

In my case.. I am not 100% sure if the external DNS is resolvable... but are not 100% sure on how to test this

instance-20220317-1005.subnet03170958.vcn03170958.oraclevcn.com"

this happens most likely because of the port settings on the VPN.. which have been configured to prevent things like ping etc..

Let me know if you have any tips.. as this is not an area of strength.. :slight_smile:

Ok… I understand what is going on now… the following is a private DNS on the Oracle OCI infrastructure… so it does not resolve from my laptop… but it does if you have sshd into the compute

instance-20220317-1005.subnet03170958.vcn03170958.oraclevcn.com

I am going to try again using the IP address instead… which I know can be reached externally…

success… yes :slight_smile:

I enrolled the identity successfully on the mobile app … now… time for the next step…

1 Like

Awesome to see you had success. Using the IP is entirely possible - for a "long term" solution using DNS is superior because if you ever have the need to move that controller you can. other than that using just the ip works fine.

1 Like

This was such a huge win… cant tell you how happy I am…