Few bugs/issues

1

Hello, I've been playing around with Ziti a bit, especially on Android and I noticed a few issues.

  1. The Android Edge client app goes too far up under the clock/notification bar instead of starting below it, like most apps. I have a Pixel 8 Pro.
  2. If you try and login to ZAC on android (using Chrome) it says "Login Failed: Unable to login to selected edge controller - OK. Please make sure the selected controller is online and accessible." - However I know my login is correct and my network has access and that ZAC is working as a Windows 10 PC with Chrome can succesfully login on the same network.
  3. After enrolling on the Android edge client, and i get a quick bubble that says "Enrolled!" I don't see the identity listed, I need to close the app and go back in.
  4. While not directly an issue... I saw this: add support for Ext JWT signer authentication · Issue #253 · openziti/ziti-tunnel-android · GitHub and was wondering if this means Android edge client users can now authenticate via IDPs such as Authentik? I'm envisioning users enabling the Ziti tunnel and being directed to the IDP's webpage and having to login (unless their login is still valid) each time they enable the tunnel, for proper user auth.
2

@007bond007
Thank you for using/trying out OpenZiti Android client.

  1. This has been fixed in the latest update (rolling out to production now)
  2. can't comment on it. (maybe @TheLumberjack ?)
  3. I added an issue for it enrolled identity not shown until UI restart · Issue #268 · openziti/ziti-tunnel-android · GitHub
  4. yes, that is correct, external login can now be used in Android client. Please try it and report any issues.

Thanks

3
  1. can't comment on it. (maybe @TheLumberjack ?)

I believe #2 may be related to this bug, which was just very recently fixed in the 3.7.1 release of ZAC.

@007bond007 if/when you get a chance, can you try upgrading to the latest ZAC (currently 3.7.1) and try again?

4

Awesome... thank you both for your quick response! @rgalletto @ekoby

  1. Fixed... looks great!
  2. Confirmed to be fixed.... however another bug popped up (see #5 below)
  3. Thankyou!
  4. Awesome... saw it listed in the latest Android update... however When trying to "Enable" the identity that is set to an auth policy using the external JWT I receive "failed to authenticate" for "Status" right after enabling it on Android. I confirmed my JWT information is correct in ZAC for Authentik. I'm wondering if in my identity what "Enrollment Type" should be for using the external JWT. See below picture for identity config:
    image
    image693×881 50.6 KB

New Issue:
5. On Chrome in Windows, under "Authentication" I no longer see the links for "Auth Policies" and "JWT Signers" at the top tabs, only "Manage Certificate Authorities", but on Android they still appear. I can manually type the URL in the browser on my desktop to get there, and then the links appear unless I go back into Authentication. Might just be missing links on the desktop version of the page. I also noticed the website doesn't seem well-optimized for mobile (I had to turn on "Desktop Mode" and zoom in where needed).
6. There appears to be a broken link on this documentation page: By JWT | OpenZiti - the "See Authenticating" link at the bottom.
7. I tried removing the test123 identity I created and then I was unable to add it back in (it says "Enrolled!" and then never appears, even after disconnecting and closing the app, but yet the barcode enrollment option in ZAC is gone) and my first identity "josh" I turned it off and back on, and it says "offline" despite the intercept still working. I'm not sure how I set off this chain of bugs!

Update: If I "clear data" in Android settings for the app, and re-enroll I can get back to working Enrollment but I'm still getting the Auth failed errors for the external JWT.

5

re 5: in my experience ZAC is prone to weird caching issues on Chrome. Clearing cache usually resolves UI issues for me

re 7: make sure your external signer is configured with external auth url -- otherwise SDK does not know how to bootstrap OIDC flow. that URL should be the base URL for .well-known/openid-configuration. I am not familiar with Authentik, but for keycloak it is https://<hostname>:<port>/realms/my_realm. Also your identity must have external ID set.

6

@ekoby

Thankyou for the reply. I came across a video on the youtube channel that walks through setting up the external signer and noted what you mentioned about, about needing the "base URL" for the .well-known - this is odd as most apps actually want the external auth URL (which Authentik provides readily). I did change this and adjust a few things based on the video... though I think I found the reason things don't work as expected.

In the video it looks like they download a "Network JWT" file, which I don't have a button for on the "JWT Signers" page. My previously created test identity, when resetting the enrollment (despite having "None" set for enrollment) allowed me a QR code and JWT file download - which is what I was using to try and enroll. I created a new test user, and these options were then unavailable - so perhaps because the users were created in an older version of ZAC. Either way I'm not sure how to get that network JWT file now - I also find this very unfriendly for users - most "VPN"-like apps allow you to just put in a "Server URL" for the VPN Server (Ziti Controller in this case) and then the IDP is automatically called (look at how NetBird does this, seamlessly).

I also noticed another bug - if I'm adding scopes to the JWT signer and I use a space, it will add a literal space in the quotes for the entry, but if I use a comma, it won't. My guess is there should be a function that strips any whitespace on the beginning/end of the entry when pressing space before committing it.

And another bug I found is I'm unable to delete my original JWT Signer from the JWT Signers page (with the "-" circle or the "..." > Delete).

7

you should be able to see network JWT link from JWT signers page. If it is not there you can always download it directly from https://<controller_address>/network-jwts. It will be in the JSON response (token field), like this:

{
    "data": [
        {
            "name": "default",
            "token": "eyJhbGciOiJSUzI1NiIs... "
        }
    ],
    "meta": {}
}
8

@rgalletto can you verify/fix this pls? Is this just a visual issue maybe or does the data contain the space?

9

The link wont show if the controller isn't at a version that supports the /network-jwts endpoint in the edge API. I believe that was added somewhere around v1.1.17 v1.2.0.

  1. I no longer see the links for "Auth Policies" and "JWT Signers" at the top tabs

I'm not able to reproduce this one unfortunately..

if I'm adding scopes to the JWT signer and I use a space, it will add a literal space in the quotes for the entry

I will take a closer look at this. Sounds like an issue with parsing/trimming pasted input.

10

Ok so I thought my controller was up to date, turns out apt-update/upgrade doesn't do it... found the article and upgraded to 1.2.2 from 1.1.15 :). Looks like I can now see the network JWT button, and I can also delete my old External JWK and I see the tabs after clearing my browser cache on Chrome.

Now I'm trying to add the network JWT to my Android app. and it seems to be crashing the app "Ziti Mobile Edge keeps stopping". Sometimes this message doesn't appear and it just brings me to the main screen of the Ziti app - but I don't see any additional connection. I'll give this a try on the desktop to see if it works there.

Update: Looks like the desktop adds it but no IDP icon, even disabling and re-enabling it. Hmm

Thankyou for all your help so far, all! I really appreciate it.

11

Ok did some more testing, the SSL seems valid (my alt cert from let's encrypt) but looking at the debug logs on the windows client I see the following (domain replaced for privacy):

[2025-01-24T04:45:32.207Z]   DEBUG ziti-sdk:ziti_ctrl.c:1071 ctrl_paging_req() ctrl[ziti.domain.net:8441] starting paging request GET[/external-jwt-signers]
[2025-01-24T04:45:32.213Z]   ERROR tlsuv:engine.c:923 openssl: handshake was terminated: error:00000005:lib(0)::reason(5)
[2025-01-24T04:45:32.213Z]   ERROR tlsuv:tls_link.c:113 TLS(000001699ae94020) handshake error error:00000005:lib(0)::reason(5)
[2025-01-24T04:45:32.213Z]   ERROR tlsuv:http.c:189 handshake failed status[3]: error:00000005:lib(0)::reason(5)
[2025-01-24T04:45:32.213Z]    WARN ziti-sdk:ziti_ctrl.c:184 ctrl_resp_cb() ctrl[ziti.domain.net:8441] request failed: -4079(software caused connection abort)
[2025-01-24T04:45:32.213Z]    WARN ziti-sdk:ziti_ctrl.c:342 internal_version_cb() ctrl[ziti.domain.net:8441] CONTROLLER_UNAVAILABLE(software caused connection abort)
[2025-01-24T04:45:32.213Z]    WARN ziti-sdk:ziti.c:1908 version_pre_auth_cb() ztx[0] failed to get controller version: CONTROLLER_UNAVAILABLE/software caused connection abort
[2025-01-24T04:45:32.213Z]    WARN ziti-sdk:ziti_ctrl.c:184 ctrl_resp_cb() ctrl[ziti.domain.net:8441] request failed: -4079(software caused connection abort)
[2025-01-24T04:45:32.213Z]    INFO ziti-sdk:ziti_ctrl.c:187 ctrl_resp_cb() ctrl[ziti.domain.net:8441] attempting to switch endpoint
[2025-01-24T04:45:32.213Z]    WARN ziti-sdk:ziti_ctrl.c:605 ctrl_next_ep() ctrl[ziti.domain.net:8441] no controllers are online
[2025-01-24T04:45:32.213Z]    WARN ziti-sdk:ziti.c:641 ext_jwt_singers_cb() ztx[0] failed to get external auth providers: software caused connection abort
12

This isn't really quite enough information, I don't think. It simply looks like the windows client can't connect to the controller. This thread is also getting a little long. Would you mind starting a new one with a clear title? We choose to use discourse so people hitting issues would be able to search the forum for similar problems, starting a new thread for the windows problem will help those people.

Are you downloading the network jwt from zac I assume? Could you share it with me via dm or email? World you send logs for us to look at?

13

Yes no problem, will start a new one. The windows and Android issues seems related though. This was the windows debug log for the connection, should I also include controller logs and Android logs? Yes it is the network JWT from Zac.

1 Like