ZDEW OIDC Enrollment Fails - Controller Ignores API Session Token After OIDC Auth

Hi Ziti Community,

I'm hoping for some guidance on an issue I'm facing with Ziti Desktop Edge (ZDE) for Windows enrollment via OIDC using Microsoft Entra ID.

My Goal:
To allow users to enroll their ZDE for Windows clients using their Microsoft Entra ID credentials.

My Setup:

  • Ziti Controller Version: v1.5.4 (from Docker, URL https://controller-url:1280)

  • Ziti Desktop Edge: Service v1.5.8,

  • Identity Provider: Microsoft Entra ID

The Problem:
The OIDC authentication flow itself appears to complete successfully. The ZDE client authenticates via Entra ID, and the Ziti controller validates the received OIDC token and issues a Ziti API session token.

However, immediately afterward, when ZDE attempts subsequent API calls (like GET /current-identity or POST /current-api-session/certificates to finalize enrollment), these calls are rejected by the controller.

What I'm Seeing in the Logs:

  • ZDE Client Logs show the successful OIDC flow and API session acquisition:
DEBUG ziti-sdk:oidc.c:927 oidc_client_set_tokens() using id_token={...<REDACTED OIDC ID TOKEN DETAILS>...}
DEBUG ziti-sdk:external_auth.c:94 ext_token_cb() received access token: <REDACTED_ACCESS_TOKEN_SNIPPET>...
VERBOSE ziti-sdk:ziti_ctrl.c:145 start_request() ctrl[https://<controller-url>/] starting POST[/authenticate]
VERBOSE ziti-sdk:ziti_ctrl.c:427 ctrl_body_cb() ctrl[https://<controller-url>/] HTTP RESPONSE: {"data":{"_links":{...},"id":"<REDACTED_api_session_id>", ...}}
DEBUG ziti-sdk:ziti_ctrl.c:394 ctrl_login_cb() ctrl[https://<your-controller-url>/] authenticated successfully session[<REDACTED_api_session_id>]

Then, subsequent calls immediately fail:

VERBOSE ziti-sdk:ziti_ctrl.c:145 start_request() ctrl[https://<controller-url>/] starting GET[/current-identity]
VERBOSE ziti-sdk:ziti_ctrl.c:427 ctrl_body_cb() ctrl[https://<controller-url>/] HTTP RESPONSE: {"error":{"code":"UNAUTHORIZED","message":"The request could not be completed. The session is not authorized or the credentials are invalid",...}}
ERROR ziti-sdk:ziti_ctrl.c:522 ctrl_body_cb() ctrl[...] API request[/current-identity] failed code[UNAUTHORIZED]...

(Similar errors occur for POST /current-api-session/certificates etc.)

  • Controller (v1.5.4) Logs show successful OIDC token validation and API session issuance:
<timestamp> DBG authMethod=ext-jwt authPolicyId=<REDACTED_oidc_auth_policy_id> expectedAudience=<REDACTED_ENTRA_APP_CLIENT_ID> extJwtSignerId=<REDACTED_ext_jwt_signer_id> ... identityId=<REDACTED_ziti_identity_id> issuer=https://login.microsoftonline.com/<REDACTED_ENTRA_TENANT_ID>/v2.0 msg=validated candidate JWT at index 0 tokenAudiences=<REDACTED_ENTRA_APP_CLIENT_ID>
<timestamp> DBG apiSessionId=<example_api_session_id> ... msg=adding apiSession strategy=instant

But then, for the ZDE's very next API calls, the controller logs this:

<timestamp> ERR authMethod=ext-jwt ... msg=encountered 0 candidate JWTs, verification cannot occur

Key Configuration Points:

  • My controller uses an internal PKI for its listener on port 1280. ZDE client trusts this CA.

  • My External JWT Signer and Auth Policy are configured for Entra ID. The auth policy allows primary.extJwt with this signer.

  • The Ziti identity exists, has externalId matching preferred_username, is linked to the azureoidc auth policy, and has roleAttributes: ["default", "users"].

What I've Tried:

  • Ensured TLS trust. Router enrollment is fine.

  • Username/password login with ziti edge login (CLI v1.5.4) works correctly with API sessions.

  • Controller DEBUG logs are enabled via -v.

My Core Question:
Why does the Ziti controller after successfully issuing an API session token from an OIDC authentication, then appear to ignore that token for the ZDE client's subsequent API calls and incorrectly fall back to ext-jwt authentication (which then fails with "0 candidate JWTs")? This prevents the OIDC enrollment from completing.

Has anyone encountered a similar situation or have insights into potential misconfigurations or known behaviors with controller v1.5.4 and OIDC API sessions?

Thanks for any assistance!

Welcome to the OpenZiti community @jamesfear

Are you running controller in HA mode by any chance?

Hi Ekoby,
Thanks for your reply and for creating such a great product!
To answer your question — no, it’s a standalone controller.

Hi @jamesfear, this is the first indication that there's "some kind of problem" imo. You need to have more than 0 candidate JWTs. This probably means your identity is mapped to an auth policy that doesn't allow for external jwt signers.

For starters, I would do a few things... First, use the default auth policy that came out of the gate. That policy allows for all authentication mechanisms. Then change your identity to use the default auth policy. Once you get that far, you should start seeing more helpful errors, like these:

May 13 14:40:38 ip-172-31-11-231 ziti[3088016]: {"authMethod":"ext-jwt","file":"github.com/openziti/ziti/controller/model/authenticator_mod_ext_jwt.go:398","func":"github.com/openziti/ziti/controller/model.(*AuthModuleExtJwt).process","level":"error","msg":"encountered 1 candidate JWTs and all failed to validate for primary authentication, see the following log messages","time":"2025-05-13T14:40:38.734Z"}
May 13 14:40:38 ip-172-31-11-231 ziti[3088016]: {"audience":"caa8ac83-c05f-491c-be60-942888f63459","authMethod":"ext-jwt","error":"audience validation failed: audience value is invalid, no audiences matched the expected audience","extJwtSignerId":"1VVFnbdJRoql6bF6RLq3KI","file":"github.com/openziti/ziti/controller/model/authenticator_mod_ext_jwt.go:86","func":"github.com/openziti/ziti/controller/model.(*candidateResult).LogResult","issuer":"https://login.microsoftonline.com/25445e86-2ae6-4434-b116-25c66c27168d/v2.0","level":"error","msg":"failed to validate candidate JWT at index 0","time":"2025-05-13T14:40:38.734Z"}

Here, the audience I configured for my ext jwt signer is "wrong" for some reason. Figuring out why it's wrong is a whole other ball of wax. When you're logging at DEBUG level, you'll see a log message that looks like this on the tunneler-side:

-- if ACCESS token --
[2025-05-13T14:47:25.339Z]   DEBUG ziti-sdk:oidc.c:927 oidc_client_set_tokens() using access_token={"aud":"822f7b18-7875-4aae-b9aa-96d0a0ea736d","iss":"https://login.microsoftonline.com/25445e86-2ae6-4434-b116-25c66c27168d/v2.0","iat":1747147345,"nbf":1747147345,"exp":1747151953,"aio":"AXQAi/8ZAAAA1lKa6iSsOMPRZDjohDuFjJhe/rlGTV6PlNMlrickIZD6jBhkNYLktES5gfCZb+k2af+SBA+qNBaP/b5luXtVR+8A4tj+xPkjFLW8jWCn+v391YFuG8ZxawJC01Yri4QiboJ6nqZwF6WIGyJlLql9kg==","azp":"caa8ac83-c05f-491c-be60-942888f63459","azpacr":"0","name":"clint.dovholuk","oid":"f956f220-6e60-47a2-81b0-f9fad280e838","preferred_username":"clint.dovholuk@work.email","rh":"1.AS0Ahl5EJeYqNESxFiXGbCcWjRh7L4J1eK5KuaqW0KDqc233AEgtAA.","scp":"auth","sid":"004e4ef9-9097-0e75-462b-64fc1e310a50","sub":"x6MQVVQlV1lNzWqazNhLGHFDXX5gsAC_LzDcxa3G47s","tid":"25445e86-2ae6-4434-b116-25c66c27168d","uti":"ttxEIKuvdEeebeluGp8RAA","ver":"2.0"}

-- if ID token --
[2025-05-13T14:47:45.192Z]   DEBUG ziti-sdk:oidc.c:927 oidc_client_set_tokens() using id_token={"aud":"caa8ac83-c05f-491c-be60-942888f63459","iss":"https://login.microsoftonline.com/25445e86-2ae6-4434-b116-25c66c27168d/v2.0","iat":1747147365,"nbf":1747147365,"exp":1747151265,"aio":"AXQAi/8ZAAAAxdOVIvJPA87nxhWQ6H4XYDBu3imjgdBmJyHXcEB+0whHNsa/S3OSMLZRBWXk4D9VjvwNAt5R2C7IZcB8dOILtoQc3VAfajFCyR5mGFpkk6P6+oyxCLhTKn7zsBaUfi2LYdrzf9Ety6F0mO4eprLhQw==","name":"clint.dovholuk","oid":"f956f220-6e60-47a2-81b0-f9fad280e838","preferred_username":"clint.dovholuk@work.email","rh":"1.AS0Ahl5EJeYqNESxFiXGbCcWjYOsqMpfwBxJvmCUKIj2NFn3AEgtAA.","sid":"004e4ef9-9097-0e75-462b-64fc1e310a50","sub":"iMIHWJEtF2GYRD7KfQKoB_7dbZ8iJBJZU74iDgZH6Zw","tid":"25445e86-2ae6-4434-b116-25c66c27168d","uti":"Q8YSMV3qAEy7XxRjR20pAA","ver":"2.0"}

Look at the token to see what/why/where you configured the ext-jwt-signer wrongly. For example, I had used email, but no email returned even though I added the email and profile scopes to my signer

May 13 14:50:45 ip-172-31-11-231 ziti[3088016]: {"authMethod":"ext-jwt","file":"github.com/openziti/ziti/controller/model/authenticator_mod_ext_jwt.go:398","func":"github.com/openziti/ziti/controller/model.(*AuthModuleExtJwt).process","level":"error","msg":"encountered 1 candidate JWTs and all failed to validate for primary authentication, see the following log messages","time":"2025-05-13T14:50:45.527Z"}
May 13 14:50:45 ip-172-31-11-231 ziti[3088016]: {"audience":"caa8ac83-c05f-491c-be60-942888f63459","authMethod":"ext-jwt","error":"claims property [email] was not found in the claims","extJwtSignerId":"1VVFnbdJRoql6bF6RLq3KI","file":"github.com/openziti/ziti/controller/model/authenticator_mod_ext_jwt.go:86","func":"github.com/openziti/ziti/controller/model.(*candidateResult).LogResult","issuer":"https://login.microsoftonline.com/25445e86-2ae6-4434-b116-25c66c27168d/v2.0","level":"error","msg":"failed to validate candidate JWT at index 0","time":"2025-05-13T14:50:45.527Z"}

so i changed over to preferred_username. After that, I can login with Entra.

Let's get the 0 candidate JWTs sorted out first before moving on? Maybe this gives you all you need though...

This is complicated, basically you can be in a 'partially authenticated' state where you have a secure connection, but some other factor prevents you from being fully authenticated. In this case, it's because you didn't successfully complete the external jwt signer auth with the controller, even though you succeeded with the IdP. So your configuration is the reason, it's not quite right for some reason.

Hope that helps

One more thing, if you're using the Ziti Desktop Edge for Windows and you modify your ext-jwt-signer (audience, scopes, whatever). make sure you toggle the identity off/on to make sure your local client has the latest settings from the controller

toggle

Hi @TheLumberjack,

Thank you so much for your time and the suggestion! Moving my OIDC identity to use the Default Auth Policy (and adding my external JWT signer to it) indeed resolved the issue , and ZDE OIDC enrollment is now working successfully. I really appreciate your help in getting me unstuck after several days of troubleshooting.

I'm now trying to better understand why this change made the difference, so I can learn more about Ziti's internal workings and avoid similar pitfalls in the future.

My Understanding of What Happened (and where I'm still a bit unclear):

  1. Initial State (Not Working):
  • My OIDC identity used a custom Auth Policy.

  • This azureoidc policy was configured with primary.extJwt.allowed: true and my external jwt signer.

  • The /authenticate call with the Entra ID JWT would succeed, and the controller would issue a Ziti API session token.

  • However, all subsequent API calls from ZDE using that Ziti API session token would fail. The controller logs showed it attempting authMethod=ext-jwt again and erroring with encountered 0 candidate JWTs.

  1. Fixed State (Working):
  • My OIDC identity now uses the Default Auth Policy.

  • I updated the Default Auth Policy to also allow primary.extJwt: true with my external jwt signer.

  • Now, after the /authenticate call and API session issuance, ZDE's subsequent API calls (like /current-identity, /current-api-session/certificates) succeed, and enrollment completes.

My Question for Deeper Understanding:

You mentioned:

"This is complicated, basically you can be in a 'partially authenticated' state where you have a secure connection, but some other factor prevents you from being fully authenticated. In this case, it's because you didn't successfully complete the external jwt signer auth with the controller, even though you succeeded with the IdP. So your configuration is the reason, it's not quite right for some reason."

And also that the Default policy "allows for all authentication mechanisms."

Could you perhaps elaborate a little more on the "interplay" or the "other factor" that was at play here?

  • Specifically, when an API session token (e.g., zt-session) is presented for subsequent requests, how does the controller use the API session's originating Auth Policy to re-authorize or validate those requests?

  • What is it about the Default Auth Policy (perhaps its typical inclusion of primary.cert.allowed: true or other primary methods) that allows an API session generated via extJwt (under that policy) to then be sufficient for subsequent non-JWT authenticated calls, whereas my custom azureoidc policy (which only had extJwt allowed as primary) did not?

  • Is it that the API session token effectively "inherits" the breadth of allowed primary methods from its originating Auth Policy for ongoing validation, and my custom policy was too narrow?

I'm trying to build a mental model of how the API session's validity and capabilities are continuously assessed against the rules of the Auth Policy that birthed it. Any further clarification on this would be incredibly helpful for my understanding of Ziti's authentication and authorization flow.

Thanks again for your invaluable assistance!

It's hard to know 'exactly' what went wrong, where, and why to be honest. I added a new entraAuthPolicy just now, and assigned this policy to my identity and it will still authenticate successfully:

I'm not sure what might have gone wrong. You could try to put the policy back (recreate it) similar to what I did for a test?

As for the deeper understanding, I'm not the pro when it comes to API sessions. I will see if I can get someone else to post back after looking at what you wrote. I might be thinking about something else and have confused the conversation...

I'm going to assume that your custom policy has secondary authentication enabled, which causes additional requirements to be met. In this case, I suspect authPolicySecondary.requireExtJwtSigner has a value assigned to it. You can verify that by looking for that value on your custom policy.

If true, the set of events that is occurring is:

  1. Authentication via ext-jwt - a custom token swap, OIDC JWT -> API Session token
  2. SDK submits requests to the controller with/ only the API Session token
  3. Controller verifies API Session token, notes the AuthPolicy requires an OIDC JWT as well (authPolicySecondary.requireExtJwtSigner is set)
  4. Controller looks for a bearer OIDC JWT in the Authorization header and doesn't find any, and denies access

As for "why," OpenZiti has two authentication factors, "Primary" and "Secondary." They are outlined here: Authentication | OpenZiti.

The short version is that primary authentication establishes "what identity you are," and secondary authentication verifies "additional requirements that must remain true." Secondary factors allow administrators to apply multiple additional factors (MFA). In some cases, the client can rectify these without administrator intervention.

Successfully passing a primary authentication factor issues you an API Session that is used to create a security context that notes what session it is, who owns it, and other details about it.

If there are no secondary factors, the API Session is fully authenticated.

If there are secondary factors, the API Session may or may not be treated as fully authenticated, depending on the factor required. For example, if the administrators require TOTP MFA to be enabled on all accounts, they can attempt to pass TOTP MFA or enroll in it for the first time. Another example is the one I mentioned above, requiring that the API Session be scoped with an OIDC token. This transforms external JWT signers into an ongoing check instead of a point-in-time token exchange, meaning that the client SDK must provide a valid JWT token on every request in addition to the API Session token.

Thank you so much, @TheLumberjack and @andrew.martinez! That was exactly the issue. It looks like I had added a secondary JWT signer at some point, and after days of troubleshooting, I completely overlooked it. Really appreciate your help! Thank you again for all your support and for the great product!