Gotcha, we are replying past eachother. So if my posts seem out of sync, it s because I’m not seeing your replies as I type mine.
no worries.
Yes, I just checked and externalId is indeed present on the identity I was trying to debug. I might have a mix of identities though ( few through third part CA and few through one time JWT from controller ). Would that be a problem?
externalIds are unique and having some through 3rd Party CAs or not should be fine.
For the client that is not working does the matching identity have an externalId that matches the common name in the certificate being used?
yes. It does. Here is the output for reference -
{
"data": [
{
"_links": {
"auth-policies": {
"href": "./auth-policies/default"
},
"authenticators": {
"href": "./identities/K9rgRr-qG/authenticators"
},
"edge-router-policies": {
"href": "./identities/K9rgRr-qG/edge-routers"
},
"enrollments": {
"href": "./identities/K9rgRr-qG/enrollments"
},
"failed-service-requests": {
"href": "./identities/K9rgRr-qG/failed-service-requests"
},
"posture-data": {
"href": "./identities/K9rgRr-qG/posture-data"
},
"self": {
"href": "./identities/K9rgRr-qG"
},
"service-policies": {
"href": "./identities/K9rgRr-qG/service-policies"
}
},
"createdAt": "2022-10-20T19:01:11.311Z",
"id": "K9rgRr-qG",
"tags": {},
"updatedAt": "2022-10-21T06:09:02.973Z",
"appData": {},
"authPolicyId": "default",
"authenticators": {},
"defaultHostingCost": 0,
"defaultHostingPrecedence": "default",
"disabled": false,
"enrollment": {},
"envInfo": {
"arch": "x86_64",
"os": "Linux",
"osRelease": "4.18.0-372.26.1.el8_6.x86_64",
"osVersion": "#1 SMP Sat Aug 27 02:44:20 EDT 2022"
},
"externalId": "myidentity.server",
"hasApiSession": true,
"hasEdgeRouterConnection": false,
"isAdmin": false,
"isDefaultAdmin": false,
"isMfaEnabled": false,
"name": "myidentity.server",
"roleAttributes": [
"all-myca"
],
"sdkInfo": {
"appId": "ziti-edge-tunnel",
"appVersion": "v0.20.6-local",
"branch": "HEAD",
"revision": "040c4dd",
"type": "ziti-sdk-c",
"version": "0.30.2"
},
"serviceHostingCosts": {},
"serviceHostingPrecedences": {},
"type": {
"_links": {
"self": {
"href": "./identity-types/Device"
}
},
"entity": "identity-types",
"id": "Device",
"name": "Device"
},
"typeId": "Device"
}
],
"meta": {
"filterableFields": [
"updatedAt",
"tags",
"isSystem",
"roleAttributes",
"name",
"type",
"isDefaultAdmin",
"createdAt",
"externalId",
"authPolicyId",
"isAdmin",
"id"
],
"pagination": {
"limit": 10,
"offset": 0,
"totalCount": 1
}
}
}
This is the exact problem I am facing. The identity cant connect to edge router(s).
So we can authenticate just fine, API Session acquired, but we cannot connect to the Edge Router because it is saying that it doesn’t know about the client certificate.
Can you show the output of ziti edge list edge-routers -j (removing cert pems here is fine)
{
"data": [
{
"_links": {
"edge-router-policies": {
"href": "./edge-routers/qZkSqNm0HQ/edge-router-policies"
},
"self": {
"href": "./edge-routers/qZkSqNm0HQ"
}
},
"createdAt": "2022-10-12T21:08:16.856Z",
"id": "qZkSqNm0HQ",
"tags": {},
"updatedAt": "2022-10-12T21:08:25.440Z",
"appData": {},
"cost": 0,
"hostname": "er-usw2-1.myrouter",
"isOnline": true,
"name": "er-usw2-1",
"noTraversal": false,
"supportedProtocols": {
"tls": "tls://er-usw2-1.myrouter:443"
},
"syncStatus": "SYNC_DONE",
"certPem": "<CERT_PEM>",
"fingerprint": "6610ced54292dc122dbc2cd66fbdb91cd43f5c2a",
"isTunnelerEnabled": true,
"isVerified": true,
"roleAttributes": [
"all-comp"
],
"unverifiedCertPem": null,
"unverifiedFingerprint": null,
"versionInfo": {
"arch": "amd64",
"buildDate": "2022-10-13T15:31:04Z",
"os": "linux",
"revision": "72978b5aa932",
"version": "v0.26.10"
}
}
],
"meta": {
"filterableFields": [
"fingerprint",
"roleAttributes",
"isVerified",
"isTunnelerEnabled",
"id",
"isSystem",
"name",
"tags",
"createdAt",
"updatedAt"
],
"pagination": {
"limit": 10,
"offset": 0,
"totalCount": 1
}
}
}
And in case it got lost in earlier replies, I have used the third party CA with rotating certs daily before and didnt have any issue. Just that, the CA was created without the externalId bits. But at that time, if I have the same identity on 2 hosts, they were getting registered as 2 separate identities in Ziti ( e.g myidentity and myidentity001 ).
Now there is only one identity as expected, but edge router doesnt like it for some reason.
Out of curiosity have you restarted the Edge Router at all? or the client? And if so in what order(s)?
yes, couple of times
restarted client didnt help, restarted router didnt help, restarted client again but still no dice.
Ok. I had a theory about a bug, but I haven’t proven it exists yet or if it is the cause of your problem. With 3rd Party CAs using claims properties, we don’t have a record of the current certificate as it is managed outside of Ziti. The first time we see the current certificate is during authentication, but we do not persist that information anywhere. It is synchronized out to the Edge Routers at authentication time.
If the Edge Router is restarted after the client authenticates, then the Edge Router would have lost the information about the certificate, and the Controller wouldn’t have a record to send back when the Edge Router reconnects.
This should be resolvable by simply restarting the client and ensuring authentication occurs. However, your last reply about your restarts doesn’t match up as you tried restarting the client last.
Additionally: I had you list the Edge Routers so I could see if there was a sync on reconnecting problem - there isn’t according to the Controller.
I am going to see if I can reproduce this issue locally. I don’t know what to ask at the moment. If you read this and have any insights feel free to respond.
.
1 Like
I had a similar suspicion, because I have another client which can connect to the same edge router with the similar setup ( ca provided cert ) but this client identity has externalId: null ( I wanted to confirm that before posting here but i can see that client can connect to the router just fine right now )
I believe this client was created with previous setup when CA didnt have the externalId bits.
@andrew.martinez Were you able to replicate this locally? If not, is there any additional info I can provide from my side?
I am actively debugging it right now with a test case and …it seems to be working for me.
I have a controller, an edge router, service, and an identity hosting the service. I added a 3rd Party CA w/ COMMON_NAME location, no parsing etc. Authenticated w/ a certificate from my 3rd party CA, and I see the SDK connecting to the Edge Router.
I haven’t started to stare and compare my setup to the information I have from you.
I spent most of yesterday looking at this. I codified it into an integration test. I cannot get it to fail with the information I have.
If you can come up with a set of reproduction steps that would greatly help.
Apologies for the delay in response. My use case was to get 2 different certificates from 2 different workloads with a same CN to represent a single identity. Could you please do that and see if that helps to replicate the issue?
I’ll take a look. I did not test two workloads using the same identity…that is interesting. I’ll probably have a chance to look at this tomorrow morning.
1 Like
I found a couple of issues. Working through them now.
1 Like
Release 0.26.11 just went out. There seems to be an issue w/ the change log generator and the way I tagged the issues as fixed, so there isn’t text in the CHANGELOG.md that highlights them, but looking at the version change shows the commits.
Let me know if it helps with your issue.
1 Like
Great. I will upgrade to that version hopefully today / by tomorrow and report back.