# Using a single port (with ALPN)

**URL:** <https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662>\
**Category:** Ziti Overlay\
**Created:** [September 24, 2023, 10:02pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662 "2023-09-24T22:02:32Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 24, 2023, 10:02pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/1 "2023-09-24T22:02:33Z")

</div>

Hi,

Release [v0.30.0](https://github.com/openziti/ziti/releases/tag/v0.30.0) added support for running routers and controllers on a single port.

How does that work exactly? Can anyone share some configuration instructions?

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [September 25, 2023, 2:02pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/2 "2023-09-25T14:02:54Z")

</div>

I don't think it's fully documented yet but in my own testing, you just change the port for the controller's control plane and web api to the same value and it should "just work". 🙂

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 25, 2023, 2:23pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/3 "2023-09-25T14:23:49Z")

</div>

Thanks!

I'm playing with exposing all the ziti services + zrok + browzer from a single 443 port with HAProxy. The plan is to use TCP passthrough for the controller and edge router, and terminate TLS in front of zac, zrok, and browzer.

I would like to use ALPN for routing the controller and edge router connections (in addition to SNI). The current ALPN protocol IDs are:

| id | purpose |
| --- | --- |
| ziti-ctrl | Control plane connections |
| ziti-link | Fabric link connections |
| ziti-edge | Client SDK connection to Edge Routers |

Any idea how these protocols map to the following default ports? I prefilled some values based on my guesses. Please let me know if this looks ok.

| connection | port | protocol |
| --- | --- | --- |
| Controller control plane | 6262 | ziti-ctrl |
| Controller edge | 1280 | ziti-edge |
| Router listener | 10080 | ziti-link |
| Router edge | 3022 | ziti-edge |
| Router wss | 3023 | h2, http/1.1 (fixed thanks to @ekoby) |

Also, can all the router ports also be collapsed into a single one?

Thanks!

---

<div class="post-metadata">

**Author:** ![ekoby](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/ekoby/32/14_2.png) [@ekoby](https://openziti.discourse.group/u/ekoby)\
**Post date:** [September 25, 2023, 2:28pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/4 "2023-09-25T14:28:46Z")

</div>

these look correct except for the last one:  
Router WSS (port 3023) is using h2, http/1.1 (standard HTTP alpn ids) because it is using standard websocket listener

and yes, router configuration can also use a single port

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 25, 2023, 2:36pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/5 "2023-09-25T14:36:36Z")

</div>

> [@ekoby](#):
>
> and yes, router configuration can also use a single port

Does that also include the WSS port, or only the listener+edge ports? 🙂

---

<div class="post-metadata">

**Author:** ![ekoby](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/ekoby/32/14_2.png) [@ekoby](https://openziti.discourse.group/u/ekoby)\
**Post date:** [September 25, 2023, 3:11pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/6 "2023-09-25T15:11:31Z")

</div>

it should work with all(wss+link+edge) on the same port

and let us know if you see something different

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 25, 2023, 3:22pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/7 "2023-09-25T15:22:08Z")

</div>

Seemed to work like a charm -- thanks!

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 25, 2023, 9:20pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/8 "2023-09-25T21:20:43Z")

</div>

I feel like I'm almost there, but there are still issues!  
I have HAProxy using SNI to passthrough TCP connections directly to the controller and router containers, while TLS terminating zac/browzer/zrok.

ZAC, BrowZer seems to be working, but zrok is giving me trouble.

Zrok is not liking me anymore. I'm able to enable and environment and create a public share, but when I visit the share I get a "404" page from the zrok frontend:

```auto
ziti-zrok-frontend-1 | [2632.500] WARNING sdk-golang/ziti.(*ContextImpl).GetService: failed to get service: no apiSession, authentication attempt failed: response status code does not match any response statuses defined for this endpoint in the swagger spec (status 404): {}
ziti-zrok-frontend-1 | [2632.500] WARNING sdk-golang/ziti.(*ContextImpl).GetService: failed to get service: no apiSession, authentication attempt failed: response status code does not match any response statuses defined for this endpoint in the swagger spec (status 404): {}
ziti-zrok-frontend-1 | [2632.501] ERROR zrok/endpoints.GetRefreshedService: error refreshing services: failed to refresh services: no apiSession, authentication attempt failed: response status code does not match any response statuses defined for this endpoint in the swagger spec (status 404): {}
ziti-zrok-frontend-1 | [2632.501] WARNING zrok/endpoints/publicProxy.NewHTTP.authHandler.func2: 172.18.0.3:34714 -> service 'yzsab93y4g7r' not found
ziti-zrok-frontend-1 | [2632.501] ERROR zrok/endpoints.GetRefreshedService: error refreshing services: failed to refresh services: no apiSession, authentication attempt failed: response status code does not match any response statuses defined for this endpoint in the swagger spec (status 404): {}
ziti-zrok-frontend-1 | [2632.501] WARNING zrok/endpoints/publicProxy.NewHTTP.authHandler.func2: 172.18.0.3:34714 -> service 'yzsab93y4g7r' not found
haproxy | 77.125.98.42:51447 [25/Sep/2023:21:16:10.168] FE_https~ BE_zrok_fe/s1 0/0/1/10/11 404 31854 - - ---- 8/1/0/0/0 0/0 "GET https://yzsab93y4g7r.zrok.domain.com/ HTTP/2.0"

```

Also, attempting to enroll an identity on my phone now gives me a "CONTROLLER\_UNAVAILABLE" error message, and the logs show:

```auto
ziti-ziti-controller-1 | [76768.826] ERROR transport/v2/tls.(*sharedListener).processConn [tls:0.0.0.0:8440]: {error=[remote error: tls: access denied]} handshake failed
haproxy | 77.125.98.42:50980 [25/Sep/2023:21:19:28.717] FE_domain.com_tcp BE_ziti_ctrl/s1 2/0/108 4057 -- 8/8/4/4/0 0/0 req.ssl_sni:'ctrl.ziti.domain.com' req.ssl_alpn:'-'

```

Weird that I get the ALPN `-`.

Will continue debugging tomorrow... don't want to give up just yet! 🙂

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 26, 2023, 6:25am UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/9 "2023-09-26T06:25:43Z")

</div>

The zrok issue has been fixed! Apparently the `identity.json` that was initially created had the previous controller URL with the old port so after manually editing the file and setting the port to `443` it now works!

@TheLumberjack/@ekoby regarding the `CONTROLLER_UNAVAILABLE` issue when trying to enroll an identity on the iOS app -- is it possible that it's using an outdated edge SDK version which cannot handle the single port endpoint?

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [September 26, 2023, 12:07pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/10 "2023-09-26T12:07:12Z")

</div>

> [@mvelbaum](#):
>
> regarding the `CONTROLLER_UNAVAILABLE` issue when trying to enroll an identity on the iOS app -- is it possible that it's using an outdated edge SDK version which cannot handle the single port endpoint?

Yes, I am told we are in the process of releasing ALPN support for iOS/MacOS. It's "imminent" but just isn't released yet.

---

<div class="post-metadata">

**Author:** ![ekoby](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/ekoby/32/14_2.png) [@ekoby](https://openziti.discourse.group/u/ekoby)\
**Post date:** [September 26, 2023, 1:39pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/11 "2023-09-26T13:39:38Z")

</div>

SDK should handle single port without updates

Edit: make sure your JWT is generated with correct controller address. You can decode it on [JWT.io](http://JWT.io).

payload `iss`(issuer) field should have the address of your controller. Something like this:

```auto
{
  "em": "ott",
  "exp": 1682269711,
  "iss": "https://fd200fd3-a2d9-457f-bc0b-f9b8ee7d2898.production.netfoundry.io:443",
  "jti": "339dbd56-5219-4b38-94de-6dd41f546a50",
  "sub": "Nm-VK7g-3c"
}

```

---

<div class="post-metadata">

**Author:** ![smilindave26](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/smilindave26/32/185_2.png) [@smilindave26](https://openziti.discourse.group/u/smilindave26)\
**Post date:** [September 26, 2023, 2:15pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/12 "2023-09-26T14:15:23Z")

</div>

It's non-trivial to edit the identity JSON file on iOS. Assuming the issue is that the controller port changed, your best bet is to re-enroll your iOS identity

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 26, 2023, 5:02pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/13 "2023-09-26T17:02:34Z")

</div>

@smilindave26 this happens when I try to enroll a new identity, and it does show me the correct controller address when I do it.

Here are the application logs from the iOS app:

```auto
(2144)[2023-09-26T16:34:20.977Z] INFO ziti-sdk:ziti_enroll.c:94 ziti_enroll() Ziti C SDK version 0.31.4 @68c3a76(HEAD) starting enrollment at (2023-09-26T16:34:20.977)
(2144)[2023-09-26T16:34:20.977Z] DEBUG ziti-sdk:jwt.c:111 load_jwt() filename is: /private/var/mobile/Containers/Shared/AppGroup/3ED4CCB6-34B0-4291-B9EB-364C66A171BB/10lFKSce6k.jwt
(2144)[2023-09-26T16:34:20.977Z] DEBUG ziti-sdk:jwt.c:77 load_jwt_file() reading JWT from file: /private/var/mobile/Containers/Shared/AppGroup/3ED4CCB6-34B0-4291-B9EB-364C66A171BB/10lFKSce6k.jwt
(2144)[2023-09-26T16:34:20.977Z] DEBUG ziti-sdk:jwt.c:104 load_jwt_file() jwt file content is: 
*<edited out>*
(2144)[2023-09-26T16:34:20.977Z] DEBUG ziti-sdk:jwt.c:41 parse_jwt_content() ecfg->jwt_signing_input is: 
*<edited out>*
(2144)[2023-09-26T16:34:20.977Z] INFO ziti-sdk:ziti_ctrl.c:409 ziti_ctrl_init() ctrl[ctrl.ziti.mydomain.com] ziti controller client initialized
(2144)[2023-09-26T16:34:20.977Z] VERBOSE ziti-sdk:ziti_ctrl.c:134 start_request() ctrl[ctrl.ziti.mydomain.com] starting GET[/.well-known/est/cacerts]
(2144)[2023-09-26T16:34:21.188Z] INFO ziti-sdk:ziti_enroll.c:41 verify_controller_jwt() verifying JWT signature
(2144)[2023-09-26T16:34:21.188Z] ERROR ziti-sdk:ziti_enroll.c:66 verify_controller_jwt() failed to verify JWT signature
(2144)[2023-09-26T16:34:21.188Z] ERROR ziti-sdk:ziti_ctrl.c:155 ctrl_resp_cb() ctrl[ctrl.ziti.mydomain.com] request failed: -53(software caused connection abort)
(2144)[2023-09-26T16:34:21.188Z] DEBUG ziti-sdk:ziti_enroll.c:141 well_known_certs_cb() base64_encoded_pkcs7 is: (null)
(2144)[2023-09-26T16:34:21.188Z] DEBUG ziti-sdk:ziti_enroll.c:147 well_known_certs_cb() err->message is: software caused connection abort
(2144)[2023-09-26T16:34:21.188Z] ERROR ziti-sdk:ziti_enroll.c:219 well_known_certs_cb() /Users/runner/work/ziti-sdk-swift/ziti-sdk-swift/deps/ziti-tunnel-sdk-c/build-iphoneos-arm64/_deps/ziti-sdk-c-src/library/ziti_enroll.c:148 - ZITI_JWT_VERIFICATION_FAILED => -7 (JWT verification failed)
[2023-09-26T16:34:21:189Z] ERROR CZiti:ZitiEnroller.swift:215 on_enroll() CONTROLLER_UNAVAILABLE
[2023-09-26T16:34:21:190Z] ERROR CZiti:Ziti.swift:332 enroll() Optional(Error Domain=ZitiError Code=-7 "CONTROLLER_UNAVAILABLE" UserInfo={NSLocalizedDescription=CONTROLLER_UNAVAILABLE})
[2023-09-26T16:34:22:290Z] INFO Ziti Mobile Edge:Array+ZitiIdentity.swift:25 updateIdentity() QRScan1695746056.jwt:10lFKSce6k CHANGED

```

I grabbed the certificate bundle that the iOS client is asking for at `https://ctrl.ziti.mydomain.com/.well-known/est/cacerts` and it contains the correct certificate chains, which include my wildcard cert for `*.ziti.mydomain.com`.

How I did it:

```bash
$ curl https://ctrl.ziti.mydomain.com/.well-known/est/cacerts | base64 -d > certs.p7b
$ openssl pkcs7 -inform DER -in certs.p7b -text -print_certs

```

Any ideas?

---

<div class="post-metadata">

**Author:** ![ekoby](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/ekoby/32/14_2.png) [@ekoby](https://openziti.discourse.group/u/ekoby)\
**Post date:** [September 26, 2023, 5:30pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/14 "2023-09-26T17:30:14Z")

</div>

before SDK can fetch pkcs7 bundle server has to be verified. SDK is using controller cert to verify it was used to signed the JWT.

Can you get controller cert and JWT and validate them on [JWT.io](http://JWT.io)?

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 26, 2023, 6:17pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/15 "2023-09-26T18:17:36Z")

</div>

> [@ekoby](#):
>
> address of your controller.

I grabbed the first certificate using: `openssl s_client -showcerts -servername ctrl.ziti.mydomain.com -connect ctrl.ziti.mydomain.com:443`. I also grabbed the private key (which is set as the `server_key` under `alt_server_certs` in the controller identity).

I then pasted the JWT I was trying to enroll and it passed validation :/.

Payload looks like this:

```auto
{
  "iss": "https://ctrl.ziti.mydomain.com:443",
  "sub": "10lFKSce6k",
  "aud": [
    ""
  ],
  "exp": 1696278247,
  "nbf": 1695673447,
  "jti": "d565e59b-6434-4e26-9e6c-c1383e40a52d",
  "em": "ott"
}

```

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [September 26, 2023, 8:34pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/16 "2023-09-26T20:34:52Z")

</div>

### The **FIRST** issue...

I've been digging through this on my side trying to replicate your issue. I _think_ I have reproduced your problem on my side. I suspect you followed the browzer example guide, which works fine for browzer but then, I suspect you changed the 'address' of the controller so the jwts all use the "non-self-signed CA"... This gets complex, hopefully this will make sense.

When configuring the controller with `alt_server_certs`, they are used for exactly what it sounds like, as "server" certs. That's fine for accessing the API or connecting to a 'server' port.... however, it's not fine if you want to change the advertised address identities use/see (as it appears you may have shown in your jwt above)... **IF** you change the `address` in the controller's configuration, you **must** change the `identity` `server_cert` and `key` section of your edge api due to how OpenZiti's clients verify the authenticity of enrollment tokens (jwts)...

When an enrollment token is delivered to a client, the jwt is signed by the controller (server). The key used to sign the jwt is the one specified in the `identity` section (not the alt\_server\_cert section). When the client tries to enroll, it will read the `iss`uer field, get the server cert from the specified `iss`uer and verify the jwt was signed by that issuer... The problem is that if you configure the external pki in `alt_server_certs`, the key will not match the server certificate presented and the enrollment will fail. That's what I believe is the first problem you have. Please confirm the identity section of your setup.

### An Example:

As an example, i have stood up: [https://ctrl.clint.demo.openziti.org:8441](https://ctrl.clint.demo.openziti.org:8441) from the LetsEncrypt PKI. My identity section first looked like this:

```auto
      ca: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-root-ca/certs/ip-172-31-11-231-edge-controller-root-ca.cert"
      key: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-intermediate/keys/ec2-13-58-222-94.us-east-2.compute.amazonaws.com-server.key"
      server_cert: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-intermediate/certs/ec2-13-58-222-94.us-east-2.compute.amazonaws.com-server.chain.pem"
      cert: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-intermediate/certs/ec2-13-58-222-94.us-east-2.compute.amazonaws.com-client.cert"
      alt_server_certs:
      - server_cert: "/etc/letsencrypt/live/clint.demo.openziti.org/fullchain.pem"
        server_key: "/etc/letsencrypt/live/clint.demo.openziti.org/privkey.pem"

```

Looks fine, self-signed CA in the identity section, LetsEncrypt in the alt\_server\_certs... But, when I changed my `address` field to match **from** `ec2-13-58-222-94.us-east-2.compute.amazonaws.com:8441` **to** : `ctrl.clint.demo.openziti.org:8441`, I could no longer enroll a single identity anywhere.

I had to update my identity section to look like this for it to work properly, which I think is "unintuitive" at best:

```auto
    identity:
      ca: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-root-ca/certs/ip-172-31-11-231-edge-controller-root-ca.cert"
      key: "/etc/letsencrypt/live/clint.demo.openziti.org/privkey.pem"
      server_cert: "/etc/letsencrypt/live/clint.demo.openziti.org/fullchain.pem"
      cert: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-intermediate/certs/ec2-13-58-222-94.us-east-2.compute.amazonaws.com-client.cert"
      alt_server_certs:
      - server_cert: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-intermediate/certs/ec2-13-58-222-94.us-east-2.compute.amazonaws.com-server.chain.pem"
        server_key: "/home/ubuntu/.ziti/quickstart/ip-172-31-11-231/pki/ip-172-31-11-231-edge-controller-intermediate/keys/ec2-13-58-222-94.us-east-2.compute.amazonaws.com-server.key"

```

Notice how I have moved the server/key and alt server/key. Also note that the 'cert' in this situation is just not used at this time. That cert and key won't match. I don't know if this will be a problem in the future, but I'll raise it as an issue to the team....

### The Second Issue...

The second issue you'll run into is that the currently deployed Windows and MacOS clients are not able to use LetsEncrypt certificates at this time. This problem is fixed in the latest unreleased versions, due to be released soon.... (I don't know when) I don't know if you're using LetsEncrypt or some other CA, but that will be the next problem you face from a MacOS perspective.

We need to tighten up the doc and experience around alt certs for sure. This wasn't obvious nor easy for me to figure out either... We'll work on getting that done too.

I hope this helps.

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 26, 2023, 9:47pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/17 "2023-09-26T21:47:47Z")

</div>

First of all, I really appreciate your time with troubleshooting this issue!  
Second, you're right on both counts - I did rename the controller address after initial deployment and I did add `alt_server_certs` when I added Let's Encrypt certs.

Swapping the `alt_server_certs` with the ones in `identity` didn't seem to help though, but that is possibly because my proxy config - I will try to go back to using different ports and see if it makes any difference. There's also another `identity` section at the root of the YAML and a `signingCert` key under `edge`/`api`/`enrollment` and I'm not sure what they are used for exactly. I wonder what else could be silently broken because of this and how can I make sure everything is correctly configured cert-wise.

Note that trying to deploy via iOS gives this controller log msg

```auto
ziti-ziti-controller-1 | [77525.262] ERROR transport/v2/tls.(*sharedListener).processConn [tls:0.0.0.0:8440]: {error=[remote error: tls: access denied]} handshake failed

```

I also tried to enroll an identity on the Windows client, and it actually was able to proceed further by actually able to download the certs:

```auto
[2023-09-26T21:22:57.509Z] INFO ziti-sdk:ziti_enroll.c:92 ziti_enroll() Ziti C SDK version 0.33.2 @e06b76c(HEAD) starting enrollment at (2023-09-26T21:22:57.509)
[2023-09-26T21:22:57.509Z] VERBOSE ziti-sdk:jwt.c:124 load_jwt_content() jwt file content is: 
eyJhbGciOiJFUzI1NiIsImtpZCI6IjUwMDMyMGMwZmEyNTY0ZDcwNTkxMDE4NTFmZWY1ODY1MTllYmUwZWEiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2N0cmwueml0aS52ZWxiYXVtLmNvbTo0NDMiLCJzdWIiOiJXb1ZMYWJKVnYiLCJhdWQiOlsiIl0sImV4cCI6MTY5NjM2ODE2NCwibmJmIjoxNjk1NzYzMzY0LCJqdGkiOiI1YWY4MjcyNi1kNjUwLTQ4ZmItYmZmMi01ZTczZmVhMWNiMzciLCJlbSI6Im90dCJ9.xU4Etve9f3WZ2aEQKN0kGpFeJl3LE94oefy5wOdbZ7XbyZMC1uqvyYX2NGdls662bshl609cLn4OO5z52FM_6g
[2023-09-26T21:22:57.509Z] DEBUG ziti-sdk:jwt.c:36 parse_jwt_content() ecfg->jwt_signing_input is: 
eyJhbGciOiJFUzI1NiIsImtpZCI6IjUwMDMyMGMwZmEyNTY0ZDcwNTkxMDE4NTFmZWY1ODY1MTllYmUwZWEiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2N0cmwueml0aS52ZWxiYXVtLmNvbTo0NDMiLCJzdWIiOiJXb1ZMYWJKVnYiLCJhdWQiOlsiIl0sImV4cCI6MTY5NjM2ODE2NCwibmJmIjoxNjk1NzYzMzY0LCJqdGkiOiI1YWY4MjcyNi1kNjUwLTQ4ZmItYmZmMi01ZTczZmVhMWNiMzciLCJlbSI6Im90dCJ9
[2023-09-26T21:22:57.510Z] DEBUG ziti-sdk:ziti_ctrl.c:408 ziti_ctrl_init() ctrl[ctrl.ziti.mydomain.com] ziti controller client initialized
[2023-09-26T21:22:57.510Z] VERBOSE ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ctrl.ziti.mydomain.com] starting GET[/.well-known/est/cacerts]
[2023-09-26T21:22:57.510Z] TRACE ziti-edge-tunnel:tun.c:303 tun_read() starting read
[2023-09-26T21:22:57.663Z] DEBUG ziti-sdk:ziti_enroll.c:39 verify_controller_jwt() verifying JWT signature
[2023-09-26T21:22:57.663Z] DEBUG ziti-sdk:ziti_enroll.c:67 verify_controller_jwt() JWT verification succeeded!
[2023-09-26T21:22:57.739Z] VERBOSE ziti-sdk:ziti_ctrl.c:168 ctrl_resp_cb() ctrl[ctrl.ziti.mydomain.com] received headers GET[/.well-known/est/cacerts]
[2023-09-26T21:22:57.739Z] VERBOSE ziti-sdk:ziti_enroll.c:144 well_known_certs_cb() base64_encoded_pkcs7 is: *<snipped>*
[2023-09-26T21:22:57.741Z] DEBUG ziti-sdk:ziti_enroll.c:157 well_known_certs_cb() CA PEM len = 15764
[2023-09-26T21:22:57.741Z] TRACE ziti-sdk:ziti_enroll.c:158 well_known_certs_cb() CA PEM:
-----BEGIN CERTIFICATE-----
*<snipped>*
[2023-09-26T21:22:57.745Z] DEBUG ziti-sdk:ziti_ctrl.c:408 ziti_ctrl_init() ctrl[ctrl.ziti.mydomain.com] ziti controller client initialized
[2023-09-26T21:22:57.745Z] VERBOSE ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ctrl.ziti.mydomain.com] starting POST[/enroll?method=ott&token=5af82726-d650-48fb-bff2-5e73fea1cb37]
[2023-09-26T21:22:57.897Z] ERROR ziti-sdk:ziti_ctrl.c:154 ctrl_resp_cb() ctrl[ctrl.ziti.mydomain.com] request failed: -4079(software caused connection abort)
[2023-09-26T21:22:57.897Z] ERROR ziti-sdk:ziti_enroll.c:235 enroll_cb() failed to enroll with controller: https://ctrl.ziti.mydomain.com:443 CONTROLLER_UNAVAILABLE (software caused connection abort)
[2023-09-26T21:22:57.897Z] ERROR ziti-edge-tunnel:ziti-edge-tunnel.c:319 tunnel_enroll_cb() enrollment failed: CONTROLLER_UNAVAILABLE(-3)
[2023-09-26T21:22:57.897Z] INFO ziti-edge-tunnel:ziti-edge-tunnel.c:227 on_command_resp() resp[0,len=56] = {"Success":false,"Error":"enrollment failed","Code":500}

```

The controller reported an error which is a bit different than the one reported when enrolling on iOS:

```auto
ziti-ziti-controller-1 | [76977.945] ERROR transport/v2/tls.(*sharedListener).processConn [tls:0.0.0.0:8440]: {error=[local error: tls: bad record MAC]} handshake failed

```

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [September 26, 2023, 9:57pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/18 "2023-09-26T21:57:33Z")

</div>

> [@mvelbaum](#):
>
> There's also another `identity` section at the root of the YAML and a `signingCert` key under `edge`/`api`/`enrollment` and I'm not sure what they are used for exactly

This section is your "control plane" section. That's relevant to how routers talk to routers, routers to controllers etc. I wouldn't think that would matter. I think maybe this week's Ziti TV can be dedicated towards a 'working session' where I basically setup a network with a couple of routers, external certs, and make sure data passes from identity to identity properly.

> [@mvelbaum](#):
>
> Note that trying to deploy via iOS gives this controller log msg

What does it mean to 'deploy via iOS'? You're just trying to enroll an identity, right? I'm very sure the MacOS (I dont' have an iphone but it's the same code, generally) and the windows clients won't work with LetsEncrypt certs until this next update comes out for them. You _could_ try the latest build of "raw" ziti-edge-tunnel though: [Release v0.22.7 · openziti/ziti-tunnel-sdk-c · GitHub](https://github.com/openziti/ziti-tunnel-sdk-c/releases/tag/v0.22.7) that should work on MacOS/Windows but it's not the nice, native UI app, it's just a command line tool.

---

<div class="post-metadata">

**Author:** ![mvelbaum](https://avatars.discourse-cdn.com/v4/letter/m/e56c9b/32.png) [@mvelbaum](https://openziti.discourse.group/u/mvelbaum)\
**Post date:** [September 26, 2023, 10:00pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/19 "2023-09-26T22:00:45Z")

</div>

> This section is your "control plane" section. That's relevant to how routers talk to routers, routers to controllers etc. I wouldn't think that would matter.

I see, so I guess for that section the `alt_server_certs` should be good enough.

> I think maybe this week's Ziti TV can be dedicated towards a 'working session' where I basically setup a network with a couple of routers, external certs, and make sure data passes from identity to identity properly.

That would be cool indeed!

> What does it mean to 'deploy via iOS'? You're just trying to enroll an identity, right?

Oh yeah, sorry, I meant to enroll (it's 1am here so please forgive the typos :)).  
I forgot to mention that I used a pre-release of the Windows client: [Release 2.1.17 · openziti/desktop-edge-win · GitHub](https://github.com/openziti/desktop-edge-win/releases/tag/2.1.17) -- not sure if this version is supposed to have the Let's Encrypt cert support or not.

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [September 26, 2023, 10:05pm UTC](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662/20 "2023-09-26T22:05:58Z")

</div>

I can send you a DM with a 2.1.18 build that _will_ work (it's the one I'm using locally, to duration test) just know that if we release a 'real' build, you'll want to either uninstall it and install the 'released' version to pick up any other updates that might be in it... I'll shoot you a link to it you can try

[Next page](https://openziti.discourse.group/t/using-a-single-port-with-alpn/1662.md?page=2)
