How to generate .jwt and .json in Ziti for Python sample - ziti-echo-server

Nick · May 3, 2023, 10:21pm

Hi:

While we were trying the sample Python echo server program from “ziti-sdk-py/sample/ziti-echo-server at main · openziti/ziti-sdk-py · GitHub”, there are some errors.

We re-used the .jwt and .json file from C# sample:

:~/Desktop/ziti_py$ python -m openziti enroll --jwt=weather.demo.jwt --identity=weather.demo.json 
(6895)[        0.018]   ERROR ziti-sdk:ziti_enroll.c:64 verify_controller_jwt() failed to verify JWT signature
(6895)[        0.018]   ERROR ziti-sdk:ziti_ctrl.c:154 ctrl_resp_cb() ctrl[nick] request failed: -103(software caused connection abort)
(6895)[        0.018]   ERROR ziti-sdk:ziti_enroll.c:222 well_known_certs_cb() /home/runner/work/ziti-sdk-c/ziti-sdk-c/library/ziti_enroll.c:141 - ZITI_JWT_VERIFICATION_FAILED => -7 (JWT verification failed)
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/openziti/__main__.py", line 62, in <module>
    args.func()
  File "/usr/local/lib/python3.10/dist-packages/openziti/__main__.py", line 23, in do_enroll
    id_json = zitilib.enroll(jwt, key=args.key, cert=args.cert)
  File "/usr/local/lib/python3.10/dist-packages/openziti/zitilib.py", line 309, in enroll
    raise RuntimeError(errorstr(retcode))
RuntimeError: JWT verification failed

:~/Desktop/ziti_py$ python Server.py weather.demo.json weather-svc
weather.demo.json
weather-svc
(6868)[        0.000]   ERROR ziti-sdk:ziti.c:216 ziti_init_opts() /home/runner/work/ziti-sdk-c/ziti-sdk-c/library/ziti.c:182 - load_config(options->config, &cfg) => -13 (Configuration is invalid)
Traceback (most recent call last):
  File "/home/nick/Desktop/ziti_py/Server.py", line 27, in <module>
    run(sys.argv[1], sys.argv[2])
  File "/home/nick/Desktop/ziti_py/Server.py", line 6, in run
    ztx = openziti.load(ziti_id)
  File "/usr/local/lib/python3.10/dist-packages/openziti/context.py", line 87, in load_identity
    return ZitiContext.from_path(path)
  File "/usr/local/lib/python3.10/dist-packages/openziti/context.py", line 77, in from_path
    return cls(zitilib.load(path))
  File "/usr/local/lib/python3.10/dist-packages/openziti/context.py", line 27, in __init__
    raise TypeError("ctx is not a valid python void pointer type")
TypeError: ctx is not a valid python void pointer type

What is the correct approach to create .jwt and .json under Python ?

And, how to run the Python echo server ?

qrkourier · May 4, 2023, 12:21am

Hey there, @Nick . Welcome back. The first error points toward an invalid token. You also mentioned re-using the token from another example. Did you happen to use that token already? Most enrollment tokens have a short-term expiration and a re single-use and are traded for a longer-term JSON identity file that’s called a “context” by the SDK when it’s loaded.

Possibilities include: the token is malformed, expired, or spent.

qrkourier · May 4, 2023, 12:49am

The final command to run the server after it's all set up is like this.

ziti-sdk-py/sample/ziti-echo-server main
 ❯ python ziti-echo-server.py </path/to/id.json> <name-of-service>

The setup steps are:

Use the token to enroll and get an identity/context JSON file. You already have the Python enroll command for this step, but you need a valid token. I used ZEDS for to make sure this will work for you. I'll assume you're using ZEDS too.

Create a Ziti service with an intercept config like this.

{
  "addresses": [
    "python.echo.ziti"
  ],
  "protocols": [
    "tcp"
  ],
  "portRanges": [
    {
      "low": 80,
      "high": 80
    }
  ]
}

Create a Ziti client identity for yourself. You'll use it to send bytes to the echo server. Make sure you add it to your tunneler, e.g., Ziti Desktop Edge for Windows.

In ZEDS, copy the name of your Ziti service to your clipboard so you can use it to run the echo server. It looks like this.

<zeds service name> <zeds app name> <base64 encoding of your Google ID>

For example:

python ./ziti-echo-server.py /tmp/ziti-id.json "echo1-svc echo-app Z29vZ2xlLW9hdXRoMnoxMTYwMTQzMjA2MDgwNTM4ODkyOZQ="

Finally, follow the sample readme to install the required PyPi packages and run the echo server.
With the echo server running in another terminal, send some bytes to the intercept address you defined in the service config.
```
nc python.echo.ziti 80 <<< "Hello, Ziggy"
```

A good result includes a log message from the echo server like "processing incoming client," and you should get an echo of the bytes that you sent with netcat/ncat.

Nick · May 16, 2023, 5:06pm

Thank you very much for the help.
However, I still have some further questions.

Currently, I only have one Linux machine, and I used the “ZEDS” to generate .jwt file.

I still have some problem if I want to re-produce the server-client application in this blog: OpenZiti Python SDK: Introduction .

My question is:
After using ZEDS and having the .jwt file, how to generate the .json file ?

I did not see where i can generate .json file in ZEDS. In the blog OpenZiti Python SDK: Introduction , we need to do “Ziti identity enrollment” with command line :

$ python -m openziti enroll --jwt id.jwt --identity id.json

Any idea to do the example in the blog ?

Thank you

TheLumberjack · May 16, 2023, 9:08pm

Hi Nick,

Yes, you need to "enroll" the jwt using the "enroll" command as you showed. When you do, you should get an id.json.

Did you try that, and it didn't succeed?

smilindave26 · May 16, 2023, 9:36pm

I did not see where i can generate .json file in ZEDS

In ZEDS, on the main screen to manage an app, there is a section called "Manage Identities". If you click "Add a new one", you can enter the name of the new identity, and after you hit Return you'll see a new button with a key on it under the "Token" column,

Clicking on that Key button will download the JWT file, which you can use to create the JSON file via the openziti enroll command above.

Edit: I answered for JWT file, not the JSON file. Updated to reflect that...

Nick · May 16, 2023, 10:17pm

Thank you all.

There are still some problems.

I did following:

Used ZEDS to generate .jwt file.
Then, I used the following command line to generate .json file.

$ python -m openziti enroll --jwt id.jwt --identity id.json

Now, locally, I have .jwt file (from ZEDS) and .json file (generated by above command line), and the service name in ZEDS.

Now, there are some error:

When I run the sample code in ziti-sdk-py/sample/ziti-echo-server at main · openziti/ziti-sdk-py · GitHub :

nick@nick:~/Desktop/ziti_py$ python Server.py may16id1.json may16server1 
Traceback (most recent call last):
  File "/home/nick/Desktop/ziti_py/Server.py", line 39, in <module>
    run(sys.argv[1], sys.argv[2])
  File "/home/nick/Desktop/ziti_py/Server.py", line 21, in run
    server = ztx.bind(service)
  File "/usr/local/lib/python3.10/dist-packages/openziti/context.py", line 62, in bind
    zitilib.bind(sock.fileno(), self._ctx, service, terminator)
  File "/usr/local/lib/python3.10/dist-packages/openziti/zitilib.py", line 265, in bind
    check_error(_ziti_bind(fd, ztx, srv, terminator_b))
  File "/usr/local/lib/python3.10/dist-packages/openziti/zitilib.py", line 219, in check_error
    raise Exception(err, msg)
Exception: (-17, 'Service not available')

nick@nick:~/Desktop/ziti_py$ python3 Server.py may16id1.json may16server1 
Traceback (most recent call last):
  File "/home/nick/Desktop/ziti_py/Server.py", line 39, in <module>
    run(sys.argv[1], sys.argv[2])
  File "/home/nick/Desktop/ziti_py/Server.py", line 21, in run
    server = ztx.bind(service)
  File "/usr/local/lib/python3.10/dist-packages/openziti/context.py", line 62, in bind
    zitilib.bind(sock.fileno(), self._ctx, service, terminator)
  File "/usr/local/lib/python3.10/dist-packages/openziti/zitilib.py", line 265, in bind
    check_error(_ziti_bind(fd, ztx, srv, terminator_b))
  File "/usr/local/lib/python3.10/dist-packages/openziti/zitilib.py", line 219, in check_error
    raise Exception(err, msg)
Exception: (-17, 'Service not available')

If I run the sample code in OpenZiti Python SDK: Introduction , there are error :

nick@nick:~/Desktop/ziti_py$ python3 Server.py 
Traceback (most recent call last):
  File "/home/nick/Desktop/ziti_py/Server.py", line 44, in <module>
    ztx: 'may16id1.json',
NameError: name 'ztx' is not defined

ekoby · May 17, 2023, 4:39pm

If you’re using ZEDS the true service name will be the extended service name that you can get from ZEDS on the service edit page. This is generated to disambiguate service name between multiple users. This service name/ID you can use in your app.

Nick · May 20, 2023, 4:28am

Thank you for your responds:

I have run the echo server successfully, by the command line:

python ziti-echo-server.py </path/to/id.json> <name-of-service>

However, I have trouble to test it with Netcat.
I am using a Windows10 machine

>ncat --version
Ncat: Version 7.93 ( https://nmap.org/ncat )

>ncat "python.echo.ziti" 80
Ncat: Could not resolve hostname "python.echo.ziti": No such host is known. . QUITTING.

>ncat "pyechoserv1 pysample Z29vZ2xlLW9hdXRoMnwxMTgzODUwMjgzNjQ4MjE4MDM4NzA=" 80
Ncat: Could not resolve hostname "pyechoserv1 pysample Z29vZ2xlLW9hdXRoMnwxMTgzODUwMjgzNjQ4MjE4MDM4NzA=": No such host is known. . QUITTING.

In the picture below, there are two identities, which are “pyechoclient” and “pyecho”. The “pyecho” is being used by the python program. And, the “pyechoclient” is being used for the Ziti Tunneler on current Windows machine.

After we login to the Tunneler with pyechoclient’s JWT file, we can even see the service “pyechoserv1 pysample Z29vZ2xlLW9h…” on port 80.

And the python echo service is listening the port 80:

Is there any idea about the error ?

>ncat --version
Ncat: Version 7.93 ( https://nmap.org/ncat )

>ncat "python.echo.ziti" 80
Ncat: Could not resolve hostname "python.echo.ziti": No such host is known. . QUITTING.

>ncat "pyechoserv1 pysample Z29vZ2xlLW9hdXRoMnwxMTgzODUwMjgzNjQ4MjE4MDM4NzA=" 80
Ncat: Could not resolve hostname "pyechoserv1 pysample Z29vZ2xlLW9hdXRoMnwxMTgzODUwMjgzNjQ4MjE4MDM4NzA=": No such host is known. . QUITTING.

Nick · May 22, 2023, 12:14am

Thank you all.

I figured it out.

Now, I can run the python echo server sample from ziti-sdk-py/sample/ziti-echo-server at main · openziti/ziti-sdk-py · GitHub.
I run the netcat command line on a MacOS machine, which loaded with Tunneler.

However, there are still two problems:

When I want to try the Tunneler on an Ubuntu machine. I still cannot use the Linux tunneler to connect a ZEDS service. I did the following:
1.1 In ZEDS, I created another identity, let’s just call it “echo-client”, in the same application that has Python-echo service.
1.2 Download the token(.JWT) file for echo-client.
1.3 In Linux terminal Run the script for your OS to install ziti-edge-tunnel in Linux | OpenZiti.
1.4 I try to enroll the local Linux machine by: sudo -u ziti tee /opt/openziti/etc/identities/ziti-id.jwt >/dev/null
1.5 Then, I hit return at the end of the sudo line (after the >/dev/null), then hit paste, then Ctrl+d. I just paste the content from echo-client’s .JWT file. And, I double checked the content.
1.6 Then, systemctl enable --now ziti-edge-tunnel.service
1.7 Finally, I got the following output:

root@nick:~# systemctl enable --now ziti-edge-tunnel.service
Job for ziti-edge-tunnel.service failed because of unavailable resources or another system error.
See "systemctl status ziti-edge-tunnel.service" and "journalctl -xeu ziti-edge-tunnel.service" for details.
root@nick:~# systemctl status ziti-edge-tunnel.service
● ziti-edge-tunnel.service - Ziti Edge Tunnel
     Loaded: loaded (/etc/systemd/system/ziti-edge-tunnel.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: resources) since Sun 2023-05-21 15:23:52 PDT; 1s ago
        CPU: 0
root@nick:~# journalctl -xeu ziti-edge-tunnel.service
░░ Subject: A stop job for unit ziti-edge-tunnel.service has finished
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A stop job for unit ziti-edge-tunnel.service has finished.
░░ 
░░ The job identifier is 59779 and the job result is done.
May 21 15:24:02 nick systemd[1]: ziti-edge-tunnel.service: Failed to load environment files: No such file or directory
May 21 15:24:02 nick systemd[1]: ziti-edge-tunnel.service: Failed to run 'start-pre' task: No such file or directory
May 21 15:24:02 nick systemd[1]: ziti-edge-tunnel.service: Failed with result 'resources'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ The unit ziti-edge-tunnel.service has entered the 'failed' state with result 'resources'.
May 21 15:24:02 nick systemd[1]: Failed to start Ziti Edge Tunnel.
░░ Subject: A start job for unit ziti-edge-tunnel.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit ziti-edge-tunnel.service has finished with a failure.
░░ 
░░ The job identifier is 59779 and the job result is failed.

The Tunneler on Windows10 is also NOT working.
2.1 The Windows Tunneler client APP was downloaded and installed.
2.2 In ZEDS, for a different application, I also create another “echo-client” identity, and download it’s .JWT file.
2.3 Then, use the Windows Tunneler APP load the .JWT file, and the “python echo” service shows up.

2.4 However, when run ncat python.echo.ziti 80, the Windows CMD always return Ncat: Could not resolve hostname "python.echo.ziti": No such host is known. . QUITTING. It seems the Windows Tunnel APP does NOT enroll the machine.

If I use MacOS Tunneler with same steps, it works out.
Anyone had the same experience can share ?
And, does my approach has any problem ?

TheLumberjack · May 22, 2023, 12:22pm

That's great, what was the issue? Glad you got it sorted.

LINUX

That command isn't quite right. I think what you're looking for there is the following. It will enroll the identity and write the output file to the same directory (changing the extension to .json):

ziti-edge-tunnel -j /opt/openziti/etc/identities/ziti-id.jwt -i /opt/openziti/etc/identities/ziti-id.json

WINDOWS
How are you running ncat? Is that a windows binary from cmd.exe or is that a WSL binary (or other). WSL + Windows is 'a little tricky' so I just want to make sure we are looking at the same thing. If you get "no such host is known", that usually means your client can't connect to the nameserver that the tunneler (ziti desktop edge for windows) provides. from cmd.exe can you run: nslookup python.echo.ziti 100.64.0.2? That result should succeed. It's important you include the server too as most DNS utilities don't behave the same (on purpose) as other apps on your system.

Let me know about those two things and we can keep going.

Topic		Replies	Views
How to generate id.jwt in Ziti console for C# sample - weather app sdk-csharp	30	1091	April 27, 2023
API version of: ziti edge enroll --jwt identity.jwt --out identity.json	7	63	July 31, 2024
Python SDK: troubleshooting ziti-echo-server.py sample SDK Questions	1	243	July 5, 2022
Windows Ziti-Tunnel Command Line	14	381	February 16, 2023
I got it working! Ziti is awesome	8	284	November 28, 2023

How to generate .jwt and .json in Ziti for Python sample - ziti-echo-server

Related topics