Configuring ziti-tunneler-client.v1

[markamind]

I need help to understand how to use ziti-tunneler-client.v1 on a Docker instance.

situation

These are all of the commands that I use to configure an example for a tunneller

ziti edge create config httpbin.client.v1 ziti-tunneler-client.v1 '{ "hostname" : "httpbin.ziti", "port" : 2000 }'
 
ziti edge create service private.httpbin.srv --configs httpbin.client.v1 -a "httpbin-services"

ziti edge create terminator "private.httpbin.srv"  "ziti-private-blue" tcp:httpbin:80

ziti edge create service-policy httpbin-dial-policy Dial --identity-roles '#postgres-clients' --service-roles '#httpbin-services'

I start the tunneller using the following command

./ziti-tunnel proxy -i /mnt/v/temp/tunneler-id.json private.httpbin.srv:2000 -v

What I find is that this command works and returns the results expected
curl http://localhost:2000/json

this command does not work.. as it cannot locate the host
curl http://httpbin.ziti:2000/json

I really have no idea why this is happening.. or how to fix it.. especially because this is what was used as the hostname.

Maybe.. there is some thing in Docker that is extra to setup.. but I have no idea of where to start to resolve this problem..

this also relates to another problem I am having when I attempt to setup an intercept.

Running the Python SDK on an Oracle Linux server

I fixed one problem.. I had a space in between the intercept and the host. this statement was the other problem.. ziti edge create service-policy httpbin-dial-policy Dial --identity-roles '#postgres-clients' --service-roles '#httpbin-services' where I needed to set the identity-roles as follows --identity-roles "@ziti-edge-router" Now I get my terminator.. yeah.. the zcurl command works now.. but I still cannot run the python script yet I am still getting the same error.. unable to locat…

Any tips would be greatly appreciated.

[TheLumberjack]

@markamind

ziti-tunneler-client.v1

ziti-tunneler-client.v1 is "old". Use intercept.v1, it's the 'newer' version of ziti-tunneler-client.v1

However, what's helpful, is if you say "this is what I wanted to do, this is how I did it, and this is what I'm observing."

Also - I really would ask you to not use "ziti edge create terminator". I really urge you to use the identities in the edge-routers/tunnelers instead.

Can you explain, what are you trying to do and how you started docker? I assume you are running httpbin as a docker container. Posting the docker-compose file helps too because then I can see what's happening better. I expect you did not add the httpbin to a network? But maybe you did.

And finally, sometimes it's easie to push all your stuff into a github repo, or somewhere else online where I can see all these files/scripts.

Create terminator is there but it's not well-documented and it'll just be harder to accomplish whatever you're trying to do. Wherever you found that example, I'd probably like to update the sample to use an identity instead.

Make sense?

[TheLumberjack]

If you want to use this example though - you can tail the logs from the tunneler, and the logs from the router. Somewhere in there will be a hint as to what is going wrong. One other thing you can do is make sure your intercept works by issuing a dig (if on mac/linux) or a powershell “dns-resolvename” (if on windows) to see if “httpbin.ziti” is returned.

so you can try a dig @100.64.0.3 httpbin.ziti (assuming your name server is on 100.64.0.3) to verify it’s in the tunneler. you could also try running ziti edge policy-advisor identities|services to make sure there’s nothing else strange going on.

[markamind]

Great feedback.

The ultimate goal is to run the Python samples on the server… I can run this locally… but experienced lots of issues when seeking to use a Docker container.

I will make a few adjustments and revert back.

[markamind]

What I want to do
To run the following Python SDK example on the server running the Docker container.. as opposed to a local desktop

python h-ziti-p.py

While I can run this example locally, I want to know how to deploy it to a server environment.. and troubleshoot problems so that I can help with resolving technical issues.

this is why I am sticking with this to learn what I am missing. There is something fundamental I have not understood yet

This is how I did it

You will find the yaml file and commands in the folder below

https://drive.google.com/drive/folders/1D_qtD4pp5rRB0jcF4bhla8m3pa2X81A2?usp=sharing

This is what I’m observing
1 Replacing ziti-tunneler-client.v1 with intercept.v1 did not change anything

• I took the ziti-tunneler-client.v1 reference from the Java Postgres example.. maybe it was an old version

2 dig @110.0.0.157 httpbin.ziti timed out

; <<>> DiG 9.11.36-RedHat-9.11.36-3.el8 <<>> @110.0.0.157 httpbin.ziti
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

3 curl http://localhost:2000/json generates the following output

{
  "slideshow": {
    "author": "Yours Truly", 
    "date": "date of publication", 
    "slides": [
      {
        "title": "Wake up to WonderWidgets!", 
        "type": "all"
      }, 
      {
        "items": [
          "Why <em>WonderWidgets</em> are great", 
          "Who <em>buys</em> WonderWidgets"
        ], 
        "title": "Overview", 
        "type": "all"
      }
    ], 
    "title": "Sample Slide Show"
  }
}

4 curl http://httpbin.ziti:2000/json

curl: (6) Could not resolve host: httpbin.ziti

[markamind]

Something that could be related is the following entry in the yaml file

there is no working_dir or volume
would this make any difference?

httpbin:
image: kennethreitz/httpbin
networks:
- zitiblue
- zitired
#ports:
# - "80:80"

[gooseleggs]

@markamind I will mock this up and see where I get to with this in an hour or so. What is h-ziti-p.py? I gather this output:

{
  "slideshow": {
    "author": "Yours Truly", 
    "date": "date of publication", 
    "slides": [
      {
        "title": "Wake up to WonderWidgets!", 
        "type": "all"
      }, 
      {
        "items": [
          "Why <em>WonderWidgets</em> are great", 
          "Who <em>buys</em> WonderWidgets"
        ], 
        "title": "Overview", 
        "type": "all"
      }
    ], 
    "title": "Sample Slide Show"
  }
}

is generic, or have you loaded something into httpbin or somewhere for this to occur?

[markamind]

You will find h-ziti-p.py in the sample folder below.

I think I have a mis- understanding of what Docker can do… when it runs from a server…

For some reason… its not identifying the intercept… this is what I have tracked down the source of the problem to be.

ziti-sdk-py/sample at main · openziti/ziti-sdk-py

[markamind]

@gooseleggs

is generic, or have you loaded something into httpbin or somewhere for this to occur?

The results are from the httpbin service running in the Docker container.

After I start up Docker.. and configure the intercepts..as follows

and for some reason I do need to manually create the terminator

ziti edge create config httpbin.intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["httpbin.ziti"], "portRanges":[{"low":2000, "high":2000}]}'

ziti edge create service private.httpbin.srv --configs httpbin.client.v1 -a "httpbin-services"

ziti edge create terminator "private.httpbin.srv"  "ziti-private-blue" tcp:httpbin:80

ziti edge create service-policy httpbin-dial-policy Dial --identity-roles '#postgres-clients' --service-roles '#httpbin-services'

I then wanted to create a tunneller to test that the intercept works

./ziti-tunnel proxy -i /mnt/v/temp/tunneler-id.json private.httpbin.srv:2000 -v

I get the right results using 'localhost" as the host
curl http://localhost:2000/json

this validates that I can connect to the service

But.. when I use the intercept.. it cannot find the host
curl http://httpbin.ziti:2000/json

So.. I am thinking.. maybe I need to run this within a container in Docker.. rather than just from the command line.. but I am not sure why not.. as I have the tunneller running and listening

[markamind]

In contrast… when I just have the Docker container for the httpbin service running by itself…

I can setup an intercept using a controller on another machine that I have setup… and then run the Python script locally which all works well.

It’s only when it runs on the server in a Docker container… for some reason… the intercept cannot be found… though the service can

[markamind]

I noticed this.

Processes (like web servers) that just need to bind on a port below 1024 do not need to run as root: they can just be granted the net_bind_service capability instead.

Not sure if this has been granted… but probably could be the cause

Docker security

[gooseleggs]

With the configuration of this…

httpbin:
image: kennethreitz/httpbin
networks:
- zitiblue
- zitired
#ports:
# - “80:80”

httpbin will not be available OUTSIDE of the docker environment. It is binding only to the zitiblue and zitired networks.

If you remove the # on the ports option, then it will create a port forward from the host, to that service, so if I had a setting of

   ports:
     - 8080:80

Then if I connect to the host port 8080, it will translate the connection through to port 80 of the container, ie http://serverIP:8080. This will then work for machines that are off the host itself as this is a host binding.

Just building the environment now.

[markamind]

I think that this makes sense… as it would prevent connection from being made.

I will also try it out myside

Thanks for your time… it’s greatly appreciated…

[markamind]

I now have http://localhost:8080/json working

but not curl http://httpbin:8080/json

curl: (6) Could not resolve host: httpbin

[gooseleggs]

@markamind

but not curl http://httpbin:8080/json

Correct - httpbin is only resolvable to the applications running within the docker-compose environment and not to all systems.

You are asking the client to listen for (intercept) httpbin.ziti on port 2000 which you are sending through the overlay network to an edge-router, and the edge router will then be looking for httpbin. Since httpbin is resolvable by the edge router (it is on the same docker network) it should get there.

Because you enabled 8080 port bindings, then localhost:8080 will work, because that container is listenting on 8080 on the host, so either 127.0.0.1:8080 OR the host IP address:8080 will work.

SO, you just need to make sure that the edge-router that you are binding too needs to be able to resolve httpbin. The client is not concerned about httpbin and does not care about it. It only needs to know to listen for httpbin.ziti and then direct it through the overlay network.

To connect it over the ziti network, the client should be quering http://httpbin.ziti:2000 as that is what you are defining:

ziti edge create config httpbin.intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["httpbin.ziti"], "portRanges":[{"low":2000, "high":2000}]}'

[gooseleggs]

@markamind I have used your docker-compose file but hashed out the postgresSQL config as that is not relevant. These are the commands that I used to get the service up and going for me. You have used some slightly different configs which I have not yet come across, but I am a little behind your learnings!

Anyway - give this a go and see what happens.

I created an identity to be used, so this is what is referenced:

ziti edge create identity user http-client -a 'http-clients' -o http.client.jwt 

Then I used these commands:

ziti edge create config httpbin.intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["httpbin.ziti"], "portRanges":[{"low":2000, "high":2000}]}'
ziti edge create config httpbin.host.v1 host.v1 '{"protocol":"tcp", "address":"httpbin", "port":80}'
ziti edge create service httpbin.svc --configs httpbin.intercept.v1,httpbin.host.v1
ziti edge create service-policy httpbin.policy.dial Dial --service-roles "@httpbin.svc" --identity-roles '#http-clients'
ziti edge create service-policy httpbin.policy.bind Bind --service-roles '@httpbin.svc' --identity-roles "@ziti-private-blue"

This gives me the following… see the web page in the background, and I can ping in the foreground

[markamind]

Thanks… I will check through everything I have setup… I must be missing something along the way.

Just to confirm… which edge router should be used… as I was not sure if it should be @ziti-private-blue or @ziti-edge-router

[markamind]

@markamind

@ziti-private-blue

Thanks.. I noticed it was @ziti-private-blue in your references

[gooseleggs]

I have used ziti-private-blue. Since it is bound to both the red and blue networks, it could be ziti-private-red or ziti-private-blue.

[gooseleggs]

Also once done, you can hash out the -ports compose config lines to make httpbin dark again.