Configuring ziti-tunneler-client.v1

I need help to understand how to use ziti-tunneler-client.v1 on a Docker instance.

situation

These are all of the commands that I use to configure an example for a tunneller

ziti edge create config httpbin.client.v1 ziti-tunneler-client.v1 '{ "hostname" : "httpbin.ziti", "port" : 2000 }'
 
ziti edge create service private.httpbin.srv --configs httpbin.client.v1 -a "httpbin-services"

ziti edge create terminator "private.httpbin.srv"  "ziti-private-blue" tcp:httpbin:80

ziti edge create service-policy httpbin-dial-policy Dial --identity-roles '#postgres-clients' --service-roles '#httpbin-services'

I start the tunneller using the following command

./ziti-tunnel proxy -i /mnt/v/temp/tunneler-id.json private.httpbin.srv:2000 -v

What I find is that this command works and returns the results expected
curl http://localhost:2000/json

this command does not workā€¦ as it cannot locate the host
curl http://httpbin.ziti:2000/json

I really have no idea why this is happeningā€¦ or how to fix itā€¦ especially because this is what was used as the hostname.

Maybeā€¦ there is some thing in Docker that is extra to setupā€¦ but I have no idea of where to start to resolve this problemā€¦

this also relates to another problem I am having when I attempt to setup an intercept.

Any tips would be greatly appreciated.

ziti-tunneler-client.v1 is ā€œoldā€. Use intercept.v1, itā€™s the ā€˜newerā€™ version of ziti-tunneler-client.v1

However, whatā€™s helpful, is if you say ā€œthis is what I wanted to do, this is how I did it, and this is what Iā€™m observing.ā€

Also - I really would ask you to not use ā€œziti edge create terminatorā€. I really urge you to use the identities in the edge-routers/tunnelers instead.

Can you explain, what are you trying to do and how you started docker? I assume you are running httpbin as a docker container. Posting the docker-compose file helps too because then I can see whatā€™s happening better. I expect you did not add the httpbin to a network? But maybe you did.

And finally, sometimes itā€™s easie to push all your stuff into a github repo, or somewhere else online where I can see all these files/scripts.

Create terminator is there but itā€™s not well-documented and itā€™ll just be harder to accomplish whatever youā€™re trying to do. Wherever you found that example, Iā€™d probably like to update the sample to use an identity instead.

Make sense?

1 Like

If you want to use this example though - you can tail the logs from the tunneler, and the logs from the router. Somewhere in there will be a hint as to what is going wrong. One other thing you can do is make sure your intercept works by issuing a dig (if on mac/linux) or a powershell ā€œdns-resolvenameā€ (if on windows) to see if ā€œhttpbin.zitiā€ is returned.

so you can try a dig @100.64.0.3 httpbin.ziti (assuming your name server is on 100.64.0.3) to verify itā€™s in the tunneler. you could also try running ziti edge policy-advisor identities|services to make sure thereā€™s nothing else strange going on.

1 Like

Great feedback.

The ultimate goal is to run the Python samples on the serverā€¦ I can run this locallyā€¦ but experienced lots of issues when seeking to use a Docker container.

I will make a few adjustments and revert back.

What I want to do
To run the following Python SDK example on the server running the Docker containerā€¦ as opposed to a local desktop

python h-ziti-p.py

While I can run this example locally, I want to know how to deploy it to a server environmentā€¦ and troubleshoot problems so that I can help with resolving technical issues.

this is why I am sticking with this to learn what I am missing. There is something fundamental I have not understood yet

This is how I did it

You will find the yaml file and commands in the folder below

https://drive.google.com/drive/folders/1D_qtD4pp5rRB0jcF4bhla8m3pa2X81A2?usp=sharing

This is what Iā€™m observing
1 Replacing ziti-tunneler-client.v1 with intercept.v1 did not change anything

  • I took the ziti-tunneler-client.v1 reference from the Java Postgres exampleā€¦ maybe it was an old version

2 dig @110.0.0.157 httpbin.ziti timed out

; <<>> DiG 9.11.36-RedHat-9.11.36-3.el8 <<>> @110.0.0.157 httpbin.ziti
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

3 curl http://localhost:2000/json generates the following output

{
  "slideshow": {
    "author": "Yours Truly", 
    "date": "date of publication", 
    "slides": [
      {
        "title": "Wake up to WonderWidgets!", 
        "type": "all"
      }, 
      {
        "items": [
          "Why <em>WonderWidgets</em> are great", 
          "Who <em>buys</em> WonderWidgets"
        ], 
        "title": "Overview", 
        "type": "all"
      }
    ], 
    "title": "Sample Slide Show"
  }
}

4 curl http://httpbin.ziti:2000/json

curl: (6) Could not resolve host: httpbin.ziti

Something that could be related is the following entry in the yaml file

there is no working_dir or volume
would this make any difference?

httpbin:
image: kennethreitz/httpbin
networks:
- zitiblue
- zitired
#ports:
# - ā€œ80:80ā€

@markamind I will mock this up and see where I get to with this in an hour or so. What is h-ziti-p.py? I gather this output:

{
  "slideshow": {
    "author": "Yours Truly", 
    "date": "date of publication", 
    "slides": [
      {
        "title": "Wake up to WonderWidgets!", 
        "type": "all"
      }, 
      {
        "items": [
          "Why <em>WonderWidgets</em> are great", 
          "Who <em>buys</em> WonderWidgets"
        ], 
        "title": "Overview", 
        "type": "all"
      }
    ], 
    "title": "Sample Slide Show"
  }
}

is generic, or have you loaded something into httpbin or somewhere for this to occur?

You will find h-ziti-p.py in the sample folder below.

I think I have a mis- understanding of what Docker can doā€¦ when it runs from a serverā€¦

For some reasonā€¦ its not identifying the interceptā€¦ this is what I have tracked down the source of the problem to be.

The results are from the httpbin service running in the Docker container.

After I start up Dockerā€¦ and configure the interceptsā€¦as follows

and for some reason I do need to manually create the terminator

ziti edge create config httpbin.intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["httpbin.ziti"], "portRanges":[{"low":2000, "high":2000}]}'

ziti edge create service private.httpbin.srv --configs httpbin.client.v1 -a "httpbin-services"

ziti edge create terminator "private.httpbin.srv"  "ziti-private-blue" tcp:httpbin:80

ziti edge create service-policy httpbin-dial-policy Dial --identity-roles '#postgres-clients' --service-roles '#httpbin-services'

I then wanted to create a tunneller to test that the intercept works

./ziti-tunnel proxy -i /mnt/v/temp/tunneler-id.json private.httpbin.srv:2000 -v

I get the right results using 'localhost" as the host
curl http://localhost:2000/json

this validates that I can connect to the service

Butā€¦ when I use the interceptā€¦ it cannot find the host
curl http://httpbin.ziti:2000/json

Soā€¦ I am thinkingā€¦ maybe I need to run this within a container in Dockerā€¦ rather than just from the command lineā€¦ but I am not sure why notā€¦ as I have the tunneller running and listening

In contrastā€¦ when I just have the Docker container for the httpbin service running by itselfā€¦

I can setup an intercept using a controller on another machine that I have setupā€¦ and then run the Python script locally which all works well.

Itā€™s only when it runs on the server in a Docker containerā€¦ for some reasonā€¦ the intercept cannot be foundā€¦ though the service can

I noticed this.

Processes (like web servers) that just need to bind on a port below 1024 do not need to run as root: they can just be granted the net_bind_service capability instead.

Not sure if this has been grantedā€¦ but probably could be the cause

With the configuration of thisā€¦

httpbin:
image: kennethreitz/httpbin
networks:
- zitiblue
- zitired
#ports:
# - ā€œ80:80ā€

httpbin will not be available OUTSIDE of the docker environment. It is binding only to the zitiblue and zitired networks.

If you remove the # on the ports option, then it will create a port forward from the host, to that service, so if I had a setting of

   ports:
     - 8080:80

Then if I connect to the host port 8080, it will translate the connection through to port 80 of the container, ie http://serverIP:8080. This will then work for machines that are off the host itself as this is a host binding.

Just building the environment now.

1 Like

I think that this makes senseā€¦ as it would prevent connection from being made.

I will also try it out myside

Thanks for your timeā€¦ itā€™s greatly appreciatedā€¦ :slight_smile:

I now have http://localhost:8080/json working

but not curl http://httpbin:8080/json

curl: (6) Could not resolve host: httpbin

Correct - httpbin is only resolvable to the applications running within the docker-compose environment and not to all systems.

You are asking the client to listen for (intercept) httpbin.ziti on port 2000 which you are sending through the overlay network to an edge-router, and the edge router will then be looking for httpbin. Since httpbin is resolvable by the edge router (it is on the same docker network) it should get there.

Because you enabled 8080 port bindings, then localhost:8080 will work, because that container is listenting on 8080 on the host, so either 127.0.0.1:8080 OR the host IP address:8080 will work.

SO, you just need to make sure that the edge-router that you are binding too needs to be able to resolve httpbin. The client is not concerned about httpbin and does not care about it. It only needs to know to listen for httpbin.ziti and then direct it through the overlay network.

To connect it over the ziti network, the client should be quering http://httpbin.ziti:2000 as that is what you are defining:

ziti edge create config httpbin.intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["httpbin.ziti"], "portRanges":[{"low":2000, "high":2000}]}'
1 Like

@markamind I have used your docker-compose file but hashed out the postgresSQL config as that is not relevant. These are the commands that I used to get the service up and going for me. You have used some slightly different configs which I have not yet come across, but I am a little behind your learnings!

Anyway - give this a go and see what happens.

I created an identity to be used, so this is what is referenced:

ziti edge create identity user http-client -a 'http-clients' -o http.client.jwt 

Then I used these commands:

ziti edge create config httpbin.intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["httpbin.ziti"], "portRanges":[{"low":2000, "high":2000}]}'
ziti edge create config httpbin.host.v1 host.v1 '{"protocol":"tcp", "address":"httpbin", "port":80}'
ziti edge create service httpbin.svc --configs httpbin.intercept.v1,httpbin.host.v1
ziti edge create service-policy httpbin.policy.dial Dial --service-roles "@httpbin.svc" --identity-roles '#http-clients'
ziti edge create service-policy httpbin.policy.bind Bind --service-roles '@httpbin.svc' --identity-roles "@ziti-private-blue"

This gives me the followingā€¦ see the web page in the background, and I can ping in the foreground

1 Like

Thanksā€¦ I will check through everything I have setupā€¦ I must be missing something along the way.

Just to confirmā€¦ which edge router should be usedā€¦ as I was not sure if it should be @ziti-private-blue or @ziti-edge-router

Thanksā€¦ I noticed it was @ziti-private-blue in your references

I have used ziti-private-blue. Since it is bound to both the red and blue networks, it could be ziti-private-red or ziti-private-blue.

1 Like

Also once done, you can hash out the -ports compose config lines to make httpbin dark again.

1 Like