Integrating OpenZiti with a Golang TLS server

I worked this one out

ziti edge create config golanghttp-intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["golanghttp.zitified"], "portRanges":[{"low":1234, "high":2345}]}'

When you setup an intercept.. you can only specify the dns.. not the type of http

I actually came across this when I was getting this working for the first time... as when I was entering the URL.. I was either putting in https.. or the broswer defaulted to https.. preventing it from working.

It was not until I setup a link with http:/abc.def specified that it actually worked..

So.. one down..one more to go

I have now worked out how to get a golang reverse proxy working.. I just need to zitify it now

Brief update.. as I have set up most of what I believe I need for a reverse goloang proxy.. but something is happening.. when I access the target over a tunneler.

http://golanghttp.zitified:2000

http: panic serving ziti-edge-router connId=2147483675, logical=ziti-sdk[router=tls://999.9999.0000.0000:8442]: EOF

I am not sure of how to debug this.. or where to look..

before I zitified the reverse proxy.. I did confirm that it was working..

It looks like it intercepted the packet.. but something happened after that

Any tips?

When I get this working… I will write up all of my learnings about this

I'll be honest, without digging into all this really hard myself, getting and looking at your code etc. I don't think I'm going to be able to provide a ton of help. I don't think I'll be able to do that sort of work. However, looking at that address it looks totally bizzare to me. I would say that the ip address is somehow 'wrong'. see how it's a "four-tuple" like an IP address, but clearly is not a valid ipv4 address? 999.9999.0000.0000 is clearly not a valid ipv4... That's the only hint I have for you.

As far as a reverse-proxy goes - that's basically what a tunneling app is. I suspect you know that and you're trying to do 'something else'. I am afraid this one won't be easy enough for me to help you out. :frowning:

If you can ask some more direct questions about ziti, describe what you're doing and how, maybe I could offer any idea of how to debug but realistically I think you'll just have to keep the deep-dive going and add breakpoints to your 'server/reverse-proxy' code and try to figure what went wrong enough to post back with something we could maybe help with.

Good luck!

1 Like

This may help..

I noticed this in the audit logs..

cannot forward payload

Apr 20 02:36:31 instance-20220416-1603 ziti -router[704352]: {"_context":"{c/cooLb2n2u|@/L1jm}\u003cInitiator\u003e","circuitId":"cooLb2n2u","error":"cannot forward payload, no destination for circuit=cooLb2n2u src=L1jm dst=OV51","file":"OPEN ¡ GitHub ziti /fabric@v0.17.88/router/handler_xgress/receive.go:35","func":"OPEN ¡ GitHub ziti /fabric/router/handler_xgress.(*receiveHandler).HandleXgressReceive","level":"error","msg":"unable to forward payload","origin":0,"seq":3,"time":"2022-04-20T02:36:31.627Z","uuid":""}

ahh.. maybe I am trying to do something I shouldn't be.

all ok.. I will keep venturing into the unknown for a while longer

Yes, that message is certainly something that I would think we might be able to help with. I'm not well-versed with how that happens. You're still using the express install right and have a "one controller and one router" setup? When you report an error it's helpful to know if it's from a log on the client or a log on the target. This looks like it's a log from the ziti-router itself indicating that the router cannot send the payload to the destination. I'm not sure how to replicate this I've not seen it before. I can point one of the people more familiar with "the fabric" to take a peek though.

1 Like

Great… I will also upload the source code in a few minutes

This is the go application that I run

package main

import (
“fmt”
“github.com/openziti/sdk-golang/ziti”
“github.com/openziti/sdk-golang/ziti/config”
“bytes”
“encoding/json”
“io/ioutil”
“log”
“net”
“time”
“net/http”
“net/http/httputil”
“net/url”
“os”
)

type requestPayloadStruct struct {
ProxyCondition string json:"proxy_condition"
}

/*
Logging
*/

// Log the typeform payload and redirect url
func logRequestPayload(requestionPayload requestPayloadStruct, proxyUrl string) {
log.Printf(“proxy_condition: %s, proxy_url: %s\n”, requestionPayload.ProxyCondition, proxyUrl)
}

/*
Reverse Proxy Logic
*/

// Serve a reverse proxy for a given url
func serveReverseProxy(target string, res http.ResponseWriter, req *http.Request) {
// parse the url
url, _ := url.Parse(target)

// create the reverse proxy
proxy := httputil.NewSingleHostReverseProxy(url)

// Update the headers to allow for SSL redirection
req.URL.Host = url.Host
req.URL.Scheme = url.Scheme
req.Header.Set("X-Forwarded-Host", req.Header.Get("Host"))
req.Host = url.Host

// Note that ServeHttp is non blocking and uses a go routine under the hood
proxy.ServeHTTP(res, req)

}

// Get a json decoder for a given requests body
func requestBodyDecoder(request *http.Request) *json.Decoder {
// Read body to buffer
body, err := ioutil.ReadAll(request.Body)
if err != nil {
log.Printf(“Error reading body: %v”, err)
panic(err)
}

request.Body = ioutil.NopCloser(bytes.NewBuffer(body))

return json.NewDecoder(ioutil.NopCloser(bytes.NewBuffer(body)))

}

// Parse the requests body
func parseRequestBody(request *http.Request) requestPayloadStruct {
decoder := requestBodyDecoder(request)

var requestPayload requestPayloadStruct
err := decoder.Decode(&requestPayload)

if err != nil {
	panic(err)
}

return requestPayload

}

// Given a request send it to the appropriate url
func handleRequestAndRedirect(res http.ResponseWriter, req *http.Request) {
requestPayload := parseRequestBody(req)

//url := getProxyUrl(requestPayload.ProxyCondition)
    url := "https://www.theage.com.au:443"

logRequestPayload(requestPayload, url)

serveReverseProxy(url, res, req)

}

// setup the ZitiListener
func createZitiListener() net.Listener {
cfg, err := config.NewFromFile(os.Args[1])
if err != nil {
panic(err)
}
options := ziti.ListenOptions{
ConnectTimeout: 5 * time.Minute,
}
listener, err := ziti.NewContextWithConfig(cfg).ListenWithOptions(os.Args[2], &options)
if err != nil {
fmt.Printf(“Error binding service %+v\n”, err)
panic(err)
}
return listener
}

/*
Entry
*/

func main() {
// Log setup values
//logSetup()

// start server
http.HandleFunc("/", handleRequestAndRedirect)
if err := http.Serve(createZitiListener(), nil); err != nil {
	panic(err)
}

//if err := http.ListenAndServe(getListenAddress(), nil); err != nil {
//	panic(err)
//}

}

This is the intercept

ziti edge create config golanghttp-intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["golanghttp.zitified"], "portRanges":[{"low":1234, "high":2345}]}'

I am not sure how to recreate the problem..

When I run the go http client application… I get this error

Apr 20 03:15:55 instance-20220416-1603 ziti -controller[704239]: {“level”:“info”,“msg”:“http: TLS handshake error from 168.138.10.79:33994: EOF”,“time”:“2022-04-20T03:15:55.465Z”}

it may have something to do with the server that I am looking to return the results from

I thought to try something publicly available first before I link it to a web app.

Is this how its setup..? I am sure I could use it to scape the website with the unzitified version

I have never done this before.. so are likely making lots of simple errors

Cannot forward payload errors happen when one side of a circuit is torn down but the other side isn’t torn down yet. We will leave one side of connection running briefly so it can deliver any data that was received by the router but hasn’t been shipped to the end client yet.

So if you’ve got with A → Router → B

Let’s say B writes the last of it’s data to the router and then closes. This trigger the circuit to be torn down. We leave the A side of the circuit up for a bit so that the last data the B wrote to router can be deliver to A. However, since B is gone, if A tries to write any data, we no longer have anywhere to deliver it and you’ll see that error.

I’m not sure if I’m following what you’re trying to do, but it looks like you’re expecting that intercept config to work with your custom SDK application.

The Ziti SDK doesn’t do anything with configs, it only makes them available to the application. The Ziti tunneler applications are SDK applications that implement the intercepting and hosting implied by those config types.

If you want to use the intercept.v1 and host.v1/v2 config types in your application you can, but you also have to implement the intercept and hosting logic yourself. You could build a customized version of one of the ziti tunneler apps if you wanted to experiment with how proxy applications work.

On the other hand if you feel like the Ziti tunnelers are missing features, you can open an issue (or work on a PR to add the feature).

Hope that’s helpful,
Paul

1 Like

Great feedback.. what I ultimately want to do is to implement the following solution suggested by @tburtchell. (details below)

When @tburtchell suggested this.. I had no idea of what to do.. so I just started with the basics..

After having worked through this.. I felt I was ready to give it a go.. but got quite stuck with how to implement a Golang reverse proxy.. as I had not even created a Goloang ssl server.. but now that I have done this.. I realised that you don't need the ssl part.. because that is what Ziti does for you..

So.. I am now faced with placing the final piece of the jig saw puzzle.. and trying to make it fit..

It is very likely that I am doing something completely wrong.

in the example where I took the original code from.. I just remembered that you needed to pass in a payload to configure the service.. maybe I did not remove all of that code .. and maybe that is what is causing the problem

Let me know if you have any further suggestions / tips / comments.

@TheLumberjack @plorenz @tburtchell

Hey… after a few days of mental rest… I realised there was a bug in the golang code :slight_smile:

See below … providing evidence that I now have the reverse proxy working…

1 Like

BIG congratulations! Now you should make a github repo and push it out for the world to use!!! :slight_smile:

1 Like

will do..

this means that any app can now be zitified.. by simply putting it behind a zitified reverse proxy

Hi, maybe this will be my off topic discussion here.

As I went across the use case of simple client-server application on go-sdk,
I stumbled upon an issue here.

The error here, I find is panic: runtime error: index out of range [1] with length 1

I would ask what the structure of conf file would look like here. Thanks.

1 Like

To execute a “zitified” app… you need to pass in the json token… along with the service name

go run simple-server.go “$HOME/golang.http.server.json” golanghttp

Hope that helps

:slight_smile:

1 Like