You could also send me an identity on your network if you wanted 
clint at openziti dot org ā¦
1 Like
I can see in your logs you shared:
[77941.634] ERROR edge/controller/handler_edge_ctrl.(*baseSessionRequestContext).verifyTerminator: {operation=[remove.terminator] terminatorId=[NZpG] error=[invalid terminator: not found]} terminator not found
[77941.634] ERROR edge/controller/handler_edge_ctrl.(*baseRequestHandler).returnError [ch{wYXvHOJa9x}->u{classic}->i{PJld}]: {token=[cb07e6cb-1861-42f0-ac0c-ae14b5d5429e] error=[invalid terminator: not found]
That is new one on meā¦ I also see youāre accessing http.ziti. Iād really love to know the exact steps to reproduce this. I havenāt even seen this particular issue.
Right now I would say you should probably delete your configs, services and service policiesā¦ wait a good 15s and try again to recreate them.
Synatx for cleaning ALL your configs/services/service-policies is:
ziti edge delete service-policies where true
ziti edge delete services where true
ziti edge delete configs where true
itās a lighter touch than reinstalling everythingā¦ Iād clean it all up and see what happens if you run the commands again?
1 Like
okā¦ will doā¦ not sure if I can tell you the stepsā¦ as itās been a trail of spaghettiā¦
will revert back shortly
trail of ziti??? (we make all the pasta jokes here)
1 Like
still not working
Here are all of the command that I run after running all of the delete commands
ziti edge create config golanghttp-intercept.v1 intercept.v1 ā{āprotocolsā:[ātcpā],āaddressesā:[āgolanghttp.zitifiedā], āportRangesā:[{ālowā:1280, āhighā:1281}]}ā
ziti edge create service golanghttp --configs āgolanghttp-intercept.v1ā
ziti edge create service-policy golanghttp-bind-policy Bind --identity-roles ā@httpzServerā --service-roles ā@golanghttpā
ziti edge create service-policy golanghttp-dial-policy Dial --identity-roles ā@httpzClientā --service-roles ā@golanghttpā
I have confirmed that the ZitiMobileEdge app has picked up the new service.
it now shows golanghttp.zitified
and the green dot is showing
One thing that could be the issueā¦ is that I am using Oracle Cloud Infrastructureā¦
When you setup a computeā¦ you need to setup a VCNā¦ which by default has very limited privileges.
Here are the ingress rules setup
While we are on thisā¦ I needed to open port 1281ā¦ to set ZITI_EDGE_ROUTER_PORT when doing the QuickStart installation.
When I had this set as 1280ā¦ it failedā¦ saying that the listener is already in use.
Soā¦ I changed it to 1281 and it worked ok
Maybe this is itā¦ ieā¦ the port short be 1281ā¦ but I could not complete the installation using this port.
I can see that I am making a ziti of a mess with this
The listener already in use error would sound like ziti was already running. I donāt expect OCI to be of consequence, you seem to have the inbound traffic rules setup just fine as well.
Why donāt I setup a new instance with a single identity you can use which I know worked for me. Getting that first service to work really is the toughest. I appreciate you sticking through this process.
Iāll establish a new server and Iāll even record the entire video and share it so you can see all the sausage get made. Iāll even give it a shot in OCI. I know it works because the NetFoundry platform deploys routers to Oracle all the time.
It shouldnāt take me terribly long to do once I discover the right place to fire up an OCI vmā¦ Stay tunedā¦
1 Like
Awesomeā¦ I know I am closeā¦ maybe not enough salt
I hit āa lotā of bumps along the way. I have to review/edit the video which I will do a bit later on tonight and Iāll post it back.
Thereās a bug with the quickstart which causes the router to only work when youāre local to the oracle environment. The āadvertisedā value is set to the hostname - which wonāt work when youāre remote. 
thatās a very difficult problem to debug. The router itās attaching to is in the TUN logs you sent me but itās hard to spot because it seems ānormalāā¦
instance-20220317-1005-edge-router@tls://instance-20220317-1005:1281
notice that itās defined by āinstance-20220317-1005:1281ā - that needs to be the ip addressā¦ The problem is that the certs generated when you enroll the router wonāt have the ip address in them. The video Iāll be sharing with you will have that.
Iāll post back in a bit with a video and with more instructions (dinner time here)ā¦
1 Like
Ok. I hope this helpsā¦ Here are the commands/notes I show in the video. @markamind I think you can start from the bullet labeled āHERE IS THE FIXā.
Things done before the video starts:
-
Create VCN
-
Create Subnet
-
Create Security List
-
opened ports 8441-8443
-
used netcat
ncto very ports were open -
create instance named openziti09b
-
opened LOCAL firewall in firewalld with:
sudo firewall-cmd --zone=public --add-port=8441/tcp --permanent sudo firewall-cmd --zone=public --add-port=8442/tcp --permanent sudo firewall-cmd --zone=public --add-port=8443/tcp --permanent sudo systemctl restart firewalld -
disable selinux - didnāt troubleshoot but SELINUX prevented the systemd units from working
sudo cp /etc/selinux/config /etc/selinux/config.back sudo sed -i 's/^SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config sudo /usr/sbin/setenforce 0 -
installed golang using instructions from here
-
installing golang via that link puts go into /usr/local/go/bin/go, so i also added
/usr/local/go/bin/to my path in .bashrc -
Installed git using:
sudo dnf install git -y
Steps in the video Discourse Help - OCI Quickstart - unlisted - YouTube
-
RERAN expressInstall with EXTERNAL_DNS set to the EXTERNAL_IP! This was important - due to Oracle Cloud. Unlike AWS oracle doesnāt give your compute instances a DNS name.
export EXTERNAL_DNS="$(curl -s eth0.me)" -
HERE IS THE FIX the quickstart is putting the hostname into the router configuration. To fix that you need to edit the configuration finding the hostname and replacing it with the external IP address. Then you need to deleting/recreate the edge router. Only do this after successfully running quickstart.
# set a variable and use this in the next three commands routerName=clintozapr09b-edge-router ziti edge delete edge-router ${routerName} ziti edge create edge-router ${routerName} -t -a "public" -o ${routerName}.jwt ziti-router enroll ~/.ziti/quickstart/clintozapr09b/${routerName}.yaml -j ${routerName}.jwt -
installed ZAC by following the ZAC install guide
Making sure it all works:
Once I had the ZAC running and fixed the bug I was able to verify the golang http example was working by running the zitified server/client and testing with my local browserā¦
-
created four identities: one for the golang server, one for the client, one for my desktop, one for my mobile.
-
i enrolled the server/client for immediate use.
-
identities were created using
-a "http-clients"so that i could make one ādialā policy and use the attributeziti edge create identity service golang.http.server -o $HOME/golang.http.server.jwt ziti edge enroll $HOME/golang.http.server.jwt ziti edge create identity service golang.http.client -a "http-clients" -o $HOME/golang.http.client.jwt ziti edge enroll $HOME/golang.http.client.jwt ziti edge create identity service golang.http.desktop.client -a "http-clients" -o $HOME/golang.http.desktop.client.jwt ziti edge create identity service golang.http.mobile.client -a "http-clients" -o $HOME/golang.http.mobile.client.jwt -
created the one needed config, the service, and two policiesā¦
ziti edge create config golanghttp-intercept.v1 intercept.v1 '{"protocols":["tcp"],"addresses":["golanghttp.zitified"], "portRanges":[{"low":1234, "high":2345}]}' ziti edge create service golanghttp --configs 'golanghttp-intercept.v1' ziti edge create service-policy golanghttp-bind-policy Bind --identity-roles '@golang.http.server' --service-roles '@golanghttp' ziti edge create service-policy golanghttp-dial-policy Dial --identity-roles '#http-clients' --service-roles '@golanghttp' -
cloned the golang sdk for demo use:
git clone GitHub - openziti/sdk-golang: Ziti SDK for Golang ā${ZITI_HOME}/sdk-golangā
cd ā${ZITI_HOME}/sdk-golang/exercises/http/server/zitifiedā -
ran the zitified simple-server
go run simple-server.go "$HOME/golang.http.server.json" "golanghttp" -
ran the zitified simple-client
go run simple-client.go $HOME/golang.http.client.json golanghttp -
opened this url in desktop browser and saw the expected results
1 Like
Ahh.. awesome..
This reminds me of a situation that happened when I was debugging this issue..
the variables "router_name" and "ZITI_EDGE_ROUTER_RAWNAME" were not set when you reload the env file
Because manually setting the ZITI_EDGE_ROUTER_RAWNAME fixed the symptom.. I did not see how this could be linked back the problem I was debugging
my learning is that I should have also tracked the impact of this down.. something for next time
Re the difference Oracle and AWS.. is interesting.. Oracle keeps all of those DNS names private for compute instances.
Thanks so much for the video.. there was quite a bit to work through.. I have succeeded in each step.. though I found a few more things.. maybe because I am using a mac
#1. enrolling the mobile identity via QR code did not work.. I will send through the logs
#2. the link does not work if you use the https prefix.. but does if you use http..
it would be great if you can provide more details about this
Nice! Success! The "simple server" does not expose TLS. It's an http-only based example.
1 Like
re: enrolling on iOS. I went through a couple of QR code-based enrollments using the NetFoundry console successfully. I plan to do one using ZAC later today. I see from the APP logs that there was an unexpected error parsing the enrollment response from the Controller - WTF CZiti:ZitiEnroller.swift:200 on_enroll() unable to decode context (WTF stands for āWhat a Terrible Failureā, and is logged when we see an unexpected code path that we can do nothing other than log a message).
A couple of things:
- Was Ziti āConnectedā when you attempted the enrollment?
- There is an issue some have reported where Turning Ziti On from Mobile will get stuck in
Connecting...If you see this, please Go to Settings/VPN and toggle Status to ON - Would you also please check
Settings/Privacy/Analytics & Improvements/Application Dataand see if you have a report from the time of the failed enrollment request? The file will start withZiti Mobile Edge-2022
Thanks.
Thanks for the feedback. Makes sense re TLS.. I always wondered what that did.. now I know
This shows that the mobile app was connected to the controller before I attempted to enrol a new identity
