Unable to compile Ziti tunnel sdk c binary using teltonika provided openwrt toolchain

Dear Team,
I am building ziti tunneler sdk c code (v0.17.25) against teltonika provided openwrt SDK.
I have used the openwrt build script which is provided in scripts folder.
The command used is in this way " ./scripts/openwrt-build.sh -s /home/buildusr/rutos-ipq40xx-rutx-gpl -t arm_cortex-a7+neon-vfpv4_gcc-8.4.0_musl_eabi "
The error i am getting is

[ 53%] Building C object _deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/src/base64.c.o
[ 54%] Building C object _deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/src/tls_engine.c.o
[ 54%] Building C object _deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/src/compression.c.o
/home/buildusr/ziti-tunnel-sdk-c-0.17.25/_deps/uv-mbed-src/src/compression.c:3:10: fatal error: zlib.h: No such file or directory
#include <zlib.h>
^~~~~~~~
compilation terminated.
_deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/build.make:215: recipe for target ‘_deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/src/compression.c.o’ failed
make[3]: *** [_deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/src/compression.c.o] Error 1
CMakeFiles/Makefile2:446: recipe for target ‘_deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/all’ failed
make[2]: *** [_deps/uv-mbed-build/CMakeFiles/uv_mbed.dir/all] Error 2
CMakeFiles/Makefile2:422: recipe for target ‘CMakeFiles/bundle.dir/rule’ failed
make[1]: *** [CMakeFiles/bundle.dir/rule] Error 2
Makefile:189: recipe for target ‘bundle’ failed
make: *** [bundle] Error 2

Build complete

Could you help me identify where the issue is ?

Thanks

What specific SDK are you trying to use?

zlib is usually built in by default, can you make sure it is included in the Teltonika SDK build?
in SDK directory:

  • make menuconfig
  • select Libraries, scroll down to zlib
  • make sure it is enabled and save configuration
  • build Teltonika SDK

Is it possible (I guess an option would likely be in the Teltonika SDK) when one cross compiles it to use the byte order one needs. For example if ziti tunnel sdk expects the byte order to be in the network order, then one can choose that option. The reason, I am asking I cross compiled Ziti version 0.17.32/36 with Teltonika 7.1.4 SDK on ubuntu 20.04 and when I deployed it on the Teltonika Router, the syn packets are dropped at the tun interface. It appears that the byte order is not what expected and the length of the packet is misread. Lwip - iphdr_len in bytes: 15360 . It should read 60 bytes

Odd. Honestly, I would have expected the code to already translate from network byte order to target byte order properly.

more context to this issue

  1. SDK v6.8.6 has no problem with endiness when complied on ubuntu 20.04 unlike SDK v7.1.4
  2. Also, not sure if that matters, but for the SDK 7.1.4 the teltonika switched libs from uclibs to musl. I could not find any confirmation to endiness being switch as well in this upgrade on the Teltonika SDK Side

Was able to compile Teltonika v6.9 and ZET v0.17.36, but I found couple of missing include files that needed to add to complete it successfully

Steps I did:

  1. Used ubuntu 18.04 as host
  2. Downloaded the 6.9 SDK https://wiki.teltonika-networks.com/gpl/RUT9XX_R_GPL_00.06.09.tar.gz
  3. Follow the procedure to install packages Legacy RUTOS Software Development Kit instructions - Teltonika Networks Wiki
  4. select this option [*] Build the OpenWrt based Toolchain
  5. make -j $nproc
  6. git clone https://github.com/openziti/ziti-tunnel-sdk-c.git; cd ziti-tunnel-sdk-c/
  7. git checkout tags/v0.17.36
  8. add #include <sys/wait.h> to this file ~/ziti-tunnel-sdk-c/programs/ziti-edge-tunnel/netif_driver/linux/utils.c
  9. add #include <stdarg.h> to this file ~/ziti-tunnel-sdk-c/programs/ziti-edge-tunnel/netif_driver/linux/utils.h
  10. mkdir build; cd build/
  11. ran this …/scripts/openwrt-build.sh -s ~/RUTX_R -t mips_34kc
  12. updated this DISABLE_LIBSYSTEMD_FEATURE:BOOL=OFF to ON in CMakeCache.txt
  13. re-ran this …/scripts/openwrt-build.sh -s ~/RUTX_R -t mips_34kc

FYI, Teltonika v7.x is till an issue though

Hi @dariuszSki ,
I have used https://wiki.teltonika-networks.com/gpl/RUTX_R_GPL_00.07.02.tar.gz (ARM) and GitHub - openziti/ziti-tunnel-sdk-c (v.18.04 tag).
I was able to compile it without any issues. I am even able to run ziti-edge-tunnel binary on Teltonika X11 box.

But the issue i faced is while it starts up. It couldn’t do the DNS configurations.
I will be looking into the code to see how it can be fixed. Also, if team wants to know the error messages, i can put it here or create a new issue for it.

Regards,
Sameer

@sameersarkar-tcl are you able to intercept services? If so, how did you compile it? I am able to compile it for RUT950 router, but when I run the binary on the router, the syn packet is forwarded to the tunnel interface but it is dropped due to the fact that the packet length is misread to be too long.

I think the dns issue is related to the fact that ZET is looking for the expected resolvers to be there and they are not present and it fails. If I understand the code correctly, it basically checks what resolvers are available by running commands. Not sure if this can work without the code update, but IP services still should work. Here is the code for DNS resolvers commands I found in the code, when I was looking at this myself
file : ziti-tunnel-sdk-c/tun.c at main · openziti/ziti-tunnel-sdk-c · GitHub

static struct dns_cmd dns_cmds[] = {
            {
                    .path = "/usr/bin/resolvectl",
                    .update_fn = dns_update_resolvectl,
            },
            {
                    .path = "/usr/bin/systemd-resolve",
                    .update_fn = dns_update_systemd_resolve,
            },
            {
                    .path = "/usr/sbin/resolvconf",
                    .update_fn = dns_update_resolvconf
            },
            {0}
    };

@dariuszSki ,
I had to manually update /etc/hosts file, which is a hack which i tried to see if the functionality is working or not.

Yes, this is the exact issue we landed up since this code was pushed in. Earlier it was working, say tags/v0.15.36. There was this option of dnsmasq which was available earlier. Now it is discontinued.

Now i am looking for those same pieces old code for dnsmasq implementation, place it again and see if it works.
Because OpenWRT don’t support resolvectl or systemd-resolve in any way.

Regards,
Sameer Sarkar

Hi Sameer,
Can you confirm if the IP intercept is working or not? the IP intercept has nothing to do with resolver. If that does not work, then there is a bigger issue.

Thanks,

James

Hi James,
FYI, For the enterprise version we are still using the older versions of ziti-edge-tunnel, even for customer deployments.
Using openziti opensource, I didn’t try IP intercept till now. I will do it now.
It will be the first time trying with combination of openziti + teltonika box.
So before i proceed, please confirm below commands are correct to create a service and bind it.

ziti@ziticontroller:/openziti$ ziti edge create config “ipintercept.test.dial” intercept.v1 ‘{“protocols”:[“tcp”],“addresses”:[“100.64.219.150”], “portRanges”:[{“low”:80, “high”:80}]}’
New config ipintercept.test.dial created with id: BHM9o1J.x
ziti@ziticontroller:/openziti$ ziti edge create config “ipintercept.test.bind” host.v1 ‘{“protocol”:“tcp”, “address”:“100.64.219.150”,“port”:8000}’
New config ipintercept.test.bind created with id: gJi9o1J.NZ
ziti@ziticontroller:/openziti$ ziti edge create service " ipintercept.web.test.service" --configs “ipintercept.test.bind,ipintercept.test.dial”
New service ipintercept.web.test.service created with id: wRSWoHk.xZ
ziti@ziticontroller:/openziti$
ziti@ziticontroller:/openziti$ ziti edge delete service " ipintercept.web.test.service"
delete of service with id wRSWoHk.xZ: OK
ziti@ziticontroller:/openziti$
ziti@ziticontroller:/openziti$
ziti@ziticontroller:/openziti$ ziti edge create service “ipintercept.web.test.service” --configs “ipintercept.test.bind,ipintercept.test.dial”
New service ipintercept.web.test.service created with id: 07Gzo1JSNZ
ziti@ziticontroller:/openziti$
ziti@ziticontroller:/openziti$ ziti edge create service-policy ipintercept.web.test.service.bind.blue Bind --service-roles “@ipintercept.web.test.service” --identity-roles “@ziti-private-blue
New service policy ipintercept.web.test.service.bind.blue created with id: bNTSo1kSN
ziti@ziticontroller:/openziti$ ziti edge create service-policy ipintercept.web.test.service.dial.teltonikaX11 Dial --service-roles “@ipintercept.web.test.service” --identity-roles “@TeltonikaX11
New service policy ipintercept.web.test.service.dial.teltonikaX11 created with id: PzhEoHJSNZ
ziti@ziticontroller:/openziti$
ziti@ziticontroller:/openziti$

Teltonika box - ziti-edge-tunnel log

[ 540.662] INFO ziti-sdk:ziti.c:1314 ziti_set_api_session() ztx[0] api session set, setting api_session_timer to 540s
[ 896.643] INFO tunnel-cbs:ziti_tunnel_cbs.c:411 new_ziti_intercept() creating intercept for service[ipintercept.web.test.service] with intercept.v1 = {“addresses”:[“100.64.219.150”],“portRanges”:[{“high”:80,“low”:80}],“protocols”:[“tcp”]}
[ 896.643] INFO tunnel-cbs:ziti_tunnel_ctrl.c:684 on_service() starting intercepting for service[ipintercept.web.test.service]
[ 896.643] INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1185 on_event() =============== service event (added) - ipintercept.web.test.service:07Gzo1JSNZ ===============
[ 924.046] INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.219.150

Fetching the app in the box.
root@Teltonika-RUTX11:~#
root@Teltonika-RUTX11:~# curl http://100.64.219.150

Hello World


                                       ##         .
                                 ## ## ##        ==
                              ## ## ## ## ##    ===
                           /""""""""""""""""\___/ ===
                      ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
                           \______ o          _,/
                            \      \       _,'
                             `'--.._\..--''

root@Teltonika-RUTX11:~#
root@Teltonika-RUTX11:~#

It seems to be working. Unless you spot any error in configuration.

But i need DNS resolution to work as well, Need your help as well to resolve it, if you have a solution.

Regards,
Sameer

@sameersarkar-tcl It looks like it is working for you. I see that your device is arm based and the one I am using is mips based. How did you compile your code?

Hi @dariuszSki ,
For MIPS Teltonika RUT9X (RUTOS) version i was able to build it using below steps; You can see that i am using 7.x Teltonika SDK version and ziti-edge-tunnel (v0.18.4 tag).
In my case i used an ubuntu:18.04 docker container to build it in isolation.
I created a “build” non-root sudo user, as i have found that SDK don’t compile for few modules if root user is used.
NOTE: Ignore the numbers before the command they are from “history” output :wink:
2 wget https://wiki.teltonika-networks.com/gpl/RUT9_R_GPL_00.07.02.tar.gz
3 tar -xf RUT9_R_GPL_00.07.02.tar.gz
5 cd rutos-ath79-rut9-gpl/
6 vim README → Follow README to build the toolchain
7 sudo apt install build-essential ccache ecj fastjar file g++ gawk; sudo apt install -y gettext git java-propose-classpath jq libelf-dev libffi-dev; sudo apt install -y libncurses5-dev libncursesw5-dev libssl1.0-dev libtool; sudo apt install -y node-gyp nodejs nodejs-dev npm python python2.7-dev python3; sudo apt install -y python3-distutils python3-setuptools rsync subversion swig; sudo apt install -y time u-boot-tools unzip wget xsltproc zlib1g-dev;
11 ./scripts/feeds update -a
14 make -j60 → I had a multicore blade server with huge procs to consume hence used as much as possible. Once complete the toolchain was available.
15 sudo apt install -y git
16 git clone GitHub - openziti/ziti-tunnel-sdk-c
17 cd ziti-tunnel-sdk-c/
21 git checkout tags/v0.18.4
25 mkdir ~/tmp
26 cd ~/tmp
27 sudo apt-get install -y build-essential libssl-dev <---- Following steps are to install latest cmake
28 cd wget https://github.com/Kitware/CMake/releases/download/v3.23.1/cmake-3.23.1.tar.gz
29 wget https://github.com/Kitware/CMake/releases/download/v3.23.1/cmake-3.23.1.tar.gz
30 tar -zxvf cmake-3.23.1.tar.gz
31 cd cmake-3.23.1
32 ./bootstrap
33 make
34 sudo make install
45 sudo apt install -y doxygen pkg-config libsystemd-dev ← I found that ziti didn’t compile if these are not installed
42 export TOOLCHAIN=/home/build/rutos-ath79-rut9-gpl/staging_dir/toolchain-mips_24kc_gcc-8.4.0_musl
43 cd ~/ziti-tunnel-sdk-c/
44 cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=toolchains/mips-openwrt.cmake .
54 make
55 ls -ltr
56 cd programs/ziti-edge-tunnel/ ← Found the binary here

@sameersarkar-tcl can you confirm that you were able to pass traffic successfully using this binary?

@dariuszSki , I don’t have a box to test this binary with me at the moment. But if i get hold of one soon, will test it and let you know.

I tried the same steps as this and hitting this error.

[ 53%] Linking C shared library libziti.so

/home/build/rutos-ath79-rut2-gpl/staging_dir/toolchain-mips_24kc_gcc-8.4.0_musl/lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/bin/ld: ../../libsodium-build/lib/libsodium.a(libsodium_la-hash_sha512.o): Relocations in generic ELF (EM: 62)

/home/build/rutos-ath79-rut2-gpl/staging_dir/toolchain-mips_24kc_gcc-8.4.0_musl/lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/bin/ld: ../../libsodium-build/lib/libsodium.a: error adding symbols: file in wrong format

collect2: error: ld returned 1 exit status

_deps/ziti-sdk-c-build/library/CMakeFiles/ziti_dll.dir/build.make:405: recipe for target '_deps/ziti-sdk-c-build/library/libziti.so' failed

make[2]: *** [_deps/ziti-sdk-c-build/library/libziti.so] Error 1

CMakeFiles/Makefile2:1259: recipe for target '_deps/ziti-sdk-c-build/library/CMakeFiles/ziti_dll.dir/all' failed

make[1]: *** [_deps/ziti-sdk-c-build/library/CMakeFiles/ziti_dll.dir/all] Error 2

Makefile:155: recipe for target 'all' failed

make: *** [all] Error 2

build@952014f02f9a:~/ziti-tunnel-sdk-c$ client_loop: send disconnect: Broken pipe

Also

GitHub - openziti/ziti-openwrt: Ziti package feed for OpenWRT seems to be empty as well

@pearl Welcome to OpenZiti discussion!

can you please try with vcpkg-less-2 branch and follow instructions in ./docs/openwrt/BUILDING.md?

Thanks, that worked, thanks for your quick help.

ubuntu@ip-172-31-31-105:~/ziti-tunnel-sdk-c$ ls -lrt programs/ziti-edge-tunnel/

total 5460

-rw-rw-r-- 1 ubuntu ubuntu 1909 Dec 19 19:12 config-utils.c

-rw-rw-r-- 1 ubuntu ubuntu 3164 Dec 19 19:12 CMakeLists.txt

-rw-rw-r-- 1 ubuntu ubuntu 307 Dec 19 19:12 wintun.cmake

-rw-rw-r-- 1 ubuntu ubuntu 13923 Dec 19 19:12 windows-service.c

-rw-rw-r-- 1 ubuntu ubuntu 21364 Dec 19 19:12 windows-scripts.c

drwxrwxr-x 2 ubuntu ubuntu 4096 Dec 19 19:12 windows

drwxrwxr-x 6 ubuntu ubuntu 4096 Dec 19 19:12 package

drwxrwxr-x 5 ubuntu ubuntu 4096 Dec 19 19:12 netif_driver

-rw-rw-r-- 1 ubuntu ubuntu 30843 Dec 19 19:12 instance.c

-rw-rw-r-- 1 ubuntu ubuntu 6674 Dec 19 19:12 instance-config.c

drwxrwxr-x 4 ubuntu ubuntu 4096 Dec 19 19:12 include

-rw-rw-r-- 1 ubuntu ubuntu 118896 Dec 19 19:12 ziti-edge-tunnel.c

-rw-rw-r-- 1 ubuntu ubuntu 2751 Dec 19 22:53 cmake_install.cmake

-rw-rw-r-- 1 ubuntu ubuntu 17703 Dec 19 22:53 Makefile

drwxrwxr-x 4 ubuntu ubuntu 4096 Dec 19 22:53 CMakeFiles

-rwxrwxr-x 1 ubuntu ubuntu 5327832 Dec 19 22:55 ziti-edge-tunnel

ubuntu@ip-172-31-31-105:~/ziti-tunnel-sdk-c$

1 Like