Not able to intercept traffic in Teltonika X11 OpenWRT box after re-introducing DNSMASQ supporting code

Dear Team,
I need your help in enabling ziti-edge-tunnel on Teltonika X11/950 openwrt using DNSMasq.

Earlier it used to work on this boxes and currently we are using v0.17.23 code. As you may be knowing that it stopped working after few updates after introduction of resolvers (systemd-resolve and resolvectl).
I took it up to re-introduce it again, but carefully put it under “OPENWRT” compiler switch.

I have pushed my code in my fork at Modified code to support DNSMASQ for OpenWRT devices · sameersarkar-tcl/ziti-tunnel-sdk-c@1d3d82c · GitHub

It did work one time, but post that it couldn’t find the interceptor within. Sharing both logs for your reference.
Could you help me, review the code, as i may sure have missed out with few pieces of code or there could be a major impact and find the problematic place.
Also, I landed up in messing up the IP intercept here. So need help there as well.

Some command outputs:

root@Teltonika-RUTX11:/tmp/hosts# 
root@Teltonika-RUTX11:/tmp/hosts# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.46.252  0.0.0.0         UG    4      0        0 wlan0
100.64.0.0      *               255.192.0.0     U     0      0        0 tun0
100.64.0.2      *               255.255.255.255 UH    0      0        0 tun0
100.64.0.10     *               255.255.255.255 UH    0      0        0 tun0
100.64.219.150  *               255.255.255.255 UH    0      0        0 tun0
192.168.1.0     *               255.255.255.0   U     0      0        0 br-lan
192.168.46.0    *               255.255.255.0   U     4      0        0 wlan0
203.160.138.65  192.168.46.252  255.255.255.255 UGH   0      0        0 wlan0
203.160.138.70  192.168.46.252  255.255.255.255 UGH   0      0        0 wlan0
root@Teltonika-RUTX11:/tmp/hosts# 
root@Teltonika-RUTX11:/tmp/hosts# 
root@Teltonika-RUTX11:/tmp/hosts# 

root@Teltonika-RUTX11:/tmp/hosts# 
root@Teltonika-RUTX11:/tmp/hosts# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.46.252  0.0.0.0         UG    4      0        0 wlan0
100.64.0.0      *               255.192.0.0     U     0      0        0 tun0
100.64.0.2      *               255.255.255.255 UH    0      0        0 tun0
100.64.0.10     *               255.255.255.255 UH    0      0        0 tun0
100.64.219.150  *               255.255.255.255 UH    0      0        0 tun0
192.168.1.0     *               255.255.255.0   U     0      0        0 br-lan
192.168.46.0    *               255.255.255.0   U     4      0        0 wlan0
203.160.138.65  192.168.46.252  255.255.255.255 UGH   0      0        0 wlan0
203.160.138.70  192.168.46.252  255.255.255.255 UGH   0      0        0 wlan0
root@Teltonika-RUTX11:/tmp/hosts# 
root@Teltonika-RUTX11:/tmp/hosts# 
root@Teltonika-RUTX11:/tmp/hosts# 

Working trace log from code:

[       80.976] VERBOSE ziti-sdk:posture.c:191 ziti_send_posture_data() ztx[0] starting to send posture data
[       80.976]   DEBUG ziti-sdk:posture.c:204 ziti_send_posture_data() ztx[0] posture checks must_send set to TRUE, new_session_id[FALSE], must_send_every_time[TRUE], new_controller_instance[FALSE]
[       80.976] VERBOSE ziti-sdk:posture.c:229 ziti_send_posture_data() ztx[0] checking posture queries on 1 service(s)
[       80.976]   DEBUG ziti-sdk:posture.c:519 ziti_pr_send_bulk() ztx[0] no change in posture data, not sending
[       82.958]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       82.958]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       82.958]    INFO tunnel-sdk:intercept.c:61 lookup_intercept_by_address() Entering this function
[       82.958]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       82.958]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       82.958]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       82.958]    INFO tunnel-sdk:tunnel_tcp.c:340 recv_tcp() 3. Flowing from here
[       82.958]   DEBUG tunnel-sdk:tunnel_tcp.c:109 new_tcp_pcb() snd_wnd: 61690, snd_snd_max: 61690, mss: 1460
[       82.958]   DEBUG tunnel-sdk:tunnel_tcp.c:392 recv_tcp() intercepted address[tcp:100.64.0.10:80] client[tcp:100.64.0.1:58658] service[basic.web.test.service]
[       82.958] VERBOSE tunnel-cbs:ziti_tunnel_cbs.c:298 ziti_sdk_c_dial() ziti_dial(name=basic.web.test.service)
[       82.958]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:364 ziti_sdk_c_dial() service[basic.web.test.service] app_data_json[172]='{"connType":null,"dst_protocol":"tcp","dst_hostname":"simple.web.test","dst_ip":"100.64.0.10","dst_port":"80","src_protocol":"tcp","src_ip":"100.64.0.1","src_port":"58658"}'
[       82.958] VERBOSE ziti-sdk:connect.c:101 conn_set_state() conn[0.2] transitioning Initial => Connecting
[       82.958] VERBOSE ziti-sdk:posture.c:191 ziti_send_posture_data() ztx[0] starting to send posture data
[       82.958]   DEBUG ziti-sdk:posture.c:204 ziti_send_posture_data() ztx[0] posture checks must_send set to TRUE, new_session_id[FALSE], must_send_every_time[TRUE], new_controller_instance[FALSE]
[       82.958] VERBOSE ziti-sdk:posture.c:229 ziti_send_posture_data() ztx[0] checking posture queries on 1 service(s)
[       82.958]   DEBUG ziti-sdk:posture.c:519 ziti_pr_send_bulk() ztx[0] no change in posture data, not sending
[       82.958]   DEBUG ziti-sdk:connect.c:514 process_connect() conn[0.2] starting Dial connection for service[basic.web.test.service] with session[cl43yb6vy7lyz8v86lvqynm3n]
[       82.958]   DEBUG ziti-sdk:connect.c:409 ziti_connect() conn[0.2] selected ch[ziti-edge-router@tls://ziti-edge-router:3022] for best latency(63 ms)
[       82.958]   DEBUG ziti-sdk:connect.c:297 on_channel_connected() conn[0.2] selected ch[ziti-edge-router@tls://ziti-edge-router:3022] status[0]
[       82.958]   DEBUG ziti-sdk:channel.c:199 ziti_channel_add_receiver() ch[0] added receiver[2]
[       83.122] VERBOSE ziti-sdk:connect.c:101 conn_set_state() conn[0.2] transitioning Connecting => Connected
[       83.122] VERBOSE tunnel-cbs:ziti_tunnel_cbs.c:94 on_ziti_connect() on_ziti_connect status: 0
[       83.122]   DEBUG tunnel-sdk:ziti_tunnel.c:216 ziti_tunneler_dial_completed() ziti dial succeeded: client[tcp:100.64.0.1:58658] service[basic.web.test.service]
[       83.126] VERBOSE ziti-sdk:connect.c:796 flush_to_client() conn[0.2] 0 bytes available
[       83.126]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       83.126]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       83.126]   DEBUG tunnel-sdk:tunnel_tcp.c:59 on_accept() on_accept: 0
[       83.127] VERBOSE ziti-sdk:connect.c:844 conn_inbound_data_msg() conn[0.2] processing crypto header(24 bytes)
[       83.127] VERBOSE ziti-sdk:connect.c:847 conn_inbound_data_msg() conn[0.2] processed crypto header
[       83.127] VERBOSE ziti-sdk:connect.c:796 flush_to_client() conn[0.2] 0 bytes available
[       83.127]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       83.127]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       83.127] VERBOSE tunnel-sdk:tunnel_tcp.c:134 on_tcp_client_data() status 0 0x322470, state=4(ESTABLISHED) flags=0x101
[       83.128] VERBOSE ziti-sdk:connect.c:796 flush_to_client() conn[0.2] 0 bytes available
[       83.199] VERBOSE ziti-sdk:connect.c:854 conn_inbound_data_msg() conn[0.2] decrypting 228 bytes
[       83.199] VERBOSE ziti-sdk:connect.c:858 conn_inbound_data_msg() conn[0.2] decrypted 211 bytes
[       83.199] VERBOSE ziti-sdk:connect.c:854 conn_inbound_data_msg() conn[0.2] decrypting 456 bytes
[       83.199] VERBOSE ziti-sdk:connect.c:858 conn_inbound_data_msg() conn[0.2] decrypted 439 bytes
[       83.199] VERBOSE ziti-sdk:connect.c:796 flush_to_client() conn[0.2] 650 bytes available
[       83.199]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:127 on_ziti_data() ziti connection sent EOF (ziti_eof=0, tnlr_eof=0)
[       83.199]   DEBUG tunnel-sdk:ziti_tunnel.c:601 ziti_tunneler_close_write() closing write connection: client[tcp:100.64.0.1:58658] service[basic.web.test.service]
[       83.199]   DEBUG tunnel-sdk:tunnel_tcp.c:217 tunneler_tcp_close_write() closing write 0x322470, state=4(ESTABLISHED) flags=0x100
[       83.199]   DEBUG tunnel-sdk:tunnel_tcp.c:223 tunneler_tcp_close_write() closed write 0x322470, state=5(FIN_WAIT_1) flags=0x120
[       83.202]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       83.202]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       83.202]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       83.202]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       83.202] VERBOSE tunnel-sdk:tunnel_tcp.c:134 on_tcp_client_data() status 0 0x322470, state=10(TIME_WAIT) flags=0x123
[       83.202]   DEBUG tunnel-sdk:tunnel_tcp.c:138 on_tcp_client_data() client sent FIN: client=tcp:100.64.0.1:58658, service=basic.web.test.service
[       83.202]   DEBUG tunnel-sdk:tunnel_tcp.c:139 on_tcp_client_data() FIN received 0x322470, state=10(TIME_WAIT) flags=0x123
[       83.202]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:159 ziti_sdk_c_close_write() closing ziti_conn tnlr_eof=0, ziti_eof=1
[       83.202]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:162 ziti_sdk_c_close_write() closing ziti_conn tnlr_eof=1, ziti_eof=1
[       83.204] VERBOSE ziti-sdk:connect.c:101 conn_set_state() conn[0.2] transitioning Connected => Closed

Not Working Logs:

        2.317]   TRACE ziti-sdk:channel.c:548 process_inbound() ch[0] message is complete seq[-1] ct[0002]
[        2.317]    INFO ziti-sdk:channel.c:629 hello_reply_cb() ch[0] connected. EdgeRouter version: v0.25.5|2cd97dc79e5d|2022-05-03T02:47:46Z|linux|amd64
[        2.317]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:798 on_ziti_event() ztx[ TeltonikaX11] router ziti-edge-router@tls://ziti-edge-router:3022 connected
[        5.052]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[        9.513]    INFO tunnel-sdk:tunnel_tcp.c:304 recv_tcp() 2.Flowing from here
[        9.513]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.10:58732->100.64.0.10:80
[        9.513]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[        9.513]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[        9.513]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[        9.513]   TRACE tunnel-sdk:intercept.c:77 lookup_intercept_by_address() Did Not Find matching address
[        9.513]   TRACE tunnel-sdk:intercept.c:88 lookup_intercept_by_address() Matched address and intercept Not Found
[        9.513]   TRACE tunnel-sdk:tunnel_tcp.c:336 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       10.052]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       10.578]    INFO tunnel-sdk:tunnel_tcp.c:304 recv_tcp() 2.Flowing from here
[       10.578]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.10:58732->100.64.0.10:80
[       10.578]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       10.578]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       10.578]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       10.578]   TRACE tunnel-sdk:intercept.c:77 lookup_intercept_by_address() Did Not Find matching address
[       10.578]   TRACE tunnel-sdk:intercept.c:88 lookup_intercept_by_address() Matched address and intercept Not Found
[       10.578]   TRACE tunnel-sdk:tunnel_tcp.c:336 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       11.535]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-api-session/service-updates]
[       11.862]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-api-session/service-updates]
[       11.862]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-api-session/service-updates] in 41300960.000 s
[       11.862] VERBOSE ziti-sdk:ziti.c:1124 check_service_update() ztx[0] not updating: last_update is same previous (2022-06-07T09:19:51.968Z == 2022-06-07T09:19:51.968Z)
[       12.658]    INFO tunnel-sdk:tunnel_tcp.c:304 recv_tcp() 2.Flowing from here
[       12.658]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.10:58732->100.64.0.10:80
[       12.658]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       12.658]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       12.658]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       12.658]   TRACE tunnel-sdk:intercept.c:77 lookup_intercept_by_address() Did Not Find matching address
[       12.658]   TRACE tunnel-sdk:intercept.c:88 lookup_intercept_by_address() Matched address and intercept Not Found
[       12.658]   TRACE tunnel-sdk:tunnel_tcp.c:336 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       15.052]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
^C
root@Teltonika-RUTX11:/tmp# 

Thanks

Hi Sameer,

The fact that the intercept lookup function is coming up empty makes me think this may not be related to your dns/dnsmasq setup at all. Can you share the full log from ziti-edge-tunnel including the messages at startup where the intercepted services are logged?

I’m assuming you have a service that should be intercepted at 100.64.0.10, but we can see that intercept_match_addr isn’t finding a match for that IP so I’d like to dig deeper into that.

@sameersarkar-tcl - Is this the same topic being discussed on https://netfoundry.zendesk.com/agent/tickets/9994 by Anirban?

It would have been great if in this tool i could attach a txt file. Atleast on asthetics point of view the ticket dashboard would look clean. Anyways pasting whole log here.

root@Teltonika-RUTX11:/tmp# ./ziti-edge-tunnel run -v 6 -i TeltonikaX11.json --dns=dnsmasq:/tmp/hosts 
[        0.000]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1834 run() DNS dnsmasq -- AM I coming here ???
[        0.000]    INFO ziti-edge-tunnel:dnsmasq_manager.c:64 get_dnsmasq_manager() smapping_dir = /tmp/hosts
[        0.000]    INFO ziti-edge-tunnel:dnsmasq_manager.c:67 get_dnsmasq_manager() dnsmasq_manager = /tmp/hosts
[        0.000]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1843 run() dnsmasq_manager = /tmp/hosts
[        0.000]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1844 run() DNS = 0xbeed8c6c
[        0.000]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip link set tun0 up) returned 0
[        0.000]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip addr add 100.64.0.1 dev tun0) returned 0
[        0.000]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route add 100.64.0.0/10 dev tun0) returned 0
[        0.000]   ERROR tunnel-sdk:ziti_tunnel.c:92 ziti_tunneler_init() Entering here
[        0.000]    INFO tunnel-sdk:ziti_tunnel.c:68 create_tunneler_ctx() Ziti Tunneler SDK (v0.18.6-local)
[        0.000]   ERROR tunnel-sdk:ziti_tunnel.c:624 run_packet_loop() Entering here
[        0.000]    INFO tunnel-cbs:ziti_dns.c:153 ziti_dns_setup() Entering this function
[        0.000]    INFO tunnel-cbs:ziti_dns.c:147 seed_dns() DNS configured with range 100.64.0.0 - 100.127.255.255
[        0.000]   DEBUG tunnel-sdk:ziti_tunnel.c:433 ziti_tunneler_intercept() intercepting address[udp:100.64.0.2:53] service[ziti:dns-resolver]
[        0.052]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:861 load_ziti_async() attempting to load ziti instance from file[TeltonikaX11.json]
[        0.052]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:868 load_ziti_async() loading ziti instance from /tmp/TeltonikaX11.json
[        0.052]    INFO ziti_log_set_level set log level: ziti_log_lvl=6 &ziti_log_lvl = 0x1dd7e8
[        0.052]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1037 load_id_cb() identity[TeltonikaX11.json] loaded
[        0.052]    WARN ziti-edge-tunnel:instance.c:37 find_tunnel_identity() Identity ztx[TeltonikaX11.json] is not loaded yet or already removed.
[        0.052]   ERROR ziti-edge-tunnel:instance-config.c:121 save_tunnel_status_to_file() Could not save the config file [/var/lib/ziti/config.json] due to semaphore lock not initialized error.
[        0.085]    INFO ziti-sdk:ziti.c:406 ziti_init_async() ztx[0] Ziti C SDK version 0.27.10 @ca8b793(HEAD) starting at (2022-06-07T09:59:17.372)
[        0.085]    INFO ziti-sdk:ziti.c:407 ziti_init_async() ztx[0] using uv_mbed[v0.14.5], tls[mbed TLS 3.1.0]
[        0.085]    INFO ziti-sdk:ziti.c:408 ziti_init_async() ztx[0] Loading from config[TeltonikaX11.json] controller[https://ziti-edge-controller:1280]
[        0.085]    INFO ziti-sdk:ziti_ctrl.c:375 ziti_ctrl_init() ctrl[ziti-edge-controller] ziti controller client initialized
[        0.085]   DEBUG ziti-sdk:ziti.c:432 ziti_init_async() ztx[0] using metrics interval: 0
[        0.085]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/version]
[        0.085]   DEBUG ziti-sdk:ziti.c:241 ziti_set_unauthenticated() ztx[0] setting api_session_state[0] to 0
[        0.085]   DEBUG ziti-sdk:ziti_ctrl.c:228 ziti_ctrl_clear_api_session() ctrl[ziti-edge-controller] clearing api session token for ziti_controller
[        0.085]   DEBUG ziti-sdk:ziti.c:836 ziti_re_auth() ztx[0] re-auth executing, transitioning to unauthenticated
[        0.085]   DEBUG ziti-sdk:ziti.c:241 ziti_set_unauthenticated() ztx[0] setting api_session_state[0] to 0
[        0.085]   DEBUG ziti-sdk:ziti_ctrl.c:228 ziti_ctrl_clear_api_session() ctrl[ziti-edge-controller] clearing api session token for ziti_controller
[        0.085]   DEBUG ziti-sdk:ziti.c:272 is_api_session_expired() ztx[0] is_api_session_expired[TRUE] - api_session is null
[        0.085]    INFO ziti-sdk:ziti.c:792 ziti_re_auth_with_cb() ztx[0] starting to re-auth with ctlr[https://ziti-edge-controller:1280] api_session_status[0] api_session_expired[TRUE]
[        0.085]   DEBUG ziti-sdk:ziti.c:234 ziti_set_auth_started() ztx[0] setting api_session_state[0] to 1
[        0.085]   DEBUG ziti-sdk:ziti.c:302 ziti_stop_api_session_refresh() ztx[0] ziti_stop_api_session_refresh: stopping api session refresh
[        0.085]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting POST[/authenticate?method=cert]
[        0.880]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/version]
[        0.880]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/version] in 41296544.000 s
[        0.880]    INFO ziti-sdk:ziti.c:1441 version_cb() ztx[0] connected to controller https://ziti-edge-controller:1280 version v0.25.5(2cd97dc79e5d 2022-05-03T02:47:46Z)
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers POST[/authenticate?method=cert]
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed POST[/authenticate?method=cert] in 41296544.000 s
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:241 ctrl_login_cb() ctrl[ziti-edge-controller] authenticated successfully session[cl43zpodj7ntc8v86s00z77fe]
[        0.990]   DEBUG ziti-sdk:ziti.c:1357 api_session_cb() ztx[0] logged in successfully => api_session[cl43zpodj7ntc8v86s00z77fe]
[        0.990]   TRACE ziti-sdk:ziti.c:1296 ziti_set_api_session() ztx[0] API supports cached_last_activity_at
[        0.990]   DEBUG ziti-sdk:ziti.c:1307 ziti_set_api_session() ztx[0] ziti api session expires in 600 seconds
[        0.990]    INFO ziti-sdk:ziti.c:1331 ziti_set_api_session() ztx[0] api session set, setting api_session_timer to 540s
[        0.990]   DEBUG ziti-sdk:ziti.c:307 ziti_schedule_api_session_refresh() ztx[0] ziti_schedule_api_session_refresh: scheduling api session refresh: -1091746792ms
[        0.990]   DEBUG ziti-sdk:ziti.c:266 ziti_set_fully_authenticated() ztx[0] setting api_session_state[1] to 3
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-identity]
[        0.990] VERBOSE ziti-sdk:ziti.c:1246 session_post_auth_query_cb() ztx[0] post auth query callback starting with status[OK]
[        0.990] VERBOSE ziti-sdk:ziti.c:1248 session_post_auth_query_cb() ztx[0] transitioning to fully authenticated
[        0.990]   DEBUG ziti-sdk:ziti.c:266 ziti_set_fully_authenticated() ztx[0] setting api_session_state[3] to 3
[        0.990]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:724 on_ziti_event() ziti_ctx[ TeltonikaX11] connected to controller
[        0.990]   DEBUG tunnel-sdk:ziti_tunnel.c:118 ziti_tunneler_exclude_route() excluding ziti-edge-controller from tunneler intercept
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.46.252 dev wlan0 proto static src 192.168.46.199 metric 4 '
[        0.990]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.65 via 192.168.46.252) returned 0
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.990]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.46.252 dev wlan0 proto static src 192.168.46.199 metric 4 '
[        0.990]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.65 via 192.168.46.252) returned 0
[        0.990]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1067 on_event() ztx[TeltonikaX11.json] context event : status is OK
[        0.990]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:1119 on_event() ztx[TeltonikaX11.json] controller connected
[        0.990]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"identity","Action":"added","Fingerprint":"TeltonikaX11","Id":{"Name":" TeltonikaX11","Identifier":"TeltonikaX11.json","FingerPrint":"TeltonikaX11","Active":true,"Loaded":true,"Config":{"ztAPI":"https://ziti-edge-controller:1280"},"ControllerVersion":"v0.25.5","IdFileStatus":true,"MfaEnabled":false,"MfaNeeded":false,"Metrics":{"Up":0,"Down":0},"MfaMinTimeout":0,"MfaMaxTimeout":0,"MfaMinTimeoutRem":0,"MfaMaxTimeoutRem":0,"MinTimeoutRemInSvcEvent":0,"MaxTimeoutRemInSvcEvent":0,"Deleted":false,"Notified":false}}
[        0.990]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"controller","Action":"connected","Identifier":"TeltonikaX11.json","Fingerprint":"TeltonikaX11"}
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-identity]
[        0.990]   DEBUG ziti-sdk:ziti.c:1263 session_post_auth_query_cb() ztx[0] refresh_interval set to 10 seconds
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-api-session/service-updates]
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:735 ctrl_paging_req() ctrl[ziti-edge-controller] starting paging request GET[/current-identity/edge-routers]
[        0.990] VERBOSE ziti-sdk:ziti_ctrl.c:740 ctrl_paging_req() ctrl[ziti-edge-controller] requesting /current-identity/edge-routers?limit=25&offset=0
[        0.990]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-identity/edge-routers?limit=25&offset=0]
[        1.128]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-identity]
[        1.128]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-identity] in 41296544.000 s
[        1.205]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-identity]
[        1.205]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-identity] in 41296544.000 s
[        1.288]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-api-session/service-updates]
[        1.288]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-api-session/service-updates] in 41296544.000 s
[        1.288] VERBOSE ziti-sdk:ziti.c:1119 check_service_update() ztx[0] ztx last_update = 2022-06-07T09:19:51.968Z
[        1.288]   DEBUG ziti-sdk:ziti_ctrl.c:735 ctrl_paging_req() ctrl[ziti-edge-controller] starting paging request GET[/services]
[        1.288] VERBOSE ziti-sdk:ziti_ctrl.c:740 ctrl_paging_req() ctrl[ziti-edge-controller] requesting /services?limit=25&offset=0
[        1.288]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/services?limit=25&offset=0]
[        1.372]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-identity/edge-routers?limit=25&offset=0]
[        1.372]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-identity/edge-routers?limit=25&offset=0] in 41296544.000 s
[        1.372]   DEBUG ziti-sdk:ziti_ctrl.c:324 ctrl_body_cb() ctrl[ziti-edge-controller] received 1/1 for paging request GET[/current-identity/edge-routers]
[        1.372]   DEBUG ziti-sdk:ziti_ctrl.c:336 ctrl_body_cb() ctrl[ziti-edge-controller] completed paging request GET[/current-identity/edge-routers] in 0.000 s
[        1.372]   TRACE ziti-sdk:ziti.c:1194 edge_routers_cb() ztx[0] connecting to ziti-edge-router(tls://ziti-edge-router:3022)
[        1.372]    INFO ziti-sdk:channel.c:219 new_ziti_channel() ch[0] (ziti-edge-router@tls://ziti-edge-router:3022) new channel for ztx[0] identity[ TeltonikaX11]
[        1.372]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:794 on_ziti_event() ztx[ TeltonikaX11] added edge router ziti-edge-router@tls://ziti-edge-router:3022@ziti-edge-router
[        1.372]   DEBUG tunnel-sdk:ziti_tunnel.c:118 ziti_tunneler_exclude_route() excluding ziti-edge-router from tunneler intercept
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.46.252 dev wlan0 proto static src 192.168.46.199 metric 4 '
[        1.372]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.70 via 192.168.46.252) returned 0
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.372]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.46.252 dev wlan0 proto static src 192.168.46.199 metric 4 '
[        1.372]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.70 via 192.168.46.252) returned 0
[        1.372]    INFO ziti-sdk:channel.c:733 reconnect_channel() ch[0] reconnecting NOW
[        1.404]   DEBUG ziti-sdk:channel.c:704 reconnect_cb() ch[0] connecting to ziti-edge-router:3022
[        1.535]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/services?limit=25&offset=0]
[        1.535]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/services?limit=25&offset=0] in 41296544.000 s
[        1.535]   DEBUG ziti-sdk:ziti_ctrl.c:324 ctrl_body_cb() ctrl[ziti-edge-controller] received 1/1 for paging request GET[/services]
[        1.535]   DEBUG ziti-sdk:ziti_ctrl.c:336 ctrl_body_cb() ctrl[ziti-edge-controller] completed paging request GET[/services] in 0.000 s
[        1.535] VERBOSE ziti-sdk:ziti.c:971 update_services() ztx[0] scheduling service refresh 10 seconds from now
[        1.535] VERBOSE ziti-sdk:ziti.c:992 update_services() ztx[0] processing service updates
[        1.535]   DEBUG ziti-sdk:ziti.c:1069 update_services() ztx[0] sending service event 1 added, 0 removed, 0 changed
[        1.535]   DEBUG tunnel-cbs:ziti_tunnel_ctrl.c:681 on_service() service[basic.web.test.service]
[        1.535]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:508 ziti_sdk_c_on_service() Entering here
[        1.535]    INFO tunnel-cbs:ziti_tunnel_cbs.c:411 new_ziti_intercept() creating intercept for service[basic.web.test.service] with intercept.v1 = {"addresses":["simple.web.test"],"portRanges":[{"high":80,"low":80}],"protocols":["tcp"]}
[        1.535]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:527 ziti_sdk_c_on_service() I am here
[        1.535]    INFO tunnel-cbs:ziti_tunnel_cbs.c:447 new_intercept_ctx() Entering this function
[        1.535]    INFO tunnel-cbs:ziti_tunnel_cbs.c:468 new_intercept_ctx() Entering this function at INTERCEPT_CFG_V1
[        1.535]    INFO tunnel-cbs:ziti_tunnel_cbs.c:469 new_intercept_ctx() Halla bol
[        1.535]    INFO tunnel-cbs:ziti_dns.c:349 ziti_dns_register_hostname() Entering this function simple.web.test
[        1.535]    INFO tunnel-cbs:ziti_dns.c:257 new_ipv4_entry() registered DNS entry simple.web.test -> 100.64.0.10
[        1.535]    INFO tunnel-sdk:ziti_tunnel.c:334 parse_address() Entered if dns is true & got addr->str = 100.64.0.10
[        1.535]    INFO ziti-edge-tunnel:dnsmasq_manager.c:41 apply_address() Entered this function simple.web.test: 100.64.0.10
[        1.535]    INFO ziti-edge-tunnel:dnsmasq_manager.c:58 apply_address() successfully written the file /tmp/hosts/simple.web.test
[        1.535]   DEBUG tunnel-sdk:ziti_tunnel.c:433 ziti_tunneler_intercept() intercepting address[tcp:100.64.0.10:80] service[basic.web.test.service]
[        1.535]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:538 ziti_sdk_c_on_service() 2. I am here
[        1.535]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:684 on_service() starting intercepting for service[basic.web.test.service]
[        1.535] VERBOSE ziti-edge-tunnel:ziti-edge-tunnel.c:1136 on_event() =============== ztx[TeltonikaX11.json] service event ===============
[        1.535]   TRACE ziti-edge-tunnel:instance.c:215 setTunnelPostureDataTimeout() Posture Query set returned a Dial policy: zIxXN82-., is_passing 1
[        1.535]   DEBUG ziti-edge-tunnel:instance.c:257 setTunnelPostureDataTimeout() service[basic.web.test.service] timeout=-1 timeoutRemaining=-1
[        1.535]   TRACE ziti-edge-tunnel:instance.c:295 setTunnelServiceAddress() intercept.v1: {"addresses":["simple.web.test"],"portRanges":[{"high":80,"low":80}],"protocols":["tcp"]}
[        1.535]   TRACE ziti-edge-tunnel:instance.c:276 to_address() Hostname: simple.web.test
[        1.535]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1192 on_event() =============== service event (added) - basic.web.test.service:mQqOfzJSxZ ===============
[        1.535]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"bulkservice","Action":"updated","Identifier":"TeltonikaX11.json","Fingerprint":"TeltonikaX11","AddedServices":[{"Id":"mQqOfzJSxZ","Name":"basic.web.test.service","Protocols":["tcp"],"Addresses":[{"IsHost":true,"HostName":"simple.web.test","Prefix":0}],"Ports":[{"High":80,"Low":80}],"OwnsIntercept":true,"IsAccessible":true,"Timeout":-1,"TimeoutRemaining":-1}],"RemovedServices":[]}
[        1.535]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"identity","Action":"updated","Fingerprint":"TeltonikaX11","Id":{"Name":" TeltonikaX11","Identifier":"TeltonikaX11.json","FingerPrint":"TeltonikaX11","Active":true,"Loaded":true,"Config":{"ztAPI":"https://ziti-edge-controller:1280"},"ControllerVersion":"v0.25.5","IdFileStatus":true,"MfaEnabled":false,"MfaNeeded":false,"Services":[{"Id":"mQqOfzJSxZ","Name":"basic.web.test.service","Protocols":["tcp"],"Addresses":[{"IsHost":true,"HostName":"simple.web.test","Prefix":0}],"Ports":[{"High":80,"Low":80}],"OwnsIntercept":true,"IsAccessible":true,"Timeout":-1,"TimeoutRemaining":-1}],"Metrics":{"Up":0,"Down":0},"MfaMinTimeout":-1,"MfaMaxTimeout":-1,"MfaMinTimeoutRem":-1,"MfaMaxTimeoutRem":-1,"MinTimeoutRemInSvcEvent":-1,"MaxTimeoutRemInSvcEvent":-1,"ServiceUpdatedTime":"2022-06-07T09:59:18.859043Z","Deleted":false,"Notified":false}}
[        2.247]   DEBUG ziti-sdk:channel.c:854 on_channel_connect_internal() ch[0] connected
[        2.247]   TRACE ziti-sdk:channel.c:371 ziti_channel_send_for_reply() ch[0] => ct[0000] seq[0] len[34]
[        2.249]   TRACE ziti-sdk:channel.c:778 on_write() on_write(0x277dc70,0)
[        2.317]   TRACE ziti-sdk:channel.c:841 on_channel_data() ch[0] on_data [len=114]
[        2.317]   TRACE ziti-sdk:channel.c:527 process_inbound() ch[0] <= ct[0002] seq[-1] len[0] hdrs[94]
[        2.317]   TRACE ziti-sdk:channel.c:537 process_inbound() ch[0] completing msg seq[-1] body+hrds=0+94, in_offset=0, want=94, got=94
[        2.317]   TRACE ziti-sdk:channel.c:548 process_inbound() ch[0] message is complete seq[-1] ct[0002]
[        2.317]    INFO ziti-sdk:channel.c:629 hello_reply_cb() ch[0] connected. EdgeRouter version: v0.25.5|2cd97dc79e5d|2022-05-03T02:47:46Z|linux|amd64
[        2.317]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:798 on_ziti_event() ztx[ TeltonikaX11] router ziti-edge-router@tls://ziti-edge-router:3022 connected
[        5.052]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[        9.513]    INFO tunnel-sdk:tunnel_tcp.c:304 recv_tcp() 2.Flowing from here
[        9.513]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.10:58732->100.64.0.10:80
[        9.513]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[        9.513]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[        9.513]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[        9.513]   TRACE tunnel-sdk:intercept.c:77 lookup_intercept_by_address() Did Not Find matching address
[        9.513]   TRACE tunnel-sdk:intercept.c:88 lookup_intercept_by_address() Matched address and intercept Not Found
[        9.513]   TRACE tunnel-sdk:tunnel_tcp.c:336 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       10.052]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       10.578]    INFO tunnel-sdk:tunnel_tcp.c:304 recv_tcp() 2.Flowing from here
[       10.578]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.10:58732->100.64.0.10:80
[       10.578]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       10.578]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       10.578]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       10.578]   TRACE tunnel-sdk:intercept.c:77 lookup_intercept_by_address() Did Not Find matching address
[       10.578]   TRACE tunnel-sdk:intercept.c:88 lookup_intercept_by_address() Matched address and intercept Not Found
[       10.578]   TRACE tunnel-sdk:tunnel_tcp.c:336 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       11.535]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-api-session/service-updates]
[       11.862]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-api-session/service-updates]
[       11.862]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-api-session/service-updates] in 41300960.000 s
[       11.862] VERBOSE ziti-sdk:ziti.c:1124 check_service_update() ztx[0] not updating: last_update is same previous (2022-06-07T09:19:51.968Z == 2022-06-07T09:19:51.968Z)
[       12.658]    INFO tunnel-sdk:tunnel_tcp.c:304 recv_tcp() 2.Flowing from here
[       12.658]   TRACE tunnel-sdk:tunnel_tcp.c:325 recv_tcp() received segment 100.64.0.10:58732->100.64.0.10:80
[       12.658]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       12.658]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       12.658]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       12.658]   TRACE tunnel-sdk:intercept.c:77 lookup_intercept_by_address() Did Not Find matching address
[       12.658]   TRACE tunnel-sdk:intercept.c:88 lookup_intercept_by_address() Matched address and intercept Not Found
[       12.658]   TRACE tunnel-sdk:tunnel_tcp.c:336 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       15.052]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
^C
root@Teltonika-RUTX11:/tmp#

EDITED BY CLINT - added triple ticks around the log

I edited your post @sameersarkar-tcl . I used triple ticks to enclose the log in a code block which looks a lot nicer :slight_smile:

they look like this
and makes a 'code block'

image

Thanks Clint, Really appreciate it.

You’re welcome! I do agree though, there are times when being able to attach a file would be much nicer.

Thanks Sameer. It sure does seem like your intercepted packet should get matched up with your service. Can you add the following lines to the address_match function and let me know what you see?

  bool address_match(const ip_addr_t *addr, const address_list_t *addresses) {
      address_t *a;
+     char ip[64], range[64];
      STAILQ_FOREACH(a, addresses, entries) {
+         ipaddr_ntoa_r(addr, ip, sizeof(ip));
+         ipaddr_ntoa_r(&a->ip, range, sizeof(range));
+         TNL_LOG(INFO, "comparing intercepted ip %s to %s", ip, range);
          if (IP_IS_V4(&a->ip) && a->prefix_len != 32) {

Thanks,
-Shawn

Thank You Shawn,
Here is the log.

root@Teltonika-RUTX11:/tmp# ./ziti-edge-tunnel run -v 6 -i TeltonikaX11.json --dns=dnsmasq:/tmp/hosts 
[        0.000]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1834 run() DNS dnsmasq -- AM I coming here ???
[        0.000]    INFO ziti-edge-tunnel:dnsmasq_manager.c:64 get_dnsmasq_manager() smapping_dir = /tmp/hosts
[        0.000]    INFO ziti-edge-tunnel:dnsmasq_manager.c:67 get_dnsmasq_manager() dnsmasq_manager = /tmp/hosts
[        0.000]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1843 run() dnsmasq_manager = /tmp/hosts
[        0.000]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1844 run() DNS = 0xbed72c6c
[        0.000]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip link set tun0 up) returned 0
[        0.000]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip addr add 100.64.0.1 dev tun0) returned 0
[        0.000]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route add 100.64.0.0/10 dev tun0) returned 0
[        0.000]   ERROR tunnel-sdk:ziti_tunnel.c:92 ziti_tunneler_init() Entering here
[        0.000]    INFO tunnel-sdk:ziti_tunnel.c:68 create_tunneler_ctx() Ziti Tunneler SDK (v0.18.6-local)
[        0.000]   ERROR tunnel-sdk:ziti_tunnel.c:624 run_packet_loop() Entering here
[        0.000]    INFO tunnel-cbs:ziti_dns.c:153 ziti_dns_setup() Entering this function
[        0.000]    INFO tunnel-cbs:ziti_dns.c:147 seed_dns() DNS configured with range 100.64.0.0 - 100.127.255.255
[        0.000]   DEBUG tunnel-sdk:ziti_tunnel.c:433 ziti_tunneler_intercept() intercepting address[udp:100.64.0.2:53] service[ziti:dns-resolver]
[        0.053]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:861 load_ziti_async() attempting to load ziti instance from file[TeltonikaX11.json]
[        0.053]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:868 load_ziti_async() loading ziti instance from /tmp/TeltonikaX11.json
[        0.053]    INFO ziti_log_set_level set log level: ziti_log_lvl=6 &ziti_log_lvl = 0x1dd7e8
[        0.053]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1037 load_id_cb() identity[TeltonikaX11.json] loaded
[        0.053]    WARN ziti-edge-tunnel:instance.c:37 find_tunnel_identity() Identity ztx[TeltonikaX11.json] is not loaded yet or already removed.
[        0.053]   ERROR ziti-edge-tunnel:instance-config.c:121 save_tunnel_status_to_file() Could not save the config file [/var/lib/ziti/config.json] due to semaphore lock not initialized error.
[        0.086]    INFO ziti-sdk:ziti.c:406 ziti_init_async() ztx[0] Ziti C SDK version 0.27.10 @ca8b793(HEAD) starting at (2022-06-07T17:47:45.371)
[        0.086]    INFO ziti-sdk:ziti.c:407 ziti_init_async() ztx[0] using uv_mbed[v0.14.5], tls[mbed TLS 3.1.0]
[        0.086]    INFO ziti-sdk:ziti.c:408 ziti_init_async() ztx[0] Loading from config[TeltonikaX11.json] controller[https://ziti-edge-controller:1280]
[        0.086]    INFO ziti-sdk:ziti_ctrl.c:375 ziti_ctrl_init() ctrl[ziti-edge-controller] ziti controller client initialized
[        0.086]   DEBUG ziti-sdk:ziti.c:432 ziti_init_async() ztx[0] using metrics interval: 0
[        0.086]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/version]
[        0.086]   DEBUG ziti-sdk:ziti.c:241 ziti_set_unauthenticated() ztx[0] setting api_session_state[0] to 0
[        0.086]   DEBUG ziti-sdk:ziti_ctrl.c:228 ziti_ctrl_clear_api_session() ctrl[ziti-edge-controller] clearing api session token for ziti_controller
[        0.086]   DEBUG ziti-sdk:ziti.c:836 ziti_re_auth() ztx[0] re-auth executing, transitioning to unauthenticated
[        0.086]   DEBUG ziti-sdk:ziti.c:241 ziti_set_unauthenticated() ztx[0] setting api_session_state[0] to 0
[        0.086]   DEBUG ziti-sdk:ziti_ctrl.c:228 ziti_ctrl_clear_api_session() ctrl[ziti-edge-controller] clearing api session token for ziti_controller
[        0.086]   DEBUG ziti-sdk:ziti.c:272 is_api_session_expired() ztx[0] is_api_session_expired[TRUE] - api_session is null
[        0.086]    INFO ziti-sdk:ziti.c:792 ziti_re_auth_with_cb() ztx[0] starting to re-auth with ctlr[https://ziti-edge-controller:1280] api_session_status[0] api_session_expired[TRUE]
[        0.086]   DEBUG ziti-sdk:ziti.c:234 ziti_set_auth_started() ztx[0] setting api_session_state[0] to 1
[        0.086]   DEBUG ziti-sdk:ziti.c:302 ziti_stop_api_session_refresh() ztx[0] ziti_stop_api_session_refresh: stopping api session refresh
[        0.086]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting POST[/authenticate?method=cert]
[        0.748]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/version]
[        0.748]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/version] in 40268688.000 s
[        0.748]    INFO ziti-sdk:ziti.c:1441 version_cb() ztx[0] connected to controller https://ziti-edge-controller:1280 version v0.25.5(2cd97dc79e5d 2022-05-03T02:47:46Z)
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers POST[/authenticate?method=cert]
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed POST[/authenticate?method=cert] in 40268688.000 s
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:241 ctrl_login_cb() ctrl[ziti-edge-controller] authenticated successfully session[cl44gg4e60ldv8s86dpx64v4t]
[        0.811]   DEBUG ziti-sdk:ziti.c:1357 api_session_cb() ztx[0] logged in successfully => api_session[cl44gg4e60ldv8s86dpx64v4t]
[        0.811]   TRACE ziti-sdk:ziti.c:1296 ziti_set_api_session() ztx[0] API supports cached_last_activity_at
[        0.811]   DEBUG ziti-sdk:ziti.c:1307 ziti_set_api_session() ztx[0] ziti api session expires in 600 seconds
[        0.811]    INFO ziti-sdk:ziti.c:1331 ziti_set_api_session() ztx[0] api session set, setting api_session_timer to 540s
[        0.811]   DEBUG ziti-sdk:ziti.c:307 ziti_schedule_api_session_refresh() ztx[0] ziti_schedule_api_session_refresh: scheduling api session refresh: -1093213160ms
[        0.811]   DEBUG ziti-sdk:ziti.c:266 ziti_set_fully_authenticated() ztx[0] setting api_session_state[1] to 3
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-identity]
[        0.811] VERBOSE ziti-sdk:ziti.c:1246 session_post_auth_query_cb() ztx[0] post auth query callback starting with status[OK]
[        0.811] VERBOSE ziti-sdk:ziti.c:1248 session_post_auth_query_cb() ztx[0] transitioning to fully authenticated
[        0.811]   DEBUG ziti-sdk:ziti.c:266 ziti_set_fully_authenticated() ztx[0] setting api_session_state[3] to 3
[        0.811]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:724 on_ziti_event() ziti_ctx[ TeltonikaX11] connected to controller
[        0.811]   DEBUG tunnel-sdk:ziti_tunnel.c:118 ziti_tunneler_exclude_route() excluding ziti-edge-controller from tunneler intercept
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.3.1 dev wlan1 proto static src 192.168.3.4 metric 3 '
[        0.811]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.65 via 192.168.3.1) returned 0
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        0.811]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.3.1 dev wlan1 proto static src 192.168.3.4 metric 3 '
[        0.811]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.65 via 192.168.3.1) returned 0
[        0.811]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1067 on_event() ztx[TeltonikaX11.json] context event : status is OK
[        0.811]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:1119 on_event() ztx[TeltonikaX11.json] controller connected
[        0.811]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"identity","Action":"added","Fingerprint":"TeltonikaX11","Id":{"Name":" TeltonikaX11","Identifier":"TeltonikaX11.json","FingerPrint":"TeltonikaX11","Active":true,"Loaded":true,"Config":{"ztAPI":"https://ziti-edge-controller:1280"},"ControllerVersion":"v0.25.5","IdFileStatus":true,"MfaEnabled":false,"MfaNeeded":false,"Metrics":{"Up":0,"Down":0},"MfaMinTimeout":0,"MfaMaxTimeout":0,"MfaMinTimeoutRem":0,"MfaMaxTimeoutRem":0,"MinTimeoutRemInSvcEvent":0,"MaxTimeoutRemInSvcEvent":0,"Deleted":false,"Notified":false}}
[        0.811]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"controller","Action":"connected","Identifier":"TeltonikaX11.json","Fingerprint":"TeltonikaX11"}
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-identity]
[        0.811]   DEBUG ziti-sdk:ziti.c:1263 session_post_auth_query_cb() ztx[0] refresh_interval set to 10 seconds
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-api-session/service-updates]
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:735 ctrl_paging_req() ctrl[ziti-edge-controller] starting paging request GET[/current-identity/edge-routers]
[        0.811] VERBOSE ziti-sdk:ziti_ctrl.c:740 ctrl_paging_req() ctrl[ziti-edge-controller] requesting /current-identity/edge-routers?limit=25&offset=0
[        0.811]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-identity/edge-routers?limit=25&offset=0]
[        0.894]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-identity]
[        0.894]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-identity] in 40227712.000 s
[        0.940]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-identity]
[        0.940]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-identity] in 40227712.000 s
[        0.986]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-api-session/service-updates]
[        0.986]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-api-session/service-updates] in 40227712.000 s
[        0.986] VERBOSE ziti-sdk:ziti.c:1119 check_service_update() ztx[0] ztx last_update = 2022-06-07T17:46:15.244Z
[        0.986]   DEBUG ziti-sdk:ziti_ctrl.c:735 ctrl_paging_req() ctrl[ziti-edge-controller] starting paging request GET[/services]
[        0.986] VERBOSE ziti-sdk:ziti_ctrl.c:740 ctrl_paging_req() ctrl[ziti-edge-controller] requesting /services?limit=25&offset=0
[        0.986]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/services?limit=25&offset=0]
[        1.035]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-identity/edge-routers?limit=25&offset=0]
[        1.035]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-identity/edge-routers?limit=25&offset=0] in 40227712.000 s
[        1.035]   DEBUG ziti-sdk:ziti_ctrl.c:324 ctrl_body_cb() ctrl[ziti-edge-controller] received 1/1 for paging request GET[/current-identity/edge-routers]
[        1.035]   DEBUG ziti-sdk:ziti_ctrl.c:336 ctrl_body_cb() ctrl[ziti-edge-controller] completed paging request GET[/current-identity/edge-routers] in 0.000 s
[        1.035]   TRACE ziti-sdk:ziti.c:1194 edge_routers_cb() ztx[0] connecting to ziti-edge-router(tls://ziti-edge-router:3022)
[        1.035]    INFO ziti-sdk:channel.c:219 new_ziti_channel() ch[0] (ziti-edge-router@tls://ziti-edge-router:3022) new channel for ztx[0] identity[ TeltonikaX11]
[        1.035]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:794 on_ziti_event() ztx[ TeltonikaX11] added edge router ziti-edge-router@tls://ziti-edge-router:3022@ziti-edge-router
[        1.035]   DEBUG tunnel-sdk:ziti_tunnel.c:118 ziti_tunneler_exclude_route() excluding ziti-edge-router from tunneler intercept
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.3.1 dev wlan1 proto static src 192.168.3.4 metric 3 '
[        1.035]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.70 via 192.168.3.1) returned 0
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   TRACE tunnel-sdk:ziti_tunnel.c:140 ziti_tunneler_exclude_route() ipv6 address compare not implemented
[        1.035]   DEBUG ziti-edge-tunnel:tun.c:257 tun_exclude_rt() default route is 'default via 192.168.3.1 dev wlan1 proto static src 192.168.3.4 metric 3 '
[        1.035]   DEBUG ziti-edge-tunnel:utils.c:32 run_command_va() system(ip route replace 203.160.138.70 via 192.168.3.1) returned 0
[        1.035]    INFO ziti-sdk:channel.c:733 reconnect_channel() ch[0] reconnecting NOW
[        1.070]   DEBUG ziti-sdk:channel.c:704 reconnect_cb() ch[0] connecting to ziti-edge-router:3022
[        1.117]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/services?limit=25&offset=0]
[        1.117]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/services?limit=25&offset=0] in 40244448.000 s
[        1.117]   DEBUG ziti-sdk:ziti_ctrl.c:324 ctrl_body_cb() ctrl[ziti-edge-controller] received 1/1 for paging request GET[/services]
[        1.117]   DEBUG ziti-sdk:ziti_ctrl.c:336 ctrl_body_cb() ctrl[ziti-edge-controller] completed paging request GET[/services] in 0.000 s
[        1.117] VERBOSE ziti-sdk:ziti.c:971 update_services() ztx[0] scheduling service refresh 10 seconds from now
[        1.117] VERBOSE ziti-sdk:ziti.c:992 update_services() ztx[0] processing service updates
[        1.117]   DEBUG ziti-sdk:ziti.c:1069 update_services() ztx[0] sending service event 1 added, 0 removed, 0 changed
[        1.117]   DEBUG tunnel-cbs:ziti_tunnel_ctrl.c:681 on_service() service[basic.web.test.service]
[        1.117]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:519 ziti_sdk_c_on_service() Entering here
[        1.117]    INFO tunnel-cbs:ziti_tunnel_cbs.c:411 new_ziti_intercept() creating intercept for service[basic.web.test.service] with intercept.v1 = {"addresses":["simple.web.test"],"portRanges":[{"high":80,"low":80}],"protocols":["tcp"]}
[        1.117]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:538 ziti_sdk_c_on_service() I am here
[        1.117]    INFO tunnel-cbs:ziti_tunnel_cbs.c:447 new_intercept_ctx() Entering this function
[        1.117]    INFO tunnel-cbs:ziti_tunnel_cbs.c:468 new_intercept_ctx() Entering this function at INTERCEPT_CFG_V1
[        1.117]    INFO tunnel-cbs:ziti_tunnel_cbs.c:469 new_intercept_ctx() Halla bol
[        1.117]    INFO tunnel-cbs:ziti_dns.c:349 ziti_dns_register_hostname() Entering this function simple.web.test
[        1.117]    INFO tunnel-cbs:ziti_dns.c:257 new_ipv4_entry() registered DNS entry simple.web.test -> 100.64.0.10
[        1.117]    INFO tunnel-sdk:ziti_tunnel.c:334 parse_address() Entered if dns is true & got addr->str = 100.64.0.10
[        1.117]    INFO ziti-edge-tunnel:dnsmasq_manager.c:41 apply_address() Entered this function simple.web.test: 100.64.0.10
[        1.117]    INFO ziti-edge-tunnel:dnsmasq_manager.c:58 apply_address() successfully written the file /tmp/hosts/simple.web.test
[        1.117]   DEBUG tunnel-sdk:ziti_tunnel.c:433 ziti_tunneler_intercept() intercepting address[tcp:100.64.0.10:80] service[basic.web.test.service]
[        1.117]   DEBUG tunnel-cbs:ziti_tunnel_cbs.c:549 ziti_sdk_c_on_service() 2. I am here
[        1.117]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:684 on_service() starting intercepting for service[basic.web.test.service]
[        1.117] VERBOSE ziti-edge-tunnel:ziti-edge-tunnel.c:1136 on_event() =============== ztx[TeltonikaX11.json] service event ===============
[        1.117]   TRACE ziti-edge-tunnel:instance.c:215 setTunnelPostureDataTimeout() Posture Query set returned a Dial policy: kvKcoSIGi, is_passing 1
[        1.117]   DEBUG ziti-edge-tunnel:instance.c:257 setTunnelPostureDataTimeout() service[basic.web.test.service] timeout=-1 timeoutRemaining=-1
[        1.117]   TRACE ziti-edge-tunnel:instance.c:295 setTunnelServiceAddress() intercept.v1: {"addresses":["simple.web.test"],"portRanges":[{"high":80,"low":80}],"protocols":["tcp"]}
[        1.117]   TRACE ziti-edge-tunnel:instance.c:276 to_address() Hostname: simple.web.test
[        1.117]    INFO ziti-edge-tunnel:ziti-edge-tunnel.c:1192 on_event() =============== service event (added) - basic.web.test.service:mQqOfzJSxZ ===============
[        1.117]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"bulkservice","Action":"updated","Identifier":"TeltonikaX11.json","Fingerprint":"TeltonikaX11","AddedServices":[{"Id":"mQqOfzJSxZ","Name":"basic.web.test.service","Protocols":["tcp"],"Addresses":[{"IsHost":true,"HostName":"simple.web.test","Prefix":0}],"Ports":[{"High":80,"Low":80}],"OwnsIntercept":true,"IsAccessible":true,"Timeout":-1,"TimeoutRemaining":-1}],"RemovedServices":[]}
[        1.117]   DEBUG ziti-edge-tunnel:ziti-edge-tunnel.c:697 send_events_message() Events Message => {"Op":"identity","Action":"updated","Fingerprint":"TeltonikaX11","Id":{"Name":" TeltonikaX11","Identifier":"TeltonikaX11.json","FingerPrint":"TeltonikaX11","Active":true,"Loaded":true,"Config":{"ztAPI":"https://ziti-edge-controller:1280"},"ControllerVersion":"v0.25.5","IdFileStatus":true,"MfaEnabled":false,"MfaNeeded":false,"Services":[{"Id":"mQqOfzJSxZ","Name":"basic.web.test.service","Protocols":["tcp"],"Addresses":[{"IsHost":true,"HostName":"simple.web.test","Prefix":0}],"Ports":[{"High":80,"Low":80}],"OwnsIntercept":true,"IsAccessible":true,"Timeout":-1,"TimeoutRemaining":-1}],"Metrics":{"Up":0,"Down":0},"MfaMinTimeout":-1,"MfaMaxTimeout":-1,"MfaMinTimeoutRem":-1,"MfaMaxTimeoutRem":-1,"MinTimeoutRemInSvcEvent":-1,"MaxTimeoutRemInSvcEvent":-1,"ServiceUpdatedTime":"2022-06-07T17:47:46.435671Z","Deleted":false,"Notified":false}}
[        1.599]   DEBUG ziti-sdk:channel.c:854 on_channel_connect_internal() ch[0] connected
[        1.599]   TRACE ziti-sdk:channel.c:371 ziti_channel_send_for_reply() ch[0] => ct[0000] seq[0] len[34]
[        1.601]   TRACE ziti-sdk:channel.c:778 on_write() on_write(0x2667930,0)
[        1.641]   TRACE ziti-sdk:channel.c:841 on_channel_data() ch[0] on_data [len=114]
[        1.641]   TRACE ziti-sdk:channel.c:527 process_inbound() ch[0] <= ct[0002] seq[-1] len[0] hdrs[94]
[        1.641]   TRACE ziti-sdk:channel.c:537 process_inbound() ch[0] completing msg seq[-1] body+hrds=0+94, in_offset=0, want=94, got=94
[        1.641]   TRACE ziti-sdk:channel.c:548 process_inbound() ch[0] message is complete seq[-1] ct[0002]
[        1.641]    INFO ziti-sdk:channel.c:629 hello_reply_cb() ch[0] connected. EdgeRouter version: v0.25.5|2cd97dc79e5d|2022-05-03T02:47:46Z|linux|amd64
[        1.641]    INFO tunnel-cbs:ziti_tunnel_ctrl.c:798 on_ziti_event() ztx[ TeltonikaX11] router ziti-edge-router@tls://ziti-edge-router:3022 connected
[        5.053]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       10.053]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       11.117]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-api-session/service-updates]
[       11.235]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-api-session/service-updates]
[       11.235]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-api-session/service-updates] in 40244448.000 s
[       11.235] VERBOSE ziti-sdk:ziti.c:1124 check_service_update() ztx[0] not updating: last_update is same previous (2022-06-07T17:46:15.244Z == 2022-06-07T17:46:15.244Z)
[       14.867]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       14.867]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       14.867]   TRACE tunnel-sdk:tunnel_tcp.c:326 recv_tcp() received segment 100.64.0.10:42814->100.64.0.10:80
[       14.867]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       14.867]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       14.867]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       14.867]   TRACE tunnel-sdk:intercept.c:81 lookup_intercept_by_address() Did Not Find matching address
[       14.867]    INFO tunnel-sdk:intercept.c:38 address_match() comparing intercepted ip 100.64.0.10 to 0.0.0.0
[       14.867]   TRACE tunnel-sdk:intercept.c:92 lookup_intercept_by_address() Matched address and intercept Not Found
[       14.867]   TRACE tunnel-sdk:tunnel_tcp.c:337 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       15.054]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       15.905]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       15.905]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       15.905]   TRACE tunnel-sdk:tunnel_tcp.c:326 recv_tcp() received segment 100.64.0.10:42814->100.64.0.10:80
[       15.905]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       15.905]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       15.905]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       15.905]   TRACE tunnel-sdk:intercept.c:81 lookup_intercept_by_address() Did Not Find matching address
[       15.905]    INFO tunnel-sdk:intercept.c:38 address_match() comparing intercepted ip 100.64.0.10 to 0.0.0.0
[       15.905]   TRACE tunnel-sdk:intercept.c:92 lookup_intercept_by_address() Matched address and intercept Not Found
[       15.905]   TRACE tunnel-sdk:tunnel_tcp.c:337 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       17.985]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       17.985]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       17.985]   TRACE tunnel-sdk:tunnel_tcp.c:326 recv_tcp() received segment 100.64.0.10:42814->100.64.0.10:80
[       17.985]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       17.985]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       17.985]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       17.985]   TRACE tunnel-sdk:intercept.c:81 lookup_intercept_by_address() Did Not Find matching address
[       17.985]    INFO tunnel-sdk:intercept.c:38 address_match() comparing intercepted ip 100.64.0.10 to 0.0.0.0
[       17.985]   TRACE tunnel-sdk:intercept.c:92 lookup_intercept_by_address() Matched address and intercept Not Found
[       17.985]   TRACE tunnel-sdk:tunnel_tcp.c:337 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       20.054]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       20.811] VERBOSE ziti-sdk:posture.c:191 ziti_send_posture_data() ztx[0] starting to send posture data
[       20.811]    INFO ziti-sdk:posture.c:197 ziti_send_posture_data() ztx[0] first run or potential controller restart detected
[       20.811]   DEBUG ziti-sdk:posture.c:204 ziti_send_posture_data() ztx[0] posture checks must_send set to TRUE, new_session_id[TRUE], must_send_every_time[TRUE], new_controller_instance[TRUE]
[       20.811] VERBOSE ziti-sdk:posture.c:229 ziti_send_posture_data() ztx[0] checking posture queries on 1 service(s)
[       20.811]   DEBUG ziti-sdk:posture.c:519 ziti_pr_send_bulk() ztx[0] no change in posture data, not sending
[       21.235]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-api-session/service-updates]
[       21.358]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-api-session/service-updates]
[       21.358]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-api-session/service-updates] in 40212544.000 s
[       21.358] VERBOSE ziti-sdk:ziti.c:1124 check_service_update() ztx[0] not updating: last_update is same previous (2022-06-07T17:46:15.244Z == 2022-06-07T17:46:15.244Z)
[       22.065]    INFO tunnel-sdk:tunnel_tcp.c:287 recv_tcp() Flowing from here
[       22.065]    INFO tunnel-sdk:tunnel_tcp.c:305 recv_tcp() 2.Flowing from here
[       22.065]   TRACE tunnel-sdk:tunnel_tcp.c:326 recv_tcp() received segment 100.64.0.10:42814->100.64.0.10:80
[       22.065]    INFO tunnel-cbs:ziti_tunnel_cbs.c:428 intercept_match_addr() matching 100.64.0.10
[       22.065]    INFO tunnel-cbs:ziti_tunnel_cbs.c:431 intercept_match_addr() I am coming here
[       22.065]    INFO tunnel-cbs:ziti_tunnel_cbs.c:434 intercept_match_addr() I am coming here domain (null)
[       22.065]   TRACE tunnel-sdk:intercept.c:81 lookup_intercept_by_address() Did Not Find matching address
[       22.065]    INFO tunnel-sdk:intercept.c:38 address_match() comparing intercepted ip 100.64.0.10 to 0.0.0.0
[       22.065]   TRACE tunnel-sdk:intercept.c:92 lookup_intercept_by_address() Matched address and intercept Not Found
[       22.065]   TRACE tunnel-sdk:tunnel_tcp.c:337 recv_tcp() no intercepted addresses match tcp:100.64.0.10:80
[       25.055]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       30.056]   TRACE tunnel-cbs:ziti_tunnel_ctrl.c:198 process_cmd() processing command[GetMetrics] with data[{"Identifier":"TeltonikaX11.json"}]
[       31.358]   DEBUG ziti-sdk:ziti_ctrl.c:133 start_request() ctrl[ziti-edge-controller] starting GET[/current-api-session/service-updates]
[       31.489]   DEBUG ziti-sdk:ziti_ctrl.c:160 ctrl_resp_cb() ctrl[ziti-edge-controller] received headers GET[/current-api-session/service-updates]
[       31.489]   DEBUG ziti-sdk:ziti_ctrl.c:308 ctrl_body_cb() ctrl[ziti-edge-controller] completed GET[/current-api-session/service-updates] in 40212544.000 s
[       31.489] VERBOSE ziti-sdk:ziti.c:1124 check_service_update() ztx[0] not updating: last_update is same previous (2022-06-07T17:46:15.244Z == 2022-06-07T17:46:15.244Z)

Hi Sameer,

I think I see the problem. Your changes to parse_address never convert the string form of the address to numeric form. The existing (non-OPENWRT) code does this with ipaddr_aton

    if (ipaddr_aton(addr->str, &addr->ip) == 0) {

We are using to test for a string that parses as an IP but it also has a side-effect when successful of storing the numeric address in addr->ip.

I think you’ll get farther along if you change your implementation to call ipaddr_aton,

Thanks,
-Shawn

Thanks Shawn for pointing it out. After adding those line, it perfectly worked.
Ziti-edge-tunnel on teltonika box was able to intercept both hostname using dnsmasq resolver and IP.
I am mending few more things into it, such as the files which gets created in the dnsmasq path don’t get deleted after the service is deleted and then ip route also don’t get removed for those services once removed.

We have to do more testing using it on other boxes as well.

I will push the updated code to my fork soon.

Regards,
Sameer Sarkar

Hi Sameer,

That’s great! I’m glad that change got you going again with DNS requests! I do think this approach of maintaining a fork will be difficult for you as we continue to develop ziti-edge-tunnel.

Eventually (and hopefully sooner rather than later) you can get the stock ziti-edge-tunnel releases working with dnsmasq on openwrt. Now that the DNS server in ZET is recursive, it should be possible to put it “behind” dnsmasq. Please raise a new issue if you want help with setting up released ziti-edge-tunnel to work with dnsmasq.

Thanks and totally agree with you.

Sure i would like to contribute. I will raise new issue for it.
FYI I have added below list of things to this code:

  1. Re-introduced dnsmasq option
  2. Fixed an issue where routes added for individual services weren't getting deleted after removal of service from the endpoint/identity/edge-tunnel ip route table,
  3. Fixed an issue where, removal of the files created in dnsmasq path (again they also weren't deleted post removal of service from the endpoint/identity/edge-tunnel.
  4. Then added support of creation of tun interface with any name (which isn't available with openziti code). This is required for teltonika boxes as it tun interface creation and in past we have observed that they have create issues in the box.

As all of them are in same code modifications. I will create a single issue for all these and raise the request.

Regards,
Sameer

Hi Shawn,
I have raised new issue Re-Introducing dnsmasq option in ziti-edge-tunnel specially for OPENWRT supported devices · Issue #386 · openziti/ziti-tunnel-sdk-c · GitHub
and pull request with it Raising pull request to merge PR #386 by sameersarkar-tcl · Pull Request #387 · openziti/ziti-tunnel-sdk-c · GitHub

Request you and maintainers to review the code.

Thanks
Sameer

Hey @sameersarkar-tcl,

I was able to set dhcp/dnsmaq on Teltonika that works with ziti-edge-tunnel pretty well so far in my testing. Can you try it for yourself and let me know how it behaves in your environment? The more we test it, the better we can improve it. Ziti already maintains a dns registry map in the code. Why to create another such mapping in a file?

Here is the dns setup in my environment.

root@Teltonika-RUT950:~# uci show |grep dns
ddns.global=ddns
ddns.global.ddns_dateformat='%F %R'
ddns.global.ddns_loglines='250'
ddns.global.upd_privateip='1'
ddns.myddns=service
ddns.myddns.lookup_host='yourhost.example.com'
ddns.myddns.domain='yourhost.example.com'
ddns.myddns.username='your_username'
ddns.myddns.password='your_password'
ddns.myddns.interface='wan'
ddns.myddns.ip_source='network'
ddns.myddns.ip_network='wan'
ddns.myddns.service_name='dyn.com'
dhcp.@dnsmasq[0]=dnsmasq
dhcp.@dnsmasq[0].domainneeded='1'
dhcp.@dnsmasq[0].boguspriv='1'
dhcp.@dnsmasq[0].filterwin2k='0'
dhcp.@dnsmasq[0].localise_queries='1'
dhcp.@dnsmasq[0].rebind_protection='1'
dhcp.@dnsmasq[0].rebind_localhost='1'
dhcp.@dnsmasq[0].local='/lan/'
dhcp.@dnsmasq[0].domain='lan'
dhcp.@dnsmasq[0].expandhosts='1'
dhcp.@dnsmasq[0].nonegcache='0'
dhcp.@dnsmasq[0].authoritative='1'
dhcp.@dnsmasq[0].readethers='1'
dhcp.@dnsmasq[0].leasefile='/tmp/dhcp.leases'
dhcp.@dnsmasq[0].resolvfile='/tmp/resolv.conf.auto'
dhcp.@dnsmasq[0].dhcpscript='/usr/sbin/dhcpinfo.sh'
dhcp.@dnsmasq[0].server='100.64.0.2'
dhcp.@dnsmasq[0].strictorder='true'
multiwan.wan.dns='auto'
multiwan.wan2.dns='auto'
multiwan.wan3.dns='auto'
multiwan.wan4.dns='auto'
operctl.dns_list=dns_list
operctl.dns_list.enabled='1'
operctl.dns_list.dns='8.8.8.8' '1.1.1.1' '208.67.222.222' '9.9.9.9' '64.6.64.6'
qos.@classify[0].comment='ssh, dns'
ucitrack.@dhcp_common[0].affects='dnsmasq' 'dhcprelay'
ucitrack.@dhcp[0].init='dnsmasq'
ucitrack.@dnsmasq[0]=dnsmasq
ucitrack.@dnsmasq[0].init='dnsmasq'
ucitrack.@ddns[0]=ddns
ucitrack.@ddns[0].init='ddns'
root@Teltonika-RUT950:~#

Everything is pretty much standard/default setup at least on my router except these two that I added

dhcp.@dnsmasq[0].server='100.64.0.2'
dhcp.@dnsmasq[0].strictorder='true

commands I used:

uci add_list dhcp.@dnsmasq[0].server="100.64.0.2"
uci set dhcp.@dnsmasq[0].strictorder="true"
uci commit dhcp
/etc/init.d/dnsmasq restart

I ran zit-edge-tunnel as this:

./ziti-edge-tunnel run -i teltonika.json
./ziti-edge-tunnel run -i teltonika.json -u 1.1.1.1

It seem to be working fine using both.

Also, my client’s dns set up is as follows:

ubuntu@ubuntu-desktop:~$ resolvectl status
Global
       Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 2 (eth0)
Current Scopes: none
     Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 3 (wlan0)
    Current Scopes: DNS
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.1.1
       DNS Servers: 192.168.1.1
        DNS Domain: lan
ubuntu@ubuntu-desktop:~$

here is the dns lookup sequence from the client captured on the router, when ziti-edge-tunnel is shutdown.

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
21:04:06.735742 IP 192.168.1.246.36758 > 192.168.1.1.53: 2178+ [1au] A? cnn.com. (36)
21:04:06.735770 IP 192.168.1.246.36758 > 192.168.1.1.53: 2178+ [1au] A? cnn.com. (36)
21:04:06.737608 IP 192.168.1.246.46711 > 192.168.1.1.53: 7826+ [1au] AAAA? cnn.com. (36)
21:04:06.737636 IP 192.168.1.246.46711 > 192.168.1.1.53: 7826+ [1au] AAAA? cnn.com. (36)
21:04:06.740302 IP 192.168.0.13.56482 > 100.64.0.2.53: 56448+ [1au] A? cnn.com. (36)
21:04:06.740760 IP 192.168.0.13.44175 > 100.64.0.2.53: 62990+ [1au] AAAA? cnn.com. (36)
21:04:11.744722 IP 192.168.1.246.46711 > 192.168.1.1.53: 7826+ [1au] AAAA? cnn.com. (36)
21:04:11.744760 IP 192.168.1.246.46711 > 192.168.1.1.53: 7826+ [1au] AAAA? cnn.com. (36)
21:04:11.746492 IP 192.168.1.246.36758 > 192.168.1.1.53: 2178+ [1au] A? cnn.com. (36)
21:04:11.746517 IP 192.168.1.246.36758 > 192.168.1.1.53: 2178+ [1au] A? cnn.com. (36)
21:04:11.748708 IP 192.168.0.13.44175 > 192.168.0.1.53: 62990+ [1au] AAAA? cnn.com. (36)
21:04:11.749091 IP 192.168.0.13.56482 > 192.168.0.1.53: 56448+ [1au] A? cnn.com. (36)
21:04:11.795167 IP 192.168.0.1.53 > 192.168.0.13.44175: 62990 4/0/1 AAAA 2a04:4e42:400::323, AAAA 2a04:4e42::323, AAAA 2a04:4e42:600::323, AAAA 2a04:4e42:200::323 (148)
21:04:11.797151 IP 192.168.1.1.53 > 192.168.1.246.46711: 7826 4/0/1 AAAA 2a04:4e42:400::323, AAAA 2a04:4e42::323, AAAA 2a04:4e42:600::323, AAAA 2a04:4e42:200::323 (148)
21:04:11.797189 IP 192.168.1.1.53 > 192.168.1.246.46711: 7826 4/0/1 AAAA 2a04:4e42:400::323, AAAA 2a04:4e42::323, AAAA 2a04:4e42:600::323, AAAA 2a04:4e42:200::323 (148)
21:04:11.802270 IP 192.168.0.1.53 > 192.168.0.13.56482: 56448 4/0/1 A 151.101.193.67, A 151.101.129.67, A 151.101.65.67, A 151.101.1.67 (100)
21:04:11.805725 IP 192.168.1.1.53 > 192.168.1.246.36758: 2178 4/0/1 A 151.101.193.67, A 151.101.129.67, A 151.101.65.67, A 151.101.1.67 (100)
21:04:11.805769 IP 192.168.1.1.53 > 192.168.1.246.36758: 2178 4/0/1 A 151.101.193.67, A 151.101.129.67, A 151.101.65.67, A 151.101.1.67 (100)

FYI, 192.168.0.1 is IP on the WAN side.

HI @dariuszSki ,
I will give it a try and get back to you.
Offcourse, if there is something like which is available by default why not leverage.

Only concern would be that the uci command need to be executed once tunneler is up and running and while it is shutdown the entry needs to be re-instated to default.

Re-instatement could be tricky one as it would depend upon which WAN interface at the moment internet is available (It could be LTE or BB1 or BB2 or Wifi)

Thanks

After the initial configuration setup, I did not have to reenter or change anything else when ziti-edge-tunnel was shutdown and was still able to resolve public dns names. I would like to see it working on different devices using dnsmasq though. The device I am testing on is connected to LTE.
Let us know what you see in your set up. Thank you for giving a try!