# Threading with uv\_queue\_work()

**URL:** <https://openziti.discourse.group/t/threading-with-uv-queue-work/150>\
**Category:** sdk-c\
**Created:** [May 26, 2021, 3:39pm UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150 "2021-05-26T15:39:07Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![oluetkeh](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/oluetkeh/32/66_2.png) [@oluetkeh](https://openziti.discourse.group/u/oluetkeh)\
**Post date:** [May 26, 2021, 3:39pm UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150/1 "2021-05-26T15:39:07Z")

</div>

per Eugene’s [earlier guidance](https://openziti.discourse.group/t/this-request-causes-50s-delay/146/6), I went ahead and implemented processing the upstream request in a separate thread (i.e. not on the event loop) and sending the response back via ziti transport on the event loop. The [latest code](https://github.com/oluetkeh/ngx_ziti_downstream/blob/add_threading/src/ngx_ziti_downstream_module.c) uses the invocation `uv_queue_work(uv_thread_loop, work, process_upstream, respond_to_client);`

I have verified via debugger that processing of the function `process_upstream(uv_work_t *work)` occurs on a separate thread. My problem at this time is that the 2nd callback `respond_to_client(uv_work_t *work, int status)`, i.e. the ‘after processing’ is never invoked. Would you have any pointers as to what I can investigate next to fix this?

Here’s relevant detail from the logs:

```
[4.583] VERBOSE ../library/connect.c:781 conn_inbound_data_msg() conn[0.1] decrypting 94 bytes
[4.583] VERBOSE ../library/connect.c:785 conn_inbound_data_msg() conn[0.1] decrypted 77 bytes
[4.583] TRACE ../library/connect.c:736 flush_to_client() conn[0.1] flushing 77 bytes to client
[4.583] DEBUG /Users/oluetkeh/Developer/ngx_ziti_downstream/src/ngx_ziti_downstream_module.c:275 ngx_ziti_downstream_on_client_data() client sent 77 bytes
[4.583] TRACE /Users/oluetkeh/Developer/ngx_ziti_downstream/src/ngx_ziti_downstream_module.c:276 ngx_ziti_downstream_on_client_data() client sent these 77 bytes:
GET / HTTP/1.1
Host: odoo.ngx.ziti
User-Agent: curl/7.64.1
Accept: */*

[4.584] DEBUG /Users/oluetkeh/Developer/ngx_ziti_downstream/src/ngx_ziti_downstream_module.c:393 talk_to_upstream() ngx_ziti_downstream: request sent upstream. 77 bytes
[4.584] DEBUG /Users/oluetkeh/Developer/ngx_ziti_downstream/src/ngx_ziti_downstream_module.c:413 talk_to_upstream() ngx_ziti_downstream: response received from upstream. 552 bytes
[4.584] TRACE /Users/oluetkeh/Developer/ngx_ziti_downstream/src/ngx_ziti_downstream_module.c:414 talk_to_upstream() response from upstream server 552 bytes:
HTTP/1.0 303 SEE OTHER
Content-Type: text/html; charset=utf-8
Content-Length: 215
Location: http://odoo.ngx.ziti/web
Set-Cookie: session_id=d0b15c0e45759a222d3d106691062108a19231d5; Expires=Tue, 24-Aug-2021 15:30:47 GMT; Max-Age=7776000; HttpOnly; Path=/
Server: Werkzeug/0.16.1 Python/3.8.6
Date: Wed, 26 May 2021 15:30:47 GMT

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>Redirecting...</title>
<h1>Redirecting...</h1>
<p>You should be redirected automatically to target URL: <a href="/web">/web</a>. If not click the link.
[4.584] DEBUG /Users/oluetkeh/Developer/ngx_ziti_downstream/src/ngx_ziti_downstream_module.c:308 process_upstream() finished upstream communication, got 552 bytes
[5.544] TRACE _deps/uv-mbed-src/src/tls_link.c:112 TLS(0x1005046c0)[2]: 100

```

thx  
Olaf

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [May 26, 2021, 4:05pm UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150/2 "2021-05-26T16:05:43Z")

</div>

Thanks for sharing the code… I took a look. Is it possible that the loop is exiting? I see a couple loops created and assigned to your `zscf->uv_thread_loop`.

I can see your log message at the end of the function so it seems like it should be firing that callback but if the loop is dying that might be why.

I might be wrong but this seems like just uv issue - not ziti so much, right?

---

<div class="post-metadata">

**Author:** ![oluetkeh](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/oluetkeh/32/66_2.png) [@oluetkeh](https://openziti.discourse.group/u/oluetkeh)\
**Post date:** [May 26, 2021, 5:24pm UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150/3 "2021-05-26T17:24:50Z")

</div>

Thanks for the response.  
I’ve added a line after the dispatch (`uv_queue_work(uv_thread_loop, work, process_upstream, respond_to_client);`) to make sure the loop has not exited. With that one can see that the code (i.e. loop) continues after the call. Also, multiple requests are handled the same way, iow new client requests coming in are being handled.

Yes, the problem I’m facing looks definitely uv specific. I’d consider libuv an integral part of OpenZiti, though. I’ll look for guidance in other forums (too).

thx,  
Olaf

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [May 26, 2021, 5:34pm UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150/4 "2021-05-26T17:34:13Z")

</div>

I totally agree that libuv is strongly-tied with the C SDK. That’s why I was wondering if you are on teh proper loop or if one of your loops is exiting - or getting overwritten.

For me in the past most of my libuv issues were “the loop isn’t started”, “the loop exited”, or “you used the wrong loop”. Perhaps if you just use “default loop” for everything for now just to rule that out it might help? If i get a chance to look more - I’ll see if anything else strikes me and comment back.

---

<div class="post-metadata">

**Author:** ![oluetkeh](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/oluetkeh/32/66_2.png) [@oluetkeh](https://openziti.discourse.group/u/oluetkeh)\
**Post date:** [June 9, 2021, 12:49am UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150/5 "2021-06-09T00:49:02Z")

</div>

Going forward I will do some more lower level debugging but probably look into implementing the threading piece without the use of uv\_queue\_work().

On that note, do you guys have any plans of hiding the uv loop interactions with ziti functions?

---

<div class="post-metadata">

**Author:** ![smilindave26](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/smilindave26/32/185_2.png) [@smilindave26](https://openziti.discourse.group/u/smilindave26)\
**Post date:** [June 9, 2021, 1:03am UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150/6 "2021-06-09T01:03:25Z")

</div>

“plans” - no. You really need to understand that you’re running in a loop context. But we’ve talked a bit about adding something similar to `async` (like you see in Node.js). Would you find something like that useful?

---

<div class="post-metadata">

**Author:** ![smilindave26](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/smilindave26/32/185_2.png) [@smilindave26](https://openziti.discourse.group/u/smilindave26)\
**Post date:** [June 9, 2021, 1:06am UTC](https://openziti.discourse.group/t/threading-with-uv-queue-work/150/7 "2021-06-09T01:06:55Z")

</div>

Note: if you use one of the Ziti SDK wrapper libs, e.g., `ziti-sdk-nodejs`, or `ziti-sdk-swift` you don’t need to understand libuv (though all is still async of course)
