# Creating & Removing Terminator Loop

**URL:** <https://openziti.discourse.group/t/creating-removing-terminator-loop/1772>\
**Category:** Uncategorized\
**Created:** [November 7, 2023, 2:03pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772 "2023-11-07T14:03:01Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![dmuensterer](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/dmuensterer/32/2451_2.png) [@dmuensterer](https://openziti.discourse.group/u/dmuensterer)\
**Post date:** [November 7, 2023, 2:03pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/1 "2023-11-07T14:03:01Z")

</div>

Hi,  
after creating a new , very simple service, I ended up in lots and lots of Logs that indicate the creation and deletion of terminators a lot of times per second.  
Any idea what this is caused by?

 ![image](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/8/8985e6346292f61a3aba5cfaa90186766ecd2d7f.png)

What additional information can I provide for more details?

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [November 7, 2023, 2:04pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/2 "2023-11-07T14:04:47Z")

</div>

Woah, that looks very abnormal! Do you know what commands you ran? Also what version of the controller is running?

Is that something you can reproduce? Collecting the logs from the identity making the terminator and the controller would be a good first step.

---

<div class="post-metadata">

**Author:** ![plorenz](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/plorenz/32/54_2.png) [@plorenz](https://openziti.discourse.group/u/plorenz)\
**Post date:** [November 7, 2023, 2:31pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/3 "2023-11-07T14:31:37Z")

</div>

Hi @dmuensterer,  
This is something I also encountered recently. I put some fixes in

- [Ensure links don't get stuck in dialing status. Only mark link status failed if closing link was current. Fixes #1471 by plorenz · Pull Request #1484 · openziti/ziti · GitHub](https://github.com/openziti/ziti/pull/1484/files)
- [Don't send close in response to close. Fixes #443 by plorenz · Pull Request #444 · openziti/sdk-golang · GitHub](https://github.com/openziti/sdk-golang/pull/444)

The first fix is in OpenZiti v0.31.0.  
The second is in the Go SDK v0.20.129.

Let me know what versions you're using, for controller/routers and sdk.  
I'm also doing some refactoring in this area, so if you're still seeing the issue with the latest code, I should have something soon that will address the issue.

Paul

---

<div class="post-metadata">

**Author:** ![dmuensterer](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/dmuensterer/32/2451_2.png) [@dmuensterer](https://openziti.discourse.group/u/dmuensterer)\
**Post date:** [November 7, 2023, 9:17pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/4 "2023-11-07T21:17:04Z")

</div>

I actually did reproduce it - but I don't know how 😆  
I had this problem a couple of days ago already while experimenting with a wildcard intercept.  
This time again - I gave a server an identity, the attribute "ssh" for the wildcard service and SSH binding and additionally I created a service via the ZAC "Quick Create" feature to intercept/dial to another server...  
Let me please check @plorenz suggestions and pull the logs!  
@plorenz I'm running Ziti SDK 0.35.2 on client/server with Ziti Edge Tunnel v0.22.11  
Controller is on v0.30.5, one router is running on v0.30.5, one is running on v.29.0.

---

<div class="post-metadata">

**Author:** ![dmuensterer](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/dmuensterer/32/2451_2.png) [@dmuensterer](https://openziti.discourse.group/u/dmuensterer)\
**Post date:** [November 8, 2023, 10:02am UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/5 "2023-11-08T10:02:49Z")

</div>

I know, what I also did - rename the service, configs and policies... Maybe that resulted in an issue somehow? Let me check if the problem persists after restarting the identity where the terminators are created, because I didn't do so after renaming...

Edit: **Welp** Problem solved after restarting the Ziti-Edge-Tunnel... Interesting 🙂

 ![image](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/b/b706817a3fd078c6d1d932435c1a6f799ccd9a8a.jpeg)  
I will create a github issue for this

---

<div class="post-metadata">

**Author:** ![plorenz](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/plorenz/32/54_2.png) [@plorenz](https://openziti.discourse.group/u/plorenz)\
**Post date:** [November 8, 2023, 2:41pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/6 "2023-11-08T14:41:06Z")

</div>

Were you able to reproduce it while running 0.31.0 on the routers?

---

<div class="post-metadata">

**Author:** ![dmuensterer](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/dmuensterer/32/2451_2.png) [@dmuensterer](https://openziti.discourse.group/u/dmuensterer)\
**Post date:** [November 8, 2023, 10:18pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/7 "2023-11-08T22:18:38Z")

</div>

No, I’ll try with v0.31.0

---

<div class="post-metadata">

**Author:** ![dmuensterer](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/dmuensterer/32/2451_2.png) [@dmuensterer](https://openziti.discourse.group/u/dmuensterer)\
**Post date:** [November 10, 2023, 9:29am UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/8 "2023-11-10T09:29:00Z")

</div>

Is there a way to gracefully ask identities to connect to another router without just restarting the router and dropping existing connections?

---

<div class="post-metadata">

**Author:** ![TheLumberjack](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/thelumberjack/32/113_2.png) [@TheLumberjack](https://openziti.discourse.group/u/TheLumberjack)\
**Post date:** [November 10, 2023, 2:37pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/9 "2023-11-10T14:37:17Z")

</div>

Depending on your definition of 'graceful', "probably". One easy mechanism is you update the edge router policy that indicates the router can be used for the particular service. For example, if it's a public router, you turn off the `#public` attribute and then it isn't public any more and all your clients stop using it. That's one definition of graceful.

There are other, more fine-grained ways of effecting this via policies but it'll depend on how you have configured the overlay as to what sorts of options to do this would be available.

I believe there has been talk of implementing some kind of message to the endpoints indicatng "please pick another router" which maybe is a bit less abrupt than removing the router from a pool as I mentioned before, but I don't know if that's been implemented yet. I think that was going to be used to support rolling upgrades. I'm just not as close to that as @plorenz is, so I'm not sure if that was implemented yet.

---

<div class="post-metadata">

**Author:** ![dmuensterer](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/dmuensterer/32/2451_2.png) [@dmuensterer](https://openziti.discourse.group/u/dmuensterer)\
**Post date:** [November 10, 2023, 2:54pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/10 "2023-11-10T14:54:24Z")

</div>

> [@TheLumberjack](#):
>
> For example, if it's a public router, you turn off the `#public` attribute and then it isn't public any more and all your clients stop using it. That's one definition of graceful.

Makes perfect sense and was exactly what I was looking for. Thanks.

---

<div class="post-metadata">

**Author:** ![dmuensterer](https://yyz2.discourse-cdn.com/free1/user_avatar/openziti.discourse.group/dmuensterer/32/2451_2.png) [@dmuensterer](https://openziti.discourse.group/u/dmuensterer)\
**Post date:** [November 21, 2023, 1:59pm UTC](https://openziti.discourse.group/t/creating-removing-terminator-loop/1772/11 "2023-11-21T13:59:51Z")

</div>

I've also noticed the following logs spamming after creating another wildcard service, without any "special" doings afterwards like renaming etc.  
But I'm still running v0.30.5 so let me upgrade to v0.31.0 as @plorenz mentioned, a couple things were fixed

 ![image](https://global.discourse-cdn.com/free1/uploads/netfoundry/original/2X/1/1f10b5fe0ca102029c64f15ce713cc3583a8b0dc.png)
