Tunneler 0.9.0 - Error - nil pagination in response to GET /services

dariuszSki · February 13, 2020, 6:08pm

I installed v0.9.0 tunneler and getting the following error everytime the service poller runs, I guess (15s):
[ 62.723] ERROR github.com/netfoundry/ziti-edge/tunnel/intercept.ServicePoller: failed to get ziti services: nil pagination in response to GET /services

ekoby · February 13, 2020, 6:34pm

Make sure controller version matches tunneler

dariuszSki · February 13, 2020, 6:43pm

./ziti-tunnel version
v0.9.0
ziti-controller version
v0.9.0
They look the same to me

plorenz · February 13, 2020, 6:52pm

Is there an error on the controller side as well?

dariuszSki · February 13, 2020, 6:55pm

#033[34m[154465.343]#033[39m #033[31m ERROR#033[39m #033[36mgithub.com/netfoundry/ziti-edge/controller/response.(*RequestResponderImpl).RespondWithError#033[39m: #033[96m{cause=[service with id dcd3d91d-9af6-437e-9ea5-7e78bd831a5b not found]} #033[39munhandled error: service with id dcd3d91d-9af6-437e-9ea5-7e78bd831a5b not found

plorenz · February 13, 2020, 7:01pm

Thank you, that’s helpful. Can you share what role attributes the identity in question has, as well as any relevant service policies and role attributes on that service?

dariuszSki · February 13, 2020, 7:15pm

{
  "meta": {
    "filterableFields": [
      "id",
      "createdAt",
      "updatedAt",
      "name",
      "fingerprint",
      "isVerified",
      "enrollmentToken",
      "enrollmentCreatedAt",
      "enrollmentExpiresAt"
    ],
    "pagination": {
      "limit": 10,
      "offset": 0,
      "totalCount": 1
    }
  },
  "data": [
    {
      "id": "d1066752-e348-4cb6-bdb5-6eeb506b11b3",
      "createdAt": "2020-02-12T21:50:35.7849366Z",
      "updatedAt": "2020-02-12T22:21:26.181338719Z",
      "_links": {
        "edge-router-policies": {
          "href": "./edge-routers/d1066752-e348-4cb6-bdb5-6eeb506b11b3/edge-routers"
        },
        "self": {
          "href": "./edge-routers/d1066752-e348-4cb6-bdb5-6eeb506b11b3"
        }
      },
      "tags": {},
      "name": "ziti-gw01",
      "fingerprint": "7B:D7:35:B9:5C:0F:39:6C:59:8C:E8:88:B5:C1:B7:9F:11:8B:0A:F8",
      "roleAttributes": [
        "dariusz"
      ],
      "isVerified": true,
      "isOnline": true,
      "enrollmentToken": null,
      "enrollmentJwt": null,
      "enrollmentCreatedAt": null,
      "enrollmentExpiresAt": null,
      "hostname": "zedeapptest01.centralus.cloudapp.azure.com:3022",
      "supportedProtocols": {
        "tls": "tls://zedeapptest01.centralus.cloudapp.azure.com:3022"
      }
    }
  ]
}

dariuszSki · February 13, 2020, 7:16pm

{
    "id": "3b1d5f9b-d4e5-4e35-9b80-b58d9c9b00dc",
    "createdAt": "2020-02-13T17:56:50.099314709Z",
    "updatedAt": "2020-02-13T18:58:45.098870003Z",
    "_links": {
      "edge-router-policies": {
        "href": "./identities/3b1d5f9b-d4e5-4e35-9b80-b58d9c9b00dc/edge-routers"
      },
      "self": {
        "href": "./identities/3b1d5f9b-d4e5-4e35-9b80-b58d9c9b00dc"
      },
      "service-policies": {
        "href": "./identities/3b1d5f9b-d4e5-4e35-9b80-b58d9c9b00dc/identities"
      }
    },
    "tags": {},
    "name": "dariusz-second",
    "type": {
      "entity": "identity-types",
      "id": "5b53fb49-51b1-4a87-a4e4-edda9716a970",
      "name": "Device",
      "_links": {
        "self": {
          "href": "./identity-types/5b53fb49-51b1-4a87-a4e4-edda9716a970"
        }
      }
    },
    "isDefaultAdmin": false,
    "isAdmin": false,
    "authenticators": {},
    "enrollment": {},
    "roleAttributes": [
      "dariusz1"
    ]
  }

dariuszSki · February 13, 2020, 7:17pm

[
  {
    "id": "dcd3d91d-9af6-437e-9ea5-7e78bd831a5b",
    "createdAt": "2020-02-12T22:22:17.902229017Z",
    "updatedAt": "2020-02-13T18:23:42.127592555Z",
    "_links": {
      "self": {
        "href": "./services/dcd3d91d-9af6-437e-9ea5-7e78bd831a5b"
      },
      "service-edge-router-policies": {
        "href": "./services/dcd3d91d-9af6-437e-9ea5-7e78bd831a5b/service-edge-router-policies"
      },
      "service-policies": {
        "href": "./services/dcd3d91d-9af6-437e-9ea5-7e78bd831a5b/identities"
      }
    },
    "tags": {},
    "name": "dariusz01",
    "endpointAddress": "tcp:127.0.0.1:22",
    "egressRouter": "d1066752-e348-4cb6-bdb5-6eeb506b11b3",
    "roleAttributes": [
      "dariusz"
    ],
    "permissions": [
      "Bind",
      "Dial"
    ],
    "configs": null,
    "config": {}
  }
]

dariuszSki · February 13, 2020, 7:24pm

[
  {
"id": "e711f1c1-17c0-4dc7-be8c-c7768d8880ea",
"createdAt": "2020-02-13T18:12:34.415029176Z",
"updatedAt": "2020-02-13T18:12:34.415029176Z",
"_links": {
  "edge-routers": {
    "href": "./edge-router-policies/e711f1c1-17c0-4dc7-be8c-c7768d8880ea/edge-routers"
  },
  "identities": {
    "href": "./edge-router-policies/e711f1c1-17c0-4dc7-be8c-c7768d8880ea/identities"
  },
  "self": {
    "href": "./edge-router-policies/e711f1c1-17c0-4dc7-be8c-c7768d8880ea"
  }
},
"tags": {},
"name": "EdgeRouterPolicy1",
"semantic": "AnyOf",
"edgeRouterRoles": [
  "#dariusz"
],
"identityRoles": null
  }
]

dariuszSki · February 13, 2020, 7:37pm

also, getting this error when trying to add an identity to a service policy through ZAC:
{“error”:{“args”:{“cause”:{},“urlVars”:{}},“cause”:{},“causeMessage”:“duplicate value ‘ServicePolicy1’ in unique index on servicePolicies store”,“code”:“UNHANDLED”,“message”:“An unhandled error occurred”,“requestId”:“d661cd77-c1e4-44b3-a93e-033aaab9f2a3”},“meta”:{“apiEnrolmentVersion”:“0.0.1”,“apiVersion”:“0.0.1”}}
will try through API to see if I get the same error

plorenz · February 13, 2020, 8:04pm

I see everything but the service policy, can you post that one, please?

dariuszSki · February 13, 2020, 8:31pm

[
  {
    "id": "abfffd81-2299-4d72-8eb1-02c1843ae466",
    "createdAt": "2020-02-13T20:25:38.203442946Z",
    "updatedAt": "2020-02-13T20:25:38.203442946Z",
    "_links": {
      "identities": {
        "href": "./service-policies/abfffd81-2299-4d72-8eb1-02c1843ae466/identities"
      },
      "self": {
        "href": "./service-policies/abfffd81-2299-4d72-8eb1-02c1843ae466"
      },
      "services": {
        "href": "./service-policies/abfffd81-2299-4d72-8eb1-02c1843ae466/services"
      }
    },
    "tags": {},
    "name": "ServicePolicy1",
    "type": "Bind",
    "semantic": "AnyOf",
    "serviceRoles": [
      "#dariusz"
    ],
    "identityRoles": [
      "#dariusz"
    ]
  }
]

plorenz · February 13, 2020, 9:15pm

I’m guessing you want type “Dial” for that policy, unless you’re trying to host that service? Either way, you shouldn’t get that failure. I’m going to see if I can replicate, but in the meantime do you want to try with a Dial policy?

dariuszSki · February 13, 2020, 11:49pm

Dial is better, but now I get the tcp reset.

[   2.457]    INFO github.com/netfoundry/ziti-edge/tunnel/intercept.updateServices: starting tunnel for newly available service dariusz01
[   2.462]    INFO github.com/netfoundry/ziti-edge/tunnel/intercept/tproxy.(*tProxyInterceptor).intercept: Adding rule iptables -t mangle -A NF-INTERCEPT [-m comment --comment dariusz01 -d 3.3.3.3/32 -p tcp --dport 2222 -j TPROXY --tproxy-mark 0x1/0x1 --on-ip=127.0.0.1 --on-port=37215]
[   2.480]    INFO github.com/netfoundry/ziti-edge/tunnel/intercept/tproxy.(*tProxyInterceptor).intercept: Adding rule iptables -t mangle -A NF-INTERCEPT [-m comment --comment dariusz01 -d 3.3.3.3/32 -p udp --dport 2222 -j TPROXY --tproxy-mark 0x1/0x1 --on-ip=127.0.0.1 --on-port=38385]
[  14.277]    INFO github.com/netfoundry/ziti-edge/tunnel/intercept/tproxy.(*tProxyInterceptor).accept.func1: received connection: 3.3.3.3:2222 --> 3.3.3.3:49230
[  19.658]   ERROR github.com/netfoundry/ziti-sdk-golang/ziti/internal/edge_impl.(*edgeConn).Connect: {connId=[1]} timeout waiting for response
[  19.658]   ERROR github.com/netfoundry/ziti-edge/tunnel.Run: zt.Dial(dariusz01) failed: timeout waiting for response

I don’t see any logs at the edge router corresponding to this request for connection, which makes sense that it comes back with a timeout.

dariuszSki · February 13, 2020, 11:54pm

when I do simple curl to port 3022 from client’s loptop, I get a response:

~/sandbox$ curl zedeapptest01.centralus.cloudapp.azure.com:3022
curl: (52) Empty reply from server

log at the edge router:

Feb 13 23:52:03 zedeapptest01 ziti-router: #033[34m[85923.820]#033[39m #033[31m ERROR#033[39m #033[36mgithub.com/netfoundry/ziti-foundation/channel2.(*classicListener).listener [tls:0.0.0.0:3022]#033[39m: error receiving hello (receive error (tls: first record does not look like a TLS handshake))

TheLumberjack · February 14, 2020, 2:36am

Curl to https not http is probably the issue?

jeremy.tellier · February 14, 2020, 3:16pm

The Duplicate Entry error was fixed in the last push.

plorenz · February 14, 2020, 4:41pm

@dariuszSki Do you have a CLI script that you’re using to set things up? I tried to reproduce the error you were seeing when you had a bind policy, but was unable to reproduce. If you’ve got a script that let’s you see the error, I can try and fix the root cause.

plorenz · February 14, 2020, 7:58pm

FYI found the issue and put up a PR with the fix here: https://github.com/netfoundry/ziti-edge/pull/84

Topic		Replies	Views
Ziti-router connection issue with controller	2	41	April 23, 2024
Ziti-tunneler unable to connect Ziti Tunnel	13	1014	May 6, 2020
Tunneler stop working Support	14	447	April 3, 2023
Terminators and service issue	3	171	October 17, 2023
Ziti-tunnel-sidecar logging ethzero.ziti.ui Ziti Tunnel	15	848	June 21, 2021

Tunneler 0.9.0 - Error - nil pagination in response to GET /services

Related Topics