Using the ziti-sdk-jvm

markamind · June 16, 2022, 10:52am

That seemed to work… but I cannot find the files

Any ideas?

markamind · June 16, 2022, 10:55am

ahh… you need to use

sudo ./gradlew run

Failed

Exception in thread “main” org.postgresql.util.PSQLException: Something unusual has occurred to cause the driver to fail. Please report this exception.
at org.postgresql.Driver.connect(Driver.java:282)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
at jdbc.postgres.App.main(App.java:37)
Caused by: java.nio.channels.UnresolvedAddressException

markamind · June 16, 2022, 10:55am

Think the problem relates to the controller

Tips?

markamind · June 16, 2022, 10:57am

PS… I did a docker down before starting it all up

TheLumberjack · June 16, 2022, 11:42am

It’s helpful for me if you put code into a ‘code block’. Personally, I use triple ticks and new lines.

it looks like this in text - and like this

It greatly helps if I can copy/paste your output. I thought Discourse had a “code block” button in their WYSIWYG editor, but I’m not seeing it.

My guess is that you didn’t edit src/main/java/jdbc/postgres/App.java and it’s still pointing to “c:/temp”. Edit that file, make it read Ziti.init("/tmp/java-identity.json", "".toCharArray(), false);.

Once you do that, copy your identity to /tmp/java-identity.json and it should “just work”. I just did these two steps in my linux environment and here’s my output:

./gradlew run
Downloading https://services.gradle.org/distributions/gradle-7.4.2-bin.zip
...........10%...........20%...........30%...........40%...........50%...........60%...........70%...........80%...........90%...........100%

Welcome to Gradle 7.4.2!

Here are the highlights of this release:
 - Aggregated test and JaCoCo reports
 - Marking additional test source directories as tests in IntelliJ
 - Support for Adoptium JDKs in Java toolchains

For more details see https://docs.gradle.org/7.4.2/release-notes.html

Starting a Gradle Daemon (subsequent builds will be faster)

> Task :run
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by retrofit2.Platform (file:/home/cd/.gradle/caches/modules-2/files-2.1/com.squareup.retrofit2/retrofit/2.9.0/d8fdfbd5da952141a665a403348b74538efc05ff/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Result from database is: a:1
Result from database is: b:2
Result from database is: c:3
Result from database is: d:4
Result from database is: e:5
Result from database is: f:6
Result from database is: g:7
Result from database is: h:8
Result from database is: i:9
Result from database is: j:0

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.4.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 21s

TheLumberjack · June 16, 2022, 12:26pm

I pushed a small change to the App.java just now. Now if your identity file doesn’t exist you’ll see an error like:

Exception in thread "main" java.lang.Exception: Identity file NOT FOUND. The first argument should be a path to the identity file. If not supplied the default path [/tmp/java-identity.json] is used.
	at jdbc.postgres.App.main(App.java:41)

TheLumberjack · June 16, 2022, 12:31pm

Execute with gradle using:

windows:

.\gradlew run --args="C:\temp\java-identity.json"

linux/mac:

./gradlew run --args="/mnt/v/temp/java-identity.json"

markamind · June 16, 2022, 12:36pm

Still the same problem .. it's weird.. I am sure its super simple..

TheLumberjack · June 16, 2022, 12:37pm

can you provide the full output from running the gradle command? I’d like to see what it’s outputting please

markamind · June 16, 2022, 12:38pm

...
opc@instance-20220518-1244 jdbc-postgres]$ sudo ./gradlew run --args="/mnt/v/temp/java-identity.json"

Task :run
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See SLF4J Error Codes for further details.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by retrofit2.Platform (file:/root/.gradle/caches/modules-2/files-2.1/com.squareup.retrofit2/retrofit/2.9.0/d8fdfbd5da952141a665a403348b74538efc05ff/retrofit-2.9.0.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of retrofit2.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Exception in thread "main" org.postgresql.util.PSQLException: Something unusual has occurred to cause the driver to fail. Please report this exception.
at org.postgresql.Driver.connect(Driver.java:282)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
at jdbc.postgres.App.main(App.java:37)
Caused by: java.nio.channels.UnresolvedAddressException
at java.base/sun.nio.ch.Net.checkAddress(Net.java:131)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.implConnect(UnixAsynchronousSocketChannelImpl.java:306)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.connect(AsynchronousSocketChannelImpl.java:210)
at org.openziti.net.nio.NetUtilsKt$connectSuspend$3.invokeSuspend(NetUtils.kt:72)
at org.openziti.net.nio.NetUtilsKt$connectSuspend$3.invoke(NetUtils.kt)
at org.openziti.net.nio.NetUtilsKt$connectSuspend$3.invoke(NetUtils.kt)
at kotlinx.coroutines.intrinsics.UndispatchedKt.startUndispatchedOrReturn(Undispatched.kt:89)
at kotlinx.coroutines.BuildersKt__Builders_commonKt.withContext(Builders.common.kt:165)
at kotlinx.coroutines.BuildersKt.withContext(Unknown Source)
at org.openziti.net.nio.NetUtilsKt.connectSuspend(NetUtils.kt:70)
at org.openziti.net.nio.AsyncSocketImpl$Connector$doConnect$1.invokeSuspend(AsyncSocketImpl.kt:47)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
at org.openziti.net.nio.AsyncSocketImpl$Connector$DefaultImpls.doConnect(AsyncSocketImpl.kt:45)
at org.openziti.net.nio.AsyncSocketImpl$DefaultConnector.doConnect(AsyncSocketImpl.kt:59)
at org.openziti.net.nio.AsyncSocketImpl$DefaultConnector.connect(AsyncSocketImpl.kt:61)
at org.openziti.net.nio.AsyncSocketImpl.connect(AsyncSocketImpl.kt:113)
at java.base/java.net.Socket.connect(Socket.java:608)
at org.postgresql.core.PGStream.createSocket(PGStream.java:241)
at org.postgresql.core.PGStream.(PGStream.java:98)
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:109)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:235)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.(PgConnection.java:223)
at org.postgresql.Driver.makeConnection(Driver.java:402)
at org.postgresql.Driver.connect(Driver.java:261)
... 3 more

Task :run FAILED

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':run'.

Process 'command '/usr/java/jdk-11.0.15.1/bin/java'' finished with non-zero exit value 1

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.

Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See Command-Line Interface

BUILD FAILED in 31s
3 actionable tasks: 1 executed, 2 up-to-date

...

TheLumberjack · June 16, 2022, 12:39pm

Let’s check to see if we’re both using the same version of java … That might be it…

jdbc-postgres: java -version
openjdk version "11.0.14.1" 2022-02-08

TheLumberjack · June 16, 2022, 12:41pm

It doesn’t need to be ‘the same’ btw - you don’t NEED that version, I’m just interested in what version you are running Also I’ll get some ziti cli commands to make sure we’ve got the right config in place.

Finally if you run docker logs pg_postgres-db_1 - what does it show?

markamind · June 16, 2022, 12:48pm

how do you find this out..not sure which container to connect to or what command to run..

sudo docker exec -it -u root docker_ziti-private-blue_1 /bin/bash
java -version
bash: java: command not found

markamind · June 16, 2022, 12:49pm

sudo docker logs pg_postgres-db_1
Error: No such container: pg_postgres-db_1

markamind · June 16, 2022, 12:52pm

sudo docker logs docker_postgres-db_1

PostgreSQL Database directory appears to contain a database; Skipping initialization

2022-06-16 11:55:38.349 UTC [1] LOG: starting PostgreSQL 14.3 (Debian 14.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-06-16 11:55:38.351 UTC [1] LOG: listening on IPv4 address “0.0.0.0”, port 5432
2022-06-16 11:55:38.351 UTC [1] LOG: listening on IPv6 address “::”, port 5432
2022-06-16 11:55:38.354 UTC [1] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432”
2022-06-16 11:55:38.362 UTC [26] LOG: database system was shut down at 2022-06-16 10:56:35 UTC
2022-06-16 11:55:38.463 UTC [1] LOG: database system is ready to accept connections
2022-06-16 12:00:20.172 UTC [1] LOG: received fast shutdown request
2022-06-16 12:00:20.260 UTC [1] LOG: aborting any active transactions
2022-06-16 12:00:20.657 UTC [1] LOG: background worker “logical replication launcher” (PID 32) exited with exit code 1
2022-06-16 12:00:20.670 UTC [27] LOG: shutting down
2022-06-16 12:00:21.345 UTC [1] LOG: database system is shut down

PostgreSQL Database directory appears to contain a database; Skipping initialization

2022-06-16 12:01:29.758 UTC [1] LOG: starting PostgreSQL 14.3 (Debian 14.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-06-16 12:01:29.760 UTC [1] LOG: listening on IPv4 address “0.0.0.0”, port 5432
2022-06-16 12:01:29.761 UTC [1] LOG: listening on IPv6 address “::”, port 5432
2022-06-16 12:01:29.766 UTC [1] LOG: listening on Unix socket “/var/run/postgresql/.s.PGSQL.5432”
2022-06-16 12:01:30.053 UTC [26] LOG: database system was shut down at 2022-06-16 12:00:21 UTC
2022-06-16 12:01:30.380 UTC [1] LOG: database system is ready to accept connections

TheLumberjack · June 16, 2022, 12:55pm

Cool you got it I was typing up how to find that name… ok - so it doesn’t seem like the connection made it to the postgres db. Can you do the same thing for what should be something like docker logs docker_ziti-private-blue_1 too?

And, let’s make sure your config is correct. If you run:

ziti edge list configs -j | grep zitified
                    "zitified-postgres"

do you see “zitified-postgres” like I show here?

Assuming you do - the next question is to run policy-advisor to make sure that identity has the proper access:

ziti edge policy-advisor identities java-identity -q
OKAY : java-identity (2) -> private-postgres (5) Common Routers: (2/2) Dial: Y Bind: N

markamind · June 16, 2022, 12:58pm

docker logs docker_ziti-private-blue_1

…
Token: de8ef858-c916-4b2f-a9cf-5b1f7051844a
Saving identity ‘default’ to /openziti/ziti-cli.json
CREATING PRIVATE ROUTER CONFIG
Router configuration file written to: /openziti/ziti-private-blue.yaml
---------- Creating edge-router ziti-private-blue…
---------- Found existing edge-router ziti-private-blue…

An update with v0.25.13 is available for ziti-router v0.25.10 from

[ 0.897] INFO ziti/ziti-router/subcmd.run: {routerId=[OdC.XtNrw] configFile=[/openziti/ziti-private-blue.yaml] revision=[a32a117f9472] version=[v0.25.10] go-version=[go1.18.2] os=[linux] arch=[amd64] build-date=[2022-05-24T19:19:48Z]} starting ziti-router
[ 0.899] WARNING edge/router/internal/edgerouter.(*Config).LoadConfigFromMap: Invalid heartbeat interval [0] (min: 60, max: 10), setting to default [60]
[ 0.899] INFO fabric/router/forwarder.(*Faulter).run: started
[ 0.899] INFO fabric/router/forwarder.(*Scanner).run: started
[ 0.903] INFO fabric/router.(*Router).showOptions: ctrl = {“OutQueueSize”:4,“MaxQueuedConnects”:1,“MaxOutstandingConnects”:16,“ConnectTimeout”:1000000000,“DelayRxStart”:false,“WriteTimeout”:0}
[ 0.903] INFO fabric/router.(*Router).showOptions: metrics = {“ReportInterval”:60000000000,“MessageQueueSize”:10}
[ 0.903] INFO fabric/router.(*Router).initializeHealthChecks: starting health check with ctrl ping initially after 15s, then every 30s, timing out after 15s
[ 0.903] INFO fabric/router.(*Router).startXlinkDialers: started Xlink dialer with binding [transport]
[ 0.972] INFO edge/router/xgress_edge.(*listener).Listen: {address=[tls:0.0.0.0:3022]} starting channel listener
[ 0.972] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {idleTime=[10s] maxWorkers=[16] poolType=[pool.listener.xgress_edge] minWorkers=[1] maxQueueSize=[1]} starting goroutine pool
[ 0.973] INFO fabric/router.(*Router).startXgressListeners: created xgress listener [edge] at [tls:0.0.0.0:3022]
[ 0.973] INFO fabric/router.(*Router).startXgressListeners: created xgress listener [tunnel] at
[ 0.974] INFO edge/router/xgress_edge.(*Acceptor).Run: starting
[ 1.375] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {poolType=[pool.link.dialer] minWorkers=[0] idleTime=[30s] maxWorkers=[32] maxQueueSize=[1000]} starting goroutine pool
[ 1.376] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {maxQueueSize=[1000] poolType=[pool.route.handler] minWorkers=[0] maxWorkers=[128] idleTime=[30s]} starting goroutine pool
[ 1.377] INFO edge/router/fabric.(*StateManagerImpl).StartHeartbeat: heartbeat starting
[ 1.377] WARNING edge/tunnel/dns.flushDnsCaches: {error=[exec: “systemd-resolve”: executable file not found in $PATH]} unable to find systemd-resolve in path, consider adding a dns flush to your restart process
[ 1.381] INFO edge/router/xgress_edge.(*CertExpirationChecker).Run: waiting 8591h56m23.345792903s to renew certificates
[ 1.683] INFO edge/router/handler_edge_ctrl.(*helloHandler).HandleReceive.func1: received server hello, replying
[ 1.990] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkProtocol=[tls] routerVersion=[v0.25.10] linkId=[5JjLPPQV28lqZfTySF8yJ1] routerId=[j-X6IlNrm] address=[tls:ziti-edge-router:10080]} dialing link
[ 1.990] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkId=[7azULBa3yCH6NkNgan2OaC] routerId=[Gaz.Ilqrw] address=[tls:ziti-fabric-router-br:10080] linkProtocol=[tls] routerVersion=[v0.25.10]} dialing link
[ 2.297] INFO edge/router/handler_edge_ctrl.(*apiSessionAddedHandler).instantSync: {strategy=[instant]} first api session syncId [cl4gz2i4z000v8al7yal22kwr], starting
[ 2.473] INFO edge/router/handler_edge_ctrl.(*apiSessionSyncTracker).Add: received api session sync chunk 0, isLast=true
[ 3.883] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {routerVersion=[v0.25.10] linkId=[7azULBa3yCH6NkNgan2OaC] routerId=[Gaz.Ilqrw]} link destination support heartbeats
[ 4.291] INFO edge/router/handler_edge_ctrl.(*apiSessionAddedHandler).applySync: finished sychronizing api sessions [count: 13, syncId: cl4gz2i4z000v8al7yal22kwr, duration: 91.663µs]
[ 4.391] WARNING channel.(*classicDialer).Create [tls:ziti-edge-router:10080]: {error=[read tcp 172.24.0.7:42960->172.24.0.8:10080: read: connection reset by peer]} error initiating channel with hello
[ 4.393] INFO channel.getRetryVersionFor: defaulting to version 2
[ 4.393] WARNING channel.(*classicDialer).Create [tls:ziti-edge-router:10080]: Retrying dial with protocol version 2
[ 5.275] WARNING channel.(*classicDialer).Create [tls:ziti-fabric-router-br:10080]: {error=[write tcp 172.24.0.7:54090->172.24.0.9:10080: write: broken pipe]} error initiating channel with hello
[ 5.275] INFO channel.getRetryVersionFor: defaulting to version 2
[ 5.275] WARNING channel.(*classicDialer).Create [tls:ziti-fabric-router-br:10080]: Retrying dial with protocol version 2
[ 7.283] ERROR channel.(*heartbeater).sendHeartbeatIfQueueFree: {error=[context deadline exceeded] channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} failed to send heartbeat
[ 10.579] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {linkId=[7azULBa3yCH6NkNgan2OaC] routerId=[Gaz.Ilqrw] routerVersion=[v0.25.10]} link destination support heartbeats
[ 10.592] INFO fabric/router.(*xlinkAccepter).Accept: accepted new link [l/7azULBa3yCH6NkNgan2OaC]
[ 10.592] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkProtocol=[tls] routerVersion=[v0.25.10] linkId=[7azULBa3yCH6NkNgan2OaC] routerId=[Gaz.Ilqrw] address=[tls:ziti-fabric-router-br:10080]} link registered
[ 10.778] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkProtocol=[tls] routerVersion=[v0.25.10] linkId=[zhUDEBEG8pvGeXKlse3Ge] routerId=[.nj1Ilqrm] address=[tls:ziti-edge-router-wss:10080]} dialing link
[ 11.181] ERROR fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {error=[error dialing outgoing link [l/5JjLPPQV28lqZfTySF8yJ1]: error dialing payload channel for [l/5JjLPPQV28lqZfTySF8yJ1]: write tcp 172.24.0.7:42980->172.24.0.8:10080: write: broken pipe] routerId=[j-X6IlNrm] address=[tls:ziti-edge-router:10080] linkProtocol=[tls] routerVersion=[v0.25.10] linkId=[5JjLPPQV28lqZfTySF8yJ1]} link dialing failed
[ 15.474] WARNING channel.(*classicDialer).Create [tls:ziti-edge-router-wss:10080]: {error=[write tcp 172.24.0.7:39916->172.24.0.6:10080: write: broken pipe]} error initiating channel with hello
[ 15.474] INFO channel.getRetryVersionFor: defaulting to version 2
[ 15.475] WARNING channel.(*classicDialer).Create [tls:ziti-edge-router-wss:10080]: Retrying dial with protocol version 2
[ 19.173] ERROR fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkProtocol=[tls] routerVersion=[v0.25.10] error=[error dialing outgoing link [l/zhUDEBEG8pvGeXKlse3Ge]: error dialing payload channel for [l/zhUDEBEG8pvGeXKlse3Ge]: write tcp 172.24.0.7:39930->172.24.0.6:10080: write: broken pipe] linkId=[zhUDEBEG8pvGeXKlse3Ge] routerId=[.nj1Ilqrm] address=[tls:ziti-edge-router-wss:10080]} link dialing failed
[ 143.987] ERROR channel.(*reconnectingImpl).Tx [u{reconnecting}->i{eDgk}]: tx error (write tcp 172.24.0.7:53534->172.24.0.4:6262: write: broken pipe). starting reconnection process
[ 143.589] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 144.074] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 143.690] ERROR channel.(*heartbeater).sendHeartbeat: {error=[timeout waiting for message to be written to wire: context deadline exceeded] channelId=[ch{ctrl}->u{reconnecting}->i{eDgk}]} failed to send heartbeat
[ 144.292] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{eDgk}]: starting
[ 143.794] ERROR channel.(*heartbeater).sendHeartbeat: {error=[timeout waiting for message to be written to wire: context deadline exceeded] channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} failed to send heartbeat
[ 144.446] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 144.448] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 144.449] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{eDgk}]: exiting
[ 144.449] ERROR channel.(*reconnectingDialer).Reconnect [u{reconnecting}->i{eDgk} @tls:ziti-controller:6262]: unable to ping (write tcp 172.24.0.7:53534->172.24.0.4:6262: write: broken pipe)
[ 144.476] ERROR channel.(*reconnectingImpl).Rx [u{reconnecting}->i{eDgk}]: {error=[tls: failed to send closeNotify alert (but connection was closed anyway): write tcp 172.24.0.7:53534->172.24.0.4:6262: write: broken pipe]} error closing peer after rx error
[ 144.480] ERROR channel.(*reconnectingImpl).Rx [u{reconnecting}->i{eDgk}]: {error=[EOF]} rx error. closed peer and starting reconnection process
[ 144.482] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkId=[2Eyo6X7fJi10bmjvyHi0xS] routerId=[j-X6IlNrm] address=[tls:ziti-edge-router:10080] linkProtocol=[tls] routerVersion=[v0.25.10]} dialing link
[ 145.063] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 145.593] ERROR channel.(*reconnectingDialer).Reconnect [u{reconnecting}->i{eDgk} @tls:ziti-controller:6262]: reconnection attempt [#1] failed (context deadline exceeded)
[ 146.382] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 146.894] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 147.487] WARNING channel.(*classicDialer).Create [tls:ziti-edge-router:10080]: {error=[write tcp 172.24.0.7:43032->172.24.0.8:10080: write: broken pipe]} error initiating channel with hello
[ 147.487] INFO channel.getRetryVersionFor: defaulting to version 2
[ 147.487] WARNING channel.(*classicDialer).Create [tls:ziti-edge-router:10080]: Retrying dial with protocol version 2
[ 147.886] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 148.896] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 149.195] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {linkId=[2Eyo6X7fJi10bmjvyHi0xS] routerId=[j-X6IlNrm] routerVersion=[v0.25.10]} link destination support heartbeats
[ 150.251] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 150.575] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {routerVersion=[v0.25.10] linkId=[2Eyo6X7fJi10bmjvyHi0xS] routerId=[j-X6IlNrm]} link destination support heartbeats
[ 150.576] INFO fabric/router.(*xlinkAccepter).Accept: accepted new link [l/2Eyo6X7fJi10bmjvyHi0xS]
[ 150.576] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkId=[2Eyo6X7fJi10bmjvyHi0xS] routerId=[j-X6IlNrm] address=[tls:ziti-edge-router:10080] linkProtocol=[tls] routerVersion=[v0.25.10]} link registered
[ 151.075] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 151.285] INFO channel.(*reconnectingImpl).Tx [u{reconnecting}->i{eDgk}]: reconnected
[ 151.289] INFO edge/router/xgress_edge_tunnel.(*Factory).NotifyOfReconnect: control channel reconnected, re-establishing hosted services
[ 151.373] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{5876}]: starting
[ 151.373] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{5876}]: exiting
[ 151.373] INFO channel.(*reconnectingImpl).Rx [u{reconnecting}->i{eDgk}]: reconnected
[ 151.408] INFO edge/router/handler_edge_ctrl.(*helloHandler).HandleReceive.func1: received server hello, replying
[ 151.914] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 151.977] INFO edge/router/handler_edge_ctrl.(*apiSessionAddedHandler).instantSync: {strategy=[instant]} api session syncId [cl4gz5p9c00518al7fra8ry4x], starting
[ 151.977] INFO edge/router/handler_edge_ctrl.(*apiSessionSyncTracker).Add: received api session sync chunk 0, isLast=true
[ 153.007] INFO edge/router/handler_edge_ctrl.(*apiSessionAddedHandler).applySync: finished sychronizing api sessions [count: 14, syncId: cl4gz5p9c00518al7fra8ry4x, duration: 60.866µs]
[ 284.373] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 284.383] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/2Eyo6X7fJi10bmjvyHi0xS}->u{classic}->i{50K3}]} heartbeat not received in time, link may be unhealthy
[ 284.508] ERROR channel.(*channelImpl).rxer [ch{l/2Eyo6X7fJi10bmjvyHi0xS}->u{classic}->i{LpN5}]: rx error (read tcp 172.24.0.7:43050->172.24.0.8:10080: read: connection reset by peer)
[ 286.277] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 286.278] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 286.285] ERROR channel.(*heartbeater).sendHeartbeat: {error=[timeout waiting for message to be written to wire: context deadline exceeded] channelId=[ch{ctrl}->u{reconnecting}->i{5876}]} failed to send heartbeat
[ 286.293] ERROR channel.(*reconnectingImpl).Tx [u{reconnecting}->i{5876}]: tx error (use of closed network connection). starting reconnection process
[ 286.299] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{5876}]: starting
[ 286.299] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{5876}]: exiting
[ 286.299] ERROR channel.(*reconnectingDialer).Reconnect [u{reconnecting}->i{5876} @tls:ziti-controller:6262]: unable to ping (use of closed network connection)
[ 284.312] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 286.394] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 284.501] ERROR channel.(*reconnectingImpl).Rx [u{reconnecting}->i{5876}]: {error=[tls: failed to send closeNotify alert (but connection was closed anyway): write tcp 172.24.0.7:53646->172.24.0.4:6262: write: broken pipe]} error closing peer after rx error
[ 286.709] ERROR channel.(*reconnectingImpl).Rx [u{reconnecting}->i{5876}]: {error=[EOF]} rx error. closed peer and starting reconnection process
[ 284.516] INFO fabric/router/handler_link.(*closeHandler).HandleClose [ch{l/2Eyo6X7fJi10bmjvyHi0xS}->u{classic}->i{LpN5}]: {routerId=[j-X6IlNrm] linkId=[2Eyo6X7fJi10bmjvyHi0xS]} link closed
[ 286.307] INFO fabric/router/handler_link.(*closeHandler).HandleClose [ch{l/2Eyo6X7fJi10bmjvyHi0xS}->u{classic}->i{50K3}]: {linkId=[2Eyo6X7fJi10bmjvyHi0xS] routerId=[j-X6IlNrm]} link closed
[ 286.884] ERROR channel.(*channelImpl).txer [ch{l/2Eyo6X7fJi10bmjvyHi0xS}->u{classic}->i{LpN5}]: {error=[write tcp 172.24.0.7:43050->172.24.0.8:10080: write: broken pipe]} write error
[ 286.829] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/2Eyo6X7fJi10bmjvyHi0xS}->u{classic}->i{LpN5}]} heartbeat not received in time, link may be unhealthy
[ 286.892] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 286.890] ERROR fabric/router/handler_link.(*errorHandler).HandleError [ch{l/2Eyo6X7fJi10bmjvyHi0xS}->u{classic}->i{LpN5}]: {error=[write tcp 172.24.0.7:43050->172.24.0.8:10080: write: broken pipe] linkId=[2Eyo6X7fJi10bmjvyHi0xS] routerId=[j-X6IlNrm]} link error, closing
[ 286.993] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 287.903] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 287.903] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 287.908] ERROR channel.(*reconnectingDialer).Reconnect [u{reconnecting}->i{5876} @tls:ziti-controller:6262]: reconnection attempt [#1] failed (context deadline exceeded)
[ 288.293] INFO edge/tunnel/intercept.SetDnsInterceptIpRange: dns intercept IP range: 100.64.0.1 - 100.127.255.254
[ 288.302] INFO edge/tunnel/intercept/host.(*interceptor).Start: starting host interceptor
[ 288.886] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 288.893] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 289.886] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 289.894] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 290.886] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 290.894] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 291.885] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 291.894] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 292.885] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 292.893] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 293.027] INFO channel.(*reconnectingImpl).Tx [u{reconnecting}->i{5876}]: reconnected
[ 293.076] INFO edge/router/xgress_edge_tunnel.(*Factory).NotifyOfReconnect: control channel reconnected, re-establishing hosted services
[ 293.079] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {routerVersion=[v0.25.10] linkId=[4GFVDtIU30xamSOah3Maqx] routerId=[j-X6IlNrm] existingLinkId=[2Eyo6X7fJi10bmjvyHi0xS] address=[tls:ziti-edge-router:10080] linkProtocol=[tls]} existing link found
[ 293.081] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {linkProtocol=[tls] routerVersion=[v0.25.10] linkId=[18xsTlc4KKYGdhzZyO2XaY] routerId=[.nj1Ilqrm] address=[tls:ziti-edge-router-wss:10080]} dialing link
[ 293.082] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {routerVersion=[v0.25.10] linkId=[1oPmMM5nTFmsKclZ95JXPc] routerId=[Gaz.Ilqrw] existingLinkId=[7azULBa3yCH6NkNgan2OaC] address=[tls:ziti-fabric-router-br:10080] linkProtocol=[tls]} existing link found
[ 293.083] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{6Mj5}]: starting
[ 293.083] INFO channel.(*reconnectingImpl).pingInstance [u{reconnecting}->i{6Mj5}]: exiting
[ 293.084] INFO channel.(*reconnectingImpl).Rx [u{reconnecting}->i{5876}]: reconnected
[ 293.275] INFO edge/router/handler_edge_ctrl.(*helloHandler).HandleReceive.func1: received server hello, replying
[ 293.382] INFO edge/router/handler_edge_ctrl.(*apiSessionAddedHandler).instantSync: {strategy=[instant]} api session syncId [cl4gz8qo9008y8al78qsfcx4t], starting
[ 293.383] INFO edge/router/handler_edge_ctrl.(*apiSessionSyncTracker).Add: received api session sync chunk 0, isLast=true
[ 293.386] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {routerVersion=[v0.25.10] linkId=[1ixOGdNSPNA1hDs1Dad77C] routerId=[.nj1Ilqrm] address=[tls:ziti-edge-router-wss:10080] linkProtocol=[tls]} unable to dial, dial already in progress
[ 293.492] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {routerVersion=[v0.25.10] linkId=[18xsTlc4KKYGdhzZyO2XaY] routerId=[.nj1Ilqrm]} link destination support heartbeats
[ 293.784] INFO fabric/router/handler_link.(*bindHandler).BindChannel: {routerId=[.nj1Ilqrm] routerVersion=[v0.25.10] linkId=[18xsTlc4KKYGdhzZyO2XaY]} link destination support heartbeats
[ 293.791] INFO fabric/router.(*xlinkAccepter).Accept: accepted new link [l/18xsTlc4KKYGdhzZyO2XaY]
[ 293.794] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {address=[tls:ziti-edge-router-wss:10080] linkProtocol=[tls] routerVersion=[v0.25.10] linkId=[18xsTlc4KKYGdhzZyO2XaY] routerId=[.nj1Ilqrm]} link registered
[ 293.885] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 293.913] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 294.386] INFO edge/router/handler_edge_ctrl.(*apiSessionAddedHandler).applySync: finished sychronizing api sessions [count: 15, syncId: cl4gz8qo9008y8al78qsfcx4t, duration: 2.002229ms]
[ 294.886] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{LqXW}]} heartbeat not received in time, link may be unhealthy
[ 294.904] WARNING fabric/router/handler_link.(*heartbeatCallback).CheckHeartBeat: {channelId=[ch{l/7azULBa3yCH6NkNgan2OaC}->u{classic}->i{52PL}]} heartbeat not received in time, link may be unhealthy
[ 353.392] INFO fabric/router/handler_ctrl.(*dialHandler).handle |link, linkDialer|: {address=[tls:ziti-edge-router-wss:10080] existingLinkId=[18xsTlc4KKYGdhzZyO2XaY] linkProtocol=[tls] routerVersion=[v0.25.10] linkId=[6WT0PfRbnO7O6ELdbJCB2g] routerId=[.nj1Ilqrm]} existing link found

markamind · June 16, 2022, 1:00pm

…

ziti edge list configs -j | grep zitified
“zitified-postgres”

markamind · June 16, 2022, 1:01pm

ziti edge policy-advisor identities java-identity -q
OKAY : java-identity (2) → private-postgres (5) Common Routers: (1/2) Dial: Y Bind: N

TheLumberjack · June 16, 2022, 1:05pm

Thank you for all those logs and commands. I’m somewhat stumped now. Things all seem to be configured fine… That’s good. The errors in the private blue are somewhat surprising to see but they don’t seem to be relevant.

Can you try ziti edge list terminators too? Let’s make sure there’s a terminator.

I’m scratching my head as to what is wrong, it’s always something simple but finding it sometimes is a challenge.

Topic		Replies	Views
Looking for ziti-sdk-js Usage Examples and API Documentation SDK Questions	1	90	February 18, 2024
Making a DB connection in a ziti overlay SDK Questions	3	241	July 7, 2022
Need help with Contributing to Ziti Building/Development	8	342	May 25, 2022
Need starter guide for newbie	2	196	March 1, 2022
New in ziti and want to learn it General Questions	3	245	March 3, 2022

Using the ziti-sdk-jvm

windows:

linux/mac:

Related Topics