Sorry for taking so long to reply!
I see that particular log has the following:
[ 0.014] FATAL edge/router/enroll.(*RestEnroller).Enroll: {cause=[could not retrieve token URL certificate: could not contact remote server [https://ziti.broken-mirror.net:1280]: Get "https://ziti.broken-mirror.net:1280": dial tcp 47.154.58.170:1280: connect: connection refused]} failed to parse JWT
I then attempted to open port 1280 on the firewall (it was not mentioned in the quickstart guide, so I only had the suggest 8440-8443 opened).
This resulted in
[ 41.568] INFO edge/router/enroll.(*RestEnroller).Enroll: registration complete
which seems to be a good sign.
However, ran-edge-router.log then complained about not being able to reach port 6262, so I opened that as well. Everything seemed to set up right (I’m assuming those aren’t needed as open ports, so I’ll try editing /etc/hosts so that it won’t hit the firewall next time).
However…
Controller starts up fine, but crashes when I start the router.
ran-edge-router.log:
[ 0.058] INFO ziti/ziti/router.run: {revision=[3d9801e73809] arch=[arm] build-date=[2023-02-13T21:49:17Z] routerId=[25ndDrUA0] go-version=[go1.19.5] configFile=[/home/pi/.ziti/quickstart/ran/ran-edge-router.yaml] os=[linux] version=[v0.27.5]} starting ziti-router
[ 0.059] INFO fabric/router/forwarder.(*Faulter).run: started
[ 0.059] INFO fabric/router/forwarder.(*Scanner).run: started
[ 0.059] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {poolType=[pool.link.dialer] minWorkers=[0] idleTime=[30s] maxQueueSize=[1000] maxWorkers=[32]} starting goroutine pool
[ 0.059] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {idleTime=[30s] poolType=[pool.route.handler] minWorkers=[0] maxWorkers=[128] maxQueueSize=[1000]} starting goroutine pool
[ 0.060] WARNING edge/router/internal/edgerouter.(*Config).LoadConfigFromMap: Invalid heartbeat interval [0] (min: 60, max: 10), setting to default [60]
[ 0.061] INFO fabric/router.(*Router).initializeCtrlEndpoints: controller endpoints file [/home/pi/.ziti/quickstart/ran/endpoints] doesn't exist. Using initial endpoints from config
[ 0.061] INFO fabric/router.(*Router).showOptions: ctrl = {"OutQueueSize":4,"MaxQueuedConnects":1,"MaxOutstandingConnects":16,"ConnectTimeout":1000000000,"DelayRxStart":false,"WriteTimeout":0}
[ 0.061] INFO fabric/router.(*Router).showOptions: metrics = {"ReportInterval":60000000000,"MessageQueueSize":10}
[ 0.061] INFO fabric/router.(*Router).initializeHealthChecks: starting health check with ctrl ping initially after 15s, then every 30s, timing out after 15s
[ 0.062] INFO fabric/router.(*Router).startXlinkDialers: started Xlink dialer with binding [transport]
[ 0.062] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {idleTime=[10s] poolType=[pool.listener.link] minWorkers=[1] maxWorkers=[16] maxQueueSize=[1]} starting goroutine pool
[ 0.062] INFO fabric/router.(*Router).startXlinkListeners: started Xlink listener with binding [transport] advertising [tls:ziti.broken-mirror.net:10080]
[ 0.063] INFO edge/router/xgress_edge.(*listener).Listen: {address=[tls:0.0.0.0:3022]} starting channel listener
[ 0.063] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {poolType=[pool.listener.xgress_edge] minWorkers=[1] maxQueueSize=[1] maxWorkers=[16] idleTime=[10s]} starting goroutine pool
[ 0.063] INFO fabric/router.(*Router).startXgressListeners: created xgress listener [edge] at [tls:0.0.0.0:3022]
[ 0.063] INFO edge/router/xgress_edge.(*Acceptor).Run: starting
[ 0.063] INFO fabric/router.(*Router).startXgressListeners: created xgress listener [tunnel] at []
[ 0.064] INFO fabric/router.(*Router).startControlPlane: router configured with 1 controller endpoints
[ 0.064] INFO fabric/router.(*Router).startControlPlane: connecting to controller at endpoing [tls:ziti.broken-mirror.net:6262]
[ 0.425] INFO edge/router/fabric.(*StateManagerImpl).StartHeartbeat: heartbeat starting
[ 0.429] INFO edge/router/xgress_edge_tunnel.(*tunneler).Start: {mode=[host]} creating interceptor
[ 0.429] INFO edge/router/xgress_edge.(*CertExpirationChecker).Run: waiting 8615h59m3.117828007s to renew certificates
[ 0.442] ERROR channel/v2.(*reconnectingImpl).Rx [u{reconnecting}->i{0Z4p}]: {error=[EOF]} rx error. closed peer and starting reconnection process
[ 0.442] INFO channel/v2.(*reconnectingImpl).pingInstance [u{reconnecting}->i{0Z4p}]: starting
[ 0.442] INFO channel/v2.(*reconnectingImpl).pingInstance [u{reconnecting}->i{0Z4p}]: exiting
[ 0.442] ERROR channel/v2.(*reconnectingDialer).Reconnect [u{reconnecting}->i{0Z4p} @tls:ziti.broken-mirror.net:6262]: unable to ping (use of closed network connection)
[ 0.457] WARNING edge/tunnel/dns.flushDnsCaches: {error=[exit status 1]} unable to flush dns caches, consider adding a dns flush to your restart process
[ 0.458] ERROR channel/v2.(*reconnectingDialer).Reconnect [u{reconnecting}->i{0Z4p} @tls:ziti.broken-mirror.net:6262]: reconnection attempt [#1] failed (read tcp 192.168.42.8:43996->47.154.58.170:6262: read: connection reset by peer)
panic: unaligned 64-bit atomic operation
goroutine 118 [running]:
runtime/internal/atomic.panicUnaligned()
runtime/internal/atomic/unaligned.go:8 +0x24
runtime/internal/atomic.Load64(0x45a8354)
runtime/internal/atomic/atomic_arm.s:280 +0x14
github.com/openziti/channel/v2.(*heartbeater).Tx(0x45a8330, 0x4586080, {0x21a28b4, 0x4ac4240})
github.com/openziti/channel/v2@v2.0.27/heartbeater.go:82 +0x1b8
github.com/openziti/channel/v2.(*channelImpl).txer(0x4ac4240)
github.com/openziti/channel/v2@v2.0.27/impl.go:422 +0x984
created by github.com/openziti/channel/v2.(*channelImpl).startMultiplex
github.com/openziti/channel/v2@v2.0.27/impl.go:287 +0x128
ran.log
[ 0.030] INFO ziti/ziti/controller.run: {os=[linux] nodeId=[ran] version=[v0.27.5] go-version=[go1.19.5] arch=[arm] revision=[3d9801e73809] build-date=[2023-02-13T21:49:17Z]} starting ziti-controller
[ 0.031] INFO fabric/events.(*Dispatcher).eventLoop: event dispatcher: started
[ 0.031] INFO storage/boltz.(*migrationManager).Migrate.func1: fabric datastore is up to date at version 5
[ 0.034] INFO fabric/controller/network.(*Network).showOptions: network = {
"CycleSeconds": 60,
"Smart": {
"RerouteFraction": 0.02,
"RerouteCap": 4,
"MinCostDelta": 15
},
"RouteTimeout": 10000000000,
"CreateCircuitRetries": 2,
"CtrlChanLatencyInterval": 10000000000,
"PendingLinkTimeout": 10000000000,
"MinRouterCost": 10,
"RouterConnectChurnLimit": 60000000000,
"InitialLinkLatency": 65000000000,
"MetricsReportInterval": 60000000000
}
[ 0.034] INFO fabric/controller.(*Controller).showOptions: ctrl = {
"OutQueueSize": 4,
"MaxQueuedConnects": 1,
"MaxOutstandingConnects": 16,
"ConnectTimeout": 1000000000,
"DelayRxStart": false,
"WriteTimeout": 0,
"NewListener": null,
"AdvertiseAddress": null
}
[ 1.711] INFO edge/controller/server.NewController: edge controller instance id: clezt21tk0000loa03d50b2ol
[ 1.712] INFO edge/controller/server.(*Controller).Initialize: initializing edge
[ 1.731] INFO storage/boltz.(*migrationManager).Migrate.func1: edge datastore is up to date at version 29
[ 1.736] INFO edge/controller/internal/policy.NewSessionEnforcer: {frequency=[5s] sessionTimeout=[30m0s]} session enforcer configured
[ 1.737] INFO edge/controller/server.(*Controller).Run: starting edge
[ 1.738] INFO fabric/metrics.GoroutinesPoolMetricsConfigF.func1.1: {poolType=[pool.listener.ctrl] minWorkers=[1] maxWorkers=[16] maxQueueSize=[1] idleTime=[10s]} starting goroutine pool
[ 1.739] INFO channel/v2.(*UnderlayDispatcher).Run: started
[ 1.755] INFO edge/controller/server.(*Controller).checkEdgeInitialized: edge initialized
[ 1.757] INFO fabric/controller/network.(*Network).Run: started
[ 2.229] INFO xweb/v2.(*Server).Start: starting ApiConfig to listen and serve tls on 0.0.0.0:1280 for server client-management with APIs: [edge-management edge-client fabric]
[ 13.173] INFO fabric/controller/handler_ctrl.(*bindHandler).BindChannel: {routerVersion=[v0.27.5] routerId=[25ndDrUA0]} router supports heartbeats
[ 13.173] INFO fabric/controller/handler_ctrl.(*CtrlAccepter).Bind: {routerId=[25ndDrUA0]} accepted new router connection [r/25ndDrUA0]
[ 13.184] INFO edge/controller/env.(*Broker).RouterConnected.func1: {routerId=[25ndDrUA0] routerName=[ran-edge-router] routerFingerprint=[0x4afad78]} broker detected edge router with id 25ndDrUA0 connecting
[ 13.184] INFO edge/controller/sync_strats.(*InstantStrategy).RouterConnected: {routerName=[ran-edge-router] sync_strategy=[instant] routerFingerprint=[0x4afad78] syncStatus=[SYNC_QUEUED] routerId=[25ndDrUA0]} edge router connected, adding to sync routerConnectedQueue
[ 13.186] INFO edge/controller/sync_strats.(*InstantStrategy).hello: {routerId=[25ndDrUA0] routerTxId=[3NS7DSSnn] routerName=[ran-edge-router] routerFingerprint=[0x4afad78] routerChannelIsOpen=[true] strategy=[instant]} edge router sync starting
[ 13.186] INFO edge/controller/sync_strats.(*InstantStrategy).hello: {routerTxId=[3NS7DSSnn] routerName=[ran-edge-router] routerFingerprint=[0x4afad78] routerChannelIsOpen=[true] strategy=[instant] routerId=[25ndDrUA0] syncStatus=[SYNC_HELLO]} sending edge router hello
[ 13.188] INFO fabric/controller/handler_ctrl.(*xctrlCloseHandler).HandleClose [ch{25ndDrUA0}->u{classic}->i{0Z4p}]: closing Xctrl instances
[ 13.188] WARNING fabric/controller/handler_ctrl.(*closeHandler).HandleClose: {routerId=[25ndDrUA0]} disconnected
[ 13.189] ERROR channel/v2.(*channelImpl).rxer [ch{25ndDrUA0}->u{classic}->i{0Z4p}]: rx error (read tcp 192.168.42.8:6262->192.168.42.1:43980: use of closed network connection)
panic: unaligned 64-bit atomic operation
goroutine 89 [running]:
runtime/internal/atomic.panicUnaligned()
runtime/internal/atomic/unaligned.go:8 +0x24
runtime/internal/atomic.Load64(0x4d6c264)
runtime/internal/atomic/atomic_arm.s:280 +0x14
github.com/openziti/channel/v2.(*heartbeater).Tx(0x4d6c240, 0x4ab9b40, {0x23d3d84, 0x47b3a70})
github.com/openziti/channel/v2@v2.0.27/heartbeater.go:82 +0x1b8
github.com/openziti/channel/v2.(*channelImpl).txer(0x47b3a70)
github.com/openziti/channel/v2@v2.0.27/impl.go:422 +0x984
created by github.com/openziti/channel/v2.(*channelImpl).startMultiplex
github.com/openziti/channel/v2@v2.0.27/impl.go:287 +0x128
Not sure where the panic is coming from. Running ziti on its own doesn’t immediately cause an error, it shows the usual CLI help text.
