Thank you for your patience and guidance!
I wiped my LXC, emptied out the folder/stack, which is in my user's home folder (~/) and started from scratch again. eg. /home/myusername/ziti/
On this machine my logged in user is UID 1000.
First I simply followed this without any changes. The docker compose up command successfully ends shortly after
INFO ziti/router/state.(*apiSessionAddedHandler).applySync: finished synchronizing api sessions
So then I added the env_file: ".env" variable to the compose file and created a .env file like the one above to bind-mount the config folder eg. /home/myusername/ziti/config. Then I try to recreate with one of these commands below:
docker compose up --force-recreate quickstart
--or--
docker compose down -v --remove-orphans
sudo rm -r config/
docker compose up
Both of these methods result in the container failing to successfully create and ends with a timeout and the system restarting over and over:
I should say, it doesn't matter if I bind-mount by using the .env file or by changing the compose.yml file to ./config:/home/ziggy. The issue seems to be specifying the bind-mount...
quickstart-1 | [ 0.336] INFO ziti/controller/server.NewController: edge controller instance id: cm26dbqts000001osth4fkwkk
quickstart-1 | [ 0.336] INFO ziti/controller.(*Controller).RegisterXmgmt: adding xmgmt *server.submgmt, enabled? true
quickstart-1 | [ 0.336] INFO ziti/controller/server.(*Controller).Initialize: initializing edge
quickstart-1 | [ 0.342] INFO ziti/controller/internal/policy.NewSessionEnforcer: {sessionTimeout=[30m0s] frequency=[5s]} session enforcer configured
quickstart-1 | [ 0.343] INFO ziti/controller/server.(*Controller).Run: starting edge
quickstart-1 | [ 0.343] INFO ziti/controller.(*Controller).Run.GoroutinesPoolMetricsConfigF.func1.1: {maxWorkers=[16] poolType=[pool.listener.ctrl] minWorkers=[1] idleTime=[10s] maxQueueSize=[1]} starting goroutine pool
quickstart-1 | [ 0.347] INFO channel/v3.(*UnderlayDispatcher).Run: started
quickstart-1 | [ 0.348] INFO ziti/controller/server.(*Controller).checkEdgeInitialized: edge initialized
quickstart-1 | [ 0.416] INFO ziti/controller/zac.ZitiAdminConsoleFactory.New: initializing ZAC SPA Handler from /ziti-console
quickstart-1 | [ 0.416] INFO xweb/v2.(*Server).Start: starting ApiConfig to listen and serve tls on 0.0.0.0:1280 for server client-management with APIs: [edge-management edge-client fabric zac]
quickstart-1 | [ 0.417] INFO ziti/controller/network.(*Network).Run: started
quickstart-1 | timed out waiting for controller: https://my_proxmox_public_ip:1280 <---------- time out here then the container restarts (and proceeds to boot loop)
quickstart-1 | Environment left intact at: /home/ziggy/quickstart
I then repeated the same process on the arm64 VPS. My logged in user is 1001 on this server.
If I don't use the .env and use the default compose.yml settings (either UID-1000 or changing it to 1001), the system successfully come up to:
INFO ziti/router/state.(*apiSessionAddedHandler).applySync: finished synchronizing api sessions
But if I bind-mount the config folder using the .env file or by specifying it in the compose.yml then the I get a similar same behavior as on LXC server with timeout and boot looping, but the log is a little different:
[ 0.803] INFO xweb/v2.(*Server).Start: starting ApiConfig to listen and serve tls on 0.0.0.0:1280 for server client-management with APIs: [edge-management edge-client fabric zac]
[ 0.804] INFO ziti/controller/network.(*Network).Run: started
[ 0.863] INFO ziti/ziti/cmd/edge.(*QuickstartOpts).run: Controller online. Continuing...
[ 0.863] ERROR ziti/ziti/router.run: {version=[v1.1.15] go-version=[go1.23.1] configFile=[/home/ziggy/quickstart/quickstart-router.yaml] arch=[arm64] revision=[0eec47ce3c80] build-date=[2024-10-02T13:03:23Z] os=[linux] error=[open /home/ziggy/quickstart/quickstart-router.yaml: no such file or directory]} error loading ziti router config
panic: open /home/ziggy/quickstart/quickstart-router.yaml: no such file or directory
goroutine 33 [running]:
github.com/openziti/ziti/ziti/router.run(0x4000806608, {0x40024874f0, 0x1, 0x3abf0dc?})
github.com/openziti/ziti/ziti/router/run.go:66 +0x1200
github.com/spf13/cobra.(*Command).execute(0x4000806608, {0x4002487490, 0x1, 0x1})
github.com/spf13/cobra@v1.8.1/command.go:989 +0x81c
github.com/spf13/cobra.(*Command).ExecuteC(0x4000806608)
github.com/spf13/cobra@v1.8.1/command.go:1117 +0x344
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/openziti/ziti/ziti/cmd/edge.(*QuickstartOpts).run.func2()
github.com/openziti/ziti/ziti/cmd/edge/quickstart.go:303 +0xa4
created by github.com/openziti/ziti/ziti/cmd/edge.(*QuickstartOpts).run in goroutine 1
github.com/openziti/ziti/ziti/cmd/edge/quickstart.go:297 +0x13f0
DEBUG: run command is: ziti edge quickstart --home /home/ziggy/quickstart --ctrl-address my_vps_public_ip --ctrl-port 1280 --router-address my_vps_public_ip --router-port 3022 --password my_password
Controller running... Configuring and starting Router...
[ 0.000] INFO ziti/ziti/cmd/edge.(*QuickstartOpts).run: permanent --home '/home/ziggy/quickstart' will not be removed on exit
[ 0.010] WARNING ziti/controller/config.LoadConfig: this environment is using a default generated trust domain [spiffe://00b244da3c76803c8ef3829afe17cf01fc33b459], it is recommended that a trust domain is specified in configuration via URI SANs or the 'trustDomain' field
[ 0.010] WARNING ziti/controller/config.LoadConfig: this environment is using a default generated trust domain [spiffe://00b244da3c76803c8ef3829afe17cf01fc33b459], it is recommended that if network components have enrolled that the generated trust domain be added to the configuration field 'additionalTrustDomains' array when configuring a explicit trust domain
[ 0.011] INFO ziti/ziti/controller.run: {version=[v1.1.15] os=[linux] go-version=[go1.23.1] build-date=[2024-10-02T13:03:23Z] nodeId=[client] revision=[0eec47ce3c80] arch=[arm64]} starting ziti-controller
[ 0.015] INFO ziti/controller/db.RunMigrations.(*migrationManager).Migrate.func1: edge datastore is up to date at version 37
[ 0.659] INFO ziti/common/metrics.ConfigureGoroutinesPoolMetrics.GoroutinesPoolMetricsConfigF.func1.1: {poolType=[pool.router.messaging] maxWorkers=[100] idleTime=[30s] maxQueueSize=[100] minWorkers=[0]} starting goroutine pool
[ 0.659] INFO ziti/controller/network.(*Network).showOptions: network = {
"CreateCircuitRetries": 2,
"CycleSeconds": 60,
"EnableLegacyLinkMgmt": false,
"InitialLinkLatency": 65000000000,
"IntervalAgeThreshold": 0,
"MetricsReportInterval": 60000000000,
"MinRouterCost": 10,
"PendingLinkTimeout": 10000000000,
"RouteTimeout": 10000000000,
"RouterConnectChurnLimit": 60000000000,
"RouterComm": {
"QueueSize": 100,
"MaxWorkers": 100
},
"Smart": {
"RerouteFraction": 0.02,
"RerouteCap": 4,
"MinCostDelta": 15
}
}
[ 0.660] INFO ziti/controller/webapis.NewFabricManagementApiFactory: initializing management api factory with 0 xmgmt instances
[ 0.660] INFO ziti/controller.(*Controller).showOptions: ctrl = {
"OutQueueSize": 4,
"MaxQueuedConnects": 1,
"MaxOutstandingConnects": 16,
"ConnectTimeout": 5000000000,
"DelayRxStart": false,
"WriteTimeout": 0,
"MessageStrategy": null,
"NewListener": null,
"AdvertiseAddress": {},
"RouterHeartbeatOptions": {
"sendInterval": 10000000000,
"checkInterval": 1000000000,
"closeUnresponsiveTimeout": 30000000000
},
"PeerHeartbeatOptions": {
"sendInterval": 10000000000,
"checkInterval": 1000000000,
"closeUnresponsiveTimeout": 30000000000
}
}
[ 0.660] INFO ziti/controller/server.NewController: edge controller instance id: cm26f7old000001qm1ulbr7mb
[ 0.660] INFO ziti/controller.(*Controller).RegisterXmgmt: adding xmgmt *server.submgmt, enabled? true
[ 0.660] INFO ziti/controller/server.(*Controller).Initialize: initializing edge
[ 0.663] INFO ziti/controller/internal/policy.NewSessionEnforcer: {frequency=[5s] sessionTimeout=[30m0s]} session enforcer configured
[ 0.663] INFO ziti/controller/server.(*Controller).Run: starting edge
[ 0.663] INFO ziti/controller.(*Controller).Run.GoroutinesPoolMetricsConfigF.func1.1: {poolType=[pool.listener.ctrl] minWorkers=[1] maxWorkers=[16] idleTime=[10s] maxQueueSize=[1]} starting goroutine pool
[ 0.664] INFO channel/v3.(*UnderlayDispatcher).Run: started
[ 0.665] INFO ziti/controller/server.(*Controller).checkEdgeInitialized: edge initialized
[ 0.697] ERROR transport/v2/tls.(*sharedListener).processConn [tls:0.0.0.0:1280]: {remote=[my_vps_public_ip:56112] error=[tls: client didn't provide a certificate]} handshake failed
[ 0.779] ERROR transport/v2/tls.(*sharedListener).processConn [tls:0.0.0.0:1280]: {remote=[my_vps_public_ip:56120] error=[tls: client didn't provide a certificate]} handshake failed
[ 0.808] INFO ziti/controller/zac.ZitiAdminConsoleFactory.New: initializing ZAC SPA Handler from /ziti-console
[ 0.809] INFO xweb/v2.(*Server).Start: starting ApiConfig to listen and serve tls on 0.0.0.0:1280 for server client-management with APIs: [edge-management edge-client fabric zac]
[ 0.810] INFO ziti/controller/network.(*Network).Run: started
[ 0.863] INFO ziti/ziti/cmd/edge.(*QuickstartOpts).run: Controller online. Continuing...
[ 0.863] ERROR ziti/ziti/router.run: {version=[v1.1.15] go-version=[go1.23.1] error=[open /home/ziggy/quickstart/quickstart-router.yaml: no such file or directory] os=[linux] arch=[arm64] build-date=[2024-10-02T13:03:23Z] revision=[0eec47ce3c80] configFile=[/home/ziggy/quickstart/quickstart-router.yaml]} error loading ziti router config
panic: open /home/ziggy/quickstart/quickstart-router.yaml: no such file or directory
goroutine 146 [running]:
github.com/openziti/ziti/ziti/router.run(0x4003032008, {0x4001a907d0, 0x1, 0x1?})
github.com/openziti/ziti/ziti/router/run.go:66 +0x1200
github.com/spf13/cobra.(*Command).execute(0x4003032008, {0x4001a90740, 0x1, 0x1})
github.com/spf13/cobra@v1.8.1/command.go:989 +0x81c
github.com/spf13/cobra.(*Command).ExecuteC(0x4003032008)
github.com/spf13/cobra@v1.8.1/command.go:1117 +0x344
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/openziti/ziti/ziti/cmd/edge.(*QuickstartOpts).run.func2()
github.com/openziti/ziti/ziti/cmd/edge/quickstart.go:303 +0xa4
created by github.com/openziti/ziti/ziti/cmd/edge.(*QuickstartOpts).run in goroutine 1
github.com/openziti/ziti/ziti/cmd/edge/quickstart.go:297 +0x13f0