Bump on openziti docker images please

1

Hello

Just been looking around at the logging information, as the call peaked my interest. However, when I run ziti version I get the following output…

ziti@74473ba406ab:/openziti$ ziti version
NAME             VERSION
ziti             v0.25.10
ziti-controller  *********************************************************************************

An update with v0.26.11 is available for ziti-controller v0.25.10 from 
https://github.com/openziti/ziti/releases/latest/

*********************************************************************************
v0.25.10
ziti-prox-c      not installed
ziti-router      *********************************************************************************

An update with v0.26.11 is available for ziti-router v0.25.10 from 
https://github.com/openziti/ziti/releases/latest/

*********************************************************************************
v0.25.10
ziti-tunnel      *********************************************************************************

An update with v0.26.11 is available for ziti-tunnel v0.25.10 from 
https://github.com/openziti/ziti/releases/latest/

*********************************************************************************
v0.25.10
ziti-edge-tunnel not installed

Doing a docker pull, it is not updating, so looks like a container build is lagging behind.

Container images I am using:

ZITI_IMAGE=openziti/quickstart
ZITI_VERSION=latest

Thanks

2

hmmmm. Not sure what’s goin’ on. I just went and checked and sure enough Docker Hub shows 0.26.11 as latest. I pushed the build anyway again so now it shows up as being only moments ago (prior it showed 17days ago)

are you using docker compose pull or docker-compose pull? I’m not sure if I’ll have an answer to this one for you… :confused:

3

It picked up the updates, but the version hasn’t bumped. I think I had this issue when I was starting out and needed to docker a docker-compose down -v and then when I brought it up it was correct. Don’t want to reset the environment, so will see what I can work through. Being a container it should just be correct (one of its superpowers). Will report back.

4

really. Ok that’s good to know. I’ll investigate. That’s certainly unexpected :slight_smile:

5

Yeah - we have been here before: Following on from quickstart - #16 by TheLumberjack about half way down. Came across it when looking around.

6

OK - found the problem.

In the docker-compose file we have

    volumes:
      - prod-ziti-fs:/openziti

Looking at the ziti.env file, it is pointing to the bin file in /openziti folder

export ZITI_BIN_DIR="/openziti/ziti-bin"

Since the volume will already exist, it wont copy anything from the original /openziti folder (if that is where the binaries actually exist prior to the container) to the volume. With Docker, if the volume doesn’t exist it will copy the /openziti folder into the volume.

So, the fix is to either one of two things
a) Point the ZITI_BIN_DIR variable back into the container
b) put a startup script to copy the updated binaries to the location
c) if /openziti is where the binaries are installed during docker build, then need to have them in a ‘staging’ folder to overwrite those in the /openziti/ziti-bin folder on startup.

Just going to hunt the github in case this has been fixed and I am running a broken version

7

Oh that makes a lot of sense. @gberl002 - can we set the ZITI_BIN_DIR on docker compose somewhere else? I imagine in the .env file?

8

Looking at the Dockerfile, everything is done in the /openziti folder. I think we need to split apart the configuration from the ziti components. To that end, move the /openziti/scripts and /openziti/ziti-bin back to a path outside of the volume. Everything else looks instance related and not generic. Will leave it for @gberl002 to resolve.

9

Yeah. I was just talking to @gberl002 about exactly that. One way to fix it for yourself for now – I think – is to cp the latest binaries out to /var/lib/docker/volumes/docker_ziti-fs/_data/ziti-bin on your own. I didn’t try that myself, but I expect it’ll work just fine if you’re feeling ambitious. If not, well, I understand that too. :slight_smile:

I’m still surprised that docker didn’t untar the “modified” layer on top of these, but it’s fine, we’ll fix that as you said. Thanks for finding that.

10

Hey @gooseleggs I just updated the docker image, there are now two locations for ziti “stuff”. One is /var/openziti/ which will house the scripts and binaries, the other is /persistent/ which will house all the rest (configs, env files, pki, etc) which is the new shared volume replacing /openziti/. This way when a new image is pulled, the scripts and binaries will be replaced but the network configs and such will remain as they are now the shared volume.

MIGRATION STEPS

Backup current files:
Back up your current files in the /openziti/ folder in the container. You can do this fairly easily with docker cp <container_name>:/openziti <local_path>. For example, I used docker cp docker_ziti-controller_1:/openziti . which just copied the openziti folder to my current path.

Add persistent volume
Add the following under the volumes section of ziti-controller in docker-compose, this would be the current docker-compose you are using, not the latest one. This will essentially copy all the files in /openziti to the /persistent directory.

ziti-fs:/persistent

As an example, the volumes in ziti-controller should look like this in your docker-compose file

    volumes:
      - ziti-fs:/openziti
      - ziti-fs:/persistent

Update path references

  1. Start up your environment docker-compose up -d

  2. Exec into controller and change paths docker exec -u 0 -it docker_ziti-controller_1 /bin/bash Note that your controller container name may be slightly different.

  3. Update paths in the following order, you can just copy and paste the whole block.

    sed -i -e 's#/openziti/ziti-bin#/var/openziti/ziti-bin#g' $(find . -maxdepth 1 -type f)
sed -i -e 's#/openziti/scripts#/var/openziti/scripts#g' $(find . -maxdepth 1 -type f)
sed -i -e 's#/openziti/pki#/persistent/pki#g' $(find . -maxdepth 1 -type f)
sed -i -e 's#/openziti#/persistent#g' $(find . -maxdepth 1 -type f)
sed -i -e 's#var/persistent#var/openziti#g' $(find . -maxdepth 1 -type f)

  4. Shut down the environment (keeping the volume so no -v) docker-compose down

Get the latest docker-compose
I’m not sure if you’re using the repo or you just have a copied version of the file. So, git pull if you’re using the repo or, you can find the file here ziti/docker-compose.yml at release-next · openziti/ziti · GitHub

Update local image
Run docker-compose pull to pull the latest quickstart image

Start up as usual
At this point you can start your environment as normal docker-compose up -d

Note: If you run into any issues along the way, you should have your backup files that were copied during the first step but of course, let me know if I can help.

11

Hey Geoff

Thanks for this. Just wondering if we can do this better. I would give it a lash, but work is crazy busy at the moment. Happy for you to reject this option, but throwing it out there

a) Leave the /openziti volume as it is
b) Modify the new startup scripts access-control.sh, run-controller.sh and run-router.sh to check if there is a /openziti/ziti-bin folder. If there is, then start a migration process which:

  • Deletes the /openziti/ziti-bin folder (as no longer required)
  • run the sed commands to update references in the conf files on the /openziti volume
  • delete the ziti-scripts folder, or as an option, make these symlinks to the new scripts folder

All the user needs to do is update the command lines in the docker-compose file, which on startup will perform all the migration work we need done.

advantages
a) it wont scare off people not familiar with docker
b) It is less error prone
c) it makes it easy as they only need to change the docker compose file (it has backward compatibillity in that they have this bug until they update the compose file)
d) It keeps in line with the container doing any modifications of itself during startup

12

@gooseleggs Thanks for getting back to me, your suggestion is appreciated and well thought out. However, as of right now we don’t have any official plans to have a built-in migration mechanism. We’re not aware of many users out there with networks set up using the docker-compose quickstart that would want to upgrade the binaries while keeping their environment (obviously you are one exception and due to your consistent support and contributions, we wanted to provide an upgrade path for you).

This is just a one-time situation and a breaking change like this is rare. With that said, at least for now, we won’t be incorporating any built-in migration, although, this certainly may change in the future.

Thanks again for your suggestion and continued support.

13

Thanks @gberl002, migrated now

1 Like