Hard coded executable paths

Building/Development
3

Hi @sabedevops! Sure, happy to answer questions on NixOS; it is rather different from most other distros.

Are the links /run/current-system/sw/bin/[binary] always installed to some /nix/store/.../bin/[binary] ?

  • The nix store typically begins with a prefix of /nix/store/, and while it is possible to change it, that tends to be fairly rare except under special use cases and is generally not advised as doing so prevents the use of community binary caches.

  • The general pattern for binaries is for to be placed in /nix/store/$HASH/bin/$BINARY. However, there are exceptions to this. Checking my own /run/current-system/sw/bin/ directory, for example, I have:

    • 1828 binaries from 320 different installed packages
    • 1795 of these binaries are found in the .../bin/ subdir
    • 33 binaries are found in other subdirectories (about 1.8%)
    • The exceptions in my case tend to be special build executables (/share/... subdir) or library tooling related executables (/libexec/... subdir)

Similarly, can you describe the purpose of /etc/static in some greater detail for NixOS?

I’m trying to understand whether the solution here is to change these hardcoded paths to depends on some #ifndef pattern, as opposed to searching $PATH. We could potentially then handle this in system packaging, or allow the user to define their own during build time. What do you think?

Some of what I'm thinking would depend on whether OpenZiti would like to have the release artifacts work directly on NixOS. Probably the answer to that may depend on level of effort, right? Maybe yes if level of effort is low, maybe not so much if level of effort is high? There are probably 3 levels of NixOS compatibility to consider with regard to direct use of the release binaries:

  1. Binaries work on NixOS as downloaded from the repo:

    • libs are compatible (statically linked), code is compatible (no FHS code deps and logic)

  2. Binaries don't work on NixOS (reason: lib refs only):

    • libs are incompatible due to FHS paths, but can be patched easily directly from release binary with minimal effort
    • code is compatible (no FHS code deps and logic)

  3. Binaries don't work on NixOS (reason: FHS code deps and logic [and optionally lib ref problem also]):

    • code is incompatible (due to FHS code deps and logic)
    • optionally: libs are incompatible due to FHS paths, but can be patched easily directly from release binary with minimal effort

Right now, we are in case (3) for the tunneler client. I haven't look at the other binaries yet, but probably they are case (3) also. If the code logic adjustments are addressed at the runtime level, then we are in case (2) and it would be fairly trivial for a NixOS user to download a release binary, patch the lib refs and directly use the binary. Case (1) would probably only be achieved if the builds are static like mentioned in the other thread here as well as the FHS code logic issue being handled at runtime.

If the hard coded paths are handled by #ifndef at build time, I think that pretty much guarantees NixOS incompatibility with release binaries because presumably the CI binary artifact releases will continue to be built with FHS paths, right?

Ultimately, we'd like to see OpenZiti packaged for Nix, in which case the tunneler code will work with or without #ifndef addition, because the libs will be appropriately linked and the FHS paths appropriately substituted for NixOS compatible paths during the build. Downside will be that NixOS users won't be able to use the upstream release binaries directly if they want, although I think that's probably not unusual on NixOS for these very reasons. It just may be awhile before we (or someone else) gets the time to package OpenZiti for Nix.

John

1 Like