Hello All!
I’m developing on a Mac and have both a service and a client endpoint built with the SDK and running successfully on the Mac. The service, however, ultimately needs to be hosted on a Raspberry Pi, and this is where I’m running into trouble.
Here’s how it goes:
% # Native build is fine
% go build -o ptzcam-mock
%
% go clean
%
% # Cross compile fails due to unsupported clang option
% CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=7 go build -o ptxcam-mock
# runtime/cgo
clang: error: argument unused during compilation: '-marm' [-Werror,-Wunused-command-line-argument]
Using the golang -x --work and -n options I’m able to trace the build process down to the following:
...
% cd $WORK
% cd b099
% TERM='dumb' clang -I /usr/local/go/src/runtime/cgo -fPIC -marm -pthread -fmessage-length=0 -I ./ -g -O2 -Wall -Werror -o ./_x001.o -c _cgo_export.c
clang: error: argument unused during compilation: '-marm' [-Werror,-Wunused-command-line-argument]**
%
% # I'm using a recent release of Clang
% clang --version
Apple clang version 14.0.0 (clang-1400.0.29.102)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
%
% # Clang indeed does not support a -m option
% clang --help | grep "\-m "
%
% # But does have a -target option that appears to serve the same purpose
% clang --help | grep "\--target="
--target=<value> Generate code for the given target
Any help getting over this hurdle would be greatly appreciated.
Hi @Ross, welcome to the community! Are you developing on an M1 mac? Can you build any of the sample apps already out there fine or are they causing you issues too?
Can we start out by trying to build one of the examples the go sdk provides to see if we can get to a common ground first?
Had you come across that SO post already, did it not set you on the right path? If you can share the source/repo, I can try to reproduce this locally or work with someone with an M1 to reproduce the problem.
Hello, and thanks much for the replies. And apologies for my late response (contending priorities).
I had visited the suggested sight and had experimented with defining CC. I ran into a permissions issue: error: error opening 'cgo-gcc-input-2579416899.d': Permission denied. Using sudo -E ... got me past that and on to the next hurdle below:
% sudo -E CC=/opt/homebrew/Cellar/llvm/13.0.1_1/bin/clang-13 CGO_ENABLED=1 GOOS=linux GOARCH=arm GOARM=6 go build --work -o ptzcam-mock
WORK=/var/folders/jb/nxj9j4hx6md1ryydd7syrt0c0000gp/T/go-build790818574
# github.com/miekg/pkcs11
Homebrew clang version 13.0.1
Target: arm-unknown-linux-gnueabihf
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/13.0.1_1/bin
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.6
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.6.4
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.8
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.8.5
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.9
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.9.3
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/6
Found candidate GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/6.3.0
Selected GCC installation: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.9
Candidate multilib: .;@m32
Selected multilib: .;@m32
"/opt/homebrew/bin/arm-linux-gnueabihf-ld" --sysroot=/Users/ross.maxfield/rpi/sysroot/stretch -pie -EL -X --eh-frame-hdr -m armelf_linux_eabi -dynamic-linker /lib/ld-linux-armhf.so.3 -o $WORK/b323/_cgo_.o /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/Scrt1.o /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/crti.o /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.9/crtbeginS.o -L/usr/lib -L/usr/lib/arm-linux-gnueabihf -L/usr/lib/gcc/arm-linux-gnueabihf/4.9 -L/usr/lib/arm-linux-gnueabihf/pulseaudio -L/usr/local/lib -L/lib/arm-linux-gnueabihf -L/Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.9 -L/Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.9/../../../../lib -L/Users/ross.maxfield/rpi/sysroot/stretch/lib/arm-linux-gnueabihf -L/Users/ross.maxfield/rpi/sysroot/stretch/lib/../lib -L/Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf -L/Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/../lib -L/Users/ross.maxfield/rpi/sysroot/stretch/lib -L/Users/ross.maxfield/rpi/sysroot/stretch/usr/lib $WORK/b323/_cgo_main.o $WORK/b323/_x001.o $WORK/b323/_x002.o $WORK/b323/_x003.o $WORK/b323/_x004.o -headerpad_max_install_names -lpthread -rpath-link=/lib/arm-linux-gnueabihf:/usr/lib/arm-linux-gnueabihf -rpath /usr/lib -rpath /usr/lib/arm-linux-gnueabihf -rpath /usr/lib/gcc/arm-linux-gnueabihf/4.9 -rpath /usr/local/lib -rpath /lib/arm-linux-gnueabihf -lrt -lm -ldl -ldl -lgcc --as-needed -lgcc_s --no-as-needed -lpthread -lc -lgcc --as-needed -lgcc_s --no-as-needed /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/gcc/arm-linux-gnueabihf/4.9/crtendS.o /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/crtn.o
/opt/homebrew/bin/arm-linux-gnueabihf-ld: $WORK/b323/_x003.o: in function `New':
pkcs11.cgo2.c:(.text+0x20): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/homebrew/bin/arm-linux-gnueabihf-ld: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/libdl.a(dlopen.o): in function `dlopen':
(.text+0xc): undefined reference to `__dlopen'
/opt/homebrew/bin/arm-linux-gnueabihf-ld: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/libdl.a(dlclose.o): in function `dlclose':
(.text+0x0): undefined reference to `__dlclose'
/opt/homebrew/bin/arm-linux-gnueabihf-ld: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/libdl.a(dlsym.o): in function `dlsym':
(.text+0xc): undefined reference to `__dlsym'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
ross.maxfield@00679-rossmaxfield ptzcam-mock %
Using Go’s -work option I narrowed in on the offending point and began experimenting there:
% sudo -E "/opt/homebrew/bin/arm-linux-gnueabihf-ld" --sysroot=$SYSROOT -pie -EL -X --eh-frame-hdr -m armelf_linux_eabi -o $WORK/b323/_cgo_.o -dynamic-linker /lib/ld-linux-armhf.so.3 $SYSROOT/usr/lib/arm-linux-gnueabihf/Scrt1.o $SYSROOT/usr/lib/arm-linux-gnueabihf/crti.o $SYSROOT/usr/lib/gcc/arm-linux-gnueabihf/4.9/crtbeginS.o -L$SYSROOT/usr/lib/arm-linux-gnueabihf -L$SYSROOT/usr/lib/gcc/arm-linux-gnueabihf/4.9 -L$SYSROOT/usr/lib -L$SYSROOT/lib/arm-linux-gnueabihf -L$SYSROOT/lib -L$SYSROOT/usr/lib/arm-linux-gnueabihf $WORK/b323/_cgo_main.o $WORK/b323/_x001.o $WORK/b323/_x002.o $WORK/b323/_x003.o $WORK/b323/_x004.o -lrt -lm -lgcc -lpthread -lc -ldl -lglib-2.0 $SYSROOT/usr/lib/gcc/arm-linux-gnueabihf/4.9/crtendS.o $SYSROOT/usr/lib/arm-linux-gnueabihf/crtn.o
Password:
/opt/homebrew/bin/arm-linux-gnueabihf-ld: /var/folders/jb/nxj9j4hx6md1ryydd7syrt0c0000gp/T/go-build2402145786/b323/_x003.o: in function `New':
pkcs11.cgo2.c:(.text+0x20): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/homebrew/bin/arm-linux-gnueabihf-ld: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/libdl.a(dlopen.o): in function `dlopen':
(.text+0xc): undefined reference to `__dlopen'
/opt/homebrew/bin/arm-linux-gnueabihf-ld: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/libdl.a(dlclose.o): in function `dlclose':
(.text+0x0): undefined reference to `__dlclose'
/opt/homebrew/bin/arm-linux-gnueabihf-ld: /Users/ross.maxfield/rpi/sysroot/stretch/usr/lib/arm-linux-gnueabihf/libdl.a(dlsym.o): in function `dlsym':
(.text+0xc): undefined reference to `__dlsym'
I’m not as far along as you are with the toolchain and pi sdk setup, but I have an idea of what might be going wrong here.
My theory is that your sysroot directory contains broken symlinks (because the symlinks were created based on the image being mounted on “/”). At least this is what I noticed on my system after I extracted the PI image onto my mac:
scarey@shawns-m1-mbp rootfs % find . -name 'libdl*' -print | xargs ls -l
-rw-r--r-- 1 scarey staff 9800 Feb 6 2019 ./lib/arm-linux-gnueabihf/libdl-2.24.so
lrwxr-xr-x 1 scarey staff 13 Feb 6 2019 ./lib/arm-linux-gnueabihf/libdl.so.2 -> libdl-2.24.so
-rw-r--r-- 1 scarey staff 8110 Feb 6 2019 ./usr/lib/arm-linux-gnueabihf/libdl.a
lrwxr-xr-x 1 scarey staff 35 Feb 6 2019 ./usr/lib/arm-linux-gnueabihf/libdl.so -> /lib/arm-linux-gnueabihf/libdl.so.2
scarey@shawns-m1-mbp rootfs % file ./usr/lib/arm-linux-gnueabihf/libdl.so
./usr/lib/arm-linux-gnueabihf/libdl.so: broken symbolic link to /lib/arm-linux-gnueabihf/libdl.so.2
My searches led me to posts from people who were apparently able to get around this error by fixing the libdl.so symlink so that it points to e.g. /Users/ross.maxfield/rpi/sysroot/stretch/lib/arm-linux-gnueabihf/libdl.so.2 in your case.
If that doesn’t help, could you please share how you set up your cross compiler and pi sdk so that I can get to the same error that you’re seeing?
Shawn,
Thanks much for your suggestion. Indeed, I recently updated my rpi sysroot image and just noticed that the softlinks are relative to /. I changed the libld.so link from /lib/arm-linux-gnueabihf/libdl.so.2 to ../../../lib/arm-linux-gnueabihf/libdl.so.2 and veolia!, joy.
btw it seems that this issue comes up often. There are several versions of a python script like this one out there that fixes up the absolute symlinks on an rpi sysroot.
I answered my own question... This SO post makes it seem like you need to set a C compiler for CGO to use: