Compiling in Docker and on baremetall.

[lgehr]

On my machine a can compile with ./build-in-docker.sh but ipfs-v0.12.0-docker can not be run.
exec: Failed to execute process './out/ipfs-v0.12.0-docker': The file does not exist or could not be executed. Maybe this is because I am on alpine linux (musl).
I wonder why ./out/ipfs-v0.12.0-docker is dynamically linked? Shouldn't go make statically linked binaries?

❱ file ./out/ipfs-v0.12.0-docker
./out/ipfs-v0.12.0-docker: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=21e7fe095bb5efc65be8ba5607b12c9c919df586, for GNU/Linux 3.2.0, not stripped

I dug a little bit into the Go 1.17 compiler bug issues. I wanted to know if it still persists in Go 1.18.

On my machine make build throws errors from three packages:

• github.com/lucas-clemente/quic-go/internal/qtls
• github.com/klauspost/compress/zstd/internal/xxhash
• github.com/cespare/xxhash/v2

In quic-go the issue is fixed in v0.26.0
In compress the issue is fixed in v.1.13.6 which is pulled in v0.16.0 of libp2p/go-libp2p-quic-transport
In cespare/xxhash/v2 the issue is fixed in v2.1.2 sadly the package which uses caspare/xxhash dgraph-io/risotto did not update xxhash yet.

I am still confused why the ipfs-folks can build their package.
Is this not the first step of the makefile where we fail?

[mrd0ll4r]

So, yeah. We build in docker precisely because of the Go compiler bugs in 1.17. They might be fixed in 1.18, but IPFS did not build with 1.18. It does now, which is nice, but I haven't tested if that fixes the plugin build errors. An easy way to test this would be to change the builder Dockerfile to use Go 1.18 and see if that works.

Re dynamic linking: Go executables are mostly statically linked, apart from some networking things, most notably name resolution. I don't know why this is, but that's the way it is. If that's the reason why the executable does not run on your machine is beyond me. Musl may be an issue, I haven't tried that yet. You could change the Dockerfile to build on Alpine, which should then hopefully work..

Re "how can they build but we can't": IPFS itself can be built with Go 1.17, but plugins can't....

Anyway. It would be nice to get it to compile with 1.18. Also, there will be a new IPFS release soon-ish, which should build with 1.18. So when that happens we have to update a few dependencies here and there, so that we build with/against that. This is the tracking issue on their end.

[lgehr]

Re dynamic linking: Go executables are mostly statically linked, apart from some networking things, most notably name resolution. I don't know why this is, but that's the way it is. If that's the reason why the executable does not run on your machine is beyond me. Musl may be an issue, I haven't tried that yet. You could change the Dockerfile to build on Alpine, which should then hopefully work..

What is strage is the error message. 'The file does not exist or could not be executed.' Does not read like a missing linking error. What I also do not undestand is why the Github-Action-job „Build“ ('go build -v -buildmode=plugin -o mexport.so') runs flawlessly.

Re "how can they build but we can't": IPFS itself can be built with Go 1.17, but plugins can't....

I still do not understand why. Because the breaking dependencies are only dependencies from our module `meplugin/metricplugin` because the seem to depend on `github.com/ipfs/go-ipfs/core`. What am I missing?

Anyway. It would be nice to get it to compile with 1.18. Also, there will be a new IPFS release soon-ish, which should build with 1.18. So when that happens we have to update a few dependencies here and there, so that we build with/against that. [This](ipfs/kubo#8640) is the tracking issue on their end.

I will try to compile on 1.18 when publishing the build with GH-Actions on GH

[mrd0ll4r]

What I also do not undestand is why the Github-Action-job „Build“ ('go build -v -buildmode=plugin -o mexport.so') runs flawlessly.

We specify Go 1.16 for the Github CI, otherwise that wouldn't be working either, I'm pretty sure :)

I still do not understand why. Because the breaking dependencies are only dependencies from our module meplugin/metricplugin because the seem to depend on github.com/ipfs/go-ipfs/core. What am I missing?

Maybe a misunderstanding between plugins and dependencies? IPFS itself, including all dependencies, can be built with 1.17 (and now 1.18 as well). Go plugins, which is some special funky mode for the go compiler (which, of course, is entirely incompatible with usual plugin systems and dynamic linking), however, were tricky in 1.16. I guess that not everything about them was broken, but with the dependencies of IPFS (which we (in part?) import transitively), it was impossible to build an IPFS plugin with 1.17.

I've tried with 1.18, but back then IPFS itself wouldn't build with that, so I didn't get very far....

The easiest way to test that now is to wait for IPFS 0.13 to be released (or try with a dependency on go-ipfs' master, I guess), update a few dependencies and tags in the docker build script, bump the Go version to 1.18 in there, and see if it builds in Docker. If yes, that's good enough for me :)