Compiler bootstrapping depends on more than you might think
15 points by abhin4v
15 points by abhin4v
When you build Go 1.4 with your modern C compiler and OS, you will get a modern binary that calls your modern C standard library and kernel system calls in the appropriate way. But when you use your newly built Go 1.4 to build Go 1.19, the Go binary it produces is one that was effectively frozen around the time Go 1.4 was released; it makes the system calls and expects the ABIs and so on that were current then, not that are current now.
Maybe I'm misreading this but it sounds like Go inherits the host compiler's syscall bindings/implementation instead of using whatever is defined in the Go 1.19 source code?
I don't think that's the case. I would make this distinction:
sizeof(int) is8 bytes
Linux and Windows both have stable ABIs, but OS X doesn't, and OpenBSD doesn't (the example given in the blog post).
In OS X and OpenBSD all the apps have to be recompiled with a new OS and a new C compiler. That is OK for C, but what about Go?
So then the point of the blog post is that bootstrapping a compiler is easier on Linux, because it has a stable ABI. On Linux, Go just targets the stable ABI, full stop. It doesn't depend on libc or C compilers.
On OS X and OpenBSD, the Go compiler doesn't have anything to target. In other words, the OS interface on OS X and Open BSD is defined using C header files that change from version to version, and the Go compiler does not understand C header files -- e.g. it has no idea what sizeof(int) is on a given platform is, and so forth.
On Linux, the Go compiler doesn't need to understand C header files, because it just repeats the binary definition of the interface in Go code, and that is known to be stable.
I believe Rust works the same way, although I haven't looked. I think it just repeats the stable ABI in Rust code. Although I'm not sure what it does on OS X either.
I was confused about the specific point that if you have Go v1 bootstrap Go v2 that Go v2 inherits the syscall interface of Go v1 instead of providing its own bindings.
The Go 1.4 compiler is written in C and makes calls to the C runtime (aka libc et al). However the Go runtime is (mostly) written in Go, even as far back as Go 1.4, and it makes direct system calls, using the ABI and system calls of the time that Go version was released (circa 2015 for Go 1.4, later for Go 1.19). So when you build a Go program it gets the Go runtime of that toolchain and its ABI requirements, and if parts of that ABI doesn't work any more on your current OS version, your Go program doesn't run.
The Go runtime can't use the OS's current API because the current API is generally usable only from C or from things that emulate it well enough to parse C header files, link to and call C functions in libc, and so on. It would be nice if OSes fully supported cross-language APIs with defined, language-independent, machine-parseable descriptions that Go, Rust, etc etc could pick up and use directly, but they don't. Some OSes (current OpenBSD and I think Illumos derived ones at least) explicitly require you to call the C library/runtime to do certain things; there is no supported non-C API/ABI.
(I'm the author of the linked-to article and I should have explained the runtime situation in more detail in that article.)
So when you build a Go program it gets the Go runtime of that toolchain
That's the part that I missed. I would have expected the build of Go maj.min to include building the runtime for maj.min and not to inherit the runtime of the toolchain. Otherwise it sounds like the runtime would be frozen at 1.4 forever?
The Go runtime can't use the OS's current API because the current API is generally usable only from C
Isn't that exactly what Go does on MacOS/BSD/etc?
The final binaries of the Go toolchain are (re)built with itself, but the first set of binaries has to be build with the initial bootstrap Go version and as far as I know they use that initial version's runtime. It would probably be challenging to do otherwise, since some knowledge of how the runtime behaves is entwined into the compiler's code generation (eg, for garbage collection, write barriers, and so on).
While Go calls the C libraries on OpenBSD, macOS, and so on, I don't think it uses the API as such in the sense of reading C struct definitions, #defines, and so on from the C header files. Instead I think it has its own versions, as seen in eg runtime/defs_openbsd_amd64.go and runtime/sys_openbsd_amd64.s. At least some of these are more or less automatically generated, but AFAIK aren't produced at compiler build time, and even on these platforms the Go runtime tries to go as close to straight system calls as possible, which means it knows specifically about system calls instead of higher level C runtime operations.
Me clicking on this link on Lobsters from Firefox 151.0.3 being told it's suspiciously old and that I should email the author: I'm good, thanks.
You should probably update if possible. You are 7 major releases behind, and are missing fixes for a lot of sandbox escapes.