There is nothing magical in resolving the local relocations. It is just that current static libraries (static archives of plain .o files) are produced directly using "ar" and don't even go through the linker... The changes to the linker so to apply the relocation finalization are less than 50 lines of code on top of the existing "ld -r" that creates a relocatable object (which despite its name, does not handle relocations).
The key point in the proposal for a static-bundle-object is to properly handle static libraries as linked objects, instead of as a bunch of plain .o files.
Regarding --whole-archive, is it correct that it would be the default and you could opt-out of it with the function-sections/gc-sections combination?
Are there cases in which function-sections doesn't work (GCs too much) but a hypothetical "file-sections" does? For example cases in which the code relies on side effects of global constructors, but those would be left out by function-sections?
> Regarding --whole-archive, is it correct that it would be the default and you could opt-out of it with the function-sections/gc-sections combination?
This is the current intention, as implemented in the up-to-date draft: https://github.com/bminor/binutils-gdb/commit/99aef6ba8db670.... Please note however that one would no longer need to specify the "--whole-archive" flag, hence resolving issues with potential duplicate placement of the static library in the linker's CLI.
> Are there cases in which function-sections doesn't work (GCs too much) but a hypothetical "file-sections" does? For example cases in which the code relies on side effects of global constructors, but those would be left out by function-sections?
Good question. In the first article in this series I discussed issues with global constructors and was pretty much waved away, being told that code should not be written this way. One of the members of the ELF committee did suggest an alternative for handling it yet pretty much mentioned that there are still missing pieces that require handling for their proposal to work (https://groups.google.com/g/generic-abi/c/sT25-xfX9yc/m/NRo0...).
Would you consider adding something like "file-sections" support to -r, preserving file boundaries as separate subsections? I have no idea how hard it would be.
Correct me if i'm wrong, but wouldn't "file-sections" be identical to generating a static bundle object per original object file, and wrapping them all inside a .a archive?
OK, now I understood the gap. There is a technical limitation for relocation resolution when the relocation is against a different section. This means that for function sections we de-facto have no relocation finalization, only conversion of symbols from "global" to "local".
Hence, for a "file-sections" flag, we would only resolve relocations within a given file, but will leave intact relocations that cross the file boundary. Accordingly, this means that "function-sections" is identical to generating a static bundle object per original object file, and bundling them all together inside a .a archive.
> Accordingly, this means that "function-sections" is identical to generating a static bundle object per original object file, and bundling them all together inside a .a archive.
Ah, how are local symbols resolved within the whole .a file? I thought it would be only within the individual object file.
OP here, it was also my opinion that the handling of static libraries should significantly be improved, and ld should have a "--static-lib" flag to properly handle it. Sadly, the ELF committee prefers a more subtle approach, hence even the proposed build of the static bundle object is done on top of the existing "ld -r", and the output is still wrapped inside a .a archive for compatibility.
I hope that once integrated to linkers the adoption of this new format will help convince that it deserves significantly better tooling.
While the previous article presented the problem domain (multiple issues with using .a archives of plain object-code files for static libraries), this article presents the technical white paper that formally proposes an alternative format (accompanied by a POC on top of GNU's ld), as well as the gist of the ELF committee's discussion about the proposal.
Given the decision of the ELF committee, the format will be judged by adoption in practice and the feedback it receives from users. In other words, your opinion is more than welcome.
It would be interesting to know how many of bugs are triaged and declared as "won't fix" in order to comply with the zero bugs policy.
Aside from that, while it might seem like an ideal engineering culture, I find it a bit extreme. The harsh SLA leaves little room for prioritization. Sometime the team is in fact working on a tough integration deadline, and medium-level bugs can wait for it to finish.
Going over my current list of bugs, some are minor and can wait to the last mile of the release and some will be resolved by new features we have in planning. I do aim to minimize the list of bugs and even my email inbox is based on a "zero inbox" policy. Still "zero" in this case is some small epsilon that is under control, and will go down to zero if no new bugs/emails arrive in the meantime. Call it a sliding window of epsilon width, but it almost never really reaches zero.
It leaves close to no room for prioritization because the prioritization is predefined via the policy. We've had bugs on edge cases - the odds of an end user hitting them was very small (but not zero). Either the product owner (or whatever you want to call them) gets to prioritize the backlog, or its someone else - in this case it sounds like the engineering team.
In the world we have more than just bugs. We also have features, and refactoring and whatnot. Prioritization should be done across all tasks, so a bug could be "medium" but the team might not even work on bugs this week unless they are a show stopper.
Isn't this policy overruling the judgement of the team/product lead and focuses too much "only" on the bugs?
Not sure what is the measurable metric here, and what will be considered a success in this trial period.
Propagating the fix downstream depends on the release cycles of all downward vendors. Giving them a heads up will help planning, but I doubt it will significantly impact the patching timeline.
It is highly more likely that companies will get stressed that the public knows they have a vulnerability, while they are still working to fix it. The pressure from these companies will probably shut this policy change down.
Also, will this policy apply also to Google's own products?
The measure would probably be whether any of the reports led to examples of downstreams either syncing prior to release via security sharing they didn't already have established or any projects preparing to sync out of normal schedule ahead of time, regardless of if that's a small or large magnitude of change. How companies would prefer the public hear about a vulnerability has always been the lowest concern out of disclosures so I don't expect it to bring anything new here.
Google's products represent 3/6 of the initial vulnerabilities following this new reporting policy in the linked reporting page.
If someone needs a wrapper for a technology, that modifies the output it provides (like meson and bazel do), maybe there is an issue with said technology.
If pkg-config was never meant to be consumed directly, and was always meant to be post processed, then we are missing this post processing tool. Reinventing it in every compilation technology again and again is suboptimal, and at least Make and CMake do not have this post processing support.
This was the point of posting the trivial little `pkg.bzl` above: Bazel doesn't need to do all this crazy stuff in `rules_cc` and `rules_foreign_cc`: those are giant piles of internal turf wars within the Blaze team that have spilled onto GitHub.
The reason why we can't have nice things is that nice things are simple and cheap and there's no money or prestige in it. `zig cc` demonstrates the same thine.
delivers portable (trivially packages up as `.deb` or anything you want), secure (no heartbleed 0x1e in `libressl`), fast (no GOT games, other performance seppeku) builds. These are zero point zero waste: fully artifact cached at the library level, fully action cached at the source level, fully composable, supporting cross-compilation and any standard compiler.
I do this in real life. It's a few hundred lines of nix and bash. I'm able to do this because I worked on Buck and shit, and I've dealt with Bazel and CMake for years, and so I know that the stuff is broken by design, there is no good reason and no plan beyond selling consulting.
This complexity theatre sells shit. It sure as hell doesn't stop security problems or Docker brainrot or keep cloud bills down.
1. "Advanced" compilation environments (meson) probably limit this ability to some extent.
2. Package managers (rpmbuild for instance) mandate build with debug symbols and they do the strip on their own so to create the debug packages. This limits our control of these steps.
