Rahul Butani rrbutani

See this gist about post-hoc action input pruning and the limitations w.r.t to caching.

buck2 has starlark APIs for (limited) dynamic deps (i.e. dynamic action creation albeit only with input subsetting): https://buck2.build/docs/rule_authors/dynamic_dependencies/

Bazel's internal dependency engine (skyframe) is capable of representing such dependencies and internal rulesets make use of this functionality (i.e. ThinLTO and C++20 module support in rules_cc) however — starlark rules have no way to express such graphs.

buck2 is very explicit re: the interactions dep files have w/caching:

• dep files can (after execution) narrow the set of inputs for an action
• the input set of the action as known to your buck2 daemon will have this narrowed set but...
  • the actual action cache key (as is given to disk/remote caches, etc.) will still have the full set of inputs
• ultimately this means that as far as remote caching for cold builds is concerned, the action at hand is still keyed on all of its inputs — including the ones that are pruned
  • this means that — for cold builds — if there have been any changes to any of the "pruned out" files since the last run of the action (+ cache population), the action will not hit in the cache
• additionally, in the event that there are analysis-level changes that result in the starlark action being reconstructed (i.e. BUILD file is edited), we also lose the narrowed set of inputs

• for changes that materially affect t

Prototypes a stage-based toolchain bootstrap with transitions using "quux-lang" — a fictitious language — as an example.

❯ bazel cquery 'filter("^//:*", deps(:foo) - filter("//:_quux_toolchain_current_bootstrap_stage_is_[0-9]*", //:*)) - //:defs.bzl - //:toolchain - //:quux_toolchain_type' --output=graph > out.dot

bazel query 'filter("^_version_is_at_.*$", ...) + true + false' --output=graph

flowchart TD
  classDef true stroke:#1F0
  classDef false stroke:#F00
  classDef m stroke:#F90
  classDef l stroke:#C0F