Buck: Java ABIs

Java ABIs

This information pertains to building Java code with Buck.

When compiling a Java rule, in addition to the library JAR that contains all of the compiled classes and resources for the rule, Buck also creates an ABI JAR containing just the resources and the class interfaces. Since ABI JARs do not contain method bodies or private members, they are smaller and change less often than library JARs, which enables Buck to do two things with them:

  • Use them in ABI rule keys to more accurately detect which rules need to be rebuilt during an incremental build.
  • Use them on the compiler's classpath instead of library JARs to get a small but noticeable performance boost since the smaller size of ABI JARs enables the compiler to load them faster than library JARs

Three Kinds of ABI JARs

Buck has three ways to create an ABI JAR, controlled by thejava.abi_generation_mode config setting:

  • From classes, by building the library JAR first and then stripping out the unnecessary bits.
  • From source, by hooking in to the compiler while it is compiling the library JAR and emitting the ABI jar partway through. (This helps reduce bottlenecks due to slow rules or low parallelism in a build graph.)
  • From source only, by examining just the text of the source code, using heuristics to infer things that can normally only be determined from looking at dependencies. (This drastically increases parallelism in the rule graph and reduces the number of cache fetches required during incremental builds.)

Requirements for Source-only ABI JARs

Source-only ABI JARs are generated using just the text of the source code for a rule, without compiling (most of) that rule's dependencies first. A few details of an ABI JAR cannot be known for certain from just the source, so Buck uses heuristics to infer these details. Even when using source-only ABIs, Buck must still use a rule's dependencies to compile the library JAR. At that time, Buck checks whether the heuristics got the right answer and fails the build if not.

Such build failures can generally be fixed with buck fix, if sometimes in a sub-optimal way. The following describes the requirements for source-only ABI generation, how buck fix fixes issues, and what options there are for more optimal fixes.

  • Annotations and constants in their own rules: All annotations and compile-time constants (including enum values) used in the interface of a rule must be present during source-only ABI generation.

    The error will suggest adding required_for_source_abi = True to any rule that defines an annotation type or a compile-time constant that is used from another rule, and buck fix will make that change. For best results, such rules should be manually inspected to ensure they are as small as possible--ideally just containing annotations, enums, and constants--and have as few dependencies as possible.

  • Packages and classes named according to Java convention: Any packages that will not be available during source-only ABI generation must have names that begin with a lowercase letter. Any top-level classes that will not be available must have names that begin with an uppercase letter. (This is extremely common practice in Java.)

    The error will suggest renaming the package or class, and this will have to be done manually. (buck fix will not attempt it.) If the package or class name in question cannot be changed, add required_for_source_abi = Trueto the build rule that defines it.

  • Member types referenced canonically: Any references to member types that will not be available during source-only ABI generation must be canonical. That is, member types are inherited by subclasses; a canonical reference refers to the member type as a member of the class in which it is defined, not one that inherits it.

    If the defining class is accessible, the error will suggest using the canonical name. Otherwise, it will suggest adding source_only_abi_deps. buck fixwill make whichever change is suggested by the error. For best results, consider making the defining class accessible so that the extra deps are not needed.

  • Superclass member types referenced unambiguously: Any references to member types that are inherited by the current class and will not be available during source-only ABI generation must be at least partially qualified.

    The error will suggest adding an import and partially qualifying the name, andbuck fix will make the change.

  • Enough deps for the compiler to run: Source-only ABI generation involves running the Java compiler front-end over a rule's code without providing most of that rule's dependencies, then working with the compiler's data model to generate the ABI. The compiler is remarkably resilient to missing dependencies, but in some cases a missing dependency will prevent it from getting far enough for Buck to generate the source-only ABI. These cases typically involve inheritance, where a type is available during source-only ABI generation, but one of its superclasses is defined in another rule that is not available.

    The error will suggest adding source_only_abi_deps to the rule so that the necessary dependencies will be available, and buck fix will make the change. For best results, limit the depth of class hierarchies (this is a best practice for Java design in many cases as well) and avoid splitting them across modules.

  • Only single-type imports: On-demand imports (star imports) and staticimports of types cannot be resolved at source-ABI generation time unless the rule containing the type in question is available.

    The error will suggest either adding a single-type import (in the case of star imports) or adding source_only_abi_deps in the case of static imports, and buck fix will make that change. For best results, consider changingstatic imports of types to non-static imports to avoid needing to add the extra dependencies.