Buck: buck query

buck query

Provide facilities to query information about the target nodes graph and lists the build targets that satisfy the query expression.

buck query allows us to combine different operators, using a query language. For example, to retrieve the list of all tests for a build target, a single buck query call suffices by combining the deps and testsof operators.

buck query "testsof(deps(//java/com/example/app:amazing))"

Query Language

The Query Language was inspired by Bazel's Query Language. buck query uses the same parser, so the lexical syntax is similar, supporting part of Bazel's query functions and a few extensions.

Lexical Syntax

Expressions in the query language are composed of the following tokens:

  • Keywords, such as deps or testsof. Keywords are the reserved words of the language. The complete set of keywords is:
  • Words, such as //foo:bar or //foo:bar+lib". If a character sequence is "quoted" (begins and ends with a single-quote ', or begins and ends with a double-quote "), it is a word. If a character sequence is not quoted, it may still be parsed as a word. Unquoted words are sequences of characters drawn from the set of alphabet characters, numerals, slash /, colon :, period ., hyphen -, underscore _, and star *. Unquoted words may not start with a hyphen or period.

    This syntax makes it so that quote marks aren't needed in most cases. Quoting "java_test" is unnecessary but harmless. Quoting is necessary when writing scripts that construct buck query expressions from user-supplied values.

    //foo:bar+wiz    # WRONG: scanned as //foo:bar + wiz.
    //foo:bar=wiz    # WRONG: scanned as //foo:bar = wiz.
    "//foo:bar+wiz"  # ok.
    "//foo:bar=wiz"  # ok.
    

    Note that this quoting is in addition to any quoting that may be required by your shell.

    buck query ' "//foo:bar=wiz" '
    
    In this example, single-quotes are used for the shell and double-quotes for buck query.

  • Punctuation, such as parens (()), period (.) and comma (,), etc. Words containing punctuation (other than the exceptions listed above) must be quoted.

Whitespace characters outside of a quoted word are ignored.

Expressions: syntax and semantics of the grammar

This is the grammar of the query language, expressed in EBNF notation:

expr ::= word
       | (expr)
       | expr intersect expr
       | expr ^ expr
       | expr union expr
       | expr + expr
       | expr except expr
       | expr - expr
       | allpaths(expr, expr)
       | attrfilter(word, word, expr)
       | buildfile(expr)
       | deps(expr)
       | deps(expr, depth)
       | filter(word, expr)
       | inputs(expr)
       | kind(word, expr)
       | labels(word, expr)
       | owner(word)
       | rdeps(expr, expr)
       | rdeps(expr, expr, depth)
       | set(word *)
       | testsof(expr)

Target Patterns

expr ::= word

Syntactically, a build target pattern is just a word. It is interpreted as an unordered set of targets. A build target pattern can be evaluated to a set containing one or more elements.

For example, the word //foo:bar evaluates to a set containing one element, while //foo/... matches all targets in every directory beneath the foo directory.

Aliases

expr ::= word

Aliases defined in .buckconfig can be used in queries. The aliases will be resolved to the corresponding build targets. Suppose there is an alias app = //apps/myapp:app , then app can be used instead //apps/myapp:app in query expressions.

buck query can also be used to resolve aliases. e.g.

$ buck query app
//apps/myapp:app

Parenthesized expressions

expr ::= (expr)

Parentheses associate subexpressions to force an order of evaluation. A parenthesized expression evaluates to the value of its argument.

Algebraic set operations: intersection, union, set difference

expr ::= expr intersect expr
       | expr ^ expr
       | expr union expr
       | expr + expr
       | expr except expr
       | expr - expr

These three operators compute the usual set operations over their arguments. Each operator has two forms, a nominal form such as intersect and a symbolic form such as ^. Both forms are equivalent; the symbolic forms are quicker to type. For example,

deps(//foo:bar) intersect deps(//baz:lib)

evaluates to the targets that appear both in the transitive closure of //foo:bar and //baz:lib. Equivalently:

deps(//foo:bar) ^ deps(//baz:lib)

The intersect (^) and union (+) operations are commutative (symmetric); except (-) is asymmetric. The parser treats all three operators as left-associative and of equal precedence, so you might want parentheses. For example, the first two of these expressions are equivalent, but the third is not:

x intersect y union z
(x intersect y) union z
x intersect (y union z)

It is strongly recommended to use parentheses where there is any danger of ambiguity in reading a query expression.

Read targets from an external source: set

expr ::= set(word *)

The set(a b c ...) operator computes the union of a set of zero or more targets, separated by whitespace (no commas).

If you are invoking buck query programmatically, e.g. run a query on a list of targets, set() is a way of grouping this list in the query. Remember to quote the targets to make sure they are parsed correctly.

All dependency paths: allpaths

expr ::= allpaths(expr, expr)

The allpaths(from, to) operator evaluates to the graph formed by paths between the sets from and to, following the dependencies between nodes. For example, the value of allpaths(//foo:bar, //foo/bar/lib:baz) is the dependency graph rooted at the single target node //foo:bar, including all target nodes that depend on //foo/bar/lib:baz.

The arguments can be expressions.
allpaths(kind(java_library, '//...'), '//foo:bar') shows all the paths between any java_library in the repository and the target //foo:bar.

This operator is best used together with the --dot parameter to generate an image. Sample usage:

$ buck query "allpaths(//foo:bar, //foo/bar/lib:baz)" --dot > result.dot
$ dot -Tpng result.dot -o image.png

Rule attribute filtering: attrfilter

expr ::= attrfilter(word, word, expr)

The attrfilter(attribute, value, expression) operator evaluates the given expression and filters the resulting build targets whose attribute contain the given value.

If the attribute is just a value, say name, it is compared to the given value. If it's a list, the target is filtered if the given value is contained in the list. If it's a dictionary, the value is searched both in the keys and values of the dictionary.

For example, attrfilter(deps, '//foo:bar', '//...') filters the build targets in the repository that depend on '//foo:bar'.

Build files of targets: buildfile

expr ::= buildfile(expr)

The buildfile(expression) operator evaluates to the build files that define the targets that result from the evaluation of expression.

In order to find the build file associated with a file, the owner operator can be combined with buildfile. For example, buildfile(owner('foo/bar/main.cpp')) first finds the targets that own foo/bar/main.cpp and then find the build files that define these targets, e.g. foo/bar/BUCK.

Transitive closure of dependencies: deps

expr ::= deps(expr)
       | deps(expr, depth)

The deps(x) operator evaluates to the graph formed by the transitive closure of dependencies of its argument set x. For example, the value of deps(//foo:bar) is the dependency graph rooted at the single target node //foo:bar, including all its dependencies.

The deps operator accepts an optional second argument, which is an integer literal specifying an upper bound on the depth of the search. So deps(//foo:bar, 1) evaluates to the direct dependencies of the target //foo:bar, and deps(//foo:bar, 2) further includes the nodes directly reachable from the nodes in deps(//foo:bar, 1), and so on. If the depth parameter is omitted, the search is unbounded, i.e. it computes the transitive closure of dependencies.

Filter targets by name: filter

expr ::= filter(word, expr)

The filter(pattern, expression) operator evaluates the given expression and filters the targets whose name matches the given regular expression pattern. For example, filter('library', deps('//foo:bar')) will filter the targets in the transitive closure of //foo:bar that contain library in their name.

Another example is filter('.*\.java$', labels(srcs, //foo:bar)) which filters the java files used to build //foo:bar. Quotation of the pattern is often required to make sure regular expressions, e.g. .*xpto, are parsed correctly.

Direct input files: inputs

expr ::= inputs(expr)

The inputs(x) operator evaluates to the files which are inputs to the argument set x, ignoring all dependencies. Note that it does not include any files required for parsing (e.g. the BUCK file), rather just the files required to actually run the build after parsing has been performed.

Filter targets by rule type: kind

expr ::= kind(word, expr)

The kind(pattern, expression) operator evaluates the given expression and filters the targets whose rule type matches the given pattern. For example, kind('java_library', deps('//foo:bar')) will filter all java_library targets in the transitive dependencies of //foo:bar.

The pattern can be a regular expression. Hence, kind('.*_test', '//...') will evaluate to all targets in the repository whose rule type ends with _test, e.g. java_test or cxx_test.

Quotation of the pattern is often required to make sure regular expressions, e.g. .*_test, are parsed correctly.

Extract content of rule attributes: labels

expr ::= labels(word, expr)

The labels(name, expr) operator evaluates to the set of build targets and file paths specified in the attribute name of the targets that result from the evaluation of the expression expr.

For example, labels(srcs, '//foo:bar') will return all files and build targets specified in the srcs attribute of the rule //foo:bar.

labels('deps', testsof('//foo:bar')) evaluates to the build targets and file paths of the deps attribute of the tests of the rule //foo:bar. Note that deps must be quoted because it is a reserved keyword of the query language.

Find targets that own specified files: owner

expr ::= owner(word)

The owner(x) operator evaluates to the targets that own the file specified as argument. For example, owner('examples/1.txt') evaluates to the target that owns the file examples/1.txt, e.g. //examples:one.

If no target that owns the file is found, the operator evaluates to the empty set.

Transitive closure of reverse dependencies: rdeps

expr ::= rdeps(expr, expr)
       | rdeps(expr, expr, depth)

The rdeps(u, x) operator evaluates to the reverse dependencies of the argument set x within the transitive closure of the set u (the universe).

The rdeps operator accepts an optional third argument, which is an integer literal specifying an upper bound on the depth of the search. So rdeps(//foo:bar, //example:baz, 1) evaluates to the targets in the transitive closure of //foo:bar that depend directly on //example:baz. If the depth parameter is omitted, the search is unbounded.

List the tests of given targets: testsof

expr ::= testsof(expr)

The testsof(x) operator evaluates to the tests associated with the targets of its argument set x. For example, the value of testsof(set('//foo:bar' '//baz:app+lib') is the set of tests associated with either //foo:bar or //baz:app+lib.

In order to obtain all the tests associated with the target and its dependencies, this operator can be combined with the deps operator, e.g. testsof(deps(//foo:bar)) first finds the transitive closure of //foo:bar, and then lists all the tests of this transitive closure.

Executing multiple queries at once

Suppose you want to know the tests associated with a set of targets. This can be done combining the testsof, deps and set operators, e.g.

buck query testsof(deps(set(target1 target2 target3)))

Suppose now that you want to now the tests for each of these targets. The above command only had the union of the tests. Instead of executing one query per target, buck query provides an interface to perform repeated queries with different inputs at the same time. To do this, first define the query expression format and then pass the list of input targets. Example:

buck query testsof(deps(%s)) target1 target2 target3

The %s in the query expression format will be replaced by each the targets and the resulting query is evaluated for each input. Combined with the --json parameter, the result of this buck query call is aggregated by input target, otherwise it returns the union of each query.

Referencing Args Files

When running queries, arguments can be stored in external files, one argument per line, and referenced with the @ symbol. This is convenient when the number of arguments is long or when the query input should be persisted to source control.

buck query testsof(deps(%s)) @/path/to/args-file

If only one query that includes all targets in the file is desired, an alternative syntax can be used:

buck query testsof(deps(%Ss)) @/path/to/args-file

In the example above, the lines of the file will be converted to a set and subsituted for the %Ss. In addition, each line's contents will be singly quoted. In the example above, if the args file contained the following:

//foo:bar
//foo:baz

Then the query produced will be equivalent to:

buck query testsof(deps(set('//foo:bar' '//foo:baz')))

Parameters

  • --dot

    Outputs the digraph representing the query results in dot format. Example usage:

    $ buck query "allpaths(//foo:bar, //path/to:other)" --dot > graph.dot
    $ dot -Tpng graph.dot -o graph.png
    

    Then, open graph.png to visualize the graph.

  • --json Outputs the results as JSON.
  • --output-attributes <attributes>

    Outputs the results as a JSON dictionary build target -> attributes map. The attributes map is a dictionary mapping the specified attributes to their values for the build target. Attributes are regular expressions (e.g. '.*' matches all attributes). If an attribute (e.g. srcs) is not defined for a build target, it is not present in the output. Example:

    $ buck query '//example/...' --output-attributes buck.type name srcs
    {
      "//example/foo:bar" : {
        "buck.type" : "cxx_library",
        "name" : "foobar",
        "srcs" : [ "example/foo/bar.cc", "example/foo/lib/lib.cc" ]
      }
      "//example/foo:main" : {
        "buck.type" : "cxx_binary",
        "name" : "main"
      }
    }
    

Examples

# For all of the following examples, assume this BUCK file exists in
# the `examples` directory.
cxx_library(
  name = 'one',
  srcs = [ '1.cpp' ],
  deps = [
    ':two',
    ':three',
  ],
)

cxx_library(
  name = 'two',
  srcs = [ '2.cpp' ],
  deps = [
    ':four',
  ],
  tests = [ ':two-tests' ]
)

cxx_library(
  name = 'three',
  srcs = [ '3.cpp' ],
  deps = [
    ':four',
    ':five',
  ],
  tests = [ ':three-tests' ],
)

cxx_library(
  name = 'four',
  srcs = [ '4.cpp' ],
  deps = [
    ':five',
  ]
)

cxx_library(
  name = 'five',
  srcs = [ '5.cpp' ],
)

cxx_test(
  name = 'two-tests',
  srcs = [ '2-test.cpp' ],
  deps = [ ':two' ],
)

cxx_test(
  name = 'three-tests',
  srcs = [ '3-test.cpp' ],
  deps = [ ':three' ],
)

Lists all the targets in the repository.

buck query "//..."
//examples:five
//examples:four
//examples:one
//examples:three
//examples:three-tests
//examples:two
//examples:two-tests

Resolve multiple aliases. Suppose there is an alias app = //apps/myapp:app and another lib = //libraries/mylib:lib.

buck query "%s" app lib --json
{
  "app": ["//apps/myapp:app"],
  "lib": ["//libraries/mylib:lib"]
}

Lists all of the rules that the one library directly depends on
buck query "deps(//examples:one, 1)"
//examples:one
//examples:three
//examples:two
Lists the JSON representation of the transitive closure of the rules that the one library depends on
buck query "deps(//examples:one)"
[
  "//examples:five",
  "//examples:four",
  "//examples:one",
  "//examples:three",
  "//examples:two"
]
Outputs a JSON representation of the tests associated with the one and three libraries.
buck query --json "testsof(deps('%s'))" //examples:one //examples:three
{
  "//examples:one": ["//examples:two-tests"],
  "//examples:three": ["//examples:three-tests"]
}

Resolve multiple aliases. Suppose there is an alias app = //apps/myapp:app and another lib = //libraries/mylib:lib.

buck query "buildfile(owner('examples/1.cpp'))"
example/BUCK