Buck: buck query

buck query

The buck query command provides functionality to query the target-nodes graph and return the build targets that satisfy your query expression.

The query language implemented in buck query, enables you to combine multiple operators in a single command. For example, to retrieve a list of all the tests for a build target, you can combine the deps() and testsof() operators into a single call to buck query.

buck query "testsof(deps('//java/com/example/app:amazing'))"

Query Language

The Buck query language was inspired by the Bazel Query Language. The Buck query language uses the same parser, so the lexical syntax is similar. The Buck query language supports a subset of Bazel's query functionality with a few extensions, such as attrfilter, inputs, and owner.

Lexical Syntax

Expressions in the query language are composed of the following tokens:

  • Keywords, such as deps or testsof. Keywords are the reserved words of the language. The complete set of keywords is:
  • Words, such as //foo:bar and //foo:bar+lib". If a character sequence is "quoted", that is, it begins and ends with a single-quote ('), or begins and ends with a double-quote ("), it is always parsed as a word. If a character sequence is not quoted, it is still parsed as a word if it adheres to certain conditions: unquoted words are sequence of characters drawn from the set of alphabet characters, numerals, forward slash (/), colon (:), period (.), hyphen (-), underscore (_), and asterisk (*). Unquoted words may not start with a hyphen or period. This syntax makes quoting unnecessary in many cases. For example, quoting "java_test" is unnecessary.

    Consider the following additional examples:

    //foo:bar+wiz    # WRONG: scanned as //foo:bar + wiz.
    //foo:bar=wiz    # WRONG: scanned as //foo:bar = wiz.
    '//foo:bar+wiz'  # Okay.
    '//foo:bar=wiz'  # Okay.
    

    Note that quoting of words is in addition to any quoting that may be required by your shell. In the following example, double-quotes are used for the shell and single-quotes for the build target expression.

    buck query " '//foo:bar=wiz' "
    

    You should always use quotes when writing scripts that construct buck query expressions from user-supplied values.

  • Punctuation, such as parentheses (()), period (.) and comma (,), etc. Words containing punctuation—other than the exceptions listed above under Words—must be quoted.

Whitespace characters outside of a quoted word are ignored.

Expressions: syntax and semantics of the grammar

This is the grammar of the query language, expressed in Extended Backus Naur Form (EBNF) notation:

expr ::= word
       | (expr)
       | expr intersect expr
       | expr ^ expr
       | expr union expr
       | expr + expr
       | expr except expr
       | expr - expr
       | allpaths(expr, expr)
       | attrfilter(word, word, expr)
       | buildfile(expr)
       | deps(expr)
       | deps(expr, depth)
       | deps(expr, depth, expr)
       | deps(expr, depth, first-order-deps(expr))
       | filter(word, expr)
       | inputs(expr)
       | kind(word, expr)
       | labels(word, expr)
       | owner(word)
       | rdeps(expr, expr)
       | rdeps(expr, expr, depth)
       | set(word *)
       | testsof(expr)

Target Patterns

expr ::= word

Syntactically, a build target pattern, or build target expression, is just a word. A build target pattern evaluates to a set containing one or more elements. It is interpreted as an unordered set of targets.

For example, the word //foo:bar resolves to a set containing one build target, while //foo/... resolves to all targets in every directory beneath the foo directory.

Aliases

expr ::= word

Aliases defined in .buckconfig can be used in queries. Aliases resolve to their corresponding build targets. Suppose there is the following alias

app = //apps/myapp:app

then app can be used instead of //apps/myapp:app in query expressions.

You can also use buck query to resolve aliases. For example:

$ buck query app
//apps/myapp:app

Parenthesized expressions

expr ::= (expr)

Parentheses associate sub-expressions to force an order of evaluation. A parenthesized expression resolves to the value of the expression it encloses.

Algebraic set operations: intersection, union, set difference

expr ::= expr intersect expr
       | expr ^ expr
       | expr union expr
       | expr + expr
       | expr except expr
       | expr - expr

These three operators compute the corresponding set operations over their arguments. Each operator has two forms, a nominal form, such as intersect, and a symbolic form, such as ^. Both forms are equivalent; the symbolic forms are just quicker to type. For example,

buck query "deps('//foo:bar') intersect deps('//baz:lib')"

and

buck query "deps('//foo:bar') ^ deps('//baz:lib')"

both return the targets that appear in the transitive closure of //foo:bar and //baz:lib.

The intersect (^) and union (+) operators are commutative (symmetric); however, the except (-) operator is asymmetric.

The parser treats all three operators as left-associative and of equal precedence, so we strongly recommend that you use parentheses if you need to ensure a specific order of evaluation. For example, the first two expressions are equivalent, but the third is not:

x intersect y union z
(x intersect y) union z
x intersect (y union z)

Group targets: set

Semantics

set(a b c ...) 

Syntax

expr ::= set(expr *)

The set(a b c ...) operator computes the union of a set of zero or more targets, separated by white space (no commas). Quote the targets to ensure that they are parsed correctly.

Example:

The following expression returns the merged set (union) of dependencies for the targets: :main and :myclass.

buck query "deps( set( ':main' ':myclass' ) )"

If you invoke buck query programmatically, such as to run a query on a list of targets, then set() is a way to group this list in the query.

All dependency paths: allpaths

Semantics

allpaths(from, to)

Syntax

expr ::= allpaths(expr, expr)

The allpaths(from, to) operator evaluates to the graph formed by paths between the sets from and to, following the dependencies between nodes. For example, the value of

buck query "allpaths('//foo:bar', '//foo/bar/lib:baz')"

is the dependency graph rooted at the single target node //foo:bar, that includes all target nodes that depend on //foo/bar/lib:baz.

The two arguments to allpaths() can themselves be expressions. For example, the command:

buck query "allpaths(kind(java_library, '//...'), '//foo:bar')"

shows all the paths between any java_library in the repository and the target //foo:bar.

We recommend using allpaths() with the --dot parameter to generate a graphviz file that can then be rendered as an image. For example:

$ buck query "allpaths('//foo:bar', '//foo/bar/lib:baz')" --dot > result.dot
$ dot -Tpng result.dot -o image.png

Graphviz is an open-source graph-visualization software tool. Graphviz uses the dot language to describe graphs.

Rule attribute filtering: attrfilter

Semantics

attrfilter(attribute, value, expression)

Syntax

expr ::= attrfilter(word, word, expr)

The attrfilter(attribute, value, expression) operator evaluates the given expression and filters the resulting build targets for those where the specified attribute contains the specified value.

In this context, the term attribute refers to an argument in a build rule, such as name, headers, srcs, or deps.

If the attribute is a single value, say name, it is compared to the specified value, and the target is returned if they match. If the attribute is a list, the target is returned if that list contains the specified value. If the attribute is a dictionary, the target is returned if the value exists in either the keys or the values of the dictionary.

For example,

buck query "attrfilter(deps, '//foo:bar', '//...')"

returns the build targets in the repository that depend on //foo:bar—or more precisely: those build targets that include //foo:bar in their deps argument list.

The match performed by attrfilter() is semantic rather than textual. So, for example, if you have the following deps argument in your build file:

cxx_binary(
  name = 'main',
  srcs = [
    'main.cpp'
  ],
  deps = [
    ':myclass',
  ],
)

Your attrfilter() clause should be:

buck query "attrfilter( deps, '//:myclass', '//...' )"

Note the double forward slash (//) before the second argument to attrfilter().

Build files of targets: buildfile

Semantics

buildfile(expression)

Syntax

expr ::= buildfile(expr)

The buildfile(expression) operator evaluates to those build files that define the targets that result from the evaluation of expression.

In order to find the build file associated with a source file, combine the owner operator with buildfile. For example,

buck query "buildfile(owner('foo/bar/main.cpp'))" 

first finds the targets that own foo/bar/main.cpp and then returns the build files, such as foo/bar/BUCK, that define those targets.

Transitive closure of dependencies: deps and first-order-deps

Semantics

deps(x)
deps(x, integer)
deps(x, integer, y)
deps(x, integer, first_order_deps(y))

Syntax

expr ::= deps(expr)
       | deps(expr, depth)
       | deps(expr, depth, expr)
       | deps(expr, depth, first_order_deps(expr))

The deps(x) operator evaluates to the graph formed by the transitive closure of the dependencies of its argument set, x, including the nodes from the argument set itself. For example, the value of

buck query "deps('//foo:bar')" 

is the dependency graph rooted at the target node //foo:bar. It includes all of the dependencies of //foo:bar. It also includes //foo:bar itself.

The deps operator accepts an optional second argument, which is an integer literal specifying an upper bound on the depth of the search. So,

deps('//foo:bar', 1) 

evaluates to the direct dependencies of the target //foo:bar, and

deps('//foo:bar', 2) 

further includes the nodes directly reachable from the nodes in deps('//foo:bar', 1), and so on. If the depth parameter is omitted, the search is unbounded, that is, it computes the entire transitive closure of dependencies.

Filter expressions and first_order_deps()

The deps() operator also accepts an optional third argument, which is a filter expression that is evaluated for each node and returns the child nodes to recurse on when collecting transitive dependencies.

This filter expression can use the first_order_deps() operator which returns a set that contains the first-order dependencies of the current node—which is equivalent to deps(<node>, 1). For example, the query,

buck query "deps('//foo:bar', 1, first_order_deps())"

is equivalent to

buck query "deps('//foo:bar', 1)"

The first_order_deps() operator can be used only inside the expression passed as an argument to deps().

Note that because deps() uses positional parameters, you must specify the second argument in order to specify the third. In this scenario, if you want the search to be unbounded, we recommend that you use 2147483647 which corresponds to Java's Integer.MAX_VALUE.

Filter targets by name: filter

Semantics

filter(pattern, expression)

Syntax

expr ::= filter(word, expr)

The filter(pattern, expression) operator evaluates the specified expression and returns the targets that have a name attribute that matches the specified regular expression pattern. For example,

buck query "filter('library', deps('//foo:bar'))"

returns the targets in the transitive closure of //foo:barthat contain the string library in their name attribute.

The filter() operator performs a partial match. So, both of the following clauses would match a target with the name main.

buck query "filter( 'main', '//...' )"

buck query "filter( 'mai', '//...' )"

Another example:

buck query "filter('.*\.java$', labels(srcs, '//foo:bar'))"

returns the java files used to build //foo:bar.

You often need to quote the pattern to ensure that regular expressions, such as .*xpto, are parsed correctly.

Direct input files: inputs

Semantics

inputs(x)

Syntax

expr ::= inputs(expr)

The inputs(x) operator returns the files that are inputs to the argument set x, ignoring all dependencies. Note that it does not include any files required for parsing, such as the BUCK file. Rather, it returns only the files required to actually run the build after parsing has been performed.

Note that input() returns only those input files indicated by the target graph. Input files that are present in the action graph but not in the target graph are not returned by input().

You could consider the inputs() and owner() operators to be inverses of each other.

Filter targets by rule type: kind

Semantics

kind(pattern, expression)

Syntax

expr ::= kind(word, expr)

The kind(pattern, expression) operator evaluates the specified expression and returns the targets where the rule type matches the specified pattern. For example,

buck query "kind('java_library', deps('//foo:bar'))"

returns all java_library targets in the transitive dependencies of //foo:bar.

The specified pattern can be a regular expression. For example,

buck query "kind('.*_test', '//...')"

returns all targets in the repository with a rule type that ends with _test, such as java_test and cxx_test.

You often need to quote the pattern to ensure that regular expressions, such as .*xpto, are parsed correctly.

To get a list of the available rule types in a given set of targets, you could use a command such as the following:

buck query : --output-attribute buck.type

which prints all the rule types in the build file in the current directory (:)—in JSON format. See --output-attribute described in the Parameters section below for more information.

Extract content of rule attributes: labels

Semantics

labels(name, expression)

Syntax

expr ::= labels(word, expr)

The labels(name, expression) operator returns the set of build targets and file paths listed in the attribute specified by the name parameter, in the targets that result from the evaluation of expression. Valid values for name include srcs, headers, and deps.

Example: Get all build targets and file paths specified in the srcs attribute for all the rules in the build file in the current directory.

buck query "labels( 'srcs', ':' )"

In performing this operation, Buck validates that any source files referenced in these attributes do, in fact, exist; Buck generates an error if they do not.

Example: Get all the build targets and file paths specified in the deps arguments in the tests of the target //foo:bar.

buck query "labels('deps', testsof('//foo:bar'))"

Note that deps must be quoted because, in addition to being a build-file attribute, it is itself a reserved keyword of the query language.

Find targets that own specified files: owner

Semantics

owner(file)

Syntax

expr ::= owner(word)

The owner(file) operator returns the targets that own the specified file. In this context, own means that the target has the specified file as an input. You could consider the owner() and inputs() operators to be inverses of each other.

Example:

buck query "owner('examples/1.txt')"

returns the targets that owns the file examples/1.txt, which could be a value such as //examples:one.

It is possible for the specified file to have multiple owners, in which case, owner() returns a set of targets.

If no owner for the file is found, owner() outputs the message:

No owner was found for <file>

Transitive closure of reverse dependencies: rdeps

Semantics

rdeps(u, x) 

Syntax

expr ::= rdeps(expr, expr)
       | rdeps(expr, expr, depth)

The rdeps(u, x) operator returns the reverse dependencies of the argument set x within the transitive closure of the set u (the universe). The returned values include the nodes from the argument set x itself.

The rdeps operator accepts an optional third argument, which is an integer literal specifying an upper bound on the depth of the search.

The following example,

buck query "rdeps('//foo:bar', '//example:baz', 1)"

returns the targets in the transitive closure of //foo:bar that depend directly on //example:baz.

If the depth parameter is omitted, the search is unbounded.

List the tests of the specified targets: testsof

Semantics

testsof(expression)

Syntax

expr ::= testsof(expr)

The testsof(expression) operator returns the tests associated with the targets specified by expression. For example,

buck query "testsof(set('//foo:bar' '//baz:app+lib')"

returns the tests associated with //foo:bar and //baz:app+lib.

To obtain all the tests associated with the target and its dependencies, you can combine the testsof() operator with the deps() operator. For example,

buck query "testsof(deps('//foo:bar'))"

first finds the transitive closure of //foo:bar, and then lists all the tests associated with the targets in this transitive closure.

Executing multiple queries at once

Suppose you want to know the tests associated with a set of targets. This can be done by combining the testsof, deps and set operators. For example,

buck query "testsof(deps(set('target1' 'target2' 'target3')))"

Suppose you now want to know the tests for each of these targets; the above command returns the union of the tests. Instead of executing one query for the entire set of targets, buck query provides a way to repeat a query with different targets using a single command. To do this, first define the query expression format and then list the input targets, separated by spaces. For example,

buck query "testsof(deps( %s ))" target1 target2 target3

The %s in the query expression is replaced by each of the listed targets, and for each target, the resulting query expression is evaluated. If you add the --json parameter, the result of the command is grouped by input target; otherwise, as in the previous example using set(), the command merges the results and returns the union of the queries.

This syntax is also useful for subcommands that take arguments that are not targets, such as owner(). Recall that the set() operator works only with targets, but the owner() operator takes a filename as its argument.

buck query "owner( %s )" main.cpp myclass.cpp myclass.h

Referencing Args Files

When running queries, arguments can be stored in external files, one argument per line, and referenced with the @ symbol. This is convenient when the number of arguments is long or when you want to persist the query input in source control.

buck query "testsof(deps(%s))" @/path/to/args-file

If you want to include all the targets in the @-file in a single query execution, you can use the following alternative syntax. Note the addition of the capital "S" in "%Ss".

buck query "testsof(deps(%Ss))" @/path/to/args-file

In the example above, the lines of the file are converted to a set and substituted for the %Ss. In addition, each line's contents are singly quoted. In the example above, if the args file contains the following:

//foo:bar
//foo:baz

Then the query expression is equivalent to:

buck query "testsof(deps(set('//foo:bar' '//foo:baz')))"

If you use multiple %Ss operators in a single query, you can specify which lines in the @-file should be used for each instance of %Ss in the query expression: use -- to separate elements that go in different sets. For example:

buck query "testsof(deps(%Ss)) union deps(%Ss)" @path/to/args-file

//foo:foo
--
//bar:bar

is equivalent to running the following:

buck query "testsof(deps(set('//foo:foo'))) union deps(set('//bar:bar'))"

Parameters

  • --dot

    Outputs the digraph representing the query results in dot format. The nodes will be colored according to their type. See graphviz.org for color definitions.

    android_aar          : springgreen2
    android_library      : springgreen3
    android_resource     : springgreen1
    android_prebuilt_aar : olivedrab3
    java_library         : indianred1
    prebuilt_jar         : mediumpurple1
    

    Example usage:

    $ buck query "allpaths('//foo:bar', '//path/to:other')" --dot > graph.dot
    $ dot -Tpng graph.dot -o graph.png
    

    Then, open graph.png to visualize the graph.

  • --json Outputs the results as JSON.
  • --output-attributes <attributes>

    Outputs the results as a JSON dictionary build target -> attributes map. The attributes map is a dictionary mapping the specified attributes to their values for the build target. Attributes are regular expressions (e.g. '.*' matches all attributes). If an attribute (e.g. srcs) is not defined for a build target, it is not present in the output.

    NOTE: There is ambiguity when using this option with '%s' style queries. It is suggested to use --output-attribute instead. Example:

    $ buck query '//example/...' --output-attributes buck.type name srcs
    {
      "//example/foo:bar" : {
        "buck.type" : "cxx_library",
        "name" : "foobar",
        "srcs" : [ "example/foo/bar.cc", "example/foo/lib/lib.cc" ]
      }
      "//example/foo:main" : {
        "buck.type" : "cxx_binary",
        "name" : "main"
      }
    }
    

  • --output-attribute <attribute>

    Outputs the results as a JSON dictionary build target -> attributes map. The attributes map is a dictionary mapping the specified attributes to their values for the build target. Attributes are regular expressions (e.g. '.*' matches all attributes). If an attribute (e.g. srcs) is not defined for a build target, it is not present in the output.

    Multiple attributes may be specified by providing the --output-attribute option multiple times.

    NOTE: The primary difference between this and --output-attributes is that --output-attribute works correctly with other multiple-argument queries. Example:

    $ buck query '//example/...' --output-attribute buck.type --output-attribute name --output-attribute srcs
    {
      "//example/foo:bar" : {
        "buck.type" : "cxx_library",
        "name" : "foobar",
        "srcs" : [ "example/foo/bar.cc", "example/foo/lib/lib.cc" ]
      }
      "//example/foo:main" : {
        "buck.type" : "cxx_binary",
        "name" : "main"
      }
    }
    

Examples

#
# For the following examples, assume this BUCK file exists in
# the `examples` directory.
#
cxx_library(
  name = 'one',
  srcs = [ '1.cpp' ],
  deps = [
    ':two',
    ':three',
  ],
)

cxx_library(
  name = 'two',
  srcs = [ '2.cpp' ],
  deps = [
    ':four',
  ],
  tests = [ ':two-tests' ]
)

cxx_library(
  name = 'three',
  srcs = [ '3.cpp' ],
  deps = [
    ':four',
    ':five',
  ],
  tests = [ ':three-tests' ],
)

cxx_library(
  name = 'four',
  srcs = [ '4.cpp' ],
  deps = [
    ':five',
  ]
)

cxx_library(
  name = 'five',
  srcs = [ '5.cpp' ],
)

cxx_test(
  name = 'two-tests',
  srcs = [ '2-test.cpp' ],
  deps = [ ':two' ],
)

cxx_test(
  name = 'three-tests',
  srcs = [ '3-test.cpp' ],
  deps = [ ':three' ],
)

Example: List all the targets in the repository.

buck query "//..."
//examples:five
//examples:four
//examples:one
//examples:three
//examples:three-tests
//examples:two
//examples:two-tests

Example: Resolve multiple aliases.

Suppose .buckconfig contains the following aliases:

app = //apps/myapp:app
lib = //libraries/mylib:lib

Then the following query

buck query "%s" app lib --json

returns

{
  "app": ["//apps/myapp:app"],
  "lib": ["//libraries/mylib:lib"]
}

Example: List all of the targets on which the one library directly depends.

$ buck query "deps(//examples:one, 1)"
//examples:one
//examples:three
//examples:two

Example: Display a JSON representation of the transitive closure of the targets on which the one library depends.

$ buck query "deps(//examples:one)"
[
  "//examples:five",
  "//examples:four",
  "//examples:one",
  "//examples:three",
  "//examples:two"
]

Example: Display a JSON representation of the tests associated with the one and three libraries.

$ buck query --json "testsof(deps('%s'))" //examples:one //examples:three
{
  "//examples:one": ["//examples:two-tests"],
  "//examples:three": ["//examples:three-tests"]
}

Example: Display the build file that contains the target which is the owner of the source file, examples/1.cpp.

$ buck query "buildfile(owner('examples/1.cpp'))"
example/BUCK