Lookups

Purpose of lookups

Tasks and Workflows often need to refer to Artifacts and Collections. Rather than having to use raw IDs everywhere, this is normally done using lookups, written in a special-purpose query language. Lookups allow traversing collections to find items stored in them. Each category of collection defines lookup names tailored to the structure and content of the collection.

Types of lookups

Some lookups resolve to a single artifact or collection, or raise an error if it cannot be found: these are called “single lookups”. For example, a common field in task data is environment, specifying a tarball or image that the task should use as its base execution environment. A single lookup that finds a bookworm image might be debian/match:codename=bookworm. This finds the debian collection (which in context is interpreted to be of the category debian:environments) and asks it to run the match lookup for codename=bookworm; the collection then returns a suitable artifact.

Some lookups resolve to multiple artifacts or collections, based on filters applied to the items in a collection: these are called “multiple lookups”. For example, a workflow might take a binary_artifacts field listing the binary artifacts that it should process. A multiple lookup that finds all the binary packages produced by the glibc source package in Debian’s trixie suite, regardless of architecture, might be:

collection: trixie@debian:suite
child_type: artifact
category: debian:binary-package
data__srcpkg_name: glibc

This finds the trixie collection of category debian:suite and then applies the given filters to all the items of that collection.

In some cases, lookups may resolve to promises rather than to real artifacts. These allow workflows to refer to artifacts that may not yet exist, but that are expected to be provided by other tasks or workflows. In conjunction with the internal collection associated with all workflows, this can be used to plug multiple sub-workflows together into more complex “pipelines”. For example, under a single root workflow, an autopkgtest sub-workflow that wants to consume the output of an sbuild sub-workflow might use:

binary_artifacts:
  collection: internal@collections
  child_type: artifact
  category: debian:upload
  name__startswith: build-

See Lookup syntax for reference documentation on writing lookups.

Lookups and workspace inheritance

Lookups are context-sensitive: the result of the lookup depends on the workspace in which it is executed, and on the user who is executing it.

Most lookups start with the identification of a collection. The lookup debian@debian:environments/match:codename=bookworm on a workflow will look for a collection named debian and of category debian:environments in the workspace where the workflow is executed. If there’s no such collection, it will try to find the collection in one of the parent workspaces by following in the inheritance chain.

However this falls short when you want to refer to a collection in a parent workspace and when the current workspace also has a collection with the same name. In that case, assuming that the parent workspace is named parent, you can prefix the lookup with //workspace=parent/ to force the lookup to start in the context of the parent workspace. To refer to a parent workspace that is in another scope (e.g. myscope), you have to specify the scope too: //scope=myscope:workspace=parent/

This kind of prefix can also be used in the collection field of multiple lookups.