Tasks
Ontology for generic tasks
While tasks are unique in theory, we can have different tasks sharing some commonalities. In the Debian context in particular, we have different ways to build Debian packages with different helper programs (sbuild, pbuilder, etc.) and we want those tasks to reuse the same set of parameters so that they can be called interchangeably.
This public interface is materialized by a generic task that can be scheduled by the users and that will run one of the available implementations that can run on one of the available workers.
This section documents those generic tasks and their interface.
There are some task_data
keys that apply to all tasks:
notifications
(optional): a dictionary containing:on_failure
(required): a specification of what to do if the task fails, formatted as an array of dictionaries as follows:channel
(required): theNotificationChannel
to use for this notificationdata
(optional): a dictionary as follows (for email channels; this may change for other notification methods):from
(optional): the email address to send this notification from (defaults to the channel’sfrom
property)to
(optional): a list of email addresses to send this notification to (defaults to the channel’sto
property)cc
(optional): a list of email addresses to CC this notification to (defaults to the channel’scc
property, if any)subject
(optional): the subject line for this notification (defaults to the channel’ssubject
property, or failing that toWorkRequest $work_request_id completed in $work_request_result
); the strings${work_request_id}
and${work_request_result}
(or$work_request_id
and$work_request_result
, provided that they are not followed by valid identifier characters) are replaced by their values
Task data key names are used in pydantic
models, and must therefore be
syntactically valid Python identifiers
(although they may collide with keywords, in which case pydantic
aliases
should be used).
Many tasks look up their execution environment from a
debian:environments collection. These
lookups have architecture
, format
, and backend
filters
automatically added to them based on the task data’s host_architecture
and backend
fields, so it is normally only necessary to specify
codename
and perhaps variant
(e.g.
debian/match:codename=bookworm:variant=autopkgtest
).
Task PackageBuild
A generic task to represent a package build, i.e. the act of transforming a source package (.dsc) into binary packages (.deb).
The task_data
associated to this task can contain the following keys:
input
(required): a dictionary describing the input datasource_artifact
(Single lookup, required): source artifact pointing to a source package, used to retrieve the source package to build.extra_binary_artifacts
: (Multiple lookup, optional). List of artifacts. If provided these binary package artifacts (debian:binary-package
ordebian:binary-packages
) are downloaded and made available to apt when installing build-dependencies.
distribution
(required ifbackend
isschroot
): name of the target distribution.environment
(Single lookup with default categorydebian:environments
), required ifbackend
is notschroot
): artifact of categorydebian:system-tarball
ordebian:system-image
artifact, depending on the backend type.QEMU
andINCUS_VM
require adebian:system-image
artifact, while the other backends require adebian:system-tarball
.backend
(optional, defaults tounshare
): Ifauto
, the task uses the default. Supported backends:incus-lxc
,incus-vm
,qemu
,schroot
, andunshare
.extra_repositories
(optional): a list of extra repositories to enable. Each repository is described by a dictionary with the following possible keys:sources_list
: a single-line for an APT’s sources.list fileauthentication_key
(optional): the ascii-armored public key used to authenticate the repository
host_architecture
(required): the architecture that we want to build for, it defines the architecture of the resulting architecture-specific .deb (if any)build_architecture
(optional, defaults to the host architecture): the architecture on which we want to build the package (implies cross-compilation if different from the host architecture). Can be explicitly set to the undefined value (Python’sNone
or JavaScript’snull
) if we want to allow cross-compilation with any build architecture.build_components
(optional, defaults toany
): list that can contain the following 3 words (cfdpkg-buildpackage --build=any,all,source
):any
: enables build of architecture-specific .deball
: enables build of architecture-independent .debsource
: enables build of the source package (.dsc)
build_profiles
: list of build profiles to enable during package build (cfdpkg-buildpackage --build-profiles
)build_options
: value ofDEB_BUILD_OPTIONS
during buildbuild_path
(optional, default unset): forces the build to happen through a path named according to the passed value. When this value is not set, there’s no restriction on the name of the path.binnmu
(optional, default unset): build a binNMU:changelog
: one line of text for the Debian changelog entrysuffix
: suffix appended to the binary package version, e.g.+b1
timestamp
(optional, default is now): changelog datemaintainer
(optional, default is uploader): changelog author
Task SystemBootstrap
A generic task to represent the bootstrapping of a Debian system out
of an APT repository. The end result of such a task is to generate
an artifact of category debian:system-tarball
.
The task_data
associated to this task can contain the following keys:
bootstrap_options
: a dictionary with a few global options:variant
(optional): maps to the--variant
command line option of debootstrapextra_packages
(optional): list of extra packages to include in the bootstrapped systemarchitecture
(required): the native architecture of the built Debian system. The task will be scheduled on a system of that architecture.
bootstrap_repositories
: a list of repositories used to bootstrap the Debian system. Note that not all implementations might support multiple repositories.types
(optional): a list of source types to enable amongdeb
(binary repository) anddeb-src
(source repository). Defaults to a list withdeb
only.mirror
(required): the base URL of a mirror containing APT repositories in$mirror/dists/$suite
suite
(required): name of the distribution’s repository to use for the bootstrapcomponents
(optional): list of components to use in the APT repository (e.g.main
,contrib
,non-free
, …). Defaults to download theRelease
from the suite and using all the Components.check_signature_with
(optional, defaults tosystem
): indicates whether we want to check the repository signature with the system-wide keyrings (system
), or with the external keyring documented in the in thekeyring
key (valueexternal
), or whether we don’t want to check it at all (valueno-check
).keyring_package
(optional): install an extra keyring package in the bootstrapped systemkeyring
(optional): provide an external keyring for the bootstrapurl
(required): URL of the external keyring to downloadsha256sum
(optional): SHA256 checksum of the keyring to validate the downloaded fileinstall
(boolean, defaults to False): if True, the downloaded keyring is installed and used in the target system.
customization_script
(optional): a script that is copied in the target chroot, executed from inside the chroot and then removed. It lets you perform arbitrary customizations to the generated system. You can use apt to install extra packages. If you want to use something more elaborated than a shell script, you need to make sure to install the appropriate interpreter during the bootstrap phase with theextra_packages
key.
Some executor backends require specific packages to be installed in the tarball/image:
incus-lxc
: Requires:extra_packages: [dbus, systemd, systemd-resolved, systemd-sysv]
, as the image has to be bootable and configure networking with systemd-networkd.
Task SystemImageBuild
This generic task is an extension of the SystemBootstrap generic task: it should generate a disk image artifact complying with the debian:system-image definition. That disk image contains a Debian-based system matching the description provided by the SystemBootstrap interface.
The following additional keys are supported:
disk_image
format
(required): desired format for the disk image. Supported values areraw
andqcow2
.filename
(optional): base of the generated disk image filename.kernel_package
(optional): name of the kernel package to install, the default value islinux-image-generic
, which is only available on Bullseye and later, on some architectures.bootloader
(optional): name of the bootloader package to use, the default value issystemd-boot
on architectures that support it.partitions
(required): a list of partitions, each represented by a dictionary with the following keys:size
(required): size of the partition in gigabytesfilesystem
(required): filesystem used in the partition, can benone
for no filesystem,swap
for a swap partition, orfreespace
for free space that doesn’t result in any partition (it will thus just offset the position of the following partitions).mountpoint
(optional, defaults tonone
): mountpoint of the partition in the target system, can benone
for a partition that doesn’t get a mountpoint.
Some executor backends require specific packages to be installed in the tarball/image or specific customization:
incus-vm
: Requires: A kernel and bootloader, which the SimpleSystemImageBuild task will install. Also:python3
andcustomization_script: /usr/share/autopkgtest/setup-commands/setup-testbed
to support theautopkgtest-virt-incus
driver used bysbuild
andautopkgtest
.qemu
: Requires: A kernel and bootloader, which the SimpleSystemImageBuild task will install.
Specifications of tasks
This section lists all the available tasks, with the input that they are accepting, the description of what they are doing, including the artifacts that they are generating.
The tasks listed in this section are those that you can use to submit work requests.
Autopkgtest task
The task_data
associated to this task can contain the following keys:
input
(required): a dictionary describing the input data:source_artifact
(Single lookup, required): thedebian:source-package
ordebian:upload
artifact representing the source package to be tested with autopkgtestbinary_artifacts
(Multiple lookup, required): a list ofdebian:binary-packages
ordebian:upload
artifacts representing the binary packages to be tested with autopkgtest (they are expected to be part of the same source package as the one identified withsource_artifact
)context_artifacts
(Multiple lookup, optional): a list ofdebian:binary-packages
ordebian:upload
artifacts representing a special context for the tests. This is used to trigger autopkgtests of reverse dependencies, wherecontext_artifacts
is set to the artifacts of the updated package whose reverse dependencies are tested, and source/binary artifacts are one of the reverse dependencies whose autopkgtests will be executed.
host_architecture
(required): the Debian architecture that will be used in the chroot or VM where tests are going to be run. The packages submitted ininput:binary_artifacts
usually have a matching architecture (but need not in the case of cross-architecture package testing, eg. testing i386 packages in an amd64 system).environment
(Single lookup with default categorydebian:environments
, required):debian:system-tarball
ordebian:system-image
artifact (as appropriate for the selected backend) that will be used to run the tests.backend
(optional): the virtualization backend to use, defaults toauto
where the task is free to use the most suitable backend. Supported:incus-lxc
,incus-vm
,qemu
, andunshare
.include_tests
(optional): a list of the tests that will be executed. If not provided (or empty), defaults to all tests being executed. Translates into--test-name=TEST
command line options.exclude_tests
(optional): a list of tests that will skipped. If not provided (or empty), then no tests are skipped. Translates into the--skip-test=TEST
command line options.debug_level
(optional, defaults to 0): a debug level between 0 and 3. Translates into-d
up to-ddd
command line options.extra_apt_sources
(optional): a list of APT sources. Each APT source is described by a single line (deb http://MIRROR CODENAME COMPONENT
) that is copied to a file in /etc/apt/sources.list.d. Translates into--add-apt-source
command line options.use_packages_from_base_repository
(optional, defaults to False): if True, then we pass--apt-default-release=$DISTRIBUTION
with the name of the base distribution given in thedistribution
key.extra_environment
(optional): a dictionary listing environment variables to inject in the build and test environment. Translates into (multiple)--env=VAR=VALUE
command line options.needs_internet
(optional, defaults to “run”): Translates directly into the--needs-internet
command line option. Allowed values are “run”, “try” and “skip”.fail_on
(optional): indicates whether the work request must be marked as failed in different scenario identified by the following sub-keys:failed_test
(optional, defaults to true): at least one test has failed (and the test was not marked as flaky).flaky_test
(optional, defaults to false): at least one flaky test has failed.skipped_test
(optional, defaults to false): at least one test has been skipped.
timeout
(optional): a dictionary where each key/value pair maps to the corresponding--timeout-KEY=VALUE
command line option with the exception of theglobal
key that maps to--timeout=VALUE
. Supported keys areglobal
,factor
,short
,install
,test
,copy
andbuild
.
Note
At this point, we have voluntarily not added any key for the
--pin-packages
option because that option is not explicit enough:
differences between the mirror used to schedule jobs and the mirror
used by the jobs result in tests that are not testing the version that
we want. At this point, we believe it’s better to submit all modified
packages explicitly via input:context_artifacts
so that we are sure
of the .deb that we are submitting and testing with. That way we can even
test reverse dependencies before the modified package is available in any
repository.
This assumes that we can submit arbitrary .deb on the command line and that they are effectively used as part of the package setup.
autopkgtest is always run with the options --apt-upgrade
--output-dir=ARTIFACT-DIR --summary=ARTIFACT-DIR/summary --no-built-binaries
.
An artifact of category debian:autopkgtest
is generated to store all output
files, and is described in the artifacts reference.
Mmdebstrap task
The mmdebstrap
task fully implements the SystemBootstrap interface.
On top of the keys defined in that interface, it also supports the
following additional keys in task_data
:
bootstrap_options
use_signed_by
(defaults to True): if set to False, then we do not pass the keyrings to APT via theSigned-By
sources.list option, instead we rely on the--keyring
command line parameter.
The keys from bootstrap_options
are mapped to command line options:
variant
maps to--variant
(and it supports more values than debootstrap, see its manual page)extra_packages
maps to--include
The keys from bootstrap_repositories
are used to build a sources.list
file that is then fed to mmdebstrap
as input.
SimpleSystemImageBuild task
The simplesystemimagebuild
task implements the SystemImageBuild interface except that it expects a single
entry in the list of partitions: the entry for the root filesystem (thus
with a mountpoint of /
).
In terms of compliance with the SystemBootstrap
interface, the
bootstrap phase only uses a single repository but the remaining
repositories are enabled after the bootstrap.
This task is implemented with the help of the debos
tool.
Lintian task
The task_data
associated to this task can contain the following keys:
input
(required): a dictionary of values describing the input data, one of the sub-keys is required but both can be given at the same time too.source_artifact
(Single lookup, optional): thedebian:source-package
ordebian:upload
artifact representing the source package to be tested with lintianbinary_artifacts
(Multiple lookup, optional): a list ofdebian:binary-package
,debian:binary-packages
, ordebian:upload
artifacts representing the binary packages to be tested with lintian (they are expected to be part of the same source package as the one identified withsource_artifact
)
Note
While it’s possible to submit only a source or only a single binary artifact, you should aim to always submit source + arch-all + arch-any related artifacts to have the best test coverage as some tags can only be emitted when lintian has access to all of them at the same time.
environment
(Single lookup with default categorydebian:environments
, required):debian:system-tarball
artifact that will be used to run lintian. Must havelintian
installed.backend
(optional): the virtualization backend to use, defaults toauto
where the task is free to use the most suitable backend. Supported options:incus-lxc
,incus-vm
,unshare
.output
(optional): a dictionary of values controlling some aspects of the generated artifactssource_analysis
(optional, defaults to True): indicates whether we want to generate thedebian:lintian
artifact for the source packagebinary_all_analysis
(optional, defaults to True): same assource_analysis
but for thedebian:lintian
artifact related toArchitecture: all
packagesbinary_any_analysis
(optional, defaults to True): same assource_analysis
but for thedebian:lintian
artifact related toArchitecture: any
packages
target_distribution
(optional): the fully qualified name of the distribution that will provide the lintian software to analyze the packages. Defaults todebian:unstable
.include_tags
(optional): a list of the lintian tags that are allowed to be reported. If not provided (or empty), defaults to all. Translates into the--tags
or--tags-from file
command line option.exclude_tags
(optional): a list of the lintian tags that are not allowed to be reported. If not provided (or empty), then no tags are hidden. Translates into the--suppress-tags
or--suppress-tags-from file
command line option.fail_on_severity
(optional, defaults tonone
): if the analysis emits tags of that severity or higher, then the task will return a “failure” instead of a “success”. Valid values are (in decreasing severity) “error”, “warning”, “info”, “pedantic”, “experimental”, “overridden”. “none” is a special value indicating that we should never fail.
The lintian runs will always use the options --display-level
">=classification"
(>=pedantic
in jessie) --no-cfg
--display-experimental --info --show-overrides
to collect the full set of
data that lintian can provide.
Note
Current lintian can generate “masked” tags (with M: prefix) when you
use --show-overrides
. For the purpose of debusine, we entirely
ignore those tags on the basis that it’s lintian’s decision to hide
them (and not the maintainer’s decision) and as such, they don’t bring
any useful information. Lintian is full of exceptions to not emit some
tags and the fact that some tags rely on a modular exception mechanism
that can be diverted to generate masked tags is not useful to package
maintainers.
For those reasons, we suggested to lintian’s maintainers to entirely stop emitting those tags in https://bugs.debian.org/1053892
Between 1 to 3 artifacts of category debian:lintian
will be generated (one
for each source/binary package artifact submitted) and they will have a
“relates to” relationship with the corresponding artifact that has been
analyzed. The debian:lintian
artifacts are described
in the artifacts reference.
Sbuild task
Regarding inputs, the sbuild
task is compatible with the ontology
defined for Task PackageBuild even though it implements only
a subset of the possible options at this time.
Currently unsupported PackageBuild
task keys:
extra_repositories
build_architecture
build_options
build_path
Output artifacts and relationships:
debian:package-build-log
: sbuild outputrelates-to:
source_artifact
relates-to:
b
,c
debian:binary-package
: one for each binary package (*.deb
) built from the source packagerelates-to:
source_artifact
debian:binary-packages
: the binary packages (*.deb
) built from the source package, grouped into a single artifactrelates-to:
source_artifact
debian:upload
:c
plus the right administrative files (.changes
,.buildinfo
) necessary for its binary uploadextends:
b
,c
relates-to:
b
,c
debusine:signing-input
: the.changes
file ready for signing if requiredrelates-to:
d
debusine:work-request-debug-logs
: debusine-specific worker logsrelates-to:
source_artifact
Piuparts task
A specific task to represent a binary package check using the
piuparts
utility.
The task_data
associated to this task can contain the following keys:
input
(required): a dictionary describing the input databinary_artifacts
(Multiple lookup, required): a list ofdebian:binary-packages
ordebian:upload
artifacts representing the binary packages to be tested. Multiple artifacts can be provided so as to support e.g. testing binary packages from split indep/arch builds.
backend
(optional, defaults tounshare
). Ifauto
, the task uses the default. Supported backends:incus-lxc
,incus-vm
,schroot
, andunshare
.environment
(Single lookup with default categorydebian:environments
, required): artifact of categorydebian:system-tarball
that will be used to run piuparts itself.base_tgz
(Single lookup with default categorydebian:environments
, required): artifact of categorydebian:system-tarball
that will be used to run piuparts tests, throughpiuparts --base-tgz
. If the artifact’s data haswith_dev: True
, the task will remove the files/dev/*
before using it.host_architecture
(required): the architecture that we want to test on.
The piuparts
output will be provided as a new artifact.
Blhc task
A task to represent a build log check using the blhc
utility.
The task_data
associated to this task can contain the following keys:
input
(required): a dictionary describing the input dataartifact
(Single lookup, required): adebian:package-build-log
artifact corresponding to the build log to be checked. The file should have a.build
suffix.
extra_flags
(optional): a list of flags to be passed to the blhc command, such as--bindnow
or--pie
. If an unsupported flag is passed then the request will fail.
The blhc
output will be provided as a new artifact of category
debian:blhc
, described in the artifacts reference.
The task returns success if `blhc`
returns an exit code of 0 or 1, and
failure otherwise.
UpdateDerivedCollection task
This is a generic server-side task that compares two collections, one of which is derived from the other, and creates any work requests necessary to update the derived collection.
The task_data
for this task may contain the following keys:
base_collection
(Single lookup, required): the “base” collection which we are using as a source of dataderived_collection
(Single lookup, required): the “derived” collection that we are updatingchild_task_data
(optional): a dictionary to use as thetask_data
of child work requests, with additional items merged into it as indicated by the specific implementation; for example, it may be useful to specify anenvironment
hereforce
(boolean, defaults to False): if True, schedule work requests for each matching artifact in the base collection regardless of whether there is already a corresponding artifact in the derived collection (for example, this might be useful when the implementation of the task has changed)
Specific tasks based on this interface are responsible for determining the relevant subsets of active items in each of the base and derived collections that are compared, for defining the desired derived item names given a set of base items, and for defining the work requests needed to perform each individual update to the derived collection.
This task takes the relevant subset of the derived collection and finds the
items in the base collection from which each of them were derived, using
derived_from
in each of the per-item data fields. (Multiple items may
be derived from the same base items.) It then compares these items to the
relevant subset of the base collection and determines the derived items that
need to be changed given the current contents of the base collection, in one
of these ways:
add: a derived item is desired but does not exist
replace: a derived item is desired with the same
name
as one that already exists, but either its base items have changed orforce
is Trueremove: a derived item exists but is not desired
The definition of a collection item’s name
guarantees that only one
active item with a given name may exist in any given collection, so it is a
convenient key to use here.
For each derived item that should be added or replaced, the task creates
suitable child work requests to create a new derived item and update the
derived collection, according to the specific implementation. It only
creates a work request if another work request with the same parameters does
not already exist, or if force
is True.
For each derived item that should be removed, the task immediately removes it from the derived collection.
If this task is part of a workflow, then each of the created work requests is created as a sub-step of it in the same workflow.
UpdateSuiteLintianCollection task
This task implements the UpdateDerivedCollection task interface, updating a derived debian:suite-lintian collection from a base debian-suite collection.
All active items in both collections are considered relevant.
Given a base debian:binary-package
artifact with an architecture other
than all
, a derived item with the name
{srcpkg_name}_{srcpkg_version}_{architecture}
is desired.
Given a base debian:source-package
artifact or for a base
debian:binary-package
artifact with Architecture: all
, derived items
with the names {srcpkg_name}_{srcpkg_version}_{architecture}
are
desired for the following values of architecture
:
source
(only for a source package artifact with no corresponding binary package artifacts)all
each architecture where another base
debian:binary-package
artifact exists for the same source package name and version
The child work requests are for Lintian tasks, with
the following task_data
in addition to anything specified in
this task’s child_task_data
:
input
:source_artifact
(Single lookup): the relevantdebian:source-package
artifact in the base collectionbinary_artifacts
(Multiple lookup): a list of the relevantdebian:binary-package
artifacts in the base collection
Each child work request has an event reaction as follows, where
{derived_collection}
is the derived_collection
from this task’s
data:
on_success:
- action: "update-collection-with-artifacts"
artifact_filters:
category: "debian:lintian"
collection: {derived_collection}
APTMirror task
This is a server-side task that updates an existing debian:suite collection to reflect the state of an external APT suite. It creates source and binary package artifacts as required.
The task_data
for this task may contain the following keys:
collection
(required): the name of thedebian:suite
collection to updateurl
(required): the base URL of the external repositorysuite
(required): the name of the suite in the external repository; if this ends with/
, then this is a flat repository andcomponents
must be omittedcomponents
(optional): if set, only mirror these componentsarchitectures
(required): if set, only mirror binary packages for this list of architecturessigning_key
(optional): ASCII-armored public key used to authenticate this suite
Todo
architectures
should be optional, but discovering the available
architectures without having to implement delicate GPG verification code
ourselves is hard; see message #49 in #848194.
The suite must have at least a Release
file to make it possible to
handle multiple architectures in a reasonable way, and if signing_key
is
specified then it must also have either an InRelease
or a
Release.gpg
file. Source and binary packages must have SHA256
checksums.
Additions and removals of collection items are timestamped as described in Collections, so tasks that need a static view of a collection (e.g. so that all tasks in the same workflow instance see the same base suite contents) can do this by filtering on collection items that were created before or at a given point in time and were not yet removed at that point in time.
Todo
Document exactly how transactional updates of collections work in general: tasks need to see a coherent state, and simple updates to collections can be done in a database transaction, but some longer update tasks where using a single database transaction is impractical may need more careful handling. See discussion in !517.
GenerateKey task
This is a signing task that generates a new key on a signing worker and stores it for later use.
The task_data
for this task may contain the following keys:
purpose
(required): the purpose of the key to generate:uefi
, oropenpgp
.description
(required): A text string with a human-readable description of the new key’s intended purpose.
The output will be provided as a debusine:signing-key artifact. The task should typically be configured with an update-collection-with-artifacts event reaction to attach the new signing key to the collection where it will be used, such as debian:suite-signing-keys.
Todo
This will need additional parameters once we start supporting HSMs.
ExtractForSigning task
This is a worker task that takes the output of the Sbuild task and extracts debusine:signing-input artifacts from them for use by the Sign task.
The task_data
for this task may contain the following keys:
input
(required): a dictionary describing the input data:template_artifact
(Single lookup, required): adebian:binary-package
artifact containing a template packagebinary_artifacts
(Multiple lookup, required): a list ofdebian:binary-package
,debian:binary-packages
, ordebian:upload
artifacts used to find the packages referred to by the template’sfiles.json
environment
(Single lookup with default categorydebian:environments
, required):debian:system-tarball
artifact that will be used to unpack binary packages using theunshare
backend
The task operates as follows:
It extracts the
/usr/share/code-signing/$binary_package_name/files.json
file from the template binary package.It checks that
files.json
uses only relative paths with no..
components.For each package in the template’s
files.json
:It checks that the package name is a syntactically-valid Debian package name.
It finds the corresponding package among the binary artifacts.
If there is a
trusted_certs
entry, it copies it into the corresponding output artifact.For each file:
It checks that the file name uses only relative paths with no
..
components, and that the resulting path within the extracted binary package does not traverse symlinks to outside the extracted binary package.It stores a copy of the file in the output artifact with the name
$package/$file
.
The output will be provided as debusine:signing-input artifacts, one for each package in the template’s
files.json
, with each artifact having a relates-to
relationship to
the template package and to the binary package from which its files were
extracted.
Sign task
This is a signing task that signs the contents of a debusine:signing-input artifact on a signing worker.
The task_data
for this task may contain the following keys:
purpose
(required): the purpose of the key to sign with:uefi
oropenpgp
(needed separately fromkey
so that the scheduler can check whether the worker has the necessary tools available)unsigned
(Single lookup, required): thedebusine:signing-input
artifact whose contents should be signedkey
(Single lookup, required): thedebusine:signing-key
artifact to sign with; must matchpurpose
The output will be provided as a debusine:signing-output artifact.
AssembleSignedSource task
This is a worker task that takes debusine:signing-output artifacts produced by Sign tasks and assembles the resulting source package.
The task_data
for this task may contain the following keys:
environment
(Single lookup with default categorydebian:environments
, required):debian:system-tarball
artifact that will be used to pack the source package using theunshare
backend.dpkg-dev
will be installed there if necessary.template
(Single lookup, required): a debian:binary-package artifact containing a source templatesigned
(Multiple lookup, required): signeddebusine:signing-output
artifacts matching the template
The task operates as follows:
It makes a copy of the
/usr/share/code-signing/$binary_package_name/source-template/
directory from the template binary package.It checks that
debian/source/format
is exactly3.0 (native)
and that neitherdebian/source/options
nordebian/source/local-options
exists.It checks that
files.json
uses only relative paths with no..
components.For each package name and file name in the template’s
files.json
, it finds the corresponding file in the signed artifacts and copies it intodebian/signatures/$package/$file.sig
. For this to work, the names of the files in thedebusine:signing-input
anddebusine:signing-output
artifacts must be composed of the binary package name, followed by/
, followed by the path in the correspondingfile
key infiles.json
.It packs the resulting assembled source package using
dpkg-source -b
, and makes a suitable.changes
file for it usingdpkg-genchanges
.
The output will be provided as a debian:source-package artifact, with a built-using
relationship to
the debian:binary-package artifacts that
were related to the input to the Sign task,
and a debian:upload artifact containing that source
package and the corresponding .changes
file.
Delay task
This wait task completes after its
delay_until
timestamp.
The task_data
for this task may contain the following keys:
delay_until
(datetime, required): a timestamp on or after which this task may be run
As with other wait tasks, the scheduler marks it running (without assigning
a worker) as soon as possible after it is pending, after which it is
considered to be waiting for its event to happen. Separately, it handles
the completion of this particular task directly, by marking it as
successfully completed if the current time is greater than or equal to its
delay_until
timestamp.
MakeSourcePackageUpload task
This worker task makes a debian:upload
artifact from a
debian:source-package
artifact. This involves unpacking the
source package and running dpkg-genchanges
on it.
The task_data
for this task may contain the following keys:
input
(required): a dictionary describing the input data:source_artifact
(Single lookup, required): adebian:source-package
artifact
since_version
(string, optional): include changelog information from all versions strictly later than this version in the.changes
file; the default is to include only the topmost changelog entrytarget_distribution
(string, optional): override the targetDistribution
field in the.changes
file to this value; the default is to use the distribution from the topmost changelog entryenvironment
(Single lookup with default categorydebian:environments
, required):debian:system-tarball
artifact that will be used to rundpkg-source
anddpkg-genchanges
using theunshare
backend.
The output is a debian:upload
artifact, with extends
and
relates-to
relationships to the input source package artifact (to
match the behaviour of debusine import-debian-artifact
).
Used by the package_upload workflow.
MergeUploads task
This worker task combines multiple debian:upload
artifacts into a
single one, in preparation for uploading them together. This involves
running mergechanges
(from devscripts) on them, or equivalent.
The task_data
for this task may contain the following keys:
input
(required): a dictionary describing the input data:uploads
(Multiple lookup, required): a list ofdebian:upload
artifacts
environment
(Single lookup with default categorydebian:environments
, required):debian:system-tarball
artifact that will be used to runmergechanges
using theunshare
backend.
The output is a debian:upload
artifact with extends
relationships to
each of the input upload artifacts.
Used by the package_upload workflow.
Debsign task
This is a signing task that signs a
debian:upload artifact on a signing worker. It is
separate from the Sign task because signing uploads is a customized
operation involving signing multiple files and possibly updating checksums
in the .changes
file to match the signed versions of other files.
The task_data
for this task may contain the following keys:
unsigned
(Single lookup, required): thedebian:upload
artifact whose contents should be signedkey
(Single lookup, required): thedebusine:signing-key
artifact to sign the upload with, which must have purposeopenpgp
The output will be provided as a debian:upload artifact, with relates-to
relations to the
unsigned and key artifacts.
Used by the package_upload workflow.
ExternalDebsign task
This wait task blocks until a user provides a signature for an upload.
The task_data
for this task may contain the following keys:
unsigned
(Single lookup, required): thedebian:upload
artifact whose contents should be signed
Running this task does not do anything. In order to complete it,
the user needs to execute debusine provide-signature
<work-request-id>
to download the files to sign, sign them locally
with debsign
, upload them back as part of a new debian:upload
artifact, and finally record that artifact as an output artifact of
the ExternalDebsign task. The web UI advises the user to run this command
when showing such a work request.
The containing workflow should then normally use an event reaction to add that output artifact to a suitable collection (usually the workflow’s internal collection).
The task does not verify the signature, since it doesn’t necessarily have the public key available. It remains the responsibility of whatever would normally verify the signature (e.g. an external upload queue) to do so.
Used by the package_upload workflow.
PackageUpload task
This server task uploads Debian packages to an upload queue.
It is the equivalent of running dput
, but since other parts of
debusine ensure that the upload is well-formed, there’s no need for
most of the complexity of dput
and we can avoid needing an
environment for it.
The task_data
for this task may contain the following keys:
input
(required): a dictionary describing the input data:upload
(Single lookup, required): adebian:upload
artifact
target
(required): the upload queue, as anftp://
orsftp://
URL
The implementation should take care to use a suitable connection
timeout. An SSH private key should be provided in the ~/.ssh/
directory of the user running debusine.
Used by the package_upload workflow.