Scheduler tags

The Debusine scheduler matches tasks and workers using sets of required and provided tags: a task can be assigned to a worker if the worker provides all the tags that the task requires.

See Tag-based scheduling for the design background for this feature.

Tag lifetime

The set of tags provided and required by a worker is computed every time the worker updates its dynamic data.

The set of tags provided and required by a task is computed when the task becomes PENDING.

Once the task is PENDING, the sets of task-provided and task-required tags are final and cannot be changed anymore. This makes space for complex use cases while keeping complexity away from the scheduler.

The final sets of task-provided and task-required tags are preserved for the rest of the lifetime of the task, as they are useful after the scheduling for UI use cases like filtering and debugging.

Tag provenance

Providing some tags can be a security sensitive operation, so some tags can only be provided by parts of Debusine that are controlled by restricted sets of users. Debusine therefore has a way to restrict contributing tags to required and provided sets by provenance.

The main security sensitive operations are adding provided tags and removing required tags, as any attack removing provided tags or adding required tags may only have the effect of making a task harder to schedule by reducing the number of candidate workers it can use. While those are the main use cases for filtering tags by provenance, provenance is still tracked for all 4 cases.

Note that there is currently no design for removing tags from the set of tags being constructed, so only the case of adding tags is considered.

This section lists the currently defined provenances for tags.

Workers providing tags

  • worker, for tags provided in dynamic worker metadata when a worker registers as active

  • admin, for tags provided in static worker metadata by the Debusine admins

  • system, from database queries

Workers requiring tags

  • admin, for tags provided in static worker metadata by the Debusine admins

Tasks providing tags

  • user, work task/workspace input in task_data

  • system, from database queries

  • workspace, from the task configuration system

Tasks requiring tags

  • user, work task/workspace input in task_data

  • system, from database queries

  • workspace, from the task configuration system

Tag ontologies

This is a reference documentation of well known tags that can be used for matching.

Tags are strings, tag values are namespaced by prefixes followed by a colon (:), and tags with a similar prefix share some amount of homogeneity and semantics. A set of tags with the same prefix is called an ontology.

Tags are matched by exact string matching, and site admins and workspace admins are free to add custom ones to provided and required sets.

Tags provided by workers

worker:build-arch

Describe the build architectures supported in the worker.

Provenances allowed: all.

  • worker:build-arch:{name}: the worker supports the given build architecture.

name is an architecture name as used in Debian, like amd64, arm64, ppc64el, and so on.

worker:cap

Describe relevant capabilities available in the worker:

Provenances allowed: all.

  • worker:cap:{name}: the worker has the named capability.

name could be the name of an installed package, the name of a suite of packages providing a significant feature, or the name of an available hardware feature.

Other capabilities in actual use:

  • worker:cap:autopkgtest: autopkgtest is available

  • worker:cap:debsign: debsign is available

  • worker:cap:mmdebstrap: mmdebstrap is available

  • worker:cap:sbsign: sbsign is available

  • worker:cap:sbuild: mmdebstrap is available

Examples of possible capabilities: qemu, dev-kvm, gpu, worker-type:worker, task:worker:sbuild:version:1, and so on.

worker:class

Provenances allowed: admin.

Assign one or more arbitrary classes to workers.

  • worker:class:{name}: the worker has been assigned the given class.

name is a classification system arbitrarily defined by the Debusine administrators.

Possible examples: small, large, trusted, official, experimental, and so on.

worker:executor

Provenances allowed: all.

Describe the executor backends available in the worker.

  • worker:executor:{name}: the given executor is available for use in the worker.

name can take values from the debusine.tasks.models.BackendType enum except for auto.

Example executor names: unshare, incus-lxc, incus-vm and qemu.

worker:task

Provenances allowed: all.

Describe the availability of a given version of a given task in the worker.

  • worker:task:{task_type}:{task_name}:version:{version}: the worker can execute the task with the given type, name and version.

worker:type

Provenances allowed: system.

Describe the type of worker:

  • worker:type:{worker_type}: type of worker, encoding the type of tasks that this worker can execute. This is provided by the database field Worker.worker_type and cannot be overridden.

Possible values:

  • worker:type:worker

  • worker:type:server

  • worker:type:signing

  • worker:type:not-assignable: this worker cannot be used for scheduling

Todo

{task_name:="sign"}:available:purpose (when not OpenPGP)

Tags provided by tasks

task:scope

Provenances allowed: system.

Describe the scope that contains the task.

  • task:scope:{scope_name}: the task is in the given scope.

task:workspace

Provenances allowed: system.

Describe the workspace that contains the task.

  • task:workspace:{scope_name}:{workspace_name}: the task is in the given workspace.

task:group

Provenances allowed: system.

Declare that the user who created the task is a member of a group.

  • task:group:{scope_name}:{workspace_name}:{group_name}: the user is a member of this group.

scope_name and group_name correspond to the scope and name of the group.

Given that a group is only optionally assigned to a workspace, workspace_name corresponds to the workspace name of the group, when present, or to the empty string otherwise. A tag for a non-workspaced group could then be, for example: task:group:debian::Debian.

task:source-package

Provenances allowed: system.

Name of the source package for this task.

  • task:source-package:{name}: name of the source package for the task.

name corresponds to a source package name.