Getting started with debusine
In this tutorial you will discover two of debusine’s building blocks: work requests and artifacts. The former lets you schedule the execution of different kind of tasks on remote workers, while the latter is a simple way to represent a set of files managed by debusine.
For the purpose of this tutorial, we will use debusine’s command line interface to:
create a build chroot
upload a source package
build binary packages out of the uploaded source package and in the build chroot that got created
Pre-requisites
You need to have access to a working debusine instance:
If you are a Debian developer, you can use debusine.debian.net. You can login there with your salsa.debian.org account, and it will automatically create your debusine account.
Otherwise, please follow the instructions from Install your first debusine instance. Once completed, the debusine instance will be available under the
debusine.internal
hostname and this is the name that we will continue to use here.
Install the client and configure it
You should first install the debusine-client
package (if needed
configure APT with one of the supported package repositories):
$ sudo apt install debusine-client
Now it’s time to create yourself a token that the client will use to
connect to the server. For this open your web browser and visit the URL
of your debusine instance (i.e. http://debusine.internal
or
https://debusine.debian.net). Then login with your user credentials (or
with the salsa.debian.org authentication) and open the user menu (in
the top right corner, or behind the “hamburger” menu) and click on the
Tokens page.
Click the link create a new token and input a comment describing the token’s intended usage (for example “On my laptop”) and click the Create button. You will receive the token, which you should copy before exiting the page as the token will only be shown once.
Finally, we can create debusine-client’s configuration file. It indicates the server URL and the token to use (please adjust the DEBUSINE_URL to match your specific case and paste the value of the token in the TOKEN variable):
$ DEBUSINE_URL="http://debusine.internal" # or DEBUSINE_URL="https://debusine.debian.net"
$ TOKEN="1e81fb5f00a489344ddfe4e34588c84d202a97ca87cc136d15d17cb3d1eb4e82"
$ DEBUSINE_FQDN=$(basename $DEBUSINE_URL)
$ mkdir -p ~/.config/debusine/client
$ cat >~/.config/debusine/client/config.ini <<END
[General]
default-server = $DEBUSINE_FQDN
[server:$DEBUSINE_FQDN]
api-url = $DEBUSINE_URL/api
token = $TOKEN
END
More information about the debusine command line interface is available in The debusine command.
Submit a work request to create a build chroot
Submitting a work request is asking the debusine server to schedule some specific task on one of its available workers. Debusine can execute many different tasks.
In order to generate a build chroot, you can rely on the mmdebstrap task. Each task requires a certain numbers of key-value
parameters that are fed to debusine create-work-request TASKNAME
as YAML data
on the standard input. Try this:
$ debusine create-work-request mmdebstrap << END
bootstrap_options:
architecture: amd64
variant: buildd
bootstrap_repositories:
- mirror: http://deb.debian.org/debian
suite: bookworm
components:
- main
- contrib
- non-free
- non-free-firmware
END
It does output some YAML structured information:
result: success
message: Work request registered on http://debusine.internal/api with id 314.
work_request_id: 314
At this point, the task has not been executed yet but it has been accepted
and will be processed as soon as a worker becomes available. You can
follow the status of the work request through the web interface
(click on Work requests in the top menu) or with debusine
show-work-request WORK_REQUEST_ID
.
$ debusine show-work-request 314
id: 314
created_at: 2024-01-24 17:02:31.855184+00:00
started_at: 2024-01-24 17:02:31.937001+00:00
completed_at: null
duration: null
status: running
result: ''
worker: 1
task_name: mmdebstrap
task_data:
[…]
artifacts: []
workspace: System
In the above example, the task is currently running. Note that if the
status is pending
it indicates that no debusine-worker
is
currently available to run the task. Check that the debusine-worker
is
connected, has been approved and has mmdebstrap
installed.
Once it has completed (as indicated by status: completed
), you will
see different values: the result field will be either success
,
failure
(task ran but returned a failure) or error
(something
unexpected happened), and more information will be available in the
artifacts
key in particular:
$ debusine show-work-request 314
id: 314
created_at: 2024-01-24 17:02:31.855184+00:00
started_at: 2024-01-24 17:02:31.937001+00:00
completed_at: 2024-01-24 17:05:48.457239+00:00
duration: 196
status: completed
result: success
[…]
artifacts:
- id: 534
workspace: System
category: debian:system-tarball
[…]
- id: 535
workspace: System
category: debusine:work-request-debug-logs
[…]
The two artifacts listed correspond to two set of files generated by the
task and each task will typically generate the same kind of files as
output. In the case of the mmdebstrap
task, you will get:
a debian:system-tarball artifact: it contains a compressed tarball with the chroot.
a debusine:work-request-debug-logs artifact: it contains various files generated by debusine to help troubleshoot issues with the task. Here it contains the input that debusine submitted to mmdebstrap, the precise command-line, and the output of the command.
The generated artifacts can be browsed and downloaded from the web interface. But they can also be downloaded from the command line with the debusine client:
$ debusine download-artifact 534
Downloading artifact and uncompressing into /home/debian
system.tar.xz
Artifacts are thus created by debusine tasks, but they can also be
provided as input to some debusine tasks. Later in this tutorial
we will refer to this artifact 534
to schedule a package build
in the build chroot that it contains.
Create an artifact by uploading a source package
The low level debusine create-artifact
command can be used to create
any arbitrary artifact but when it comes to Debian source packages (.dsc)
or Debian uploads (.changes), debusine offers a more convenient
interface with debusine import-debian-artifact [FILE|URL]
. You can
refer to a local file or to a remote URL.
For instance, you can create and upload an artifact for the “hello” source package with:
$ debusine import-debian-artifact http://deb.debian.org/debian/pool/main/h/hello/hello_2.10-3.dsc
[...]
message: New artifact created in http://debusine.internal/api in workspace System with id 536.
artifact_id: 536
Submit a work request to build binary packages
Among the available tasks, the sbuild one can build
binary packages out of a source package (provided through the parameter
source_artifact_id
below the input
key) and a build chroot
(provided through the parameter environment
).
Let’s generate the YAML file with the work request specification by taking care
to refer to the IDs of the artifacts that we created in the former steps
(536
for the source package, 534
for the build chroot):
$ cat > work-request-hello.debusine << END
build_components:
- any
- all
host_architecture: amd64
input:
source_artifact_id: 536
environment: 534
END
Then submit the job:
$ debusine create-work-request sbuild < work-request-hello.debusine
Again you can follow the status of the work request on the web interface or with the command line, and once completed you will be able to download the binary packages that have been built.
The sbuild tasks generates debian:upload, debian:binary-packages and debian:package-build-log artifacts.
On top of the contained files, the artifact category also defines the
structure of the metadata that is associated to the artifact. You can
inspect those metadata and the file listing on the web interface or on
the command line with debusine show-artifact ARTIFACT_ID
(here 539
is the artifact ID of the debian:upload
artifact created by the
task):
$ debusine show-artifact 539
id: 539
workspace: System
category: debian:upload
created_at: '2024-01-24T21:07:04.975882+00:00'
data:
type: dpkg
changes_fields:
Date: Mon, 26 Dec 2022 16:30:00 +0100
[…]
download_tar_gz_url: http://debusine.internal/artifact/539/?archive=tar.gz
files_to_upload: []
expire_at: null
files:
hello_2.10-3_amd64.buildinfo:
size: 5511
checksums:
sha256: 422aef340c827d2ed2b38c353f660b70e754509bc0ddb0952975090d9f25caaa
type: file
url: http://debusine.internal/artifact/539/hello_2.10-3_amd64.buildinfo
hello_2.10-3_amd64.changes:
size: 1889
checksums:
sha256: d5d694b42b94587d38a5f883fe1fc5d44368ffe974ac3d506d55bcbef0ab0767
type: file
url: http://debusine.internal/artifact/539/hello_2.10-3_amd64.changes
hello_2.10-3_amd64.deb:
size: 53084
checksums:
sha256: 069754b87d7a546253554813252dacbd7a53e959845cc9f6e8f4c1c8fe3746c5
type: file
url: http://debusine.internal/artifact/539/hello_2.10-3_amd64.deb
hello-dbgsym_2.10-3_amd64.deb:
size: 35096
checksums:
sha256: 1550fcd93105a3cf8fddfc776fda0fbebb51dd7c2d2286eeabc43cb37896ad1e
type: file
url: http://debusine.internal/artifact/539/hello-dbgsym_2.10-3_amd64.deb