Release history

0.7.2 (2024-11-13)

Quality

  • Use hello from bookworm in piuparts integration test.

0.7.1 (2024-11-12)

Quality

  • Fetch packages from matching suites in integration tests.

0.7.0 (2024-11-12)

Server

  • Unblock reverse-dependencies when aborting a work request.

  • Upgrade to Django 4.2.

  • Implement an admin role for scopes.

  • Validate group names.

  • Add debusine-admin group management command.

  • Add make_signed_source workflow.

  • Add API for monitoring worker status.

  • Add roles for workspaces.

  • Handle scopes in workspace management commands.

  • Add an initial set of permission predicates.

  • Add scope visibility permission check.

  • Use workspace permissions in collection lookup.

  • Force evaluation of lazy request.user in AuthorizationMiddleware.

  • Don’t ignore failed elements of multiple lookups.

  • Make the default workspace public.

  • Improve command-line handling of constraint violations.

  • Add singleton collections.

  • Add permission for creating workspaces.

  • Add lintian workflow.

  • Fix debusine-admin create_workspace --default-expiration-delay command-line parsing.

  • Support lookups that match items of multiple types.

  • Add piuparts workflow.

  • Add qa workflow.

  • Implement signing_template_names in sbuild workflow.

  • Add same_work_request lookup filter to debian:package-build-logs collection.

  • Add debian_pipeline workflow.

  • Add CopyCollectionItems task.

Web UI

  • Disallow public access to work requests in private workspaces.

  • Prototype implementation of scopes in URLs.

  • Handle workspaces with the same name in different scopes.

  • Remove workspace/ segment from URLs.

Client

  • Implement scope support.

  • Correctly download artifacts with directories in file paths.

Worker

  • SystemBootstrap task:

    • Allow keyring URLs starting with file:///usr/share/keyrings/.

    • Write non-ASCII-armored keyrings to .gpg rather than .asc.

  • Sbuild task:

    • Relax binnmu_maintainer validation in dynamic data to avoid failures if DEBUSINE_FQDN is under a non-email-suitable domain.

    • Drop unnecessary sbuild:host_architecture from dynamic metadata.

  • Add DebDiff task.

Signing

  • Sign task:

    • Fail if signing failed.

    • Use detached signatures when signing UEFI files.

    • Take multiple unsigned artifacts and sign them all with the same key.

  • Register Debsign task, which previously existed but was unusable.

Documentation

Quality

  • Add more type annotations for tasks.

  • Fix test failures in non-English locales.

  • Skip simplesystemimagebuild test with UML >= 6.11um1 for now.

0.6.0 (2024-10-10)

Server

  • Tighten up handling of creating artifacts with files that already exist.

  • Add Wait task type.

  • Add Delay task.

  • Add ExternalDebsign task and a corresponding API view to allow a client to provide a signature to it.

  • Add a system for coordinating multiple sub-workflows within a higher-level workflow.

  • Introduce scopes.

  • Introduce a basic application context.

  • Run workflow orchestrators via Celery.

  • Add autopkgtest workflow.

  • Add debusine-admin scope command.

  • Add retry-with-delays action for use in on_failure event reactions.

  • sbuild workflow:

    • Support build profiles.

    • Add retry_delays, which can be used for simplistic retries of dependency-wait failures.

  • Let nginx gzip-compress text responses.

  • Add PackageUpload task.

  • Add package_upload workflow.

Web UI

Client

  • Add debusine provide-signature command.

  • Allow debusine import-debian-artifact to upload individual .deb packages.

  • Correct imported package relations.

  • Don’t download large artifacts as tarballs.

Worker

  • Add MakeSourcePackageUpload task.

  • Add MergeUploads task.

  • Sbuild task:

    • Support build_profiles.

    • Don’t permit architecture-independent binary-only NMUs.

    • Fix architecture field of created debian:binary-packages artifacts.

    • Export DEB_BUILD_OPTIONS for nocheck and nodoc profiles.

    • Set a default maintainer for binary-only NMUs.

  • Apply some environment constraints to the Piuparts task’s base_tgz lookup.

  • Register ExtractForSigning task, which previously existed but was unusable.

  • Fix unshare executor compatibility with Debian environments from before the start of the /usr merge.

  • Fall back to the worker’s host architecture for the purpose of environment lookups if the task doesn’t specify one.

  • Log progress through the main steps of each task.

Signing

Documentation

  • Document signing workers and tasks.

  • Add design for permission management.

  • Add design for reverse-dependencies-autopkgtest workflow.

  • Add design for task configuration, work request statistics, and other build-related features.

  • Add short introduction to Debusine concepts tying everything together.

  • Move explanation of expiration logic to a separate Expiration of data page.

  • Simplify explanation of artifacts.

  • Move information about Task types to a separate page.

  • Move information about collection data models to a separate page.

Quality

  • Use vulture to find dead code.

  • Sort imports automatically using isort.

  • Make coverage reports briefer.

0.5.0 (2024-09-03)

Server

  • Avoid N+1 queries when resolving multiple lookups.

  • Automatically drop privileges when running debusine-admin or debusine-signing as root.

  • Mark retried work requests as blocked if necessary.

  • Add an API endpoint to review manual unblocks.

  • Unassign pending or running work requests when disabling a worker.

  • Fix ineffective debian:environments uniqueness constraint.

  • Adjust the sbuild workflow to allow storing build logs in a new debian:package-build-logs collection.

  • Default to a five-second timeout when sending email, to avoid hangs if the local mail transport agent is broken.

  • Don’t buffer output to log files.

  • Validate new work requests when creating them.

Web UI

  • Link to work request and build log in artifact list.

  • Add a framework of UI shortcuts and sidebar information, allowing a more attractive and consistent presentation of resources such as artifacts and work requests.

  • Redirect user to original URL after login.

  • If an artifact has only one file, download that file by default instead of a tarball.

  • Show input artifacts in work request views.

  • Add a user-friendly view of files in artifacts.

  • Fix error when viewing an artifact with multiple related build logs.

  • Use pygments to render text content.

  • Redesign work request detail view.

  • Use work request labels in the UI.

  • Add UI to review work requests blocked on manual approval.

  • Add a view of registered workers and their running work requests.

  • Fix collection search paging.

  • Add a view of the task queue.

Client

  • Only accept valid artifact categories in debusine create-artifact.

  • Don’t process downloads one byte at a time.

  • Retry some HTTP requests.

Worker

Signing

  • Add support for static (not extracted under wrap) PKCS#11 keys.

  • Add OpenPGP key generation and signing support.

Documentation

  • Document that workers need sbin directories in their PATH.

  • Clarify data model details for the workflow hierarchy.

  • Improve documentation for debusine-admin manage_worker disable.

  • Fix documentation of creating a collection in Set up APT mirroring.

  • Add design for coordinating sub-workflows.

  • Add design for package upload task and workflow.

Quality

  • Support building debusine itself with nocheck and nodoc build profiles.

  • Add pre-commit configuration.

  • Fix various ResourceWarnings.

  • Convert Python packaging to hatchling.

  • Add many more type annotations.

  • Use dbconfig-pgsql for database configuration, avoiding services restarting indefinitely after initial installation.

  • Ensure that debusine starts after and stops before a PostgreSQL service running on the same machine.

  • Make task-killing tests more reliable.

0.4.1 (2024-06-28)

Server

  • Make debusine:test artifact instantiable.

Web UI

  • Introduce a common base layout with a right sidebar.

  • Implement labels for artifacts.

  • Add specialized view for showing build log artifacts.

Worker

  • Run sbuild with --bd-uninstallable-explainer=dose3 and parse its output.

Quality

  • Fix license classifier in setup.cfg.

0.4.0 (2024-06-24)

Server

  • Add API endpoint to retry work requests.

  • Implement retrying workflows.

  • Give the scheduler Celery worker a different node name.

  • Switch to RedisPubSubChannelLayer.

Web UI

  • Add UI to retry work requests.

Worker

  • Add binary-only NMU support to sbuild task.

  • Use arch-test to provide better defaults for system:architectures.

Signing

  • Add a new signing service. This currently supports generating keys (though currently only in software, as opposed to an HSM) and signing UEFI Secure Boot images with them. A few more pieces still need to be assembled before this is useful.

Documentation

  • Document HTTPS setup.

  • Document signing worker.

Quality

  • Remove now-unnecessary autopkgtest schroot creation from integration tests.

  • Add a “playground” system to manage test object creation and to allow discussion of UI prototypes.

  • Use HTTPS in integration tests.

  • Bump timeout for mmdebstrap integration tests.

  • Reorganize test cases for improved type-safety.

  • Fix cleanup order in an integration test which caused failures on slow architectures.

0.3.2 (2024-06-03)

Server

  • Rename some leftovers of “internal” naming for server tasks.

  • Added method to check if a work request can be retried.

  • Fix Architecture: all matching in sbuild workflow.

Web UI

  • Second iteration on collection UI design.

  • Add base template support for django.contrib.messages.

Quality

  • Fix several race conditions and timeouts that caused autopkgtest failures on slow architectures.

0.3.1 (2024-05-28)

Server

  • Namespace collections under workspaces.

  • Refresh worker from database before marking it disconnected, so that we don’t lose changes made using debusine-admin edit_worker_metadata.

  • Add backend capability to retry aborted or failed work requests.

  • sbuild workflow:

    • Fix task data for Architecture: all work requests.

    • Specify the backend in environment lookups.

    • Defer environment resolution.

Web UI

  • Fix typo resulting in HTTP 500 error in collection detail view.

Worker

  • Handle systemd 256 in incus-lxc executor.

  • Handle dangling /etc/resolv.conf symlinks in environments in the unshare executor.

  • Fix mmdebstrap task to specify the architecture of the chroot.

Documentation

  • Fix several errors in the “Getting started with debusine” tutorial.

  • Adjust “The debusine command” reference to refer to self-documenting --help output.

Quality

  • Skip some integration tests for architectures that weren’t in bookworm.

  • Add enums for artifact and collection categories, to guard against typos.

0.3.0 (2024-05-23)

Highlights:

  • The focus of this milestone is on automatic orchestration of building blocks, to allow tasks to be scheduled for all items of a collection. For example, Debusine can now automatically schedule Lintian tasks for all packages in a suite.

  • Added collections and workflows.

  • Added a new lookup syntax, taking advantage of collections.

Server

  • Add infrastructure for collections.

  • Implement debian:environments collection.

  • Implement debian:suite-lintian collection.

  • Add debusine-admin create_collection command.

  • Store tokens only in a hashed form.

  • Implement debian:suite collection.

  • Move the scheduler to a dedicated Celery worker.

  • Generalize work request notifications into event reactions.

  • Implement basic building blocks of workflows.

  • Implement synchronization points.

  • Implement workflow orchestrators.

  • Implement workflow callbacks.

  • Add --default-file-store options to debusine-admin create_workspace and debusine-admin manage_workspace.

  • Restrict creation of non-worker tasks via the API.

  • Add debusine-admin create_file_store command.

  • Implement scheduling priorities.

  • Implement update-collection-with-artifacts event reaction.

  • Implement collection item lookup syntax and semantics.

  • Implement aptmirror server task.

  • Implement updatesuitelintiancollection task to update a debian:suite-lintian collection from debian:suite.

  • Implement debusine:workflow-internal collection.

  • Add debusine-admin create_work_request command.

  • Implement sbuild and update_environments workflows.

  • Add a _system user for use by scripts.

  • Implement expiry of collection items.

  • Add APIs to create workflow templates and workflows.

  • Add debusine-admin create_workflow command.

  • Add debusine-admin delete_workspace command.

  • Implement expiry of work requests.

Web UI

  • Fix ordering of work requests by task name.

  • Improve rendering of multi-line strings in task data.

  • Show workflow information for work requests that are part of workflows.

  • Show task type in work request lists.

  • Improve handling of expired artifacts in autopkgtest/lintian views.

  • Order a work request’s artifacts by ID within each category.

  • Show the user who created a work request in the work request detail view.

  • Show a notice when a work request’s artifacts have expired.

  • Add workspace detail and collection views.

Client

  • Separate YAML input and output more clearly when running debusine create-artifact or debusine create-work-request.

  • Add debusine manage-work-request command to adjust work request priorities.

  • Add debusine create-workflow-template and debusine create-workflow commands.

Worker

  • Add support for passing extra packages to the sbuild task.

  • Exit cleanly on failure to report a completed work request to the server.

  • Restrict mmdebstrap and simplesystemimagebuild tasks to workers that support the requested architecture, as was done for other tasks in 0.2.1.

  • Only consider the autopkgtest task to have succeeded on exit codes 0, 2, and 8.

  • Remove network-related files that mmdebstrap copies from the host.

  • Allow sbuild to produce no .changes file, so that users can examine the log files of failed builds.

  • Improve “Unexpected artifact type” error from the image cache.

  • Rename autopkgtest task’s environment key to extra_environment.

  • Rename environment_id to environment in all tasks, and support the new lookup syntax.

  • Drop insecure sbuild_options from sbuild task.

  • Rename task data fields in autopkgtest, blhc, lintian, piuparts, sbuild, and updatesuitelintiancollection tasks to support the new lookup syntax, removing _id from key names and accepting single or multiple lookups as appropriate.

  • Correctly tag sid tarballs and images as codename=sid.

  • Don’t purge build-dependencies after build in the sbuild task.

Documentation

  • Move unimplemented features to a new “Development blueprints” section.

  • Add design practices.

  • Rework “Where to start” section in “Contribute to debusine”.

  • Clarify parameters to piuparts task.

  • Clarify the role of Incus when installing a debusine instance.

  • Add design for tasks that update collections.

  • Document work request scheduling and associated worker metadata.

  • Add design for workflows.

  • Document image caching and cleanup.

  • Add design for scheduling priorities.

  • Add design for collection item lookups.

  • Add design for sbuild workflow.

  • Add design for update_environments workflow.

  • Add how-to for setting up APT mirroring.

  • Add example script to automate Incus configuration for workers.

  • Document packages required for Incus VMs.

  • Add example script to populate a debusine instance with example data.

  • Document environment requirements for executor backends.

  • Update “Getting started with debusine” tutorial to use workflows and collections.

  • Add more documentation of worker behaviour.

Quality

  • Validate the summary in debian:lintian artifacts.

  • Drop compatibility with Debian bullseye; debusine now requires Python >= 3.11.

  • Enforce pydantic models for WorkRequest.workflow_data and WorkRequest.event_reactions.

  • Use pydantic models for autopkgtest and lintian views.

  • Fix some tests on non-amd64 architectures.

  • Auto-format HTML templates using djlint.

  • Add infrastructure for more semantic testing of HTML output.

0.2.1 (2024-03-07)

Server

  • Add a Celery worker for server-side tasks.

Client

  • Trim down dependencies slightly.

Worker

  • Require KVM access for simplesystemimagebuild task.

  • Change container to instance in Incus templates.

  • Log task completion.

  • Restrict tasks to workers that support the requested architecture.

Documentation

  • Improve home page slightly.

Quality

  • Enforce mypy project-wide, including all Django components.

0.2.0 (2024-02-29)

Highlights:

  • Added artifact file storage system.

  • Debian developers can use Debusine to run various QA tasks against packages they are preparing. Those tasks can be scheduled through the API or through the web UI.

Note that it is not possible to directly migrate a database installed using 0.1.0. Migrations from this release to future releases will be possible.

Server

  • Implement file storage.

  • Implement artifact handling.

  • Implement expiration of artifacts and their files.

  • Run database migrations on debusine-server package upgrade.

  • Add debusine-admin monthly_cleanup command, run from a systemd timer.

  • Link work requests to workspaces.

  • Add debusine-admin create_user, debusine-admin list_users, and debusine-admin manage_user commands.

  • Link tokens to users.

  • Allow email notifications if a work request fails.

  • Depend on python3-daphne.

  • Ensure all local artifacts are JSON-serializable.

  • Add debusine-admin create_workspace, debusine-admin list_workspaces, and debusine-admin manage_workspace commands.

  • Use WorkRequest workspace in artifacts.

  • Add default expiration delay to workspaces.

  • Add API to list work requests.

  • Make sure the Django app’s secret key is never publicly readable.

  • Mark workers as disconnected on debusine-server startup.

  • Use Restart=on-failure rather than Restart=always in debusine-server.service.

  • Add debusine-admin info command to help with setting up deployments.

  • Add daily artifact cleanup timer.

  • Use pydantic models for artifact data.

  • Add remote, read-only file storage backend for external Debian archives.

Web UI

  • Add web UI for work requests and workspaces.

  • Add login/logout support to web UI, allowing access to non-public workspaces.

  • Allow registering/removing user API keys using the web UI.

  • Allow uploading artifacts using the web UI.

  • Refinements to web UI for work requests.

  • Make Django aware of HTTP/HTTPS state of requests.

  • Fix download error with empty artifact file and document mmap usage.

  • Implement integration with Salsa Single Sign-On.

  • Add lintian view.

  • Polish various aspects of the web UI.

  • Add autopkgtest view.

  • Fetch images for tasks directly, not via a tarball.

Client

  • Rename client’s configuration key from debusine to api-url.

  • Add --data option to debusine create-work-request.

  • Rename debusine work-request-status to debusine show-work-request.

  • Add debusine on-work-request-completed to allow running a command when a work request completes.

  • debusine.client: Drop obsolete silent keyword, and stricter prototype tests.

  • Add debusine --debug option to debug HTTP traffic.

  • Implement a package downloader (dget).

  • Implement a paginated listing API client.

  • Add API client method for listing all work requests.

  • Add debusine list-work-requests command.

  • Add debusine import-debian-artifact command.

Worker

  • Modify sbuild task to use artifacts.

  • Add pre-upload consistency checks on sbuild results.

  • Rename worker’s configuration key from debusine-url to api-url.

  • Upload sbuild log files even if the .dsc file did not exist.

  • Add piuparts task.

  • Add lintian task.

  • Add autopkgtest task.

  • Add mmdebstrap task.

  • Avoid trying to add debusine-worker user in postinst if it already exists.

  • Add image caching for executor backends.

  • Add unshare executor.

  • Port the autopkgtest and piuparts tasks to unshare.

  • Use Restart=on-failure rather than Restart=always in debusine-worker.service.

  • Make tasks check whether their tools are installed.

  • Use a lock to protect execution of the work request.

  • Add blhc task.

  • Add simplesystemimagebuild task.

  • Use pydantic models for task data.

  • Log exceptions in task preparation and clean-up.

  • Add Incus executor (for both containers and VMs).

  • Add a qemu executor, currently only for autopkgtest and sbuild tasks.

Documentation

  • Drop the “slug” field and the “repository” type.

  • Document debian:package-build-log artifact in ontology.

  • Document using local.py to change settings.

  • Create an overview document with an elevator-pitch-style introduction.

  • Add initial design for autopkgtest and lintian tasks.

  • Add initial design for system tarball artifacts and debootstrap-like tasks.

  • Add initial design for tasks building system disk images.

  • Update the description of the sbuild task.

  • Restructure the documentation following the Diátaxis principles.

  • Clarify copyright notice, contributor status and list of contributors.

  • Enable the Sphinx copybutton plugin.

  • Add some documentation for the Python client API.

  • Improve the “Getting started with debusine” tutorial.

  • Add documentation for debusine-admin commands.

  • Add “Install your first debusine instance” tutorial.

  • Add initial design for collections.

  • Refine design for workflows.

Quality

  • Harmonize license to be GPL-3+ everywhere.

  • Support pydantic 1 and 2.

  • Apply mypy, pyupgrade, and shellcheck consistently.

  • Sync (Build-)Depends with setup.cfg.

0.1.0 (2022-09-09)

Initial release. Includes a server that can drive many workers over a worker-initiated websocket connection, where the workers use the server’s API to get work requests and provide results. There is an sbuild task that workers can run.