============================= Workflow for use by Debian CI ============================= We want to allow ci.debian.net to use Debusine as a backend. Our existing :task:`Autopkgtest` task and :workflow:`autopkgtest` workflow can do most of the hard work, but will need some changes. Overall model ============= ci.debian.net should have its own workspace and its own :collection:`debian:qa-results` collections. The ci.debian.net operators should have the ``OWNER`` role on this workspace, while bot accounts should have the ``CONTRIBUTOR`` role. This workspace can have suitable :ref:`expiration policies `, retaining logs for a few months and basic result history indefinitely (although ci.debian.net will keep its own copies of the history as well, at least to begin with). ``debci`` will start workflows as needed, wait for them to complete, and collect the results. Until we have outbound webhooks to notify ``debci`` of workflow completion, it should simply poll occasionally for the status of workflows whose results it has not yet collected. In future, it might be possible to have ``britney`` dispatch requests to Debusine directly, and nothing in this design should make that more difficult. However, ``debci`` has quite a bit of intelligence of its own as well as the ci.debian.net frontend, so we recommend that it be kept in the loop at least for the first iteration of this work. Looking up packages =================== We can assume that Debusine's mirrored suites are reasonably up to date, and use lookups into them. This won't be quite perfect, especially for packages that have been uploaded very recently; but the integration between ``britney`` and ci.debian.net already knows which version of a package was tested and will repeat tests until it gets a result from the desired version, so we can tolerate some slack here. ``debci`` can also choose to use lookups with versions so that workflows will fail immediately if the desired version is unavailable. Being able to test lookups from the client side (:issue:`691`) could help with this. Pinning packages ================ ci.debian.net currently uses the ``autopkgtest --pin-packages`` option. For example, a test run triggered by changes to source packages ``foo`` and ``bar`` from unstable would use ``--pin-packages=unstable=src:foo,src:bar``. However, as noted in the :task:`Autopkgtest` task, we have deliberately not added support for that until now due to concerns about it not being explicit enough. We could add support for this anyway and tolerate the unexpected behaviour. However, with a few small changes, it should be possible to get nearly-equivalent results using something similar to this: .. code-block:: yaml context_artifacts: - collection: sid@debian:suite category: debian:binary-package error_on_empty: true lookup__binaries_from_source_version: "foo_1.0-1_amd64" - collection: sid@debian:suite category: debian:binary-package error_on_empty: true lookup__binaries_from_source_version: "bar_2.0-2_amd64" This requires adding ``error_on_empty: true`` to :ref:`dictionary lookups ` to make it be an error if the lookup returns no results, and adding the ``binaries_from_source_version`` multiple lookup filter to the :collection:`debian:suite` collection to match the current versions of all binaries from the given source package version on the given architecture (including ``Architecture: all`` if the architecture is not ``all``). When used, this would be more precise: Debusine would know exactly which binary artifacts were requested, automatically recording them in the dynamic task data. The required extensions seem worthwhile in general. ``britney`` does not currently pass the source version of pinned packages in its request to ``debci`` (unless it were to parse the ``trigger`` field, which seems undesirable). We could change that protocol, or we could add a similar unversioned ``binaries_from_source`` multiple lookup filter (e.g. ``lookup__binaries_from_source: "foo_amd64"``), which would at least still record the binary artifacts in dynamic task data. Unfortunately, this is still not quite enough. `autopkgtest(1) `__ has this note about ``--pin-packages``: **Attention**: This does not currently resolve some situations where dependencies of the given packages can only be resolved in ``RELEASE``. In this case the apt pinning will be removed and package installation will be retried with the entirety of ``RELEASE``, unless ``--no-apt-fallback`` is specified. The Debusine task cannot handle this itself, unless we were to add something complicated like preemptive dependency resolution or parsing the output of ``autopkgtest`` for failures to resolve dependencies. As a result, while we can still use ``context_artifacts`` to increase precision, there seems no alternative but to add ``pin_packages`` parameters to the :task:`Autopkgtest` task and the :workflow:`autopkgtest` workflow, with notes about their limitations. It does appear to be possible to pass ``--pin-packages=unstable=`` (without a package list) to ``autopkgtest`` along with additional binaries, which should have the same fallback behaviour. Extra APT sources and signing keys ================================== The ``extra_repositories`` option (defined in the same way as in the generic :task:`PackageBuild` task) should already be good enough for this. APT retries =========== ``debci`` uses ``autopkgtest --setup-commands`` to set ``Acquire::Retries "10";`` in APT's configuration. This seems like a good idea for robustness, but there's no need for this to be done at the task level; we can simply use ``customization_script`` to adjust the environments we use. This could be done specifically for ``autopkgtest``, but we might as well do it across the board. Backend selection ================= ``incus-lxc`` should be suitable for most packages. :ref:`task-configuration`, maintained by the owners of the relevant workspace, can be used to configure some packages to use ``incus-vm`` instead. Regression tracking =================== Debusine supports tracking and analyzing regressions, controlled by various options to QA-related workflows, particularly ``update_qa_results`` and ``enable_regression_tracking``. ``britney`` already does similar work, and the Debian release team doesn't want to outsource that. However, ``debci`` needs to be able to schedule reference test runs, and it makes sense to use parts of the same mechanism to do so even without the regression analysis step. This should be done using the following parameters: .. code-block:: yaml qa_suite: forky@debian:suite reference_qa_results: forky@debian:qa-results update_qa_results: true This will skip running the task if the reference :collection:`debian:qa-results` collection already contains an analysis from the last 30 days with a version matching the one published in ``qa_suite``. However, reference runs sometimes break for other reasons such as external data or the passage of time (e.g. expired certificates), so ``debci`` needs to be able to force an update. This will be done by changing the definition of the ``update_qa_results`` parameter to :workflow:`autopkgtest` as follows: * ``update_qa_results`` (string, defaults to ``no``): whether to update reference QA results. Allowed values are ``no``, ``yes``, and ``force``. When set to ``yes``, the workflow runs QA tasks and updates the collection passed in ``reference_qa_results`` with the results, unless that collection already contains a current matching result. ``force`` is like ``yes``, but does not check whether the collection already contains a current matching result. For compatibility, boolean False is equivalent to ``no``, and boolean True is equivalent to ``yes``. Between them, ``britney/scripts/debci-put.py`` and ``debci`` need to arrange for the above parameters to be passed to Debusine when running ``migration-reference/0`` tests. ``debci`` will then collect the results and store them in the same kind of way that it collects all other results from Debusine.