========================== Signing repository indexes ========================== As part of adding :ref:`repository hosting ` to Debusine, it needs to be able to sign repository indexes. Changes to collection managers ============================== The collection manager interface gains an optional ``created_at`` parameter when adding items to collections, allowing us to control the time when an item is considered to have been added. This is important for repository indexes, since they are all considered to have been created at the time specified in the :task:`GenerateSuiteIndexes` task's ``generate_at`` parameter. It also gains an optional ``replaced_by`` parameter, defaulting to the next-most-recently-created collection item with the same name, if any. If ``replaced_by`` is set either explicitly or implicitly, then the collection manager sets the new item's ``removed_at``, ``removed_by_user``, and ``removed_by_workflow`` fields to the replacing item's corresponding ``created_*`` fields. It is an error for ``replaced_by`` to have a creation date older than the new item's ``created_at``. These changes allow retroactively inserting items into the history of a collection, which we can use for backfilling snapshots. The :task:`GenerateSuiteIndexes` task already supports this, but to make the same arrangements work properly for signed index files we need to push them down to a lower level. Changes to event reactions ========================== The :ref:`action-update-collection-with-artifacts` action (and, for symmetry, the :ref:`action-update-collection-with-data` action) gains an optional ``created_at`` parameter, which is passed when adding the new collection item. Changes to :collection:`debian:archive` collection ================================================== The collection gains a new data field: * ``signing_keys`` (list of strings, optional): the fingerprints of the :asset:`debusine:signing-key` assets to sign repository indexes in this archive with, which must each have purpose ``openpgp`` :collection:`debian:suite` collections are only valid as items in an archive if they have the same workspace. Since archives are :ref:`singleton collections `, this has the effect of ensuring that a suite can be in at most one archive. Changes to :collection:`debian:suite` collection ================================================ The collection gains a new data field: * ``signing_keys`` (list of strings, optional): the fingerprints of the :asset:`debusine:signing-key` assets to sign repository indexes in this suite with, which must each have purpose ``openpgp`` This allows overriding the list of signing keys on a per-suite basis, for situations where using the same keys for the whole archive is not appropriate. If ``signing_keys`` is not set, then the containing :collection:`archive ` controls which signing keys are used. .. task:: SignRepositoryIndex SignRepositoryIndex task ======================== This is a :ref:`signing task ` that signs a :artifact:`debian:repository-index` artifact on a signing worker. It is separate from the :task:`Sign` task to make it easier to add the signed indexes to the suite without needing additional helper tasks. The ``task_data`` for this task may contain the following keys: * ``suite_collection``: the suite whose indexes should be signed * ``unsigned`` (:ref:`lookup-single`, required): the :artifact:`debian:repository-index` artifact whose contents should be signed * ``mode`` (required): ``detached`` or ``clear`` * ``signed_name`` (required): the file name to use in the output artifact The task looks up the suite's ``signing_keys`` field; if that is not set, it falls back to the ``signing_keys`` field of its containing archive, if any. It signs the unsigned artifact with all the signing keys corresponding to fingerprints in that list, creating a single signed output file. The output will be provided as a :artifact:`debian:repository-index` artifact, with ``relates-to`` relations to the unsigned artifact. .. workflow:: update_suite Workflow ``update_suite`` ========================= This workflow updates metadata for a single :collection:`suite `. Initially this will just involve generating and signing basic indexes for the suite, but later it may also generate supplementary files such as ``Contents-*``. * ``task_data``: * ``suite_collection``: the suite whose indexes should be generated The workflow creates a :task:`GenerateSuiteIndexes` task, with task data as follows: * ``suite_collection``: ``{suite_collection}`` * ``generate_at``: the transaction timestamp at which the workflow orchestrator is being run (this needs special care to preserve idempotency, since any later runs of the same workflow orchestrator would have a different transaction timestamp) If the suite or, failing that, its containing archive (if any) has a non-empty ``signing_keys`` field, the workflow additionally creates two :task:`SignRepositoryIndex` tasks with dependencies on the :task:`GenerateSuiteIndexes` task. They have task data as follows: * ``suite_collection``: ``{suite_collection}`` * ``unsigned``: the :artifact:`debian:repository-index` artifact for the ``Release`` file produced by the :task:`GenerateSuiteIndexes` task * ``mode``: ``detached`` for the first task, or ``clear`` for the second * ``signed_name``: ``Release.gpg`` for the first task, or ``InRelease`` for the second These tasks each have event reactions that add the output :artifact:`debian:repository-index` artifact to the suite at the appropriate path (``Release.gpg`` or ``InRelease``), setting ``created_at`` to the transaction timestamp at which the workflow orchestrator is being run. The ``InRelease`` task depends on the ``Release.gpg`` task, in order that the "current" view of a suite can arrange to only show a snapshot once an ``InRelease`` file exists for it. .. note:: We use two tasks rather than making both signatures in a single task because otherwise the event reactions would have no way of distinguishing between the two output artifacts. Changes to :workflow:`update_suites` workflow ============================================= The workflow does nothing if the workspace does not have a (singleton) :collection:`debian:archive` collection. Key generation stage -------------------- If the archive does not have ``signing_keys`` set in its data, then the workflow first creates a :task:`GenerateKey` task, with task data as follows: * ``purpose``: ``openpgp`` * ``description``: a suitable description of the new key, identifying the workspace It then creates a :ref:`workflow callback ` with ``step`` set to ``generated-key`` and a dependency on the :task:`GenerateKey` task, and stops populating the workflow graph at that point. When called, that callback sets the ``signing_keys`` field in the archive's data to be a list containing only the fingerprint of the new :asset:`debusine:signing-key` asset. Main stage ---------- Instead of creating :task:`GenerateSuiteIndexes` tasks directly, the workflow creates an :workflow:`update_suite` sub-workflow for each suite that it considers to need updating, with task data as follows: * ``suite_collection``: the suite whose indexes should be generated * ``signing_keys``: the suite's ``signing_keys``, if present; otherwise, the archive's ``signing_keys`` If the key generation stage above created a :task:`GenerateKey` task, then it adds the associated workflow callback as an additional dependency of the :workflow:`update_suite` sub-workflow. Key management ============== By default, each :collection:`archive ` has its own OpenPGP signing key, which is used to sign indexes in its :collection:`suites `. This provides a reasonable default for common cases, where needing to rotate keys for an archive for any reason has limited consequences. The signing key is generated automatically the first time metadata updates for the suites in an archive are needed. An administrator can manually change the configuration for the relevant collections to use different signing keys. For example: * During key rollovers, a suite's indexes may be signed using multiple keys. * In some high-value cases, different suites in the same archive use different signing keys that are generated manually and have carefully-selected expiry periods: Debian itself is an example of this. * All the workspaces in a scope may be controlled by the same customer, and it may be simpler for them all to use the same signing key. .. todo:: Debusine currently only supports OpenPGP signing with software-encrypted keys. It should gain support for generating keys on PKCS#11 tokens and signing using those keys, and for exporting those keys under wrap in order to deal with hardware security models with a limited number of object slots.