Suite relationships

Goals

This blueprint defines a structured way to record relationships between collections, starting with debian:suite collections.

Requirements

Record relationships from one collection to other collections. Possible relationships with its cardinality:

forked_from: the suite this suite was forked from (0..1)

based_on: the original base suite this one derives from (0..1)

requires: suites whose APT sources are required to use/build/test this suite (0..N, ordered)

targeting: the suite where packages in this suite are expected to land (0..1)

default_qa_results: the debian:qa-results collection to use by default for this suite (0..1)

Work across workspace inheritance

Allow relationships from suites to non-suite collections (e.g. to define the default debian:qa-results collection)

Remove (or make optional, while preserving functionality) the need to specify suites manually in places such as qa_suite or reference_qa_results.

For requires, the relationship is ordered. When generating the sources.list or equivalent, entries are emitted in ascending position order.

Use cases

Support transitions

One of the goals of collection relationships is to support transitions.

Once the relationships between collections are added, Debusine can infer automatically which is the suite, the qa_suite or the regression_tracking_qa_results. This will help supporting transitions.

Simplify workflows

Initially all the workflow variables will remain the same. But, if they are not set, when possible, they will be inferred from the relationships. At a later stage, the variables could be removed (if left there, they can be used to override the relationships)

DebianPipelineWorkflow

look up the primary_suite from (vendor, codename)

define suite_under_test = suite if not None else primary_suite (in other words: DebianPipelineWorkflow.suite if the user has specified it, else it is the primary_suite)

Infer: * qa_suite: suite_under_test.targeting if not None else primary_suite * regression_tracking_qa_results: qa_suite.suite_default_qa_results * check_installability_suite: same as qa_suite

Use suite_under_test.requires when building extra_repositories

Auto document / simplify personal repositories

This is a particular case of Simplify workflows use case.

Debusine users can create a personal workspace and create debian:suite in there to publish personal repositories.

At the moment, they are isolated from other suites. Adding suite relationships will help testing by avoiding the need to define duplicated values in the workflow. It will also help that if a repository changes from requiring forky to requiring testing it will be a single change in the relationships instead of having to tweak different variables.

This view should indicate other APT sources that are required to use suites in the archive, according to requires:

URL: https://<debian-archive-host>/<scope>/<workspace>/

URL example: https://deb.debusine.debian.net/debian/r-cjwatson-debusine/

The SuiteRootView (/<scope>/<workspace>/dists/<suite>/) will include the APT configuration using requires.

Lookups

No changes in lookups are required.

Implementation plan

Add data model

class CollectionRelation(models.Model):
    """Model relations between collections."""

    objects: CollectionRelationManager = CollectionRelationManager()

    Types: TypeAlias = _CollectionRelationTypes

    source = models.ForeignKey(
        Collection, on_delete=models.CASCADE, related_name="relations"
    )

    #: to implement constraints based on category
    source_category = models.CharField(max_length=255)

    target = models.ForeignKey(
        Collection, on_delete=models.CASCADE, related_name="targeted_by"
    )
    #: to implement constraints based on category
    target_category = models.CharField(max_length=255)

    workspace = models.ForeignKey(Workspace, on_delete=models.CASCADE)

    type = models.CharField(max_length=32, choices=_CollectionRelationTypes.choices)

    position = models.PositiveIntegerField(null=True, blank=True)

    # Include fields to record who (token, it could be a worker via API)
    # and when the relation was created or modified.

    class Meta(TypedModelMeta):
        base_manager_name = "objects"

        constraints = [
            # Enforce that `requires` relations have a position, others not
            models.CheckConstraint(
                name="%(app_label)s_%(class)s_position",
                check=(
                    Q(
                        type=_CollectionRelationTypes.SUITE_REQUIRES,
                        position__isnull=False,
                    )
                    | Q(
                        type__in=[
                            _CollectionRelationTypes.SUITE_FORKED_FROM,
                            _CollectionRelationTypes.SUITE_BASED_ON,
                            _CollectionRelationTypes.SUITE_TARGETING,
                            _CollectionRelationTypes.SUITE_QA_RESULTS,
                        ],
                        position__isnull=True,
                    )
                ),
            ),
            models.UniqueConstraint(
                name="%(app_label)s_%(class)s_suite_requires_no_duplicate_position",
                fields=["source", "position"],
                condition=Q(type=_CollectionRelationTypes.SUITE_REQUIRES),
            ),
            models.UniqueConstraint(
                name="%(app_label)s_%(class)s_cardinality_checks_max_1",
                fields=["source", "type"],
                condition=(
                    Q(type=_CollectionRelationTypes.SUITE_FORKED_FROM)
                    | Q(type=_CollectionRelationTypes.SUITE_BASED_ON)
                    | Q(type=_CollectionRelationTypes.SUITE_TARGETING)
                    | Q(type=_CollectionRelationTypes.SUITE_QA_RESULTS)
                ),
            ),
            models.CheckConstraint(
                name="%(app_label)s_%(class)s_categories_correct",
                check=(
                    # SUITE_QA_RESULTS: collection must be a suite,
                    # target must be qa-results
                    Q(
                        type=_CollectionRelationTypes.SUITE_QA_RESULTS,
                        source_category=CollectionCategory.SUITE,
                        target_category=CollectionCategory.SUITE_QA_RESULTS,
                    )
                    # SUITE_FORKED_FROM: both must be suites
                    | Q(
                        type=_CollectionRelationTypes.SUITE_FORKED_FROM,
                        source_category=CollectionCategory.SUITE,
                        target_category=CollectionCategory.SUITE,
                    )
                    # SUITE_REQUIRES: both must be suites
                    | Q(
                        type=_CollectionRelationTypes.SUITE_REQUIRES,
                        source_category=CollectionCategory.SUITE,
                        target_category=CollectionCategory.SUITE,
                    )
                    # SUITE_TARGETING: both must be suites
                    | Q(
                        type=_CollectionRelationTypes.SUITE_TARGETING,
                        source_category=CollectionCategory.SUITE,
                        target_category=CollectionCategory.SUITE,
                    )
                ),
            ),
            models.CheckConstraint(
                name="%(app_label)s_%(class)s_source_no_self_referenced",
                check=~Q(source_id=F("target_id")),
            ),
        ]

CRUD: add CLI support

The API will be defined at a later point, following Debusine endpoint conventions.

CLI commands follow the existing Debusine object verb style. Since relationships are between collections, the object is collection relation.

List relations

$ debusine collection relation list --workspace WORKSPACE \
    --from NAME@CATEGORY --to NAME@CATEGORY --type TYPE

The output should include, for each relation: source (from), target (to), type, and position (when applicable). Ordered by from, to, type and position.

The optional parameters --from, --to and --type act like a filter.

Edit relation targets

This command edits the target list for a given source collection and relation type. It mimics the workspace inheritance interface.

$ debusine collection relation --workspace WORKSPACE --yaml \
    FROM_NAME@CATEGORY TYPE \
    --append [COLLECTION [COLLECTION...]] \
    --prepend [COLLECTION [COLLECTION...]] \
    --remove [COLLECTION [COLLECTION...]] \
    --set [COLLECTION [COLLECTION...]] \
    --edit

Parameters:

FROM-NAME@CATEGORY: source collection whose outgoing relations will be edited

TYPE: relation type to edit (fork_parent, requires, goal, default_qa_results)

–append`: add the specified collection(s) to the end of the target list

–prepend`: add the specified collection(s) to the beginning of the target list

–remove`: remove the specified collection(s) from the target list

–set`: replace the complete target list with the specified collections

–edit`: edit the complete target list using $EDITOR (only option that does not accept a COLLECTION argument)

The command edits the set of outgoing relations for the specified (from, type) pair.

For relation types with multiple targets (such as requires), the order of the target list is significant and defines the ordering of relations.

For relation types with cardinality 0..1 (such as fork_parent, goal, and default_qa_results), the list can contain at most one target. If more than one target is provided, the operation fails (API is called, API returns a suitable error and client displays the error).

For edit, the editor is opened with one target collection per line, written as NAME@CATEGORY. For example:

# Here you can edit the targets of the relation list.
#
# This is a YAML list of collection identifiers written as NAME@CATEGORY.
# Each entry represents a target collection for RELATION_TYPE.
#
# The order of entries is significant for ordered relation types
# (such as "requires").
#
# Lines starting with '#' are ignored.

 - trixie@debian:suite
 - trixie-security@debian:suite

The user may reorder lines, remove lines, or add new target collections.

For example, replacing trixie-security@debian:suite with trixie-proposed-updates@debian:suite and placing it before trixie@debian:suite:

# [INTRODUCTORY COMMENT]

- trixie-proposed-updates@debian:suite
- trixie@debian:suite

Each entry in the YAML sequence represents a target collection for the specified (FROM, TYPE) relation set.

When the editor exits, the server atomically updates the relations so that the targets and their order exactly match the provided list. Relations that no longer appear in the list are deleted, and new relations are created for newly added targets.

Note that empty list removes all relations of the specified type:

# [ INTRODUCTORY COMMENT ]

[]

An empty file is treated the same way: deletes all relations.

Display and manage collection relationships on the Web

List relations of a collection

URL structure: /<scope>/<wname>/collection/<source_category>/<source_name>/relation/?relation_type=REL_TYPE

URL example: /debian/base/collection/debian:suite/trixie/relation/?relation_type=goal

The view will contain (if no relation_type query parameter is included):

Title: Relations of <source_name>@<source_category>

With a relation_type query parameter:

Title: Relations of type "REL_TYPE" of <source_name>@<source_category>

List with the following columns:

Type of relation

Target: target collection

Position

Details. With a link on each relation to display, on demand, creation and modification metadata (created_at, created_by_user, modified_at, modified_by_user)

Actions: [ Delete ] (if the user has permissions to do so). Open /relation/<id>/delete/ to confirm the deletion. See Delete section.

[ Add new relation ] (would open the Create view)

Note

The list of relationships can also be included in the existing view: /<scope>/<wname>/collection/<source_category>/<source_name>/ under a new HTML header Relations. E.g. in the existing https://debusine.debian.net/debian/base/collection/debian:suite/trixie/ after Artifacts header.

Create

(View available if user has permissions to create relations)

URL structure: /<scope>/<wname>/collection/<source_category>/<source_name>/relation/add/
URL example: /debian/base/collection/debian:suite/trixie/relation/add/

The view will contain:

Title: Create relation for <source_name>@<source_category>

Form with fields:

Target: select widget, list collections visible in this workspace (including collections from inherited workspaces)

Type of relation: Fork parent, Requires, Goal, QA Results

For Requires relations: the position is not entered manually when creating it. New relations are appended at the end.

Reorder relations

(View available if user has permissions to edit relations)

URL structure: /<scope>/<wname>/collection/<source_category>/<source_name>/relation/<relation_type>/edit/

URL example: /debian/base/collection/debian:suite/trixie/relation/requires/edit/

The view will find the from collection based on the scope, workspace, source_category and source_name.

The view will display the existing outgoing relations for the given from collection and relation_type and allow the user to reorder the to targets using a UI similar to the inheritance chain widget.

Delete

(View available if user has permissions to delete the relation, linked from the List of relations and/or Reorder relations).

URL structure (GET) /<scope>/<wname>/relation/<id>/delete/

URL example (GET) /debian/base/relation/15/delete/

URL structure (POST) /<scope>/<wname>/relation/<id>/delete/

URL example (POST) /debian/base/relation/15/delete/

The GET view will display the relation to be deleted and ask for a final confirmation, then submit a POST request for the deletion.

If the deleted relation belongs to an ordered relation type (such as requires), the positions of the remaining relations are atomically reindexed to keep them contiguous.

Permissions

Collections belong to a workspace. Workspace owners can manage the relations of collections in that workspace.