.. _suite-relationships: =================== Suite relationships =================== Goals ===== This blueprint defines a structured way to record relationships between collections, starting with :collection:`debian:suite` collections. Requirements ============ * Record relationships from one collection to other collections. Possible relationships with its cardinality: * ``forked_from``: the suite this suite was forked from (0..1) * ``based_on``: the original base suite this one derives from (0..1) * ``requires``: suites whose APT sources are required to use/build/test this suite (0..N, ordered) * ``targeting``: the suite where packages in this suite are expected to land (0..1) * ``default_qa_results``: the ``debian:qa-results`` collection to use by default for this suite (0..1) * Work across workspace inheritance * Allow relationships from suites to non-suite collections (e.g. to define the default :collection:`debian:qa-results` collection) * Remove (or make optional, while preserving functionality) the need to specify suites manually in places such as ``qa_suite`` or ``reference_qa_results``. For ``requires``, the relationship is ordered. When generating the ``sources.list`` or equivalent, entries are emitted in ascending ``position`` order. Use cases ========= Support transitions ------------------- One of the goals of collection relationships is to support transitions. Once the relationships between collections are added, Debusine can infer automatically which is the ``suite``, the ``qa_suite`` or the ``regression_tracking_qa_results``. This will help supporting transitions. Simplify workflows ------------------ Initially all the workflow variables will remain the same. But, if they are not set, when possible, they will be inferred from the relationships. At a later stage, the variables could be removed (if left there, they can be used to override the relationships) * ``DebianPipelineWorkflow`` * look up the ``primary_suite`` from ``(vendor, codename)`` * define ``suite_under_test = suite if not None else primary_suite`` (in other words: ``DebianPipelineWorkflow.suite`` if the user has specified it, else it is the ``primary_suite``) * Infer: * ``qa_suite``: ``suite_under_test.targeting if not None else primary_suite`` * ``regression_tracking_qa_results``: ``qa_suite.suite_default_qa_results`` * ``check_installability_suite``: same as ``qa_suite`` * Use ``suite_under_test.requires`` when building ``extra_repositories`` Auto document / simplify personal repositories ---------------------------------------------- This is a particular case of **Simplify workflows** use case. Debusine users can create a personal workspace and create ``debian:suite`` in there to publish personal repositories. At the moment, they are isolated from other suites. Adding suite relationships will help testing by avoiding the need to define duplicated values in the workflow. It will also help that if a repository changes from requiring ``forky`` to requiring ``testing`` it will be a single change in the relationships instead of having to tweak different variables. This view should indicate other APT sources that are required to use suites in the archive, according to ``requires``: * URL: ``https://///`` * URL example: ``https://deb.debusine.debian.net/debian/r-cjwatson-debusine/`` The ``SuiteRootView`` (``///dists//``) will include the APT configuration using ``requires``. Lookups ======= No changes in lookups are required. Implementation plan =================== Add data model -------------- .. code-block:: python class CollectionRelation(models.Model): """Model relations between collections.""" objects: CollectionRelationManager = CollectionRelationManager() Types: TypeAlias = _CollectionRelationTypes source = models.ForeignKey( Collection, on_delete=models.CASCADE, related_name="relations" ) #: to implement constraints based on category source_category = models.CharField(max_length=255) target = models.ForeignKey( Collection, on_delete=models.CASCADE, related_name="targeted_by" ) #: to implement constraints based on category target_category = models.CharField(max_length=255) workspace = models.ForeignKey(Workspace, on_delete=models.CASCADE) type = models.CharField(max_length=32, choices=_CollectionRelationTypes.choices) position = models.PositiveIntegerField(null=True, blank=True) # Include fields to record who (token, it could be a worker via API) # and when the relation was created or modified. class Meta(TypedModelMeta): base_manager_name = "objects" constraints = [ # Enforce that `requires` relations have a position, others not models.CheckConstraint( name="%(app_label)s_%(class)s_position", check=( Q( type=_CollectionRelationTypes.SUITE_REQUIRES, position__isnull=False, ) | Q( type__in=[ _CollectionRelationTypes.SUITE_FORKED_FROM, _CollectionRelationTypes.SUITE_BASED_ON, _CollectionRelationTypes.SUITE_TARGETING, _CollectionRelationTypes.SUITE_QA_RESULTS, ], position__isnull=True, ) ), ), models.UniqueConstraint( name="%(app_label)s_%(class)s_suite_requires_no_duplicate_position", fields=["source", "position"], condition=Q(type=_CollectionRelationTypes.SUITE_REQUIRES), ), models.UniqueConstraint( name="%(app_label)s_%(class)s_cardinality_checks_max_1", fields=["source", "type"], condition=( Q(type=_CollectionRelationTypes.SUITE_FORKED_FROM) | Q(type=_CollectionRelationTypes.SUITE_BASED_ON) | Q(type=_CollectionRelationTypes.SUITE_TARGETING) | Q(type=_CollectionRelationTypes.SUITE_QA_RESULTS) ), ), models.CheckConstraint( name="%(app_label)s_%(class)s_categories_correct", check=( # SUITE_QA_RESULTS: collection must be a suite, # target must be qa-results Q( type=_CollectionRelationTypes.SUITE_QA_RESULTS, source_category=CollectionCategory.SUITE, target_category=CollectionCategory.SUITE_QA_RESULTS, ) # SUITE_FORKED_FROM: both must be suites | Q( type=_CollectionRelationTypes.SUITE_FORKED_FROM, source_category=CollectionCategory.SUITE, target_category=CollectionCategory.SUITE, ) # SUITE_REQUIRES: both must be suites | Q( type=_CollectionRelationTypes.SUITE_REQUIRES, source_category=CollectionCategory.SUITE, target_category=CollectionCategory.SUITE, ) # SUITE_TARGETING: both must be suites | Q( type=_CollectionRelationTypes.SUITE_TARGETING, source_category=CollectionCategory.SUITE, target_category=CollectionCategory.SUITE, ) ), ), models.CheckConstraint( name="%(app_label)s_%(class)s_source_no_self_referenced", check=~Q(source_id=F("target_id")), ), ] CRUD: add CLI support ---------------------- The API will be defined at a later point, following Debusine endpoint conventions. CLI commands follow the existing Debusine ``object verb`` style. Since relationships are between collections, the object is ``collection relation``. List relations ~~~~~~~~~~~~~~ .. code-block:: shell $ debusine collection relation list --workspace WORKSPACE \ --from NAME@CATEGORY --to NAME@CATEGORY --type TYPE The output should include, for each relation: source (``from``), target (``to``), ``type``, and ``position`` (when applicable). Ordered by ``from``, ``to``, ``type`` and ``position``. The optional parameters ``--from``, ``--to`` and ``--type`` act like a filter. Edit relation targets ~~~~~~~~~~~~~~~~~~~~~~ This command edits the target list for a given source collection and relation type. It mimics the ``workspace inheritance`` interface. .. code-block:: shell $ debusine collection relation --workspace WORKSPACE --yaml \ FROM_NAME@CATEGORY TYPE \ --append [COLLECTION [COLLECTION...]] \ --prepend [COLLECTION [COLLECTION...]] \ --remove [COLLECTION [COLLECTION...]] \ --set [COLLECTION [COLLECTION...]] \ --edit Parameters: * ``FROM-NAME@CATEGORY``: source collection whose outgoing relations will be edited * ``TYPE``: relation type to edit (``fork_parent``, ``requires``, ``goal``, ``default_qa_results``) * `--append``: add the specified collection(s) to the end of the target list * `--prepend``: add the specified collection(s) to the beginning of the target list * `--remove``: remove the specified collection(s) from the target list * `--set``: replace the complete target list with the specified collections * `--edit``: edit the complete target list using ``$EDITOR`` (only option that does not accept a COLLECTION argument) The command edits the set of outgoing relations for the specified ``(from, type)`` pair. For relation types with multiple targets (such as ``requires``), the order of the target list is significant and defines the ordering of relations. For relation types with cardinality ``0..1`` (such as ``fork_parent``, ``goal``, and ``default_qa_results``), the list can contain at most one target. If more than one target is provided, the operation fails (API is called, API returns a suitable error and client displays the error). For ``edit``, the editor is opened with one target collection per line, written as ``NAME@CATEGORY``. For example: .. code-block:: yaml # Here you can edit the targets of the relation list. # # This is a YAML list of collection identifiers written as NAME@CATEGORY. # Each entry represents a target collection for RELATION_TYPE. # # The order of entries is significant for ordered relation types # (such as "requires"). # # Lines starting with '#' are ignored. - trixie@debian:suite - trixie-security@debian:suite The user may reorder lines, remove lines, or add new target collections. For example, replacing ``trixie-security@debian:suite`` with ``trixie-proposed-updates@debian:suite`` and placing it before ``trixie@debian:suite``: .. code-block:: yaml # [INTRODUCTORY COMMENT] - trixie-proposed-updates@debian:suite - trixie@debian:suite Each entry in the YAML sequence represents a target collection for the specified ``(FROM, TYPE)`` relation set. When the editor exits, the server atomically updates the relations so that the targets and their order exactly match the provided list. Relations that no longer appear in the list are deleted, and new relations are created for newly added targets. Note that empty list removes all relations of the specified type: .. code-block:: yaml # [ INTRODUCTORY COMMENT ] [] An empty file is treated the same way: deletes all relations. Display and manage collection relationships on the Web ------------------------------------------------------ List relations of a collection ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * URL structure: ``///collection///relation/?relation_type=REL_TYPE`` * URL example: ``/debian/base/collection/debian:suite/trixie/relation/?relation_type=goal`` The view will contain (if no ``relation_type`` query parameter is included): * Title: ``Relations of @`` With a ``relation_type`` query parameter: * Title: ``Relations of type "REL_TYPE" of @`` List with the following columns: * Type of relation * Target: target collection * Position * Details. With a link on each relation to display, on demand, creation and modification metadata (``created_at``, ``created_by_user``, ``modified_at``, ``modified_by_user``) * Actions: ``[ Delete ]`` (if the user has permissions to do so). Open ``/relation//delete/`` to confirm the deletion. See ``Delete`` section. ``[ Add new relation ]`` (would open the **Create** view) .. note:: The list of relationships can also be included in the existing view: ``///collection///`` under a new HTML header **Relations**. E.g. in the existing ``https://debusine.debian.net/debian/base/collection/debian:suite/trixie/`` after **Artifacts** header. Create ~~~~~~ (View available if user has permissions to create relations) * URL structure: ``///collection///relation/add/`` * URL example: ``/debian/base/collection/debian:suite/trixie/relation/add/`` The view will contain: * Title: ``Create relation for @`` Form with fields: * Target: select widget, list collections visible in this workspace (including collections from inherited workspaces) * Type of relation: ``Fork parent``, ``Requires``, ``Goal``, ``QA Results`` For ``Requires`` relations: the position is not entered manually when creating it. New relations are appended at the end. Reorder relations ~~~~~~~~~~~~~~~~~ (View available if user has permissions to edit relations) * URL structure: ``///collection///relation//edit/`` * URL example: ``/debian/base/collection/debian:suite/trixie/relation/requires/edit/`` The view will find the ``from`` collection based on the ``scope``, ``workspace``, ``source_category`` and ``source_name``. The view will display the existing outgoing relations for the given ``from`` collection and ``relation_type`` and allow the user to reorder the ``to`` targets using a UI similar to the inheritance chain widget. Delete ~~~~~~ (View available if user has permissions to delete the relation, linked from the ``List of relations`` and/or ``Reorder relations``). * URL structure (GET) ``///relation//delete/`` * URL example (GET) ``/debian/base/relation/15/delete/`` * URL structure (POST) ``///relation//delete/`` * URL example (POST) ``/debian/base/relation/15/delete/`` The GET view will display the relation to be deleted and ask for a final confirmation, then submit a POST request for the deletion. If the deleted relation belongs to an ordered relation type (such as ``requires``), the positions of the remaining relations are atomically reindexed to keep them contiguous. Permissions =========== Collections belong to a workspace. Workspace owners can manage the relations of collections in that workspace.