.. _contribute-to-debusine:

======================
Contribute to Debusine
======================

Contributions to Debusine are greatly appreciated!
You can contribute in many ways, be it writing documentation and blog
posts, or fixing bugs and implementing new features.

Join the community
------------------
You can reach us using IRC on the #debusine channel at irc.debian.org.
That channel is also available as a (bridged) `Matrix room
<https://matrix.to/#/#_oftc_#debusine:matrix.org>`__. There's also
a #debusine-notifications channel where you can see all activities
happening in the GitLab project (it's also bridged in a `Matrix
room <https://matrix.to/#/!ugwLXLsYADeccIzBNd:matrix.org>`__).

You can also `file issues
<https://salsa.debian.org/freexian-team/debusine/-/issues>`_ in GitLab's
interface. There are two templates to help you file bug reports and feature
requests.

While Freexian is driving the development forward by paying a few
developers, rest assured that external contributions are welcome.

Where to start?
---------------

Starting small
~~~~~~~~~~~~~~

Before starting to work on bigger projects, it probably makes sense
to get used to our :ref:`development workflows <reference-contributors>`
by tackling a few `small issues
<https://salsa.debian.org/freexian-team/debusine/-/issues/?sort=created_date&state=opened&label_name%5B%5D=Quick%20fix>`_
(they have the *Quick fix* label).

Looking for issues
~~~~~~~~~~~~~~~~~~

If you are looking for bigger issues to tackle, you will find plenty in
the `issue tracker
<https://salsa.debian.org/freexian-team/debusine/-/issues/>`_.

The paid developers are usually focused on all the issues planned in the
`current milestone
<https://salsa.debian.org/freexian-team/debusine/-/milestones>`_, working
through the list by order of priority (from labels P1 to P3). If you want
to help with the current milestone, you should probably start with a
low-priority issue.

Otherwise you can also find `backlogged issues
<https://salsa.debian.org/freexian-team/debusine/-/issues/?sort=created_date&state=opened&label_name%5B%5D=Backlogged&first_page_size=20>`_
that are not part of any milestone and that paid developers will not
handle at this point.

Add a new feature
~~~~~~~~~~~~~~~~~

If you want to work on a new feature that is not yet planned, then you
should first create a new issue using the ``Feature Request`` template
(see :ref:`design-with-issues`).

For larger features that require significant design work, it is
recommended to first write a *development blueprint* (see
:ref:`design-with-blueprints`).

A particular case where we expect and invite contributions is that of new
workflows.  See :ref:`contribute-workflow` for more details of how to go
about doing that.

.. _developer-setup:

How to contribute
-----------------

.. note:: See :ref:`runtime environment <runtime-environment>` to understand
   the runtime environment.

Set up your environment
~~~~~~~~~~~~~~~~~~~~~~~

Ready to contribute?  Here's the quickest way to set up Debusine for local
development, with everything in a single container so that your host system
isn't affected:

  #. :ref:`Set up Incus <set-up-incus>`, then start a development container:

     .. code-block:: console

       $ incus launch images:debian/trixie/cloud debusine-dev
       $ incus exec debusine-dev -- cloud-init status --wait

  #. Install ``git`` in the container:

     .. code-block:: console

       $ incus exec debusine-dev -- apt --update install git

  #. Optionally, share a directory from your host system with the container
     so that you can conveniently edit Debusine source code from your host.
     For example, if you keep source code under the ``src`` subdirectory of
     your home directory:

     .. code-block:: console

       $ incus config device add debusine-dev src \
           disk source=$HOME/src path=/home/debian/src shift=true

     ``shift=true`` may require some configuration to get ID mapping to work
     properly; see the `upstream docs
     <https://linuxcontainers.org/incus/docs/main/userns-idmap/>`_.

  #. Enter the container:

     .. code-block:: console

       $ incus exec debusine-dev -- su - debian

     Follow the rest of these instructions from inside the container.

  #. Clone Debusine locally (if you shared a source directory from your host
     system earlier, you should ``cd`` to somewhere under that source
     directory first):

     .. code-block:: console

       $ git clone https://salsa.debian.org/freexian-team/debusine.git
       $ cd debusine

  #. For a quick startup, run this command:

     .. code-block:: console

       $ bin/quick-setup.sh

     It will install required packages with apt and put a sample
     configuration file in place. It will also create a user
     and a database in PostgreSQL. The user will be named
     after your current Unix user and will own the new database
     so that it can connect to the newly created database without any
     other authentication.

  #. Populate the database with:

     .. code-block:: console

       $ ./manage.py migrate

Make and test your changes
~~~~~~~~~~~~~~~~~~~~~~~~~~

  #. Install ``pre-commit``:

     .. code-block:: console

       $ sudo apt install pre-commit

  #. Set up ``pre-commit`` to run automatically when you commit (optional):

     .. code-block:: console

       $ pre-commit install

     This runs some checks (equivalent to ``make check``) every time you run
     ``git commit``.  This downloads linters from upstream repositories; if
     you prefer to avoid that on your development machine, you can skip this
     step and rely on CI running it for you instead (but with longer
     latency).  See :ref:`pre-commit` for more details.

     Make sure you run this in the same environment where you're going to
     run ``git commit``.  If you commit inside the container, then you
     should install ``pre-commit`` inside the container; if you commit
     directly on your host system (perhaps via your editor or IDE), then you
     should install ``pre-commit`` directly on your host system.

  #. Switch to a new branch:

     .. code-block:: console

       $ git checkout -b name-of-your-bugfix-or-feature

  #. Develop your new feature, ideally following the rules of :ref:`Test-Driven Development <tdd>`.
     Remember to :ref:`update the release notes <updating-release-notes>` as
     needed.

  #. Run quick static analysis checks on your code (optional):

     .. code-block:: console

       $ make check

     As above, this downloads linters from upstream repositories, so you can
     skip it if you're very cautious about what code runs on your machine,
     but it spots many problems more quickly than the full unit test suite
     does.

  #. Check that all unit tests pass:

     .. code-block:: console

       $ make coverage

     This runs tests using ``pytest`` and ensures that you have maintained
     100% test coverage.  If you get errors, make sure to fix them.  You can
     also run ``pytest`` manually if you need to use more specific options;
     for example, ``-k`` is useful to run specific tests, and ``--pdb`` is
     useful for interactive debugging.

     .. note::
        If you get errors like ``OSError: [Errno 38] Function not
        implemented``, then it means that you are lacking /dev/shm
        with proper permissions.

Run Debusine components
~~~~~~~~~~~~~~~~~~~~~~~

.. note::

    Most Debusine changes can be tested much more easily and comprehensively
    by running the test suite, and if that's all you need to do then you can
    safely skip these steps.  However, if you need to look at the effects of
    user interface changes in a web browser, or if you need to experiment
    with task changes in a real worker, then it can be useful to have
    something closer to a full Debusine deployment.

Each of these commands should be run from a separate shell inside your
container (so ``incus shell debusine-dev -- su - debian`` and change to the
directory where you cloned Debusine, as above).

  #. Start a local test server:

     .. code-block:: console

       $ ./manage.py runserver
       [...]
       Starting development server at http://127.0.0.1:8000/
       Quit the server with CONTROL-C.

     Visit the URL returned to have access to the test website.

  #. If you want to serve apt repositories as well, start a second server on
     port ``8081``:

     .. code-block:: console

       $ ./manage.py runserver 8081
       [...]
       Starting development server at http://127.0.0.1:8001/
       Quit the server with CONTROL-C.

  #. If you want to run tasks, then start the Celery workers:

     .. code-block:: console

       $ python3 -m celery -A debusine.project worker -l INFO -Q scheduler --concurrency 1

     .. code-block:: console

       $ python3 -m celery -A debusine.project worker -l INFO

  #. Configure the worker:

     .. code-block:: console

       $ mkdir -p ~/.config/debusine/worker
       $ cat <<EOF > ~/.config/debusine/worker/config.ini
       [General]
       api-url = http://localhost:8000/api
       log-level = DEBUG
       EOF

  #. Run the worker with ``sbin`` dirs in ``PATH``, e.g.:

     .. code-block:: console

       $ PATH="/usr/sbin:/sbin:$PATH" LANG=C.UTF-8 python3 -m debusine.worker

  #. Enable the worker. You can find the name of the worker
     (by default is the hostname):

     .. code-block:: console

       $ ./manage.py list_workers

     And then enable it:

     .. code-block:: console

       $ ./manage.py manage_worker enable NAME

  #. Create a local Django superuser on your test instance:

     .. code-block:: console

        $ ./manage.py createsuperuser

  #. Login as that user using the link in the top right of the initial REST
     framework site.

  #. If you want to use the debusine client, set it up via:

     .. code-block:: console

       $ python3 -m debusine.client setup

     Set the default workspace to ``System``.

  #. Many of the Debusine worker tasks require an *environment*, which is
     a tarball or system image to pass to an executor such as ``unshare`` or
     ``incus``). You can create one via:

     .. code-block:: console

       $ python3 -m debusine.client work-request create mmdebstrap <<EOF
         bootstrap_options:
           architecture: amd64
           extra_packages:
           - devscripts
           variant: minbase
         bootstrap_repositories:
         - components:
           - main
           mirror: http://deb.debian.org/debian
           suite: trixie
       EOF

     Note that the worker needs the package ``mmdebstrap`` installed.
     If the worker does not have the package installed the work request
     will be in ``Pending`` and never run. If needed install the package and
     restart the worker.

     You can follow the progress (e.g. ``Pending``, ``Running``, or
     ``Completed``) via the web interface (browse to the scope, then
     click on the navigation bar for "Work Requests" to see it).

  #. In order to create workflows or tasks (and also other operations such as
     creating workflow templates) you will need to give permissions to the user:

    .. code-block:: console

        $ ./manage.py group create debusine/SystemAdmins
        $ ./manage.py workspace grant_role debusine/System owner SystemAdmins
        $ ./manage.py group members debusine/SystemAdmins --add YOUR-USER-NAME

Contribute your changes
~~~~~~~~~~~~~~~~~~~~~~~

It is usually easiest to run these steps directly on your host system, to
avoid needing to copy SSH keys around.

  #. If you haven't done so already, create a guest account on `Salsa
     <https://salsa.debian.org>`_ (a GitLab instance run by the Debian
     project) by visiting this page:
     https://salsa.debian.org/users/sign_up

     Follow all the steps to confirm your email, fill in your profile, and
     `set up your SSH keys
     <https://salsa.debian.org/help/user/ssh.md>`_.

     You might want to have a look at `Salsa's
     documentation <https://wiki.debian.org/Salsa/Doc>`_ and `GitLab's
     documentation <https://salsa.debian.org/help>`_ if you have questions
     about something.

     Debian Developers can skip this step, as they already have an account
     on this service.

  #. If you haven't done so already, visit the `Debusine project's page
     <https://salsa.debian.org/freexian-team/debusine>`_ and fork it to your
     own account.  See `GitLab's help on how to fork a project
     <https://salsa.debian.org/help/user/project/repository/forking_workflow.md#user-content-create-a-fork>`_.
     Then, add your fork as a remote, replacing ``your-account`` with your
     Salsa username:

     .. code-block:: console

       $ git remote add -f $USER git@salsa.debian.org/your-account/debusine.git

  #. Push your branch to your repository:

     .. code-block:: console

       $ git push -u $USER name-of-your-bugfix-or-feature

  #. Submit your work to us by creating a
     `merge request <https://salsa.debian.org/freexian-team/debusine/-/merge_requests/>`_.
     The previous ``git push`` step will give you a URL you can use for
     this.  Alternatively, you can visit the Debusine project fork hosted in
     your own account and create a merge request from there (either through
     the “Branches” page, or through the “Merge requests” page). See
     `GitLab's help on merge requests
     <https://salsa.debian.org/help/user/project/merge_requests/creating_merge_requests.md>`_
     if needed.

     Make sure to address any issues identified by the continuous
     integration system.  The result of its “pipeline” can be directly
     seen in the merge request (and in the commits pushed to your own
     repository).  You can push additional changes to your branch as needed
     for this (``git push name-of-your-bugfix-or-feature``).  If you need to
     amend or rebase commits, then use ``git push --force-with-lease
     name-of-your-bugfix-or-feature``.


Test latest ``devel`` branch code
---------------------------------

If you want to test the code in the ``devel`` branch, you can install packages
from a repository that is automatically generated by our GitLab CI
pipeline.  To avoid confusion, we recommend doing this in a separate
container from your normal development container.

.. include:: /common/add-snapshot-repository.rst

Then run:

.. code-block:: console

   $ sudo apt update
   $ sudo apt install debusine-server debusine-worker debusine-client

And follow the instructions to set up :ref:`set-up-debusine-server`,
:ref:`set-up-debusine-worker` or :ref:`set-up-debusine-client`.

Write access to the git repository
----------------------------------

`Project members
<https://salsa.debian.org/freexian-team/debusine/-/project_members>`_ have
write access to the main git repository. They can thus clone the
repository with this URL:

.. code-block:: console

   $ git clone git@salsa.debian.org:freexian-team/debusine.git

From there they can push their changes directly. They are however free to
use a fork and request review anyway when they prefer.

Django Debug Toolbar
--------------------

`Django Debug Toolbar <https://django-debug-toolbar.readthedocs.io/en/latest/>`_ can be used but only on development branches.
There are commented out lines in
``debusine/project/settings/development.py`` and
``debusine/project/urls.py`` which can be used to add the support.