Contribute to Debusine

Contributions to Debusine are greatly appreciated! You can contribute in many ways, be it writing documentation and blog posts, or fixing bugs and implementing new features.

Join the community

You can reach us using IRC on the #debusine channel at irc.debian.org. That channel is also available as a (bridged) Matrix room. There’s also a #debusine-notifications channel where you can see all activities happening in the GitLab project (it’s also bridged in a Matrix room).

You can also file issues in GitLab’s interface. There are two templates to help you file bug reports and feature requests.

While Freexian is driving the development forward by paying a few developers, rest assured that external contributions are welcome.

Where to start?

Starting small

Before starting to work on bigger projects, it probably makes sense to get used to our development workflows by tackling a few small issues (they have the Quick fix label).

Looking for issues

If you are looking for bigger issues to tackle, you will find plenty in the issue tracker.

The paid developers are usually focused on all the issues planned in the current milestone, working through the list by order of priority (from labels P1 to P3). If you want to help with the current milestone, you should probably start with a low-priority issue.

Otherwise you can also find backlogged issues that are not part of any milestone and that paid developers will not handle at this point.

Add a new feature

If you want to work on a new feature that is not yet planned, then you should first create a new issue using the Feature Request template (see Design new features with issues).

For larger features that require significant design work, it is recommended to first write a development blueprint (see Design new features with development blueprints).

A particular case where we expect and invite contributions is that of new workflows. See Contribute a new workflow for more details of how to go about doing that.

How to contribute

Note

See runtime environment to understand the runtime environment.

Set up your environment

Ready to contribute? Here’s the quickest way to set up Debusine for local development, with everything in a single container so that your host system isn’t affected:

Set up Incus, then start a development container:
$ incus launch images:debian/trixie/cloud debusine-dev
$ incus exec debusine-dev -- cloud-init status --wait
Install git in the container:
$ incus exec debusine-dev -- apt --update install git
Optionally, share a directory from your host system with the container so that you can conveniently edit Debusine source code from your host. For example, if you keep source code under the src subdirectory of your home directory:
$ incus config device add debusine-dev src \
    disk source=$HOME/src path=/home/debian/src shift=true
shift=true may require some configuration to get ID mapping to work properly; see the upstream docs.
Enter the container:
$ incus exec debusine-dev -- su - debian
Follow the rest of these instructions from inside the container.
Clone Debusine locally (if you shared a source directory from your host system earlier, you should cd to somewhere under that source directory first):
$ git clone https://salsa.debian.org/freexian-team/debusine.git
$ cd debusine
For a quick startup, run this command:
$ bin/quick-setup.sh
It will install required packages with apt and put a sample configuration file in place. It will also create a user and a database in PostgreSQL. The user will be named after your current Unix user and will own the new database so that it can connect to the newly created database without any other authentication.
Populate the database with:
$ ./manage.py migrate

Make and test your changes

Install pre-commit:
$ sudo apt install pre-commit
Set up pre-commit to run automatically when you commit (optional):
$ pre-commit install
This runs some checks (equivalent to make check) every time you run git commit. This downloads linters from upstream repositories; if you prefer to avoid that on your development machine, you can skip this step and rely on CI running it for you instead (but with longer latency). See Pre-commit hooks for more details.

Make sure you run this in the same environment where you’re going to run git commit. If you commit inside the container, then you should install pre-commit inside the container; if you commit directly on your host system (perhaps via your editor or IDE), then you should install pre-commit directly on your host system.
Switch to a new branch:
$ git checkout -b name-of-your-bugfix-or-feature
Develop your new feature, ideally following the rules of Test-Driven Development. Remember to update the release notes as needed.
Run quick static analysis checks on your code (optional):
$ make check
As above, this downloads linters from upstream repositories, so you can skip it if you’re very cautious about what code runs on your machine, but it spots many problems more quickly than the full unit test suite does.
Check that all unit tests pass:
$ make coverage
This runs tests using pytest and ensures that you have maintained 100% test coverage. If you get errors, make sure to fix them. You can also run pytest manually if you need to use more specific options; for example, -k is useful to run specific tests, and --pdb is useful for interactive debugging.

Note

If you get errors like OSError: [Errno 38] Function not implemented, then it means that you are lacking /dev/shm with proper permissions.

Run Debusine components

Note

Most Debusine changes can be tested much more easily and comprehensively by running the test suite, and if that’s all you need to do then you can safely skip these steps. However, if you need to look at the effects of user interface changes in a web browser, or if you need to experiment with task changes in a real worker, then it can be useful to have something closer to a full Debusine deployment.

Each of these commands should be run from a separate shell inside your container (so incus shell debusine-dev -- su - debian and change to the directory where you cloned Debusine, as above).

Start a local test server:
$ ./manage.py runserver
[...]
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Visit the URL returned to have access to the test website.
If you want to serve apt repositories as well, start a second server on port 8081:
$ ./manage.py runserver 8081
[...]
Starting development server at http://127.0.0.1:8001/
Quit the server with CONTROL-C.
If you want to run tasks, then start the Celery workers:
$ python3 -m celery -A debusine.project worker -l INFO -Q scheduler --concurrency 1
$ python3 -m celery -A debusine.project worker -l INFO
Configure the worker:
$ mkdir -p ~/.config/debusine/worker
$ cat <<EOF > ~/.config/debusine/worker/config.ini
[General]
api-url = http://localhost:8000/api
log-level = DEBUG
EOF
Run the worker with sbin dirs in PATH, e.g.:
$ PATH="/usr/sbin:/sbin:$PATH" LANG=C.UTF-8 python3 -m debusine.worker
Enable the worker. You can find the name of the worker (by default is the hostname):
$ ./manage.py list_workers
And then enable it:
$ ./manage.py manage_worker enable NAME
Create a local Django superuser on your test instance:
$ ./manage.py createsuperuser
Login as that user using the link in the top right of the initial REST framework site.
If you want to use the debusine client, set it up via:
$ python3 -m debusine.client setup
Set the default workspace to System.
Many of the Debusine worker tasks require an environment, which is a tarball or system image to pass to an executor such as unshare or incus). You can create one via:
$ python3 -m debusine.client work-request create mmdebstrap <<EOF
  bootstrap_options:
    architecture: amd64
    extra_packages:
    - devscripts
    variant: minbase
  bootstrap_repositories:
  - components:
    - main
    mirror: http://deb.debian.org/debian
    suite: trixie
EOF
Note that the worker needs the package mmdebstrap installed. If the worker does not have the package installed the work request will be in Pending and never run. If needed install the package and restart the worker.

You can follow the progress (e.g. Pending, Running, or Completed) via the web interface (browse to the scope, then click on the navigation bar for “Work Requests” to see it).
In order to create workflows or tasks (and also other operations such as creating workflow templates) you will need to give permissions to the user:
$ ./manage.py group create debusine/SystemAdmins
$ ./manage.py workspace grant_role debusine/System owner SystemAdmins
$ ./manage.py group members debusine/SystemAdmins --add YOUR-USER-NAME

Contribute your changes

It is usually easiest to run these steps directly on your host system, to avoid needing to copy SSH keys around.

If you haven’t done so already, create a guest account on Salsa (a GitLab instance run by the Debian project) by visiting this page: https://salsa.debian.org/users/sign_up

Follow all the steps to confirm your email, fill in your profile, and set up your SSH keys.

You might want to have a look at Salsa’s documentation and GitLab’s documentation if you have questions about something.

Debian Developers can skip this step, as they already have an account on this service.
If you haven’t done so already, visit the Debusine project’s page and fork it to your own account. See GitLab’s help on how to fork a project. Then, add your fork as a remote, replacing your-account with your Salsa username:
$ git remote add -f $USER git@salsa.debian.org/your-account/debusine.git
Push your branch to your repository:
$ git push -u $USER name-of-your-bugfix-or-feature
Submit your work to us by creating a merge request. The previous git push step will give you a URL you can use for this. Alternatively, you can visit the Debusine project fork hosted in your own account and create a merge request from there (either through the “Branches” page, or through the “Merge requests” page). See GitLab’s help on merge requests if needed.

Make sure to address any issues identified by the continuous integration system. The result of its “pipeline” can be directly seen in the merge request (and in the commits pushed to your own repository). You can push additional changes to your branch as needed for this (git push name-of-your-bugfix-or-feature). If you need to amend or rebase commits, then use git push --force-with-lease name-of-your-bugfix-or-feature.

Test latest `devel` branch code

If you want to test the code in the devel branch, you can install packages from a repository that is automatically generated by our GitLab CI pipeline. To avoid confusion, we recommend doing this in a separate container from your normal development container.

$ sudo tee /etc/apt/sources.list.d/debusine.sources <<END
Types: deb
URIs: https://freexian-team.pages.debian.net/debusine/repository/
Suites: trixie
Components: main
Architectures: all
Signed-By: |
  -----BEGIN PGP PUBLIC KEY BLOCK-----
  .
  mQENBGUAlgcBCAD0AhJt2KVLlrJ+Bx8VcjxCNGcOEMPOY1Db+F+8TiaYrSqZ07kx
  VOJYATlNAqCkQqCu89E9WvqKTg2kyNZchyArwi4bUhEfA+BT+KBeb4dGcoAdwUNX
  c4hx94M/Ow2hAStIni2TUYau99e5sY2QIFb03Lb3dShlQA2EesjQcvnpZCDBcVHW
  kYGj8zjh+i0QdtGVYJlXI/IskSJKexJuiOQ9uIGtsJ+VAm2hD1dTDWvGoUp2quWP
  uHGBouyzFkFQpCOXKBnfpF4h7LMFJSx9KNV/rSMZgLf0F9ukeGnjsST3okQGz2Sz
  E5WdoJuaLOgj/eCeTG2lHD1sdzyndQtG9sKJABEBAAG0Q0RlYnVzaW5lIFNhbHNh
  IFJlcG9zaXRvcnkgPHJhcGhhZWwrZGVidXNpbmUrc2Fsc2FyZXBvQGZyZWV4aWFu
  LmNvbT6JAU4EEwEKADgWIQSQH9XztPO3VNW3w993YWGYlEiOAAUCZQCWBwIbLwUL
  CQgHAwUVCgkICwUWAgMBAAIeAQIXgAAKCRB3YWGYlEiOAO8eCACwcqdDbVigGiv8
  x9iUOp59/6XMG/mtdSjzkkK5LVUEq9bkQAdLRbzb43p0vLl7E/7Yt3on62D6N3QB
  1ap710QsDRIN/CoKRi6sjHCiAqpSUC9LcwgLmnrLwUHDPDrkpsnWmQ7YINtV1P6E
  BFgdqqtw/1avMyliBbh3/XNRQkYq6xkcEZGIftqwX8GYB3M+cheefYe24H/JwLJm
  gerMz8MVEGKMzDm0OaLjWixV6Ga4Wkzo5ya1zg5OmROULAj3CsKDN76OCEwGGMBs
  kNsTB9aAisP8D6wW6ZXX5eChRLRCmKUEB5zpHJ++jdJ0yY38kF+N17w5eos+rc4s
  bMld2SQP
  =XfKL
  -----END PGP PUBLIC KEY BLOCK-----
END
$ sudo apt update

Then run:

$ sudo apt update
$ sudo apt install debusine-server debusine-worker debusine-client

And follow the instructions to set up Set up a Debusine server, Set up a Debusine worker or Set up debusine-client.

Write access to the git repository

Project members have write access to the main git repository. They can thus clone the repository with this URL:

$ git clone git@salsa.debian.org:freexian-team/debusine.git

From there they can push their changes directly. They are however free to use a fork and request review anyway when they prefer.