Models

Data models for the db application.

class debusine.db.models.Artifact(*args, **kwargs)[source]

Database model of an artifact.

An artifact is a set of files constituting the output of some process. Different artifacts have been output by the same process should share a common type.

An artifact can be associated with arbitrary key-value data used to convey some meta-information about the files or about the process that generated those files.

exception DoesNotExist
exception MultipleObjectsReturned
data

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

file_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

kind

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
class debusine.db.models.File(*args, **kwargs)[source]

Database model of an artifact file.

A file is always associated to a single artifact.

exception DoesNotExist
exception MultipleObjectsReturned
artifact

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

artifact_id
clean()[source]

Enforce that we reference a file somewhere.

Either remotely through the url field or locally with the path field.

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>
path

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

save(*args, **kwargs)[source]

Initialize the size field when possible.

size

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

url

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class debusine.db.models.Token(*args, **kwargs)[source]

Database model of a token.

A token contains a key and other related data. It’s used as a shared key between debusine server and clients (workers).

This token model is very similar to rest_framework.authtoken.models.Token. The bigger difference is that debusine’s token’s owner is a CharField, the rest_framework owner is a OneToOne foreign key to a user.

exception DoesNotExist
exception MultipleObjectsReturned
comment

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

disable()[source]

Disable the token and save it

enable()[source]

Enable the token and save it

enabled

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=True, **kwargs)
get_previous_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=False, **kwargs)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

key

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <debusine.db.models.TokenManager object>
owner

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

save(*args, **kwargs)[source]

Save the token. If key is empty it generates a key.

worker

Accessor to the related object on the reverse side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

Place.restaurant is a ReverseOneToOneDescriptor instance.

class debusine.db.models.TokenManager(*args, **kwargs)[source]

Manager for Token model.

get_token_or_none(token_key)[source]

Return the token with token_key or None.

get_tokens(owner=None, key=None)[source]

Return all the tokens filtered by a specific owner and/or token.

To avoid filtering by owner or token set them to None

class debusine.db.models.WorkRequest(*args, **kwargs)[source]

Database model of a request to execute a task.

Time-consuming operations offloaded to Workers and using Artifacts (and associated Files) as input and output.

Submission API needs to check if the request is valid using ontological rules - e.g. whether the specified distribution for a build task exists.

Avoid exposing the status of tasks to the admin interface to avoid runaway changes whilst the scheduler process is running.

The WorkRequest uses the non-Django tasks module to do the checks on whether a task can run on a particular worker.

New WorkRequest database entries default to WorkRequest.Statuses.PENDING.

Once the WorkRequest is assigned to a worker and is running starts running the status is changed to WorkRequest.Statuses.RUNNING.

If the WorkRequest is aborted, the Scheduled.Task status is WorkRequest.Statuses.ABORTED.

If the task finish on the Worker the WorkRequest status will be WorkRequest.Statuses.COMPLETED and a WorkRequest.Result is then set, WorkRequest.Results.PASSED or WorkRequest.Results.FAILED.

digraph {
   Statuses_PENDING -> Statuses_RUNNING -> Statuses_COMPLETED;
   Statuses_PENDING -> Statuses_ABORTED;
   Statuses_PENDING -> Statuses_RUNNING -> Statuses_ABORTED;
}

WorkRequest.started_at is set when the WorkRequest moves from WorkRequest.Statuses.PENDING to WorkRequest.Statuses.RUNNING. WorkRequest.completed_at is set when the Task moves from WorkRequest.Statuses.RUNNING to WorkRequest.Statuses.COMPLETED.

exception DoesNotExist
exception MultipleObjectsReturned
class Results(value)[source]

An enumeration.

ERROR = 'error'
FAILURE = 'failure'
NONE = ''
SUCCESS = 'success'
class Statuses(value)[source]

An enumeration.

ABORTED = 'aborted'
COMPLETED = 'completed'
PENDING = 'pending'
RUNNING = 'running'
assign_worker(worker)[source]

Assign worker and save it

completed_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

property duration

Return duration in seconds between started_at and completed_at

get_next_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=True, **kwargs)
get_previous_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=False, **kwargs)
get_result_display(*, field=<django.db.models.fields.CharField: result>)
get_status_display(*, field=<django.db.models.fields.CharField: status>)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

mark_aborted()[source]

Worker has aborted the task after request from UI

Task will typically be in CREATED or RUNNING status.

mark_completed(result)[source]

Worker has finished executing the task

mark_running()[source]

Worker has begun executing the task

objects = <debusine.db.models.WorkRequestManager object>
result

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

started_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

task_data

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

task_name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

worker

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

worker_id
class debusine.db.models.WorkRequestManager(*args, **kwargs)[source]

Manager for WorkRequest model.

aborted()[source]

Return a QuerySet of tasks in aborted status

completed()[source]

Return a QuerySet of tasks in completed status

pending(exclude_assigned=False, worker=None)[source]

Return a QuerySet of tasks in WorkRequest.Statuses.PENDING status

QuerySet is ordered by created_at

Filter out the assigned pending ones if exclude_assigned=True, and include only the WorkRequest for worker.

PENDING is the default status of a task on creation.

running(worker=None)[source]

Return a QuerySet of tasks in running status

class debusine.db.models.Worker(*args, **kwargs)[source]

Database model of a worker.

exception DoesNotExist
exception MultipleObjectsReturned
assigned_work_requests

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

connected()[source]

Return True if the Worker is connected.

connected_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

dynamic_metadata

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

dynamic_metadata_updated_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_registered_at(*, field=<django.db.models.fields.DateTimeField: registered_at>, is_next=True, **kwargs)
get_previous_by_registered_at(*, field=<django.db.models.fields.DateTimeField: registered_at>, is_next=False, **kwargs)
id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

mark_connected()[source]

Update and save relevant Worker fields after connecting.

mark_disconnected()[source]

Update and save relevant Worker fields after disconnecting.

metadata()dict[source]

Return all metadata with static_metadata and dynamic_metadata merged.

If the same key is in static_metadata and dynamic_metadata: static_metadata takes priority.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <debusine.db.models.WorkerManager object>
registered_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

set_dynamic_metadata(metadata: dict)[source]

Save metadata and update dynamic_metadata_updated_at

static_metadata

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

token

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

Restaurant.place is a ForwardOneToOneDescriptor instance.

token_id
class debusine.db.models.WorkerManager(*args, **kwargs)[source]

Manager for Worker model.

connected()[source]

Return connected workers.

classmethod create_with_fqdn(fqdn, token)[source]

Return a new Worker with its name based on @p fqdn with @p token.

get_worker_by_token_key_or_none(token_key)[source]

Return a Worker identified by its associated secret token.

get_worker_or_none(worker_name)[source]

Return the worker with worker_name or None.

waiting_for_work_request()[source]

Return workers that can be assigned a new work request.

The workers without any associated pending or running work request don’t have anything to run right now and are thus waiting for a work request.

debusine.db.models.random_uuid()[source]

Generate a random UUID and return it as a string.