Models

Data models for the db application.

class debusine.db.models.File(*args, **kwargs)

Database model of a file.

Model different attributes of the file.

From outside the class use the property hash and do not use sha256 field. This allows, if ever needed, to change the hash algorithm only modifying this class without changing the users of this class.

exception DoesNotExist

exception MultipleObjectsReturned

classmethod calculate_hash(file_path: pathlib.Path) → bytes

Return hash for the file.

fileinstore_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

filestore_set

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

classmethod from_local_path(local_path) → debusine.db.models.File

Return a File with the fields.

property hash_digest

Return the default hash digest of the File.

Use this property instead of the field to allow the algorithm to be changed in the future.

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>

sha256

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

size

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

class debusine.db.models.FileInStore(*args, **kwargs)

Database model used as “through” from FileStore.

Keeps the relationship between stores, files and attached data.

exception DoesNotExist

exception MultipleObjectsReturned

data

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

file

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

file_id

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>

store

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

store_id

class debusine.db.models.FileStore(*args, **kwargs)

Database model of a FileStore.

FileStore has files attached to it.

BackendChoices

alias of debusine.db.models.FileStoreBackendChoices

exception DoesNotExist

exception MultipleObjectsReturned

backend

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

configuration

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

fileinstore_set

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

files

Accessor to the related objects manager on the forward and reverse sides of a many-to-many relation.

In the example:

class Pizza(Model):
    toppings = ManyToManyField(Topping, related_name='pizzas')

Pizza.toppings and Topping.pizzas are ManyToManyDescriptor instances.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

get_backend_display(*, field=<django.db.models.fields.CharField: backend>)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <django.db.models.manager.Manager object>

class debusine.db.models.FileStoreBackendChoices(value)

Enumerate all the Backend choices.

LOCAL = 'Local'

class debusine.db.models.Token(*args, **kwargs)

Database model of a token.

A token contains a key and other related data. It’s used as a shared key between debusine server and clients (workers).

This token model is very similar to rest_framework.authtoken.models.Token. The bigger difference is that debusine’s token’s owner is a CharField, the rest_framework owner is a OneToOne foreign key to a user.

exception DoesNotExist

exception MultipleObjectsReturned

comment

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

disable()

Disable the token and save it.

enable()

Enable the token and save it.

enabled

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=True, **kwargs)

get_previous_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=False, **kwargs)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

key

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <debusine.db.models.TokenManager object>

owner

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

save(*args, **kwargs)

Save the token. If key is empty it generates a key.

worker

Accessor to the related object on the reverse side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

Place.restaurant is a ReverseOneToOneDescriptor instance.

class debusine.db.models.TokenManager(*args, **kwargs)

Manager for Token model.

get_token_or_none(token_key)

Return the token with token_key or None.

get_tokens(owner=None, key=None)

Return all the tokens filtered by a specific owner and/or token.

To avoid filtering by owner or token set them to None

class debusine.db.models.WorkRequest(*args, **kwargs)

Database model of a request to execute a task.

Time-consuming operations offloaded to Workers and using Artifacts (and associated Files) as input and output.

Submission API needs to check if the request is valid using ontological rules - e.g. whether the specified distribution for a build task exists.

Avoid exposing the status of tasks to the admin interface to avoid runaway changes whilst the scheduler process is running.

The WorkRequest uses the non-Django tasks module to do the checks on whether a task can run on a particular worker.

New WorkRequest database entries default to WorkRequest.Statuses.PENDING.

Once the WorkRequest is assigned to a worker and is running starts running the status is changed to WorkRequest.Statuses.RUNNING.

If the WorkRequest is aborted, the Scheduled.Task status is WorkRequest.Statuses.ABORTED.

If the task finish on the Worker the WorkRequest status will be WorkRequest.Statuses.COMPLETED and a WorkRequest.Result is then set, WorkRequest.Results.PASSED or WorkRequest.Results.FAILED.

WorkRequest.started_at is set when the WorkRequest moves from WorkRequest.Statuses.PENDING to WorkRequest.Statuses.RUNNING. WorkRequest.completed_at is set when the Task moves from WorkRequest.Statuses.RUNNING to WorkRequest.Statuses.COMPLETED.

exception DoesNotExist

exception MultipleObjectsReturned

class Results(value)

An enumeration.

ERROR = 'error'

FAILURE = 'failure'

NONE = ''

SUCCESS = 'success'

class Statuses(value)

An enumeration.

ABORTED = 'aborted'

COMPLETED = 'completed'

PENDING = 'pending'

RUNNING = 'running'

assign_worker(worker)

Assign worker and save it.

completed_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

created_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

property duration

Return duration in seconds between started_at and completed_at.

get_next_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=True, **kwargs)

get_previous_by_created_at(*, field=<django.db.models.fields.DateTimeField: created_at>, is_next=False, **kwargs)

get_result_display(*, field=<django.db.models.fields.CharField: result>)

get_status_display(*, field=<django.db.models.fields.CharField: status>)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

mark_aborted()

Worker has aborted the task after request from UI.

Task will typically be in CREATED or RUNNING status.

mark_completed(result)

Worker has finished executing the task.

mark_running()

Worker has begun executing the task.

objects = <debusine.db.models.WorkRequestManager object>

result

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

started_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

status

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

task_data

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

task_name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

worker

Accessor to the related object on the forward side of a many-to-one or one-to-one (via ForwardOneToOneDescriptor subclass) relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Child.parent is a ForwardManyToOneDescriptor instance.

worker_id

class debusine.db.models.WorkRequestManager(*args, **kwargs)

Manager for WorkRequest model.

aborted()

Return a QuerySet of tasks in aborted status.

completed()

Return a QuerySet of tasks in completed status.

pending(exclude_assigned=False, worker=None)

Return a QuerySet of tasks in WorkRequest.Statuses.PENDING status.

QuerySet is ordered by created_at.

Filter out the assigned pending ones if exclude_assigned=True, and include only the WorkRequest for worker.

PENDING is the default status of a task on creation.

running(worker=None)

Return a QuerySet of tasks in running status.

running_or_pending_exists(worker) → bool

Return True if there are running or pending work requests.

class debusine.db.models.Worker(*args, **kwargs)

Database model of a worker.

exception DoesNotExist

exception MultipleObjectsReturned

assigned_work_requests

Accessor to the related objects manager on the reverse side of a many-to-one relation.

In the example:

class Child(Model):
    parent = ForeignKey(Parent, related_name='children')

Parent.children is a ReverseManyToOneDescriptor instance.

Most of the implementation is delegated to a dynamically defined manager class built by create_forward_many_to_many_manager() defined below.

connected()

Return True if the Worker is connected.

connected_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

dynamic_metadata

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

dynamic_metadata_updated_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

get_next_by_registered_at(*, field=<django.db.models.fields.DateTimeField: registered_at>, is_next=True, **kwargs)

get_previous_by_registered_at(*, field=<django.db.models.fields.DateTimeField: registered_at>, is_next=False, **kwargs)

id

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

mark_connected()

Update and save relevant Worker fields after connecting.

mark_disconnected()

Update and save relevant Worker fields after disconnecting.

metadata() → dict

Return all metadata with static_metadata and dynamic_metadata merged.

If the same key is in static_metadata and dynamic_metadata: static_metadata takes priority.

name

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

objects = <debusine.db.models.WorkerManager object>

registered_at

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

set_dynamic_metadata(metadata: dict)

Save metadata and update dynamic_metadata_updated_at.

static_metadata

A wrapper for a deferred-loading field. When the value is read from this object the first time, the query is executed.

token

Accessor to the related object on the forward side of a one-to-one relation.

In the example:

class Restaurant(Model):
    place = OneToOneField(Place, related_name='restaurant')

Restaurant.place is a ForwardOneToOneDescriptor instance.

token_id

class debusine.db.models.WorkerManager(*args, **kwargs)

Manager for Worker model.

connected()

Return connected workers.

classmethod create_with_fqdn(fqdn, token)

Return a new Worker with its name based on fqdn, with token.

get_worker_by_token_key_or_none(token_key)

Return a Worker identified by its associated secret token.

get_worker_or_none(worker_name)

Return the worker with worker_name or None.

waiting_for_work_request()

Return workers that can be assigned a new work request.

The workers without any associated pending or running work request don’t have anything to run right now and are thus waiting for a work request.

Worker’s token must be enabled.