plainbox.impl.session – session handling

Sessions are central state holders and one of the most important classes in PlainBox. Since they are all named alike it’s a bit hard to find what the actual responsibilities are. Here’s a small shortcut, do read the description of each module and class for additional details though.

SessionState

This a class that holds all of the state and program logic. It SessionManager is a class that couples SessionState and SessionStorage. It has the methods required to alter the state by introducing additional jobs or results. It’s main responsibility is to keep track of all of the jobs, their results, if they are runnable or not (technically what is preventing them from being runnable) and to compute the order of execution that can satisfy all of the dependencies.

It holds a number of references to other pieces of PlainBox (jobs, resources and other things) but one thing stands out. This class holds references to a number of JobState objects that couple a JobResult and JobDefinition together.

SessionStateLegacyAPI

This class is a subclass of SessionState with additional methods for suspend and resume. It should not be used in new applications and it will be removed eventually, once the new manager-based API settles in. There are two classes that actually implement this API, one based on the original implementation and another one based on the new implementation. The data they create is not compatible with each other. Currently the original implementation is used. This will change very soon.

JobState

A coupling class between JobDefinition and JobResult. This class also knows why a job cannot be started in a particular session, by maintaining a set of “inhibitors” that prevent it from being runnable. The actual inhibitors are managed by SessionState.

SessionStorage

This class knows how properly to save and load bytes and manages a directory for all the filesystem entries associated with a particular session. It holds no references to a session though. Typically the class is not instantiated directly but instead comes from helper methods of SessionStorageRepository.

SessionStorageRepository

This class knows how to enumerate possible instances of SessionStorage from a given location in the filesystem. It also knows how to obtain a default location using XDG standards.
class plainbox.impl.session.JobReadinessInhibitor(cause, related_job=None, related_expression=None)

Class representing the cause of a job not being ready to execute.

It is intended to be consumed by UI layers and to provide them with enough information to render informative error messages or other visual feedback that will aid the user in understanding why a job cannot be started.

There are four possible not ready causes:

UNDESIRED:
This job was not selected to run in this session
PENDING_DEP:
This job depends on another job which was not started yet
FAILED_DEP:
This job depends on another job which was started and failed
PENDING_RESOURCE:
This job has a resource requirement expression that uses a resource produced by another job which was not started yet
FAILED_RESOURCE:
This job has a resource requirement that evaluated to a false value

All causes apart from UNDESIRED use the related_job property to encode a job that is related to the problem. The PENDING_RESOURCE and FAILED_RESOURCE causes also store related_expression that describes the relevant requirement expression.

There are three attributes that can be accessed:

cause:
Encodes the reason why a job is not ready, see above.
related_job:
Provides additional context for the problem. This is not the job that is affected, rather, the job that is causing the problem.
related_expression:
Provides additional context for the problem caused by a failing resource expression.
FAILED_DEP = 2
FAILED_RESOURCE = 4
PENDING_DEP = 1
PENDING_RESOURCE = 3
UNDESIRED = 0
cause_name
class plainbox.impl.session.JobState(job)

Class representing the state of a job in a session.

Contains two basic properties of each job:

  • the readiness_inhibitor_list that prevent the job form starting
  • the result (outcome) of the run (IJobResult)

For convenience (to SessionState implementation) it also has a reference to the job itself. This class is a pure state holder an will typically collaborate with the SessionState class and the UI layer.

can_start()

Quickly check if the associated job can run right now.

classmethod from_json_record(record)

Create a JobState instance from JSON record

get_readiness_description()

Get a human readable description of the current readiness state

job

the job associated with this state

on_result_changed

Event fired when the result associated with this job state changes

readiness_inhibitor_list

the list of readiness inhibitors of the associated job

result

the result of running the associated job

class plainbox.impl.session.SessionManager(state, storage)

Manager class for coupling SessionStorage with SessionState.

This class allows application code to manage disk state of sessions. Using the checkpoint() method applications can create persistent snapshots of the SessionState associated with each SessionManager.

checkpoint()

Create a checkpoint of the session.

After calling this method you can later reopen the same session with SessionManager.load_session().

classmethod create_with_job_list(job_list=None, repo=None, legacy_mode=False)

Create a session manager with a fresh session.

This method populates the session storage with all of the well known directories (using WellKnownDirsHelper.populate())

Parameters:
  • job_list – If specified then this will be the initial list of jobs known by the session state object. This can be specified for convenience but is really optional since the application can always add more jobs to an existing session.
  • repo – If specified then this particular repository will be used to create the storage for this session. If left out, a new repository is constructed with the default location.
  • legacy_mode – Propagated to create() to ensure that legacy (single session) mode is used.
Ptype job_list:

list of IJobDefinition.

Ptype repo:

SessionStorageRepository.

Ptype legacy_mode:
 

bool

Returns:

fresh SessionManager instance

classmethod create_with_state(state, repo=None, legacy_mode=False)

Create a session manager by wrapping existing session state.

This method populates the session storage with all of the well known directories (using WellKnownDirsHelper.populate())

Parameters:
  • stage – A pre-existing SessioState object.
  • repo – If specified then this particular repository will be used to create the storage for this session. If left out, a new repository is constructed with the default location.
  • legacy_mode – Propagated to create() to ensure that legacy (single session) mode is used.
Ptype repo:

SessionStorageRepository.

Ptype legacy_mode:
 

bool

Returns:

fresh SessionManager instance

destroy()

Destroy all of the filesystem artifacts of the session.

This basically calls remove()

classmethod load_session(job_list, storage, early_cb=None)

Load a previously checkpointed session.

This method allows one to re-open a session that was previously created by SessionManager.checkpoint()

Parameters:
  • job_list – List of all known jobs. This argument is used to reconstruct the session from a dormant state. Since the suspended data cannot capture implementation details of each job reliably actual jobs need to be provided externally. Unlike in create_session() this list really needs to be complete, it must also include any generated jobs.
  • storage – The storage that should be used for this particular session. The storage object holds references to existing directories in the file system. When restoring an existing dormant session it is important to use the correct storage object, the one that corresponds to the file system location used be the session before it was saved.
  • early_cb – A callback that allows the caller to “see” the session object early, before the bulk of resume operation happens. This method can be used to register callbacks on the new session before this method call returns. The callback accepts one argument, session, which is being resumed. This is being passed directly to plainbox.impl.session.resume.SessionResumeHelper.resume()
Ptype storage:

SessionStorage

Raises:

Anything that can be raised by load_checkpoint() and resume()

Returns:

Fresh instance of SessionManager

state

SessionState associated with this manager

storage

SessionStorage associated with this manager

class plainbox.impl.session.SessionMetaData(title=None, flags=None, running_job_name=None, app_blob=None, app_id=None)

Class representing non-critical state of the session.

The data held here allows applications to reason about sessions in general but is not relevant to the runner or the core in general

FLAG_INCOMPLETE = 'incomplete'
FLAG_SUBMITTED = 'submitted'
app_blob

Custom, application specific binary blob.

The type and value of this property is irrelevant as it is not inspected by plainbox at all. Reasonable applications will not make use of this property for storing large amounts of data. If you are tempted to do that, please redesign your application or propose changes to plainbox.

app_id

Application identifier

A string identifying the application that stored app_blob. It is recommended to use reverse domain names or UUIDs.

flags

a set of flags that are associated with this session.

This set is persisted by persistent_save() and can be used to keep track of how the application wants to interpret this session state.

Intended usage is to keep track of “testing finished” and “results submitted” flags. Some flags are added as constants to this class.

running_job_name

id of the running job

Note

This property has a confusing name. It actually refers to job ID, not name.

This property should be updated to keep track of the name of the job that is being executed. When either plainbox or the machine it was running on crashes during the execution of a job this value should be preserved and can help the GUI to resume and provide an error message.

The property MUST be set before starting the job itself.

title

the session title.

Title is just an arbitrary string that can be used to distinguish between multiple sessions.

The value can be changed at any time.

class plainbox.impl.session.SessionState(job_list)

Class representing all state needed during a single program session.

This is the central glue/entry-point for applications. It connects user intents to the rest of the system / plumbing and keeps all of the state in one place.

The set of utility methods and properties allow applications to easily handle the lower levels of dependencies, resources and ready states.

SessionState has the following instance variables, all of which are currently exposed as properties.

Variables:
  • job_list (list) –

    A list of all known jobs

    Not all the jobs from this list are going to be executed (or selected for execution) by the user.

    It may change at runtime because of local jobs. Note that in upcoming changes this will start out empty and will be changeable dynamically. It can still change due to local jobs but there is no API yes.

    This list cannot have any duplicates, if that is the case a DependencyDuplicateError is raised. This has to be handled externally and is a sign that the job database is corrupted or has wrong data. As an exception if duplicates are perfectly identical this error is silently corrected.

  • job_state_map (dict) –

    mapping that tracks the state of each job

    Mapping from job id to JobState. This basically has the test result and the inhibitor of each job. It also serves as a plainbox.impl.job.JobDefinition.id-> job lookup helper.

    Directly exposed with the intent to fuel part of the UI. This is a way to get at the readiness state, result and readiness inhibitors, if any.

    XXX: this can loose data job_list has jobs with the same id. It would be better to use job id as the keys here. A separate map could be used for the id->job lookup. This will be fixed when session controller branch lands in trunk as then jobs are dynamically added to the system one at a time and proper error conditions can be detected and reported.

  • desired_job_list (list) –

    subset of jobs selected for execution

    This is used to compute run_list. It can only be changed by calling update_desired_job_list() which returns meaningful values so this is not a settable property.

  • run_list (list) –

    sorted list of jobs to execute

    This is basically a superset of desired_job_list and a subset of job_list that is topologically sorted to allowing all desired jobs to run. This property is updated whenever desired_job_list is changed.

  • resource_map (dict) –

    all known resources

    A mapping from resource id to a list of plainbox.impl.resource.Resource objects. This encapsulates all “knowledge” about the system plainbox is running on.

    It is needed to compute job readiness (as it stores resource data needed by resource programs). It is also available to exporters.

    This is computed internally from the output of checkbox resource jobs, it can only be changed by calling update_job_result()

  • metadata (dict) – instance of SessionMetaData
add_job(new_job, recompute=True)

Add a new job to the session

Parameters:
  • new_job – The job being added
  • recompute – If True, recompute readiness inhibitors for all jobs. You should only set this to False if you’re adding a number of jobs and will otherwise ensure that _recompute_job_readiness() gets called before session state users can see the state again.
Returns:

The job that was actually added or an existing, identical job if a perfect clash was silently ignored.

Raises DependencyDuplicateError:
 

if a duplicate, clashing job definition is detected

The new_job gets added to all the state tracking objects of the session. The job is initially not selected to run (it is not in the desired_job_list and has the undesired inhibitor).

The new_job may clash with an existing job with the same id. Unless both jobs are identical this will cause DependencyDuplicateError to be raised. Identical jobs are silently discarded.

Note

This method recomputes job readiness for all jobs

desired_job_list

List of jobs that are on the “desired to run” list

This is a list, not a set, because the dependency solver algorithm retains as much of the original ordering as possible. Having said that, the actual order can differ widely (for instance, be reversed)

get_estimated_duration(manual_overhead=30.0)

Provide the estimated duration of the jobs that have been selected to run in this session (maintained by calling update_desired_job_list).

Manual jobs have an arbitrary figure added to their runtime to allow for execution of the test steps and verification of the result.

Returns:(estimate_automated, estimate_manual)

where estimate_automated is the value for automated jobs only and estimate_manual is the value for manual jobs only. These can be easily combined. Either value can be None if the value could not be calculated due to any job lacking the required estimated_duration field.

job_list

List of all known jobs.

Not necessarily all jobs from this list can be, or are desired to run. For API simplicity this variable is read-only, if you wish to alter the list of all jobs re-instantiate this class please.

job_state_map

Map from job id to JobState that encodes the state of each job.

metadata

metadata object associated with this session state.

on_job_added

Signal sent whenever a job is added to the session.

This signal is fired after on_job_state_map_changed()

on_job_removed

Signal sent whenever a job is removed from the session.

This signal is fired after on_job_state_map_changed()

on_job_result_changed

Signal fired after a job get changed (set)

This signal is fired each time a result is presented to the session.

This signal is fired after on_job_state_map_changed()

on_job_state_map_changed

Signal fired after job_state_map is changed in any way.

This signal is always fired before any more specialized signals such as on_job_result_changed() and on_job_added().

This signal is fired pretty often, each time a job result is presented to the session and each time a job is added. When both of those events happen at the same time only one notification is sent. The actual state is not sent as it is quite extensive and can be easily looked at by the application.

resource_map

Map from resource id to a list of resource records

run_list

List of jobs that were intended to run, in the proper order

The order is a result of topological sorting of the desired_job_list. This value is recomputed when change_desired_run_list() is called. It may be shorter than desired_run_list due to dependency errors.

set_resource_list(resource_id, resource_list)

Add or change a resource with the given id.

Resources silently overwrite any old resources with the same id.

trim_job_list(qualifier)

Discard jobs that are selected by the given qualifier.

Parameters:qualifier – A qualifier that selects jobs to be removed
Ptype qualifier:
 IJobQualifier
Raises ValueError:
 If any of the jobs selected by the qualifier is on the desired job list (or the run list)

This function correctly and safely discards certain jobs from the job list. It also removes the associated job state (and referenced job result) and results (for jobs that were resource jobs)

update_desired_job_list(desired_job_list)

Update the set of desired jobs (that ought to run)

This method can be used by the UI to recompute the dependency graph. The argument ‘desired_job_list’ is a list of jobs that should run. Those jobs must be a sub-collection of the job_list argument that was passed to the constructor.

It never fails although it may reduce the actual permitted desired_job_list to an empty list. It returns a list of problems (all instances of DependencyError class), one for each job that had to be removed.

update_job_result(job, result)

Notice the specified test result and update readiness state.

This function updates the internal result collection with the data from the specified test result. Results can safely override older results. Results also change the ready map (jobs that can run) because of dependency relations.

Some results have deeper meaning, those are results for local and resource jobs. They are discussed in detail below:

Resource jobs produce resource records which are used as data to run requirement expressions against. Each time a result for a resource job is presented to the session it will be parsed as a collection of RFC822 records. A new entry is created in the resource map (entirely replacing any old entries), with a list of the resources that were parsed from the IO log.

Local jobs produce more jobs. Like with resource jobs, their IO log is parsed and interpreted as additional jobs. Unlike in resource jobs local jobs don’t replace anything. They cannot replace an existing job with the same id.

plainbox.impl.session.SessionStateLegacyAPI

alias of SessionStateLegacyAPICompatImpl

class plainbox.impl.session.SessionStorage(location)

Abstraction for storage area that is used by SessionState to keep some persistent and volatile data.

This class implements functions performing input/output operations on session checkpoint data. The location property can be used for keeping any additional files or directories but keep in mind that they will be removed by SessionStorage.remove()

This class indirectly collaborates with SessionSuspendHelper and SessionResumeHelper.

break_lock()

Forcibly unlock the storage by removing a file created during atomic filesystem operations of save_checkpoint().

This method might be useful if save_checkpoint() raises LockedStorageError. It removes the “next” file that is used for atomic rename.

classmethod create(base_dir, legacy_mode=False)

Create a new SessionStorage in a random subdirectory of the specified base directory. The base directory is also created if necessary.

Parameters:
  • base_dir – Directory in which a random session directory will be created. Typically the base directory should be obtained from SessionStorageRepository.get_default_location()
  • legacy_mode – If False (defaults to True) then the caller is expected to handle multiple sessions by itself.

Note

Legacy mode is where applications using PlainBox API can only handle one session. Creating another session replaces whatever was stored before. In non-legacy mode applications can enumerate sessions, create arbitrary number of sessions at the same time and remove sessions once they are no longer necessary.

Legacy mode is implemented with a symbolic link called ‘last-session’ that keeps track of the last session created using legacy_mode=True. When a new legacy-mode session is created the target of that symlink is read and recursively removed.

load_checkpoint()

Load checkpoint data from the filesystem

Returns:

data from the most recent checkpoint

Return type:

bytes

Raises:
  • IOError, OSError – on various problems related to accessing the filesystem
  • NotImplementedError – when openat(2) is not available
location

location of the session storage

remove()

Remove all filesystem entries associated with this instance.

save_checkpoint(data)

Save checkpoint data to the filesystem.

The directory associated with this SessionStorage must already exist. Typically the instance should be obtained by calling SessionStorage.create() which will ensure that this is already the case.

Raises:
  • TypeError – if data is not a bytes object.
  • LockedStorageError – if leftovers from previous save_checkpoint() have been detected. Normally those should never be here but in certain cases that is possible. Callers might want to call break_lock() to resolve the problem and try again.
  • IOError, OSError – on various problems related to accessing the filesystem. Typically permission errors may be reported here.
  • NotImplementedError – when openat(2), renameat(2), unlinkat(2) are not available on this platform. Should never happen on Linux.
session_file

pathname of the session state file

class plainbox.impl.session.SessionStorageRepository(location=None)

Helper class to enumerate filesystem artefacts of current or past Sessions

This class collaborates with SessionStorage. The basic use-case is to open a well-known location and enumerate all the sessions that are stored there. This allows to create SessionStorage instances to further manage each session (such as remove them by calling :meth:SessionStorage.remove()`)

classmethod get_default_location()

Compute the default location of the session state repository

Returns:${XDG_CACHE_HOME:-$HOME/.cache}/plainbox/sessions
get_last_storage()

Find the last session storage object created in this repository.

Returns:SessionStorage object associated with the last session created in this repository using legacy mode.

Note

This will only return storage objects that were created using legacy mode. Nonlegacy storage objects will not be returned this way.

get_storage_list()

Enumerate stored sessions in the repository.

If the repository directory is not present then an empty list is returned.

Returns:list of SessionStorage representing discovered sessions
location

pathname of the repository

Previous topic

plainbox.impl.secure.rfc822 – RFC822 parser

Next topic

plainbox.impl.session.jobs – jobs state handling

This Page