Warning
This section needs maintenance
This skeleton represents a typical application based on Plainbox. It enumerates the essential parts of the APIs from the point of view of an application developer.
Instantiate plainbox.impl.runner.JobRunner so that we can run jobs
Instantiate plainbox.impl.session.SessionState so that we can keep track of application state.
Allow the user to select jobs that should be executed and update session state by calling plainbox.impl.session.SessionState.update_desired_job_list()
For each job in plainbox.impl.SessionState.run_list:
Check if we want to run the job (if we have a result for it from previous runs) or if we must run it (for jobs that cannot be persisted across suspend)
Check if the job can be started by looking at plainbox.impl.session.JobState.can_start()
Call plainbox.impl.runner.JobRunner.run_job() with the current job and store the result.
- optionally ask the user to perform some manipulation
- optionally ask the user to qualify the outcome
- optionally ask the user for additional comments
Call plainbox.impl.session.SessionState.update_job_result() to update readiness of jobs that depend on the outcome or output of current job.
Call plainbox.impl.session.SessionState.checkpoint() to ensure that testing can resume after system crash or shutdown.
Instantiate the selected state exporter, for example plainbox.impl.exporters.json.JSONSessionStateExporter so that we can use it to save test results.
- optionally pass configuration options to customize the subset and the presentation of the session state
Call plainbox.impl.exporters.SessionStateExporterBase.get_session_data_subset() followed by plainbox.impl.exporters.SessionStateExporterBase.dump() to save results to a file.
Call plainbox.impl.session.SessionState.close() to remove any nonvolatile temporary storage that was needed for the session.
Class representing all state needed during a single program session.
The general idea is that you feed the session with a list of known jobs and a subset of jobs that you want to run and in return get an ordered list of jobs to run.
It is expected that the user will select / deselect and run jobs. This class can react to both actions by recomputing the dependency graph and updating the read states accordingly.
As the user runs subsequent jobs the results of those jobs are exposed to the session with update_job_result(). This can cause subsequent jobs to become available (not inhibited by anything). Note that there is no notification of changes at this time.
The session does almost nothing by itself, it learns about everything by observing job results coming from the job runner (plainbox.impl.runner.JobRunner) that applications need to instantiate.
The session can save check-point data after each job is executed. This allows the system to survive and continue after a catastrophic failure (broken suspend, power failure) or continue across tests that require the machine to reboot.
Internally it ties into plainbox.impl.depmgr.DependencySolver for resolving dependencies. The way the session objects are used allows them to return various problems back to the UI level - those are all the error classes from plainbox.impl.depmgr:
Normally none of those errors should ever happen, they are only provided so that we don’t choke when a problem really happens. Everything is checked and verified early before starting a job so typical unit and integration testing should capture broken job definitions (for example, with cyclic dependencies) being added to the repository.
There are two issues that are known at this time:
Checkbox has a concept of a job. Jobs are named units of testing work that can be executed. Typical jobs range from automated CPU power management checks, BIOS tests, semi-automated peripherals testing to all manual validation by following a script (intended for humans).
Jobs are distributed in plain text files, formated as a loose RFC822 documents where typically a single text file contains a few dozen different jobs that belong to one topic, for example, all bluetooth tests.
Tests have a number of properties that will not be discussed in detail here, they are all documented in plainbox.impl.job.JobDefinition. From the architecture point of view the four essential properties of a job are name, plugin and requires and depends. Those are discussed in detail below.
The name field must be unique and is referred to by other parts of the system (such as whitelists). Typically jobs follow a simple naming pattern ‘category/detail’, eg, ‘networking/modem_connection’. The name must be _unique_ and this is enforced by the core.
The plugin field is an archaism from Checkbox and a misnomer (as Plainbox does not have any plugins). In the Checkbox architecture it would instruct the core which plugin should process that job. In Plainbox it is a way to encode what type of a job is being processed. There is a finite set of types that are documented below.
This value is used for fully automated jobs. Everything the job needs to do is automated (preparation, execution, verification) and fully handled by the command that is associated with a job.
This value is used for fully manual jobs. It has no special handling in the core apart from requiring a human-provided outcome (pass/fail classification)
This value is used for special job generator jobs. The output of such jobs is interpreted as additional jobs and is identical in effect to loading such jobs from a job definition file.
There are two practical uses for such jobs:
Some local jobs are used to generate a number of jobs for each object. This is needed where the tested machine may have a number of such objects and each requires unique testing. A good example is a computer where all network tests are explicitly “instantiated” for each network card present.
This is a valid use case but is rather unfortunate for architecture of Plainbox and there is a desire to replace it with equally-expressive pattern jobs. The advantage is that unlike local jobs (which cannot be “discovered” without enduring any potential side effects that may be caused by the job script command) pattern jobs would allow the core to determine the names of jobs that can be generated and, for example, automatically determine that a pattern job needs to be executed as a dependency of a phantom (yet undetermined) job with a given name.
The solution with “pattern” jobs may be executed in future phases of Plainbox development. Currently there is no support for that at all.
Currently Plainbox cannot determine job dependencies across local jobs. That is, unless a local job is explicitly requested (in the desired job list) Plainbox will not be able to run a job that is generated by a local job at all and will treat it as if that job never existed.
Some local jobs are used to create a form of informal “category”. Typically all such jobs have a leading and trailing double underscore, for example ‘__audio__’. This is currently being used by Checkbox for building a hierarchical tree of tests that the user may select.
Since this has the same flaws as described above (for pattern jobs) it will likely be replaced by an explicit category field that can be specified each job.
This value is used for special “data” or “environment” jobs. Their output is parsed as a list of RFC822 records and is kept by the core during a testing session.
They are primarily used to determine if a given job can be started. For example, a particular bluetooth test may use the _requires_ field to indicate that it depends (via a resource dependency) on a job that enumerates devices and that one of those devices must be a bluetooth device.
For all intents and purposes it is equivalent to “manual”. The actual difference is that a user is expected to perform some physical manipulation before an automated test.
For all intents and purposes it is equivalent to “manual”. The actual difference is that a user is expected to perform manual verification after an automated test.
The depends field is used to express dependencies between two jobs. If job A has depends on job B then A cannot start if B is not both finished and successful. Plainbox understands this dependency and can automatically sort and execute jobs in proper order. In many places of the code this is referred to as a “direct dependency” (in contrast to “resource dependency”)
The actual syntax is not strictly specified, Plainbox interprets this field as a list of tokens delimited by comma or any whitespace (including newlines).
A job may depend on any number of other jobs. There are a number of failure modes associated with this feature, all of which are detected and handled by Plainbox. Typically they only arise when during Checkbox job development (editing actual job files) and are always a sign of a human error. No released version of Checkbox or Plainbox should ever encounter any of those issues.
The actual problems are:
In all of those cases the core removes the offending job and tries to work regardless of the problem. This is intended more as a development aid rather than a reliability feature as no released versions of either project should cause this problem.
The command field is used when the job needs to call an external command. Typically all shell jobs define a command to run.
“Manual” jobs can also define a command to run as part of the test procedure.
The user field is used when the job requires to run as a specific user (e.g. root).
The job command will be run via pkexec to get the necessary permissions.
The environ field is used to pass additional environmental keys from the user session to the new environment set up when the job command is run by another user (root, most of the time).
The actual syntax is not strictly specified, Plainbox interprets this field as a list of tokens delimited by comma or any whitespace (including newlines).