Setting up a LAVA pipeline instance

Initial considerations

  1. The default setup of the LAVA packages and codebase is for the current dispatcher and Deploying Distributed Instances.
  2. A LAVA pipeline instance can have existing remote worker support alongside but uses a completely different mechanism to identify remote workers and run jobs on pipeline devices.
  3. If both systems are enabled, devices can support both pipeline and current JSON submissions. It is not yet possible to disable JSON submissions. If there is no relevant configuration for a device other than pipeline support, a JSON submission would be accepted but would stay in Submitted state until cancelled.
  4. The default setup provides both mechanisms, the only step required to allow pipeline submissions to devices connected to http://localhost is to have pipeline devices available.
  5. Helpers will be developed in due course but currently, pipeline setup is principally a manual task for admins.
  6. If only pipeline devices are to be supported, the dispatchers running lava-slave do not need to have the lava-server package installed. Each dispatcher does need to be able to connect to the ZMQ port specified in the lava-master configuration of the instance (which is then the only machine related to that instance which has lava-server installed). The lava-server package on the master should be installed as a single master instance of LAVA.
  7. The ZMQ protocol incorporates buffering at each end such that either the lava-master or the lava-slave service can be restarted at any time without affecting currently running jobs or requiring any changes or restarts at the other end of the connection. There are no other connections required between the slave and the master and the outgoing request from the slave is initiated by the slave, so it should be possible for the slave to be behind a local firewall, as long as the relevant ports are open for outgoing traffic. i.e. the slave pulls from the master, the master cannot push to the slave.

Detailed changes

The pipeline design designates the machine running Django and PostgreSQL as the lava-master and all other machines connected to that master which will actually be running the jobs are termed lava-slave machines.

If this slave has no devices which will be used by the current dispatcher, only by the pipeline, just install lava-dispatcher:

$ sudo apt install lava-dispatcher
  1. Change the init script for lava-slave (/etc/init.d/lava-slave) to point at the relevant lava-master.

  2. Change the port numbers, if required, to match those in use on the lava-master.

  3. Restart lava-slave once the changes are complete:

    $ sudo service lava-slave restart
    
  4. The administrator of the master will then be able to allocate pipeline devices to this slave.

Note

For security reasons, the slave does not declare the devices connected to it to the master. The slave actually needs no knowledge of what is connected or where. All this information is stored solely in the database of the master. Once this data is entered by the admin of the master, the slave then needs to connect and the admin can then select that slave for the relevant devices. Once selected, the slave can immediately start running pipeline jobs on those devices.

The administrator of the master will require the following information about the devices attached to each slave:

  1. Confirmation that a suitable template already exists, for each device i.e. Adding support for a device of a known type
  2. A completed and tested device dictionary for each device.

This information contains specific information about the local network setup of the slave and will be transmitted between the master and the slave in clear text over ZMQ. Any encryption would need to be arranged separately between the slave and the master. Information typically involves the hostname of the PDU, the port number of the device on that PDU and the port number of the serial connection for that device. The slave is responsible for ensuring that these ports are only visible to that slave. There is no need for any connections to be visible to the master.

Table Of Contents

Previous topic

Worked example of migrating a known device

Next topic

Test Writer use cases

This Page