Pipeline

Author:Howard Butler
Contact:howard@hobu.co
Date:03/22/2016

Pipelines are the operative construct in PDAL – you must construct pipelines to perform data translation operations, filter data, or simply to provide data to an endpoint. This document describes some features and background of PDAL Pipelines and give some examples you can use to execute translation operations.

See also

pipeline command is used to invoke JSON pipeline operations via the command line.

Warning

As of PDAL 1.2, JSON is now the preferred specification language for PDAL pipelines. XML read support is still available at 1.2, but JSON is preferred.

Composing pipelines

A Pipeline is composed as an array of pdal::Stage , with the first stage at the beginning and the last at the end. There are two primary building blocks in PDAL, pdal::Stage and pdal::PointView. pdal::Reader, pdal::Writer, and pdal::Filter are all subclasses of pdal::Stage.

pdal::PointView is the substrate that flows between stages in a pipeline and transfers the actual data as it moves through the pipeline. A pdal::PointView contains a pdal::PointTablePtr, which itself contains a list of pdal::Dimension objects that define the actual channels that are stored in the pdal::PointView.

PDAL provides four types of stages – pdal::Reader, pdal::Writer, pdal::Filter, and pdal::MultiFilter – with the latter being hardly used (just filters.merge) at this point. A Reader is a producer of data, a Writer is a consumer of data, and a Filter is an actor on data.

Note

As a C++ API consumer, you are generally not supposed to worry about the underlying storage of the PointView, but there might be times when you simply just “want the data.” In those situations, you can use the pdal::PointView::getBytes() method to stream out the raw storage.

Usage

While pipeline objects are manipulable through C++ objects, the other, more convenient way is through an JSON syntax. The JSON syntax mirrors the arrangement of the Pipeline, with options and auxiliary metadata added on a per-stage basis.

We have two use cases specifically in mind:

  • a command-line application that reads an JSON file to allow a user to easily construct arbitrary writer pipelines, as opposed to having to build applications custom to individual needs with arbitrary options, filters, etc.
  • a user can provide JSON for a reader pipeline, construct it via a simple call to the PipelineManager API, and then use the pdal::Stage::read() function to perform the read and then do any processing of the points. This style of operation is very appropriate for using PDAL from within environments like Python where the focus is on just getting the points, as opposed to complex pipeline construction.
{
  "pipeline":[
    "/path/to/my/file/input.las",
    "output.las"
  ]
}

Note

https://github.com/PDAL/PDAL/blob/master/test/data/pipeline/ contains test suite pipeline files that provide an excellent example of the currently possible operations.

Stage Types

pdal::Reader, pdal::Writer, and pdal::Filter are the C++ classes that define the stage types in PDAL. Readers follow the pattern of readers.las or readers.oci, Writers follow the pattern of writers.las or readers.oci, with Filters using filters.reprojection or filters.crop.

Note

Readers, Writers, and Filters contains a full listing of possible stages and descriptions of their options.

Note

Issuing the command pdal info --options will list all available stages and their options. See info command for more.

Options

Options are the mechanism that PDAL uses to inform pdal::Stage entities how to process data. The following example sorts the data using a Morton ordering using filters.mortonorder and writes out a LASzip file as the result. We use options to define the compression function for the writers.las pdal::Stage.

{
  "pipeline":[
    "uncompressed.las",
    {
      "type":"filters.mortonorder"
    }
    {
      "type":"writers.las",
      "filename":"compressed.laz",
      "compression":"true"
    }
  ]
}

Syntax Specification

See 1   Draft PDAL JSON Pipeline Specification for more detail