The job definitions for Jenkins Job Builder are kept in any number of YAML files, in whatever way you would like to organize them. When you invoke jenkins-jobs you may specify either the path of a single YAML file, or a directory. If you choose a directory, all of the .yaml (or .yml) files in that directory will be read, and all the jobs they define will be created or updated.
Jenkins Job Builder understands a few basic object types which are described in the next sections.
The most straightforward way to create a job is simply to define a Job in YAML. It looks like this:
- job:
name: job-name
That’s not very useful, so you’ll want to add some actions such as Builders, and perhaps Publishers. Those are described later. There are a few basic optional fields for a Job definition:
- job:
name: job-name
project-type: freestyle
defaults: global
disabled: false
display-name: 'Fancy job name'
concurrent: true
workspace: /srv/build-area/job-name
child-workspace: /srv/build-area/job-name
quiet-period: 5
block-downstream: false
block-upstream: false
retry-count: 3
Job Parameters: |
|
---|
If you need several jobs defined that are nearly identical, except perhaps in their names, SCP targets, etc., then you may use a Job Template to specify the particulars of the job, and then use a Project to realize the job with appropriate variable substitution. Any variables not specified at the project level will be inherited from the Defaults.
A Job Template has the same syntax as a Job, but you may add variables anywhere in the definition. Variables are indicated by enclosing them in braces, e.g., {name} will substitute the variable name. When using a variable in a string field, it is good practice to wrap the entire string in quotes, even if the rules of YAML syntax don’t require it because the value of the variable may require quotes after substitution. In the rare situation that you must encode braces within literals inside a template (for example a shell function definition in a builder), doubling the braces will prevent them from being interpreted as a template variable.
You must include a variable in the name field of a Job Template (otherwise, every instance would have the same name). For example:
- job-template:
name: '{name}-unit-tests'
Will not cause any job to be created in Jenkins, however, it will define a template that you can use to create jobs with a Project definition. It’s name will depend on what is supplied to the Project.
The purpose of a project is to collect related jobs together, and provide values for the variables in a Job Template. It looks like this:
- project:
name: project-name
jobs:
- '{name}-unit-tests'
Any number of arbitrarily named additional fields may be specified, and they will be available for variable substitution in the job template. Any job templates listed under jobs: will be realized with those values. The example above would create the job called ‘project-name-unit-tests’ in Jenkins.
The jobs: list can also allow for specifying job-specific substitutions as follows:
- project:
name: project-name
jobs:
- '{name}-unit-tests':
mail-to: developer@nowhere.net
- '{name}-perf-tests':
mail-to: projmanager@nowhere.net
If a variable is a list, the job template will be realized with the variable set to each value in the list. Multiple lists will lead to the template being realized with the cartesian product of those values. Example:
- project:
name: project-name
pyver:
- 26
- 27
jobs:
- '{name}-{pyver}'
If there are templates being realized that differ only in the variable used for its name (thus not a use case for job-specific substitutions), additional variables can be specified for project variables. Example:
- job-template:
name: '{name}-{pyver}'
builders:
- shell: 'git co {branch_name}'
- project:
name: project-name
pyver:
- 26:
branch_name: old_branch
- 27:
branch_name: new_branch
jobs:
- '{name}-{pyver}'
If you have several Job Templates that should all be realized together, you can define a Job Group to collect them. Simply use the Job Group where you would normally use a Job Template and all of the Job Templates in the Job Group will be realized. For example:
- job-template:
name: '{name}-unit-tests'
builders:
- shell: unittest
publishers:
- email:
recipients: '{mail-to}'
- job-template:
name: '{name}-perf-tests'
builders:
- shell: perftest
publishers:
- email:
recipients: '{mail-to}'
- job-group:
name: '{name}-tests'
jobs:
- '{name}-unit-tests':
mail-to: developer@nowhere.net
- '{name}-perf-tests':
mail-to: projmanager@nowhere.net
- project:
name: project-name
jobs:
- '{name}-tests'
Would cause the jobs project-name-unit-tests and project-name-perf-tests to be created in Jenkins.
Many of the actions of a Job, such as builders or publishers, can be defined as a Macro, and then that Macro used in the Job description. Builders are described later, but let’s introduce a simple one now to illustrate the Macro functionality. This snippet will instruct Jenkins to execute “make test” as part of the job:
- job:
name: foo-test
builders:
- shell: 'make test'
If you wanted to define a macro (which won’t save much typing in this case, but could still be useful to centralize the definition of a commonly repeated task), the configuration would look like:
- builder:
name: make-test
builders:
- shell: 'make test'
- job:
name: foo-test
builders:
- make-test
This allows you to create complex actions (and even sequences of actions) in YAML that look like first-class Jenkins Job Builder actions. Not every attribute supports Macros, check the documentation for the action before you try to use a Macro for it.
Macros can take parameters, letting you define a generic macro and more specific ones without having to duplicate code:
# The 'add' macro takes a 'number' parameter and will creates a
# job which prints 'Adding ' followed by the 'number' parameter:
- builder:
name: add
builders:
- shell: "echo Adding {number}"
# A specialized macro 'addtwo' reusing the 'add' macro but with
# a 'number' parameter hardcoded to 'two':
- builder:
name: addtwo
builders:
- add:
number: "two"
# Glue to have Jenkins Job Builder to expand this YAML example:
- job:
name: "testingjob"
builders:
# The specialized macro:
- addtwo
# Generic macro call with a parameter
- add:
number: "ZERO"
# Generic macro called without a parameter. Never do this!
# See below for the resulting wrong output :(
- add
Then <builders /> section of the generated job show up as:
<builders>
<hudson.tasks.Shell>
<command>echo Adding two</command>
</hudson.tasks.Shell>
<hudson.tasks.Shell>
<command>echo Adding ZERO</command>
</hudson.tasks.Shell>
<hudson.tasks.Shell>
<command>echo Adding {number}</command>
</hudson.tasks.Shell>
</builders>
As you can see, the specialized macro addtwo reused the definition from the generic macro add. Whenever you forget a parameter from a macro, it will not be expanded and left as is, which will most probably cause havoc in your Jenkins builds.
Defaults collect job attributes (including actions) and will supply those values when the job is created, unless superseded by a value in the ‘Job’_ definition. If a set of Defaults is specified with the name global, that will be used by all Job (and Job Template) definitions unless they specify a different Default object with the defaults attribute. For example:
- defaults:
name: global
description: 'Do not edit this job through the web!'
Will set the job description for every job created.
You can define variables that will be realized in a Job Template.
- defaults:
name: global
arch: 'i386'
- project:
name: project-name
jobs:
- 'build-{arch}'
- 'build-{arch}':
arch: 'amd64'
- job-template:
name: 'build-{arch}'
builders:
- shell: "echo Build arch {arch}."
Would create jobs build-i386 and build-amd64.
If you want to use lists or dicts variables you can use {obj:key}.
For example:
- project:
name: test_custom_distri
distributions: !!python/tuple [precise, jessie]
architectures: !!python/tuple &architectures
- amd64
- i386
axis_a:
type: user-defined
name: architectures
values: *architectures
jobs:
- '{name}-source'
- job-template:
name: '{name}-source'
project-type: matrix
axes:
- axis:
type: user-defined
name: distribution
values: '{obj:distributions}'
- axis: '{obj:axis_a}'
Custom application specific yamls tags are supported to provide enhancements when reading yaml configuration.
These allow inclusion of arbitrary files as a method of having blocks of data managed separately to the yaml job configurations. A specific usage of this is inlining scripts contained in separate files, although such tags may also be used to simplify usage of macros or job templates.
The tag !include will treat the following string as file which should be parsed as yaml configuration data.
Example:
- job: name: test-job-1 builders: !include include001.yaml.inccontents of include001.yaml.inc:
- timeout-wrapper - pre-scm-shell-ant - copy-files
The tag !include-raw will treat the following file as a data blob, which should be read into the calling yaml construct without any further parsing. Any data in a file included through this tag, will be treated as string data.
Example:
- job: name: test-job-include-raw-1 builders: - shell: !include-raw include-raw001-hello-world.sh - shell: !include-raw include-raw001-vars.shcontents of include-raw001-hello-world.sh:
#!/bin/bash # # Sample script showing how the yaml include-raw tag can be used # to inline scripts that are maintained outside of the jenkins # job yaml configuration. echo "hello world" exit 0contents of include-raw001-vars.sh:
#!/bin/bash # # sample script to check that brackets aren't escaped # when using the include-raw application yaml tag VAR1="hello" VAR2="world" VAR3="${VAR1} ${VAR2}" [[ -n "${VAR3}" ]] && { # this next section is executed as one echo "${VAR3}" exit 0 }
The tag !include-raw-escape treats the given file as a data blob, which should be escaped before being read in as string data. This allows job-templates to use this tag to include scripts from files without needing to escape braces in the original file.
Example:
- template-job: name: test-job-include-raw-{num} builders: - shell: !include-raw-escape include-raw001-hello-world.sh - shell: !include-raw-escape include-raw001-vars.sh - project: name: test-job-template-1 num: 1 jobs: - 'test-job-include-raw-{num}'contents of include-raw001-hello-world.sh:
#!/bin/bash # # Sample script showing how the yaml include-raw tag can be used # to inline scripts that are maintained outside of the jenkins # job yaml configuration. echo "hello world" exit 0contents of include-raw001-vars.sh:
#!/bin/bash # # sample script to check that brackets aren't escaped # when using the include-raw application yaml tag VAR1="hello" VAR2="world" VAR3="${VAR1} ${VAR2}" [[ -n "${VAR3}" ]] && { # this next section is executed as one echo "${VAR3}" exit 0 }
The bulk of the job definitions come from the following modules.
The jenkins job builder modules are executed in sequence.