YARN(1)

NAME
yarn - scenario testing of Unix command line tools

SYNOPSIS

DESCRIPTION
yarn is a scenario testing tool: you write a scenario describing how a user uses your software and what should happen, and express, using very lightweight syntax, the scenario in such a way that it can be tested automatically. The scenario has a simple, but strict structure:

    GIVEN some setup for the test
    WHEN thing that is to be tested happens
    THEN the post-conditions must be true

As an example, consider a very short test scenario for verifying that a backup program works, at least for one simple case.

    SCENARIO backups can be restored
    GIVEN some live data in a directory
    AND an empty backup repository
    WHEN a backup is made
    THEN the data case be restored
    FINALLY cleanup

Note the addition of AND: you can have multiple GIVEN, WHEN, and THEN statements. The AND keyword makes the text be more readable. SCENARIO is also necessary, and gives the title.

FINALLY is for cleanups. The FINALLY steps will be run regardless of whether the scenario succeeds or not.

Scenarios are meant to be written in somewhat human readable language. However, they are not free form text. In addition to the GIVEN/WHEN/THEN structure, the text for each of the steps needs a computer-executable implementation. This is done by using IMPLEMENTS. The backup scenario from above might be implemented as follows:

    IMPLEMENTS GIVEN some live data in a directory
    rm -rf "$TESTDIR/data"
    mkdir "$TESTDIR/data"
    echo foo > "$TESTDIR/data/foo"

    IMPLEMENTS GIVEN an empty backup repository
    rm -rf "$TESTDIR/repo"
    mkdir "$TESTDIR/repo"

    IMPLEMENTS WHEN a backup is made
    backup-program -r "$TESTDIR/repo" "$TESTDIR/data"

    IMPLEMENTS THEN the data can be restored
    mkdir "$TESTDIR/restored"
    restore-program -r "$TESTDIR/repo" "$TESTDIR/restored"
    diff -rq "$TESTDIR/data" "$TESTDIR/restored"

    IMPLEMENTS FINALLY cleanup
    echo nothing to do, actually

Each "IMPLEMENTS GIVEN" (or WHEN, THEN, FINALLY) is followed by a regular expression on the same line, and then a shell script that gets executed to implement any step that matches the regular expression. The implementation can extract data from the match as well: for example, the regular expression might allow a file size to be specified.

The above example is a bit silly, of course: why go to the effort to obfuscate the various steps? The answer is that the various steps, implemented using IMPLEMENTS, can be combined in many ways, to test different aspects of the program being tested.

Moreover, by making the step descriptions be human language text, matched by regular expressions, most of the test can hopefully be written, and understood, by non-programmers. Someone who understands what a program should do, could write tests to verify its behaviour. The implementations of the various steps need to be implemented by a programmer, but given a well-designed set of steps, with enough flexibility in their implementation, that quite a good test suite can be written.

The shell commands in an IMPLEMENTS section are run in the directory in which the user ran yarn. The environment variable SRCDIR is set to the fully qualified path to that directory.

OPTIONS

ENVIRONMENT

DATADIR
    Fully qualified pathname to a temporary directory, in which the tests can use files. The temporary directory is removed at the end of the test execution, unless the user specifies otherwise with --snapshot.

SRCDIR
    Fully qualitifed pathname to the directory in which the user ran yarn. This is useful when the tests want to change the directory.

EXAMPLE
To run yarn on all the scenarios in your current directory:

    yarn *.scenario

All the files will be treated together as if they had been one file.

To add a shell library to be included when running any IMPLEMENTS section:

    yarn --shell-library mylib.sh *.scenario

You can repeat --shell-library as many times as necessary.

SEE ALSO
cmdtest(1), cliapp(5).

The README.yarn file has more details on the scenario testing language.