Quality Assurance

Industrial process jocks maintain a subtle difference between the phrases “quality control” (QA) and “quality assurance” (QC). I believe that what syqada does automatically should be called quality assurance, and QC tasks embedded in a workflow should be called quality control. This section discusses the specification and use of syqada’s automatic QA.

The protocol (or task) file can define a QA behavior for the results of a task whose jobs have all completed with successful return codes. The default behavior, output, simply checks that every job wrote a file into the batchroot’s output directory. Others perform more rigorous tests. The exec behavior requires an installation that admits it is OK with people running arbitrary code. I’m trying to figure out how this could be used for harm rather than good; I guess it would involve an unscrupulous administrator writing a program to harvest user information and planting it in a workflow somewhere. Such an administrator would probably be capable of tampering with the installation as well, so let’s think about whether this persnicketiness is worthwhile.

Below are listed the possible standard QA options. More than one of these can be run by joining them with ampersand, e.g.:

qa = match word character & exec /my/fully/specified/QA_program

If more than one qa is specified, QA will terminate upon the first failure.

QA Options

qa = none

Perform no checks on output

qa = output

Check that every job wrote a file into the batchroot’s output directory

qa = count

Check that every job produced the same number of files in the output directory

qa = non_empty

Check that every job wrote at least a byte to the same number of files in the output directory

qa = match [words | characters | lines] ...

Check that the lexically first output file of each job produced the same number of words, characters, or lines (or combinations of more than one of those)

qa = exec QA_program

Run QA_program with each job’s lexically first output file as input and expect 0 return code from the output of all jobs

Command Line Usage

To override QA, the option –noQA may be given on the command line. Performing additional QA over what is specified in the protocol can be accomplished on a completed step using syqada manage:

syqada manage batch-directory --qa match=lines