Script Based Unit Testing

Leeland's picture

== ShUnit ==

:Target: All POSIX (Linux/BSD/UNIX-like OSes)
:URL: http://shunit.sourceforge.net/
:Latest Version: 1.3 (15 Jan 2005)

:A unit testing framework for the Bourne shell family of shells modeled after JUnit. This has not been updated since 2005 and I think the project is defunct.

== shUnit2 ==

:Target: All POSIX (Linux/BSD/UNIX-like OSes)
:Latest Version: 2.1.1 (13 Jul 2007)
:URL: http://shunit2.sourceforge.net/
:Documentation: http://shunit2.svn.sourceforge.net/svnroot/shunit2/trunk/source/2.1/doc/...

:shUnit2 is a xUnit based unit testing framework for shell scripts (eg. sh, bash) that is modeled after the JUnit framework. It is designed to make unit testing in shell as simple as possible.

''This is the one I am using right now and has a very active mailing list and development support.''

=== Frame Work ===

Beginning in August of 2007 I began researching into creating a unit test framework to support the Install Tools Team work. As almost all of the Install Tools are one form of script or another we needed a testing set up that was concentrated on scripts. I chose to use mostly KSH based scripts. Although everything runs in BASH scripts by sticking to KSH only syntax I figure we will have the ability to support more platforms.

I created a posix style directory structure in the IS-1.6 starting at '''opt/test'''.

==== test/README.txt ====
The testing sets in this directory are for doing unit and system testing on the Install System Tools and installation environments for Qpass.

These tests are intended only for use by the Install Tools Team members or directly by the Install Tools themselves. These tests should NOT be run by hand unless directed to do so by a team member.

NONE of these tests are guaranteed to produce reliable results except in specific environments or under very controlled conditions. You are welcome to examine them for your own purposes but without any promise of support. They are included with the Install Tools as some of the tools call on specific tests at various points, or because they might be needed for diagnosing specific issues, or for continuous integration support.

The tests are broken up into logical sub-sets to allow running individual tests or sections of tests individually and to keep the more dangerous tests in their own containment areas.

The categories are:

; light safe tests (bin-light-safe) : small unit tests which exercise individual script(s)/function(s) ON THE LOCAL MACHINE that complete within a few seconds (less than 3 seconds per test), require little CPU/IO resources and that DO NOT ALTER any files or directories.

; light destructive tests (bin-light-destructive) : small unit tests which exercise individual script(s)/function(s) ON THE LOCAL MACHINE that complete within a few seconds (less than 3 seconds per test), require little CPU/IO resources and that DO ALTER/DELETE/CREATE files and/or directories.

; heavy safe tests (bin-heavy-safe) : larger or complex unit tests which exercise individual or compound scripts or functions ON THE LOCAL OR REMOTE HOSTS which may require noticeably lengthy amounts of time or CPU/IO resources and that DO NOT ALTER any files or directories.

; heavy destructive tests (bin-heavy-destructive) : larger or complex unit tests which exercise individual or compound scripts or functions ON THE LOCAL OR REMOTE HOSTS which may require noticeably lengthy amounts of time or CPU/IO resources and that DO ALTER/DELETE/CREATE files and/or directories.

Control scripts that run testing suites and helper scripts are in the bin directory and library functions are in the lib directory.

Remember Test-Driven Design is NOT about writing tests it is about better design. These tests should be used to help enforce the DRY principle (http://en.wikipedia.org/wiki/Don't_repeat_yourself).

- The Qpass Install Tools Team

==== Conventions ====

# Test scripts are named based off the category and the name of what they are testing. Tests of library includes
# Recommended to write one test script per major function.
# Recommended to write one over wrapper script for an application or library.
# If possible write tests to cross check for structural problems as well as general results. For example if a particular variable is being referenced in a sub-script and gets pushed up to a higher level for any reason include a test that checks to make sure that variable is not being referenced in the library or application script to keep future enhancements from accidentally referencing it again.
# Follow TDD (Test Driven Design http://c2.com/cgi/wiki?TestDrivenDevelopment) principles:
## '''BEFORE ALTERING CODE''' write the test(s) to demonstrate the function is implemented properly or the bug is fixed
## Confirm the test(s) '''FAILS'''
## Check in test(s) to the version control system
## Get a quick peer code review of the test(s) '''BEFORE ALTERING CODE'''
##* If any changes come from the review check in the updated test(s) to the version control system
##* Using version control you can actually work on updating the code before the test review as you can revert the changes at any time to demonstrate the testing at a pre-alteration state (don't forget to make a copy of the changes before doing this).
## Do the work needed to get the test to pass
## Once code is passing the test(s) check in changes to the version control system
## Get a complete peer code review of the changes and test(s)
##* If any changes come from the review check in the changes to the version control system
## Refactor if needed
##* If any changes are made check in the changes to the version control system

==== Sample Unit Test Script ====

'''NOTES:'''
# Notice that the test script exits with exit $__shunit_testsFailed. This is important so that larger wrapper scripts can test the return results and know if any tests failed within the function. An exit status of zero is both a script true and means no failures.
# There should be a test for the initial function design that confirms the function is doing its job. But any bug should have one or more specific sets of functions to explicitly exercise the bug and thus to confirm that the bug does not come back.

#! /bin/sh

#-----------------------------------------------------------------------------
# ENV Setup stuff
#
# Locate the test base directory
if [ -z "$TEST_HOME" ] ; then
    TEST_HOME="$(cd ${0%/*} ; pwd)"
    TEST_HOME="${TEST_HOME%%test/*}test"
fi
# Locate the IS base directory
if [ -z "$IS_TOOLS_HOME" ] ; then
    IS_TOOLS_HOME="$(cd ${0%/*} ; pwd)"
    IS_TOOLS_HOME="${IS_TOOLS_HOME%%opt/*}"
fi

# Script containing the function(s) being tested
TARGET_SCRIPT=${IS_TOOLS_HOME}opt/bin/dmRenderBuddy
SAFE_WORK_DIR=/tmp/${0##*/}

[ ! -e $TARGET_SCRIPT ] && echo "${0##*/} : Failed target script ($TARGET_SCRIPT) does not exist" && return 1

echo "TARGET_SCRIPT=$TARGET_SCRIPT"
echo "SAFE_WORK_DIR=$SAFE_WORK_DIR"

#-----------------------------------------------------------------------------
# Helper Functions
#

function  internal_Helper () {
}

#-----------------------------------------------------------------------------
# suite tests
#

function test_SOME_ITEM_MAIN_TEST () {
}

function test_SPECIFIC_BUG_IN_FUNCTION_BUGID () {
}

#-----------------------------------------------------------------------------
# suite functions
#

oneTimeSetUp()
{
  # clean up test dir
  rm -r $SAFE_WORK_DIR 2>/dev/null
  mkdir -p $SAFE_WORK_DIR
  RESULT=$?
  [ ${RESULT} -ne 0 ] && echo "problem  with command mkdir -p ${SAFE_WORK_DIR}" && exit 1
}

# load and run shUnit2
. ${TEST_HOME}/lib/sh/shunit2

exit $__shunit_testsFailed

BODY {
counter-reset: section; /* Create a section counter scope */
}
H2:before { /* negate the TOC H2 header to prevent accidental increment */
counter-increment: section; /* Add 1 to section */
content: "Section " counter(section) ". ";
}
#toc H2:before { /* put in the TOC H2 header formating */
counter-increment: section 0; /* stop section from incrementing here */
content: " "; /* Remove the section notation from the contents header */
}
H2 {
counter-reset: subsection1; /* Set subsection1 to 0 */
color: #800517 ; /* Firebrick */
}
#toc H2 { /* make the TOC H2 header formating right */
color: black ;
}
H3:before {
counter-increment: subsection1;
content: counter(section) "." counter(subsection1) " ";
}
H3 {
counter-reset: subsection2; /* Set subsection2 to 0 */
color: #306754 ; /* Medium Sea Green */
}
H4:before {
counter-increment: subsection2;
content: counter(section) "." counter(subsection1) "." counter(subsection2) " ";
}
H4 {
counter-reset: subsection3; /* Set subsection to 0 */
color: #AF7817; /* Dark Goldenrod */
}
H5:before {
counter-increment: subsection3;
content: counter(section) "." counter(subsection1) "." counter(subsection2) "." counter(subsection3) " ";
}

Thread Slivers eBook at Amazon