-Test(1) -Test(1) NAME -Test - run tests SYNOPSIS -Test [ file-name... ][ option... ] -Test -INDependent [ file-name... ][ option... ] -Test -List [ option... ] -Test -Help DESCRIPTION The -Test command is used to run tests. If no files are named(5,8), all relevant tests are run. By default both automatic and manual tests are run. You may name directories on the command line, and all relevant tests in(1,8) that directory tree in(1,8) the change will be run. It is an error(8,n) if(3,n) there are no relevant tests. Each architecture must be tested separately. This is because there may be subtle problems that are only revealed on some architectures. Some projects may also have different code for different architectures. The status of the last test run is remembered so that tests are not run if(3,n) there is no need. (This does not apply to -REGression tests, unfor- tunately.) Tests must be re-run if(3,n) the test previously failed, if(3,n) the test file(1,n) has changed, if(3,n) there has been a build, and for each archi- tecture. TEST COMMAND CONFIGURATION The command used to execute tests is defined by the test_command field in(1,8) the project configuration file(1,n) (see aepconf(5) for more informa- tion), this defaults to using the Bourne shell if(3,n) not set. The current directory will be the top of the appropriate directory tree. If tests require temporary files, they should create them in(1,8) /tmp, as a test cannot expect to have write(1,2) permission in(1,8) the current directory. If you want to use a more sophisticated test engine, rather than a sim- ple shell script, but this test engine does not return result codes suitable for use with , you could wrap it in(1,8) a shell script which re- writes the exit(3,n,1 builtins) status into the values expects. You could also achieve the same results by writing a more complex test_command in(1,8) the project config(1,5) file. It is also possible to write(1,2) test commands which are able to test more than one file(1,n) at once. This is controlled by the batch_test_command field of the project config(1,5) file. In this case, the ${output} substi- tution indicates the name of a file(1,n) the test command must create, in(1,8) aetest(5) format, to contain the results of the tests run. This is often used on systems with multiple CPUs or the ability to distribute jobs across several computers on a network. Substitutions All of the aesub(1,5)(5) substitutions are available in(1,8) the test commands. Some of them are of particular note: ARCHitecture This substitution is replaced by the name of the architecture to be tested. Search_Path This substitution is replaced by a colon separated list of absolute paths to search when looking for test support files. Search_Path_Executable This substitution is replaced by a colon separated list of absolute paths to search when looking for executable support files (library files and sub-commands). Most of the time(1,2,n) $Search_Path_Executable are exactly the same. How- ever, during ``aegis -t -bl'' they will be different, with $Seach_Path starting at the development directory (the test being run) and $Seach_Path_Executable starting at the baseline (the executable being run). Test Result Codes As each test is run (via the test_command field in(1,8) the project config(1,5) file(1,n)), determines whether the test succeeded or failed by looking at its exit(3,n,1 builtins) status. This exit(3,n,1 builtins) status is mostly as expected for UNIX commands. Success A test should exit(3,n,1 builtins) 0 to indicate success, i.e. that the specific function under test worked as expected. Failure A test should exit(3,n,1 builtins) 1 to indicate failure, i.e. that the specific function under test did not work as expected. No Result A test should exit(3,n,1 builtins) 2 to indicate no result, i.e. that the specific function under test could not be exercised because something else went wrong. For example, running out of disk space when creating the test input files in(1,8) the /tmp directory. Actually, any exit(3,n,1 builtins) code other than 0 or 1 will be interpreted as ``no result''. However, always using 0, 1 or 2 means that if(3,n) a new result code is required by a later release of , your existing tests will con- tinue to work. OPTIONS The following options are understood: -FOrce This option may be used to specify that all tests should be run, even if(3,n) the status of the last test run indicates that there is no need to run a specific test. -PErsevere This option may be used to specify that all tests should be run, even if(3,n) some fail. Defaults to the user's persevere_pref- erence if(3,n) not specified, see aeuconf(5) for more information. -No_PErsevere This option may be used to specify that the test run should stop after the first failure. Defaults to the user's perse- vere_preference if(3,n) not specified, see aeuconf(5) for more information. -PROGress This option may be used to specify that progress messages should be issued before each test run or before each batch test run in(1,8) case batch_test_command field specified in(1,8) project con- fig file(1,n) (see aeuconf(5) for more information). -No_PROGress This option may be used to specify that progress messages should be suppressed. This is the default. -SUGgest [ number ] The `` -Integrate_Pass'' command collects test correlation sta- tistics when changes are integrated. This option may be used to request that suggest which tests should be run, using these testing correlations. If no number is specified, 10 tests will be suggested. This option implies the -REGression option. -SUGgest_Noise number This option may be used to control the amount of noise injected into the test selection performed by the -SUGgest option. The number is a percentage of noise to be injected. Defaults to 10 if(3,n) not specified. The injection of noise ensures that a vari- ety of tests are run on subsequent runs, and also some from left-field as a sanity check. RECOMMENDED ALIAS The recommended alias for this command is csh% alias aet ' -t \!* -v' sh$ aet(){ -t "$@" -v} ERRORS It is an error(8,n) if(3,n) the change is not in(1,8) one of the being developed or being integrated states. It is an error(8,n) if(3,n) the change is not assigned to the current user. It is an error(8,n) if(3,n) your have no relevant tests and no relevant exemp- tion. SEE ALSO aeb(1) build a change aeca(1) modify the attributes of a change aedb(1) begin development of a change aeib(1) begin integration of a change aent(1) add a new test to a change aecp(1) copy an existing test into a change aepconf(5) project configuration file(1,n) format aeuconf(5) user configuration file(1,n) format Reference Manual -Test(1)