New program documentation and test

For newly created programs, some documentation and test work items should be created, to aid the end-user and future program maintainers. In creating these docs and tests, it is best to take off your 'developer' hat, and put on your 'end-user' hat, followed by your 'maintainer' hat, to gain the view point of someone who has never seen or worked with your program. Your program is new to the end-user or maintainer, and nothing should be assumed about what they know about how it works.

There are three main work items to create: end-user documentation, maintainer documentation, and test code.

End-user documentation

The primary (and typically only) end-user documentation to create is text information like what you see in a unix 'man' page. In freesurfer, as of Feb 2011, the code in the dev tree and v5.1 branch supports putting 'man page'-like information in an xml file, which is then embedded into the .c/.c++/csh program and printed for you in a nice format (also supports automatic creation of a wiki page and pdf doc, TBD). Detailed instructions on how to do this are here. To see some nice examples of this help text output, type:

mri_convert --help | less

or for a simpler example:

mris_inflate --help

But first, to keep things simple, ignoring those instructions and XML for the moment, using your favorite text editor, create a text file containing the follow sections:

Name

Simply the name of your program and a very short description (ie. mri_convert - program to convert volume file formats)

Synopsis

The command-line calling sequence (ie. mri_convert [options] <in vol> <out vol>)

Description

Verbose description of the wonders of your program.

Positional Arguments

If any, arguments that must be included, without flags, typically an input vol and and output vol. Say 'None' otherwise.

Required Flagged Arguments

Flagged args like --input, if thats what you have chosen instead of positional args. 'None' otherwise.

Optional Flagged Arguments

Flagged args, but optional. Note that the XML format allows including additional help text to be associated with an arg. For example, see the <intro>, <argument> and <explanation> tags of the 'Applying Transforms' section of mri_convert.help.xml. 'None' if empty.

Example(s)

The more the merrier.

Reference(s)

If you have a paper associated with this program, include a reference here. While not necessarily useful for command-line help usage, obviously having a larger context for your program is a big help.

Reporting

Put this text: Report bugs to <freesurfer@nmr.mgh.harvard.edu>

Maintainer documentation

Imagine that a big, steaming pile of source code has been plopped on your desk, and you've been told 'it doesnt work, fix it'. This pile is entirely new to you, and hopefully this excites you, despite the fact that the original author is no longer around and doesnt respond to emails. Of course the first thing you do is read the .help.xml that is associated with it. Now that you know what the program is supposed to do, you open what appears to be the main .c file, and become even more excited to discover that it has been fully documented (in doxygen format). The file header contains a brief one-line description, followed by a longer description and maybe a reference, all of which you recognize from the .help.xml file.

But when you open the other .c files, you see documentation in the header about what that module is doing, and that fits-in with what you know about the program overall. Also, there are addtional notes targetted at the maintainer, such as assumptions made, globals used, problems encountered, things not obvious, TODOs, HACKs, etc. Within the code, each function has a doxygen formatted header, or at the very least some description of what the heck the function is supposed to take as input and supply as output. And of course variable names and function names are meaningful, and not painfully short or obscure.

There might even be functions which test the routines in this file, which provide an alternative form of 'documentation' because they show what is expected. There might even be a developer_notes.txt file that contains helpful developer info, as is the case with the scuba app. Or an umbrello.xmi file, loadable by the umbrello UML modeller (v1.5.61 or later) to view the relationship between classes (see qdecproject/umbrello.xmi for an example).

In no time at all, the verbosity of this text, gifted by the developer of the code to you the maintainer, leads you to the source of problem, and the pile of code retains its status as a glistening glorious tower of working code.

Testing

In a nutshell, to be included with each program is at least one script, named something like test_mri_convert, which when run will execute your program in the way it is typically used, using example input data, and produces output which is then compared against existing known-good output data. The script returns value 0 for pass, non-zero for fail. See the script dev/mri_convert/test_mri_convert for an example of this. This script is called during the 'make check' call in the nightly build. See the targets check_PROGRAMS and TESTS in dev/mri_convert/Makefile.am for an example. Also note the foo targets, which are an unfortunate necessary hack to get the 'make dist' target to work (the source code tarball bundle).

There can of course be multiple test programs. For example, see the test suite for scuba, which is a GUI app. It has a test program associated with each module. This is in-lieu of actually running scuba, as it can difficult (but not impossible) to create an automated test for a gui app.

Ideally the tests should not take much more than 20 minutes. If it does take longer (as is the case with test_mri_robust_register), then merely comment-out the TESTS section. While it may not run nightly, at least it is there for a maintainer or the developer to run when they think something is broken.

Author: Nick Schmansky