Model Documentation

I would like to try and address a few questions about model documentation. This is often said to be an important part of building models for Integrated Water Resources Management because there are usually a lot of people involved from various disciplines and backgrounds.

Typically you will see two different forms of documentation out there, including user documentation and programmer documentation. The user documentation is usually created to assist the end user interact with the input controls and view the results. The latter is usually there to help the user and/or programmer how the program actually works.

So, first and foremost: What is model documentation?

Model documentation is written text that accompanies the computer model. It either explains how it operates or how to use it or both, and may mean different things to people in different roles (Wikipedia, 2011).

Why do people think documentation is important?

The US Army Corps of Engineers (which happens to develop some very important software in the water resources modeling industry) says good documentation is needed and necessary. It should be assembled for every model and it should be done well (Johnson, 1982).

There are two main reasons why people might stop using a model:

• It is not trusted
• It is not needed

To address the first issue, you need someone qualified to review the model other than the primary author of the model. Often times, model documentation is critical for such a review.

While it is true that model documentation is important, we need to keep in mind that it will never replace the model itself. The first and foremost factor is a well designed and calibrated model that is shown to represent reality. It is also important to understand that good documentation will not create a need for the model if it is not already there.

Many others have noted the importance of model documentation.

"Documentation is not fun. It is hard, nasty and boring business, but this in no way makes it any less critical, particularly in a heavily user-oriented environment." (Brewer, 1976) This quote is important when considering modeling for an IWRM application since the environment in these situations is usually very user-oriented.

What constitutes good documentation?
The following information should be included in model documentation:

1. Abstract/Introduction/Background
2. The underlying methodology (theory)
3. Model limitations (and capabilities)
4. Data requirements
5. Input specifications (how is data put into the model?)
6. Summary of model output and any processing of results
7. Example application of the model

An interesting article I found in my research was written by Saul Gass called, "Documentation for a Model: A Hierarchical Approach" (Gass, 1981). While recognizing that good documentation is very difficult for a large-scale math model, it is critical for the legacy of the model. Documentation should be integral with the construction of the model. The costs, resources, review procedures, and milestones should be specified at the beginning of the modeling effort. The hierarchical approach involves development of documentation in 4 categories:

1. Operations (requirements for basic execution of the runs)
2. Use (Math theory, data requirements, and model processes)
3. Maintenance (modeling scenarios, maintenance log)
4. Assessment (assessment, applications, summary, history)

The final document should end up with all 4 features. This can be graphically represented as a bookcase with the 4 categories represented on each shelf of the bookcase.

Source: Gass, 1981

But we can't forget that one of the most important steps in developing good documentation is to first set up a plan for the organization of the documentation so you can incorporate the work into your schedule and budget.

What is preventing people from writing good documentation?

• Sufficient funds were never allocated for documentation in the first place
• It is difficult and time consuming to translate model logic and formulation to written descriptions
• The person writing the logic might not have the patience or ability to assist with documentation
• Poor documentation is more common than good documentation so people might be lacking a good example to draw from

I would love to hear comments from anyone out there. 

I would bet some other modelers out there have some experience they would like to share that would significantly add to this post. 

Thank you.

Bibliography:


Johnson, William. US Army Corps of Engineers. "Documentation Needs for Water Resources Models", 1982. Link

Wikipedia. "Software Documentation". 2011. http://en.wikipedia.org/wiki/Software_documentation

Brewer, G.D. Documentation: An Overview and Design Strategy. Simulation & Games 7, 3 (1976), 261-280.

Gass, S.I. Computer Model Documentation: A Review and An Approach. NBS Special Pub. 500-39, U.S. Gov. Printing Office, 1979.

Gass, S.I., "Documentation for a Model: A Hierarchical Approach". 1981

House, P.W., and McLeod, J. Large-Scale Models for Policy Evaluation. John Wiley & Sons, New York, 1977.