Sunday, August 5, 2007

doc authoring system

When I work on a project, I generally try to write documentation in parallel with the code, even for projects that I don't intend to open source. Firstly, having that resource available frees up my mental bandwidth for thinking about the current problem I'm trying to solve. Secondly, it's a substitute (some might say a bad substitute) for certain kinds of design documentation, although I try hard to write docs with the end user in mind. And third, it's a way to sanity check ideas, such as when I'm designing some protocol. Having that stuff written down helps me see the big picture for what I'm trying to do, and providing it right away in the reference manual seems beneficial.

A while ago, I settled on DocBook/XSL as my authoring system. I use an extension of DocBook that I wrote which makes my personal style of structuring a CL project reference guide easier to build. It also provides some conveniences for the particular way that the syntax of CL functions/macros/etc are shown. The extensions are a set of XSL templates implementing some higher-level constructs that go along with the DocBook stylesheets; these get transformed into the DocBook vocabulary for later processing.

Here's an example of the help source for a function called MAKE-POINT in a package called LPC, using the extension vocabulary that I mentioned above:
<function name="make-point">
<argument name="coords">
One of:
<itemizedlist mark="bullet" spacing="compact">
A <refclhs>list</refclhs> of alternating x/y coordinate values.
An <refclhs>integer</refclhs> x coordinate value.
<notarg name="&optional"/>
<argument name="y-coord">
An <refclhs>integer</refclhs> y coordinate value.
This function returns a <refclhs>vector</refclhs> representing one or
more points.
It's possible to mix tags from my extension vocabulary (e.g., function, syntax,
reftopic) with DocBook/XSL tags (e.g., itemizedlist, listitem).

This is what the end result looks like:

The output format is HTML Help, so the above screenshot shows part of the help viewer (it doesn't show the search/index tab nor the toolbar). My development platform is Windows XP. Not that there is anything wrong with portable solutions, and you know, raw HTML is often as good a choice as anything else. But HTML Help has features that I want, it's fairly customizable, and I can't think of many situations where a Windows user wouldn't have the viewer. Underlying this is the fact that I generally assume a Windows-based development environment for my work.

One future improvement I'd like to explore some day is to extract CL doc strings from the code and merge that in with other help content. I hear the SBCL developers do something along these lines to generate their manual, one difference being that I believe they are using texinfo or at least migrating in that direction, so I ought to take a look at what they do.


michaelw said...

DOCUMENTATION-TEMPLATE might have all the code you need to extract docstrings and function signatures. I also have a local patch which pipes the docstrings through cl-markdown, if available. WFM, YMMV.

Jack Unrue said...

Thanks for the pointer. Yeah, it seems like a lot of that could be useful, just not the raw HTML output.

David Lichteblau said...

In atdoc , I have also used a lot of XSLT. Lisp is only responsible for the docstring extraction, everything else is done using xsltproc. It generates HTML directly though, without going through docbook.

Anonymous said...

Perhaps you might be interested in


Anonymous said...

well, maybe i'm old-fashioned but i prefer documented code over undocumented code with a separate documentation.

Jack Unrue said...


There is no reason one can't have both. And I didn't say that I leave my code undocumented :-)

Jack Unrue said...

Hi David,

I'll put atdoc on my list of stuff to investigate further, thanks.

I use xsltproc as well (I forget if the binary I have is from a MinGW port or GnuWin32 or something else).