obriencj

is a pile of bits

Python, Sphinx, and GitHUB

This fine Sunday was spent in a coffee shop, trying to determine how I wanted to generate the API documentation for brine.

Of course there is no doc utility provided as part of Python proper. There’s pydoc for the command-line, but nothing that will extract static html for easy reference online, etc. Other tools have spun into being to fill this void – and each project must pick their favorite when it comes time to ship docs.

Enter the contenders

I began with epydoc. It was easy enough to setup, but I ran afoul with some odd behavior almost immediately – it insisted on documenting the abc.ABCMeta class as though it were a part of my project. I wasted almost an hour trying to make the --external-api options work to link to the abc module in its proper place, but to no avail. On top of this, I noticed annoyingly that the option to disable timestamps in the generated pages never made it into a release, though it was committed in 2008!

Next up was pdoc. pdoc is gloriously simple, designed to be a zero-configuration utility. In this it exceeds; it’s basically a static HTML version of the pydoc output. My issue was that I didn’t see a way to embed more documentation (examples, explanations, etc) in the output without putting it into the docstring of the main module. Indeed the pdoc project page is itself generated by pdoc, with its lengthy expository body embedded entirely in one enormous docstring 1. I also couldn’t get it to filter out the _cellwork utility module while still loading the barrel module. Frustratingly it was either all submodules or no submodules.

Finally I took a look at Sphinx. pdoc and Sphinx are in essence polar opposites. Pdoc will generate the API page for a module, and do nothing else. Sphinx will do just about anything at all as part of the documentation generation – API page creation is just one of its many extensions. Above all, Sphinx is a reST (reStructuredText) processor. As a massive convenience Sphinx offers a sphinx-apidoc quickstart that will generate the base setup for building API pages. It took only a little while before I had nice looking docs being created. A bit more hacking and I’d added targets to convert my existing README.md into an overview.rst for inclusion into my blossoming doc tree.

In the end I settled on sphinx. Out of the three tools I tried, it was the only one that got me to where I wanted to go. After just a bit of quality coffee time I had the expository prose and the correct subset of modules presented nicely.

Hosting the docs

GitHub Pages (neĆ© github.io) offers gratis per-project hosting. Simply create a gh-pages branch in your project’s repository; anything pushed on that branch will show up at your project’s URL. This blog is hosted on GitHub Pages, so it seems sensible to host the docs for my projects here as well.

Easy mode

The previously mentioned sphinx-apidoc quickstart provided me with a Makefile. I hacked together a deploy target for it that operates very similarly to the one in Octopress. To whit, it generates the documentation in the format of my choosing (dirhmtl in this case) and then pushes it into the gh-pages branch of my repository. From there it’s served up for the world to see. You can see my revisions here.


  1. I believe this is working as designed from the pdoc philosophy, and I can get behind that to some extent. But I think there are limits to just how much text you can sensibly cram into a docstring.