Integrating doxygen and confluence

October 19, 2010 4 Comments by Richard

FireBreath has a new website

This last week FireBreath has taken a big step — we have a new website at http://firebreath.org. The reasons for the move are varied, but what it comes down to is that Google Code’s wiki, while pretty decent for small projects, simply isn’t powerful enough to handle documentation for a project like this well.  Granted, it can do it, but not as well as other options.

Creating an integrated documentation system

My primary goal with this website is to provide a completely integrated documentation system for users of FireBreath that is easy for both me and others to update.  I am fairly happy with my solution so far which relies on Atlassian Confluence and Doxygen.  To find out why and how I chose this combination, as well as the steps I took to get here, read on.  Also, if you’re in a hurry and just want to see the end result, skip to the last couple sections of this post under “Doxygen/Confluence Nirvana” =]

Choosing Confluence

I should add a quick plug here for Atlassian Confluence. No, I don’t work for them, and previous to this I had only used their more well known solution JIRA, which is also not bad.  The main reason I had never really looked into them is that, well, I’m cheap, and those are commercial products.  However, as it turns out, Atlassian provides free licenses for open source projects!  Awesome!  It should also be mentioned that the commercial license for small organizations is pretty reasonable as well.

I installed it first just to test, and I did have to learn a bit of new wiki syntax; however, the syntax is easily understandable and powerful.  With the help of 4 or 5 vim search/replace regex strings, it probably took me all of an hour to convert 30+ wiki pages from google code format to confluence, and I can customize the theme and fonts!  When I found Gliffy (also free for open source projects) I was hooked and immediately applied for the open source license.  Less than 48 hours later I had it.

Creating doxygen docs

In a recent survey that we ask FireBreath users to fill out, many users mentioned frustration with the lack of good class documentation in FireBreath, and a few suggested something generated from Source Code.  After doing a little research, I chose Doxygen as the tool to use; in fact, it’s the only real C++ documentation generation tool I found.  It took a little learning, but soon I had class documentation generated from FireBreath source code and started adding documentation comments.

Writing good documentation for use with doxygen is difficult, but if you work at it you can come up with some pretty good stuff.  However, the index in doxygen isn’t quite what I wanted.

Integrating the doxygen docs in confluence

After I tweaked the stylesheet to make it more readable, I found that there was still something I didn’t like about the docs; we have lots of documentation written in our wiki, tutorials and explanations that don’t make sense to put in the code.  However, it’s in a completely different system from the class documentation generated by doxygen, so even if our docs link to each other, the experience is far from ideal!

We definitely need the docs integrated into the same system.

The iframe approach

The first thing I tried was embedding an iframe in a confluence wiki page.  This isn’t too hard; you just have to set up a user macro or enable the {iframe} macro.  I set a minimum height so that it was readable, and it wasn’t too bad.

The problem is, it doesn’t use the full available height, it has weird scroll bars, and different sized screens really throw off the look.

The {html-include} macro

The next thing I tried was enabling the html-include macro, adding classdocs.firebreath.org into the whitelist, and then using the following code:

{html-include:url=http://classdocs.firebreath.org/patched/classes.html}

This actually looked pretty good!  Everything was integrated, my class list was there!  I was pretty excited!  Well, at least until I tried clicking on a link.  All the links are relative, and html-include doesn’t rewrite the links.  So, naturally, none of them worked!  Also, since the <head> section of the page was missing, we had no stylesheet.

The missing stylesheet issue wasn’t that hard to fix; I simply copied the majority of the stylesheet from doxygen (which I had tweaked anyway to match the color scheme) and put it into a user macro.

<style>
/* contents of doxygen.css here */
</style>

I also added style tags to hide the tab navigation, since it looked just a bit out of place.

sed to the rescue!

So I now had class docs that I could put into a confluence page, but none of my links worked!  After a little thought, I came up with the following bash script:

#!/bin/bash
 
for fl in *.html; do
 mv $fl $fl.old
 sed 's/href="\([^#]\)/href="http:\/\/classdocs.firebreath.org\/\1/g' $fl.old &gt; $fl.stage2
 sed 's/http:\/\/classdocs.firebreath.org\/javascript:/javascript:/g' $fl.stage2 &gt; $fl.stage3
 sed 's/src="/src="http:\/\/classdocs.firebreath.org\//g' $fl.stage3 &gt; $fl
 rm -f $fl.old
 rm -f $fl.stage2
 rm -f $fl.stage3
done

This script processes all of the links in the page and adds an absolute path to each!  Fantastic, now my links work!

Except…. well, I still wasn’t happy with it.  There had to be a better way!  Sure, the links work, but when you click on it you leave the page!

Doxygen-Confluence nirvana

I looked at a lot of options.  I downloaded the Atlassian plugin framework and looked at writing a plugin to help me.  I emailed Atlassian support.  I looked at a lot of things.  Finally, I gave up and broke out the python.  I have attached the resulting script.  Here is what it does:

Uses the following config options:

  • Input path to doxygen docs
  • Output path
  • Parent pageId for:
    • Classes
    • Structs
    • Namespaces
    • Files
  • Confluence URL
  • Username/password passed on the command line

Performs the following general steps:

  1. Expects that doxygen has been run and has generated docs/html and docs/xml directories
  2. Loads and parses docs/xml/index.xml
  3. Loop through all entries in the XML file
    1. Get the refid from the xml to figure out which HTML file exists for that entry
    2. Generate a “readable name” for each entry (replace “::” with a space, since confluence doesn’t allow “:” in page names)
    3. Add an entry to the search/replace list to map the html filename to the new page name
  4. Process all .html files found; search for .html paths and replace then with the new page name path
  5. Loop through all found pages
    1. use XML-RPC to get the page from confluence
    2. If none was found, create a new page
    3. Set the parent of the page (see config options above)
    4. Set the content of the page with:
      • {doxygen_init} – injects the CSS
      • {html-include} – with a web path to the output .html files
    5. Store the page in Confluence
  6. Remove all pages that are descendants of one of the root pages that weren’t added this session

And voila!  Confluence now has all of the doxygen pages in, and since we rewrote all of the links they all point to each other.

Clean-up

If you, like me, think that including every class in the project could be a bit overkill and only include the ones that have documentation, you may want to remove the .html files that doxygen generates for the “non-visible” classes.  These aren’t linked to, so you don’t normally see them, but they do show up in the .xml file. Since part of the check the script does is to see if the expected .html file exists, you can fix this by simply deleting all .html files in the directory that nothing links to.  The best way I have found to do this is with the following script:

#!/bin/bash
for fl in *.html; do
 grep $fl * &gt; /dev/null
 if  (( $? )); then
  rm -v $fl
 fi
done

The code

Attachment: doc2confluence.py.gz

Enjoy!