PeteSearch: Writing a BHO installer

Candle
Conceptually, installing an Internet Explorer plugin is very simple. You just have to copy your DLL onto the user's system, and then add some registry settings so IE will load it. In practice there's a lot of house-keeping involved to do it nicely, so users can easily uninstall and manage the add-on for example, or to check that the machine meets the minimum requirements. I wanted to use an existing framework that would make that easy, so I did some research.

NSIS was initially very attractive, since I'd used it in the past, and found it fairly user-friendly and robust. It produced a .exe though, and I've noticed that .msi packages seem to be a more modern approach, so I looked into alternatives. The most promising was WIX, an open-source system from Microsoft for taking an XML description of what needed to be installed, and turning that into a .msi.

The first hurdle was getting used to WIX's declarative approach to installation. The wxs scripts are more like makefiles, in that they declare which files need to be installed, along with conditions and dependencies, rather than asking for a procedural list of steps, as NSIS and more traditional installers do. WIX's way makes it easier to repair installations and lets an uninstaller be created from the same script.

I found a great tutorial by Gábor DEÁK JAHN, and pretty quickly I was able to set up a script that copied over my single DLL to a new folder in Program Files. To create an MSI installer, you take a wxs XML script, run the candle command to compile it, and then run light to package it into a .msi.

The really hard bit proved to be working out how to write to the registry. As I discussed earlier, the old way to set up the needed registry information was through a process called self-registration, where code within the DLL would be executed, and that was expected to write to the registry programmatically. These days, that approach is strongly discouraged within an installer, and instead you're expected to list the registry keys and values you're altering as part of the script.

This was tough, since PeteSearch relies on ATL to parse a proprietary format called .rgs to set up the required registry keys, and there's no easy way to translate it to the required wxs XML. I was hopeful that a tool that ships with WIX, called tallow, would capture the registry changes in the right form, but when I ran it with the -s option, it crashed with an exception. This seems to be a known bug, with no solution, so I was left to hand-translate the .rgs file.

Once I got the hang of it, this didn't take too long. The .rgs file is organized into a hierarchy of keys, each level separated by curly brackets, and at each level multiple keys can be set. Translating meant taking each key/value pair, and turning it into a full <registry> tag in the wix script. For example, this in the .rgs file:

HKCR
{
PeteSearch.PeteSearch.1 = s 'PeteSearch Class'
{
CLSID = s '{00e71626-0bef-11dc-8314-0800200c9a66}'
}

becomes the following two wix tags:

One thing that caught me out was that most of the .rgs entries were writing to the default member of the key, rather than to a named member as I initially assumed. The only one that was different was the ThreadingModel value, which was written to a named member of InprocServer32. The only non-constant value I had to write was the InprocServer32 default, which specifies the location of the dll, which I was able to reference as [INSTALLDIR]PeteSearch.dll

Once I'd converted over the registry settings, I had a functional .msi that would install the dll. The only thing left was to make sure the minimum requirements were checked before the installation went ahead. PeteSearch doesn't work on anything earlier than Windows 2000, and only runs on IE7, so after some experimentation I was able to add the following two condition tags as direct children of the <product> tag in my script to enforce that:

There's a couple of gotchas I ran into; first the condition text has to be guarded in a CDATA block, or the angle brackets get interpreted as part of the XML. Secondly, when checking the IE version, I had to enclose the constant I was comparing against in quotes, or the comparison always failed.

Putting this all together was enough to create a fully functional installer. The wxs script is available through Sourceforge, or you can download it directly here. There's no UI apart from a progress dialog, I hope to sort out a basic license and confirmation screen in the future, to give more user feedback, and I'll cover that when I do.

More posts on porting Firefox add-ons to IE