19 July 2008
Custom Tools for Visual Studio
Possibly the coolest thing I've done so far as a programmer is to produce a Visual Studio CustomTool (think plugin) that allows source files from a custom language to be compiled into a C# project. The need for such a beast arose when i was porting an old database system, from a platform called CRS (think MS Access, but older and much more powerful) to C#. There were 100k+ lines of code that needed converting, so an automated tool seemed faster and saner than doing it all by hand.
The converter was written in Python courtesy of the pyparsing module, and i initially went about trying to find a way to automatically run it from the Visual Studio build process. Hooking into the MSBuild process didn't seem all that appealing, as it required messing around with the build (project) files, or creating a butt-load of .map files, as far as i could gather. At some stage during my googling, I stumbled across this mysterious CustomTool concept, which promised not just CRS code files compiled inside my C# project, but also a technique that would work across all Visual Studio editions (including express, so i could work from home).
Custom tools work by generating a module based on some arbitrary source file. The module is regenerated each time the source file changes. You can see custom tools in action with strongly typed data sets, and resource files... they generate the .Designer.cs files you see in the Solution Explorer tree.
Apparently creating a custom tool is easy, and it sure would be if there was actually any thorough and coherent documentation on the process. I've struggled through many narrow articles, broken links and sample code that could have been documented a lot better, all tied together with some heavy googling. I'm hoping this article will provide a little more breadth on the topic.
To get your custom tool working, there are only a few things you need to do:
Implement IVsSingleFileGenerator
Implementing IVsSingleFileGenerator interface should be easy, since there are only 2 methods in it. Of course the MSDN reference is typically scant and not much help on its own. And if you want to read it from your local MSDN installation, you have to first install the Visual Studio SDK for your version of Visual Studio. In fact it's a good idea to install this, as it has a sample custom tool project with some useful wrapper classes.
From the SDK's "SingleFileGenerator" sample project, copy BaseCodeGenerator.cs into your tool project and inherit from it. It converts the IVsSingleFileGenerator interface into something much simpler to implement. You could probably figure out how do write your own BaseCodeGenerator equivalent, but MS has already done it for you, so why reinvent the wheel? Many articles on the net will tell you that you should derive your tool class from BaseCodeGeneratorWithSite.cs, but unless you need the extra features it reveals, there's no need. BaseCodeGenerator requires your tool project to reference Microsoft.VisualStudio.Shell and Microsoft.VisualStudio.Shell.Interop. BaseCodeGeneratorWithSite requires a heap more references, good luck with that.
As a side note, there seems to be a bit of history to this BaseCodeGenerator/WithSite class set that makes me think Microsoft doesn't want us writing our own custom tools. Apparently it was included in the .NET 1.1 libraries, but made internal for 2.0+. At the time it was made internal, Microsoft released the code for it to the GotDotNet website, but later obliterated when they replaced the whole site with CodePlex late in 2007. Most articles I came across still reference the GotDotNet version, which now redirects to a lame CodePlex apology page. So now the only place to get these classes seems to be the SDK... makes sense, but what a run-around it's been hunting it down.
Register for COM
Now, merely implementing the interface is not enough. Because custom tools operate through COM, we need to make our tool play nicely with COM. Firstly, your class must have it's own GUID, which can be generated with the uuidgen command line tool that comes with Visual Studio, or from the IDE's Tools menu, neither of which seem to be available in the Express edition of C#. The GUID is associated with your tool class via an attribute, eg:
Once you have compiled your tool into a dll, there are a ton of ways to register your custom tool for COM. The easiest way is to have Visual Studio do it for you during the dll compilation: check "Register fo COM interop" under project properties | Build. This is equivalent to running "regasm /codebase", which means the dll doesn't have to be in a pathed directory, but if you later move or delete the dll, Visual Studio won't be able to find it.
Alternatively, you could copy your dll into the global assembly cache (GAC), either with gacutil or just copying into c:\windows\assembly. But to do that, your dll assembly needs a "strong name", ie it needs to be signed. Check "Sign the assembly" under project properties | Signing, then generate a new key file from the dropdown box underneath.
But I didn't do any of these (except for signing the assembly - now I can't remember if I actually needed to). Instead, I generated the registry entries required for COM registration:
The reason I used a regfile instead of just calling regasm in the post-built event is because regasm is not in a nicely-defined folder that works across all Visual Studio editions. It doesn't even come with 2005 Express.
Register with Visual Studio
This needs a few registry entries:
Once again I put thes registry entries in my project reg file, which is merged in the post-build event discussed above.
Actually, Daniel Cazzulino describes a really nice way to have these registry entries generated by regasm. I haven't done this, but it might make it worthwhile using regasm or the "Register for COM Interop" property, and ditching the reg file. Yeat another option, sigh.
Use it!
If you've set your tool project up like me, all you have to do to "install" it is compile the project. Enter the tool name in a source file's CustomTool property and watch the magic happen!
Other Bits
Your tool can generate warning and error messages that appear in the Error List, and even goto the source when double clicked. Call GeneratorError(0,msg,line-1,column-1) or GenerateWarning(...). Note that if either of line/column aren't valid, the double click feature won't work. Note these parameters are 0-based indexes, but are displayed (correctly) as 1-based index in the Error List.
You can even automatically associate your custom tool with a particular file extension.
Summary
So there you go, a cenceptually-simple process, made painful by the combinatorial explosion of options at each step of the journey, and documentation scattered around the tubes in blogs like this :)
My focus has been compile-to-install (then ditch the source project if need be), which is why I've chosen to do some step differently to others. But hopefully by now you'll have a better understanding of all the options, and can make an informed choice as to how you construct and deploy your next custom tool.
The converter was written in Python courtesy of the pyparsing module, and i initially went about trying to find a way to automatically run it from the Visual Studio build process. Hooking into the MSBuild process didn't seem all that appealing, as it required messing around with the build (project) files, or creating a butt-load of .map files, as far as i could gather. At some stage during my googling, I stumbled across this mysterious CustomTool concept, which promised not just CRS code files compiled inside my C# project, but also a technique that would work across all Visual Studio editions (including express, so i could work from home).
Custom tools work by generating a module based on some arbitrary source file. The module is regenerated each time the source file changes. You can see custom tools in action with strongly typed data sets, and resource files... they generate the .Designer.cs files you see in the Solution Explorer tree.
Apparently creating a custom tool is easy, and it sure would be if there was actually any thorough and coherent documentation on the process. I've struggled through many narrow articles, broken links and sample code that could have been documented a lot better, all tied together with some heavy googling. I'm hoping this article will provide a little more breadth on the topic.
To get your custom tool working, there are only a few things you need to do:
- Produce a dll that implements IVsSingleFileGenerator
- Register the dll for COM
- Register the dll with Visual Studio
- Configure your source files to use the dll
Implement IVsSingleFileGenerator
Implementing IVsSingleFileGenerator interface should be easy, since there are only 2 methods in it. Of course the MSDN reference is typically scant and not much help on its own. And if you want to read it from your local MSDN installation, you have to first install the Visual Studio SDK for your version of Visual Studio. In fact it's a good idea to install this, as it has a sample custom tool project with some useful wrapper classes.
From the SDK's "SingleFileGenerator" sample project, copy BaseCodeGenerator.cs into your tool project and inherit from it. It converts the IVsSingleFileGenerator interface into something much simpler to implement. You could probably figure out how do write your own BaseCodeGenerator equivalent, but MS has already done it for you, so why reinvent the wheel? Many articles on the net will tell you that you should derive your tool class from BaseCodeGeneratorWithSite.cs, but unless you need the extra features it reveals, there's no need. BaseCodeGenerator requires your tool project to reference Microsoft.VisualStudio.Shell and Microsoft.VisualStudio.Shell.Interop. BaseCodeGeneratorWithSite requires a heap more references, good luck with that.
As a side note, there seems to be a bit of history to this BaseCodeGenerator/WithSite class set that makes me think Microsoft doesn't want us writing our own custom tools. Apparently it was included in the .NET 1.1 libraries, but made internal for 2.0+. At the time it was made internal, Microsoft released the code for it to the GotDotNet website, but later obliterated when they replaced the whole site with CodePlex late in 2007. Most articles I came across still reference the GotDotNet version, which now redirects to a lame CodePlex apology page. So now the only place to get these classes seems to be the SDK... makes sense, but what a run-around it's been hunting it down.
Register for COM
Now, merely implementing the interface is not enough. Because custom tools operate through COM, we need to make our tool play nicely with COM. Firstly, your class must have it's own GUID, which can be generated with the uuidgen command line tool that comes with Visual Studio, or from the IDE's Tools menu, neither of which seem to be available in the Express edition of C#. The GUID is associated with your tool class via an attribute, eg:
[Guid( "EB35D84B-279D-4BAC-B587-8BE5FB28D889" )]Then you need to make your class "COM visible" by checking "Make assembly COM-visible" under project properties | Application | Assembly information.
public class CrsCodeGenerator : BaseCodeGenerator
{
Once you have compiled your tool into a dll, there are a ton of ways to register your custom tool for COM. The easiest way is to have Visual Studio do it for you during the dll compilation: check "Register fo COM interop" under project properties | Build. This is equivalent to running "regasm
Alternatively, you could copy your dll into the global assembly cache (GAC), either with gacutil or just copying into c:\windows\assembly. But to do that, your dll assembly needs a "strong name", ie it needs to be signed. Check "Sign the assembly" under project properties | Signing, then generate a new key file from the dropdown box underneath.
But I didn't do any of these (except for signing the assembly - now I can't remember if I actually needed to). Instead, I generated the registry entries required for COM registration:
regasm CrsCodeGenerator.dll /regfileand copied the output into a reg file that i then included in the project. This reg file is imported during build by including the following lines in my post-build event command line:
regedit.exe /s "$(ProjectDir)CrsCodeGenerator.reg"This doesn't require any unusual path environment assumptions... regedit will pretty much always be in a pathed directory. The copy command copies the compiled dll into Visual Stuido's home folder, so there is no need to mess with the GAC.
copy "$(TargetPath)" "$(DevEnvDir)"
The reason I used a regfile instead of just calling regasm in the post-built event is because regasm is not in a nicely-defined folder that works across all Visual Studio editions. It doesn't even come with 2005 Express.
Register with Visual Studio
This needs a few registry entries:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\8.0\Generators\{FAE04EC1-301F-11d3-BF4B-00C04F79EFBC}\You've probably guessed this only registers for Visual Studio 8.0 (2005). Adjust they key if you want to use your tool with other versions of VIsual Studio. The GUID in path corresponds to C# projects. A different GUID is required if you want your tool to work with VB projects, go look it up! The tool name, I suspect, needs to be short with no spaces, but I haven't tried otherwise. I used "CrsCustomTool". When this text is entered into the CustomTool property of a project file, our tool is run on that file. I'm not sure what the requirements are on the short title, I just put "CRS to C# Code Generator", it doesn't seem to be used anywhere I could see. The GUID is taken from the Guid attribute you generated for your tool class.]
@=""
"CLSID"="{}"
"GeneratesDesignTimeSource"=dword:00000001
Once again I put thes registry entries in my project reg file, which is merged in the post-build event discussed above.
Actually, Daniel Cazzulino describes a really nice way to have these registry entries generated by regasm. I haven't done this, but it might make it worthwhile using regasm or the "Register for COM Interop" property, and ditching the reg file. Yeat another option, sigh.
Use it!
If you've set your tool project up like me, all you have to do to "install" it is compile the project. Enter the tool name in a source file's CustomTool property and watch the magic happen!
Other Bits
Your tool can generate warning and error messages that appear in the Error List, and even goto the source when double clicked. Call GeneratorError(0,msg,line-1,column-1) or GenerateWarning(...). Note that if either of line/column aren't valid, the double click feature won't work. Note these parameters are 0-based indexes, but are displayed (correctly) as 1-based index in the Error List.
You can even automatically associate your custom tool with a particular file extension.
Summary
So there you go, a cenceptually-simple process, made painful by the combinatorial explosion of options at each step of the journey, and documentation scattered around the tubes in blogs like this :)
My focus has been compile-to-install (then ditch the source project if need be), which is why I've chosen to do some step differently to others. But hopefully by now you'll have a better understanding of all the options, and can make an informed choice as to how you construct and deploy your next custom tool.
# posted by Mark @ 4:01 pm 
