I got my ICodeParser implementation!

Note: this entry has moved.

Well, sort of :S. After "reflectoring" quite a while, I found the VsCodeDomGenerator, VsCodeDomProvider and VsCodeDomParser in the Microsoft.VisualStudio.dll assembly. After trying to create those classes through reflection, using all sorts of hacks and ugly tricks, I was about to give up, as everything failed with exceptions, as it expected a DTE ProjectItem and the like.
But then, I remembered that if you develop a custom tool using the MS provided helper classes, you get a member called CodeProvider, which is a reference to a valid CodeDomProvider. One of the cool things of the BaseCodeGeneratorWithSite is that it automatically "knows" which provider to give you depending on the current project type, so don't need to worry about file extensions and the like. Unlike what you may think, the base implementation doesn't "switch" according to the file extension, but does a more "complicate" operation: being a "sited" component (as the class name suggests), it can query for services from the IDE like any "sited" component would, through the GetService method, and that's what it does:

protected virtual CodeDomProvider CodeProvider {
    get {
        if (codeDomProvider == null) {
            IVSMDCodeDomProvider vsmdCodeDomProvider = (IVSMDCodeDomProvider) GetService(CodeDomServiceGuid);
            if (vsmdCodeDomProvider != null) {
                codeDomProvider = (CodeDomProvider) vsmdCodeDomProvider.CodeDomProvider;
            }
            Debug.Assert(codeDomProvider != null, "Get CodeDomProvider Interface failed.");
        }
        return codeDomProvider;
    }

Now I started to get excited!!! It turns out that this member gives you a VsCodeDomProvider instead of the appropriate VBCodeProvider or CSharpCodeProvider, which always return null from their CreateParser method. You guessed right, the VsCodeDomProvider returns an instance of the VsCodeDomParser!! Now everything I needed was grasp it, get the project item selected, parse it, and play with the CodeDom before writing it back!!!

Well, that turned out to be harder than it sounds. First, getting the project item. How do you get that from inside a custom tool?!?! No help whatsoever. Add-ins are blessed there, but no luck with custom tools... until after a good time hacking here and there, I found a way, through the marvelous GetService method:

// Get the item.
EnvDTE.ProjectItem item = base.GetService(typeof(EnvDTE.ProjectItem)) 
    as EnvDTE.ProjectItem;

Now I can get the selected file, open it and parse it:

// Create the parser for the item.
ICodeParser parser = base.CodeProvider.CreateParser();

if (item == null || parser == null)
  throw new ArgumentException("This custom tool can only be applied to code items.");

// Parse the file as CodeDom.
using (FileStream fs = new FileStream(item.get_FileNames(0), FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
  using (StreamReader sr = new StreamReader(fs))
  {
    CodeCompileUnit unit = parser.Parse(sr);
    //Play with the CodeDom!!!!

At this point, I had to do some cleaning on the resulting unit because the parser sets the LinePragma property on every member/class/statement, which results in code full of preprocessor directives such as #ExternalSource (VB) and #line (C#). This is just a mater of iterating the CodeCompileUnit and setting that property to null.

After such a success, I ran to create a project, and two samples for round-tripping VB and C# code, which is only natural if you have a CodeDom at hand. I even submitted it to GotDotNet about a week ago (didn't get online yet, don't know why).
I was all too happy, really... until I saw the code generated from that unit. The parser fails to parse methods, converts array fields into simple (single item) elements, completely loses properties and all custom attributes. As I had done my early tests from an XSD passed through Chris Sells XsdClassesGenerator, or, what is the same, through XSD.EXE, I didn't see these failures early on my testings... :(. Well, but I don't lose my faith that maybe Whidbey will give me a much better CodeProvider, right?

Update:My library and tests can be downloaded from GotDotNet

9 Comments

  • Where can i find your sample???

  • This is gr8, but will this work without IDE installed on machine?

  • Hi,



    I'm trying to implement some preprocessor, and your example seems very helpful.



    I have one question, though: If all you're using the item for is to get the file name and read the file, why don't you use the file name and contents passed to GenerateCode() as arguments?



    Thanks for the tip,



    Shai.

  • Ok.



    Besides the limitations you've already noted, there's a problem with implementing a preprocessor this way, that is beyond the parser's implementation.



    I coded a preprocessor that took a C# source, say xx.cs, and produced a C# source, xx.decorated.cs (the hope was to have attributes control the preprocessor). The problem is that Studio then tried to compile _both_ files, generating a load of multiple definitions.



    I tried to correct it by setting the source (xx.cs) action to "None" and "Content". The first failed, because then nothing is done with the file. The second, because then the CodeProvider we get is one with no parser (this also has the feature that the code is not regenerated in "rebuild" -- only when the source changes).



    I am using VS.NET 2002; perhaps things are a little better in the land of 2003?

  • Well, I'm afraid it's not any better in 2003. With luck, you could use partial classes in Whidbey.

    Maybe you can use a different namespace for the generated class? Like Xxx.Decorated.X, ... pretty ugly though :S

  • @Himanshu:

    If your product is under GPL, you can use some classes of SharpDevelop to make your own implementation of a ICodeParser. Here the easiest way:



    You need the following DLL:

    SharpDevelop\bin\ICSharpCode.SharpRefactory.dll



    // File CodeDomUtil.cs

    using ICSharpCode.SharpRefactory.Parser;

    // some more usings...



    namespace CodeDomUtil

    {

    public class CSharpParser : ICodeParser

    {

    public CSharpParser()

    {

    }



    public CodeCompileUnit Parse(System.IO.TextReader codeStream)

    {

    Parser p = new Parser();

    Lexer lex = new Lexer(new StringReader(codeStream.ReadToEnd()));

    p.Parse(lex);

    CodeDOMVisitor visit = new CodeDOMVisitor();

    visit.Visit(p.compilationUnit, null);

    return visit.codeCompileUnit;

    }

    }

    }



    This parser won't parse everything of your file since it is not fully implemented but for some usages it is enough. And if it is not, you are free to replace the empty methods in the implementation...

  • Very interesting Sam. "My" product (sf.net/projects/mvp-xml) is under CPL (Common Public License). Don't know about the incompatibilities of CPL/GPL...

  • ICodeParser parser;

    CodeCompileUnit codecompile;

    CSharpCodeProvider cprov;

    TextReader rdfile;

    string filename;

    try

    {

    cprov = new CSharpCodeProvider();

    parser = cprov.CreateParser();

    if ( parser!=null)

    {

    MessageBox.Show("Parser is not created");

    this.Closed();

    }



    filename=@"D:\urvish\NET TUTORIAL\CodeDOMDemystified\CodeDom\Class1.cs";

    rdfile= File.OpenText(filename);

    //string str=rdfile.ReadLine();

    if (rdfile!=null && parser!=null)

    {

    codecompile=parser.Parse(rdfile);

    }





    }

    catch(Exception ex)

    {

    MessageBox.Show(ex.Message );

    }



    in this block i always get parser as null

    regards

  • reply at urchis@yahoo.com

    thaks

Comments have been disabled for this content.