Eric J. Smith's Weblog

CodeSmith Generator 6.0–Status, Website and Future

ejsmith — Thu, 10 Nov 2011 20:51:22 GMT

We are on the home stretch of Generator 6.0 and things have come together really nicely. This is going to be a great release! We have been mostly working on performance, memory utilization and bug fixes, and we are doing really good on all fronts now. The feedback we have been getting is very positive and everyone seems to love the new version. We are really excited to get this release out the door, but we have some house cleaning to finish first.

I have been working on making updates to our website code that will allow us to sell Generator 6.0. The website code has always sort of been a low priority and it was starting to really show. So we are doing some major refactoring and cleanup work to make things clean and simple.

Now that the Generator 6.0 project is starting to wind down, we are beginning to think about what comes next. We have received feedback from a few people saying that they would like to see a stand alone template editor in addition to the integrated editor. The good news is that the way we built the new editor would make it so that we could reuse almost everything in a stand alone editor. We would like to hear from more people about whether or not we should spend the time and resources to create a stand alone version of the new editor. And if so, should the editor take on a different form than the old CodeSmith Studio? Should it be more of a notepad-esque editor that lets you edit a single template per instance or should it be a full blown VS clone like CodeSmith Studio was? Let us know what you think by sending an email to support@codesmithtools.com

CodeSmith Generator 6.0 Beta Survey

ejsmith — Thu, 08 Sep 2011 21:10:38 GMT

We are currently working on dogfooding the new template editor by updating and building some new templates. Along the way, we are trying to find and fix as many pain points as we possibly can. We are also working on profiling and memory usage reductions and trying to get everything running smoothly for the final release.

We have gotten a lot of great feedback from the community and want to encourage everyone to keep it coming. In order to try and get a little more organized with our feedback, we have created a beta survey and everyone that completes the survey will get a discount on their next purchase with us. This could be used towards upgrading to CodeSmith Generator 6.0 *HINT* *HINT*. :-) Also, please remember that if you purchase CodeSmith Generator 5.x now, you will get a free upgrade to CodeSmith Generator 6.0 when it is released.

CodeSmith Generator 6.0 Beta 1 Released!

ejsmith — Thu, 11 Aug 2011 23:30:47 GMT

It’s been a long time coming, but CodeSmith Generator 6.0 Beta 1 has finally been released today! I think a lot of people believe CodeSmith Tools is a lot bigger than we actually are (there are only 4 of us currently), but we have been working hard on this release for about 8 months now and it is pretty close to a complete rewrite. We have rewritten the entire engine, lexer, parser, property grid, editor, intellisense, and then integrated it all right into Visual Studio 2010!

This release gives us a great foundation to build on for future enhancements as I have learned a LOT since I ~~painted myself into a corner~~ built the original template engine. Our new engine is very well structured and should make it MUCH easier to make changes to both the template syntax as well as improving our editing experience. I really hope you guys like it!

Download CodeSmith Generator 6.0 Beta 1

CodeSmith Generator 6.0–Editor in Action

ejsmith — Wed, 13 Jul 2011 05:54:19 GMT

I thought the best way to show our progress so far would be to show you a quick video of the new editor in action. So without further adieu, here is the new CodeSmith Generator 6.0 editor:

* NOTE: Click the full screen button on the bottom right of the video to watch the video in higher resolution.

Unable to display content. Adobe Flash is required.

In general, I think our editing experience is already about 10 times better than it was before. Not even counting the fact that it’s integrated right into Visual Studio now. We still have some work to do, but we will be releasing a beta version soon.

CodeSmith Generator 6.0–Status and IntelliSense

ejsmith — Wed, 23 Mar 2011 18:06:38 GMT

Work on Generator 6.0 is progressing pretty nicely. We are code complete on our new parser, new compiler, new caching mechanism, out of process template execution, and WPF based property grid. There is a lot to talk about there, but I have also started working on implementing IntelliSense and I wanted to talk about that today.

Better working IntelliSense has been our biggest request probably from day one. People tend to take IntelliSense for granted because Microsoft has done a great job with it in Visual Studio and it almost becomes transparent. We just use it and expect it to work without thinking about what goes into it. I was just as guilty of this as everyone else until I started working on implementing it in CodeSmith Studio. Think about it for a second, you have a collection of text files and you need to make sense of them… what goes into that? Here is a simplified list of the steps and parts required for implementing IntelliSense.

Tokenize – the first step is to take that text file and create a series of tokens from it.
- This sounds easy, but in and of itself it can be pretty hard because of ambiguities in languages where certain things have different meanings depending on their context.
- In our case, things are even more complicated by the fact that we are mixing random languages together.
- Also, keep in mind that this needs to be happening in real time as the user types without slowing the editor down.
Parse – next we read the tokens in, make sense of them, validate them and construct an abstract syntax tree along the way so that we can have some sort of structure that can be more easily understood, traversed and transformed.
- This is expensive to build and needs to happen in a background thread and constantly be updated as the user is typing.
Gather Type Information – we need a database of reflected and parsed type information from any referenced assemblies and source files so that we can use that information in resolving and knowing what members are available for any given type.
- It is expensive to reflect over every single type in every single referenced assembly. So you need to cache this information in a structure that makes it really fast to load and search.
Gather Context – when an IntelliSense request is made, we need to figure out where we are and provide as much information about our surroundings as possible.
- Due to the fact that active typing is going on, we have to use a combination of token scanning and looking at our most recent AST to provide this information.
- Also due to active typing, we have to work with an AST that has information that is slightly out of date and we will need to translate that out of date information to the current source.
Resolve – now we need to take all of the information that has been gathered and use it to figure out what we are looking at and what options are available to show in the IntelliSense list.
- There are a lot of rules that come into play here like member scope and precedence for extension methods to name a couple.

Another thing I would like to address is that many people in the past have said to us… "just use Microsoft's IntelliSense". Unfortunately, it's not that easy. Microsoft does not expose their IntelliSense services and, even if they did, we most likely would not be able to use them because they would be very much geared around their use cases which would make them unusable for ours.

As you can see, there is a LOT of work that goes into providing that handy little drop down menu that magically pops up and makes it so that you don't have to memorize an entire framework library. We are completely re-writing every single one of those pieces in CodeSmith Generator 6.0 and our end goal is to make it so good that you completely take us for granted because it just magically works.

CodeSmith Generator 6.0 Template Parser Progress Report - Part 2

ejsmith — Wed, 02 Feb 2011 19:38:13 GMT

In this post I’m going to talk about how the new template parser in CodeSmith Generator 6.0 creates an AST (Abstract Syntax Tree) and then uses the visitor pattern to iterate over the nodes in the AST and do transformations.

The grammar rules I talked about in my previous post are used to both validate the syntax of the template code and to create typed AST nodes. Here is a very simple template:

<%@ CodeTemplate Language="C#" %>
<%= DateTime.Now %>

The resulting AST from the parse looks like this:

DirectiveNode Name=CodeTemplate
AttributeNode Language=C#
ExpressionNode
ValueNode DateTime.Now

Each node in the AST implements a visitor pattern and I’m actually using a visitor to generate the AST visualization above. The visitor recursively goes through each node in the hierarchy and calls the Visit method for that specific node type. So I end up with a visitor class that looks like this:

public class VisualizeTemplateVisitor : TemplateAstVisitorBase
{
    …
    public override void Visit(DirectiveNode node)
    {
        _builder.Append("DirectiveNode Name=");
        _builder.AppendLine(node.Value.Trim());
    }
    public override void Visit(AttributeNode node)
    {
        _builder.Append(" AttributeNode ");
        _builder.Append(node.Name.Trim());
        _builder.Append("=");
        _builder.AppendLine(Format(node.Value));
    }
    …
}

I simply call the Accept method on the visitor and pass it the root node that I want to visit and it creates the output above:

string nodeVisualization = VisualizeTemplateVisitor.Accept(templateNode)

This pattern is used extensively in the new parser. Here are some of the visitors I have so far in the new parser:

CollapseWhitespaceVisitor – used to traverse the AST and remove whitespace and line break nodes from lines that only contain template code. This visitor actually modifies the AST while it is visiting it.
PopulateParseResultVisitor – used to gather information about the template like what sub-templates are referenced, what include files are referenced, what assembly references are declared, what language the template is using (ie. C#, VB, JS) and many other things.
GenerateCompileUnitVisitor – used to generate a CodeDOM compile unit from the AST that can be fed into the .NET compiler and create an assembly.

By creating an AST and using the visitor pattern, I’m able to do any sort of transformation that I want by simply creating a new visitor and plugging it into the template compilation pipeline. As you can imagine, this ends up being a much cleaner, simpler and extensible solution than what we had before. We may even expose the compilation pipeline with MEF in CodeSmith Generator 6.0 so that anyone could make changes to the generated code or implement new template directives.

CodeSmith Generator 6.0 Template Parser Progress Report - Part 1

ejsmith — Wed, 05 Jan 2011 00:01:08 GMT

I am currently working on the new parser for CodeSmith Generator 6.0. When I wrote the original template parser back in 2004, I really had no idea what I was doing. :-) I had no idea what an AST was or what an LL(*) parser was, etc. That being said, I think I did an OK job and it has certainly served it’s purpose, but it can’t be easily changed and things are more complicated than they need to be.

I have been working on our new parser for a few weeks now and I have a pretty good start. Here is a list of things that we want to get out of the new parser:

Structure - Generate an AST as an intermediate step so that we can do transformations using a Visitor pattern and even LINQ queries.
Grammar - Create a grammar for the syntax so that we can give grammar errors.
Testability - Make it easy to unit test.
Maintainability - Make it easy to implement changes and add new features.

I decided to make use of the parser framework that Actipro includes with their SyntaxEditor control. It is an LL(*) parser framework that lets you define the grammar right in C#. Here is what a sample grammar production looks like:

codeExpression.Production = @expressionStart + @expressionText["expr"] + @expressionEnd
> Ast<ExpressionNode>()
.SetProperty(n => n.Value, AstFrom("expr"));

This code is defining the grammar rule that makes up a code expression <%= SomeValue %> in a template. It says that there will be an expression start token followed by some expression text and then an expression end token. It is also constructing an AST node for the expression. The parser framework has really helped get our new parser of the ground quickly and I highly recommend it.

The cool thing about defining a grammar is that if there are errors in the template, you will get much more intuitive error information about what the parser saw and what it was expecting to see. Previously, we pretty much just parsed the template, generated code from it and tried to compile it. Here is an example of how we will be providing much better errors in Generator 6.0. Here is a template that has a missing % at the end of the first line:

<%@ CodeTemplate Language="C#" >
<%= DateTime.Now %>

Here is the error that Generator 5.3 gives:
error CodeSmith0104: You must specify a valid template language.

And here is the error message that the new parser gives:
Line 2, Char 3: %> expected.

In my next post I will talk about the AST and how we are using the visitor pattern to do transformations on the parsed template content.

P.S. If you haven’t figured it out by now, I’m really bad about writing to my blog. Rather than break out the random excuse generator, I’m just going to start posting. Happy New Year!

CodeSmith Insight Launch Party

ejsmith — Thu, 30 Sep 2010 22:07:42 GMT

CodeSmith Tools is excited to announce the upcoming release of our new product, CodeSmith Insight, and we would like to invite you to come celebrate! Join us at the Addison Conference Center‎ on Tuesday, October 12th, from 6pm to 9pm, for the:

CodeSmith Insight Launch Party

Prizes and Giveaways
There will be over $1,000 in prizes, including: Apple iPad, Crucial C300 128GB SSD, a new Microsoft Xbox 360 Slim, Nintendo Wii, and more! Plus all attendees will receive a free CodeSmith Insight T-Shirt, as well as a 50% Lifetime Discount for their CodeSmith Insight subscription!

Games, Competition, and Food
This will be a Retro Game Party. There will be a series of old school arcade machines, including: Tetris, Streetfighter 2, Pac-Man, Mortal Kombat, NBA Jam, Metal Slug, Galaga, and more! High scores will be kept throughout the evening, and at the end of the party the leaders will battle it out for the aforementioned prizes! Also, there will be free pizza and drinks for all attendees.

What is CodeSmith Insight?
CodeSmith Insight is a help desk software with advanced application integration. This means that Insight can handle your email and user feedback, as well as your application errors and crash reports, all from a single convenient location. Insight is useful to anyone who makes software.

Please RSVP at our party website!

Be sure to visit our website and RSVP if you are coming, as there are a limited number of spots and they are filling up fast! http://party.codesmithtools.com/

We hope to see you there,
The CodeSmith Tools Team

PLINQO at the Dallas .NET Users Group

ejsmith — Tue, 07 Jul 2009 19:23:25 GMT

This Thursday July 9th, Shannon Davidson and Tom DuPont will be talking about PLINQO at the Dallas .NET Users Group at Microsoft's Las Colinas Office in Irving, Texas.

If you are in the area, we would love for you to stop by and see what PLINQO is all about. You might even win a free copy of CodeSmith!

When:
Thursday, July 9th
Doors open at 6:00PM
Presentation starts at 6:30PM

Where:
Microsoft's Las Colinas Office
LC1 Building (Right Tower)
1st Floor, MPR Room,
(Left of the main security desk)

7000 State Highway 161
Irving, TX 75039
Map

CodeSmith 4.0 Released

ejsmith — Wed, 08 Nov 2006 19:11:00 GMT

CodeSmith 4.0 has been officially released today at the Dev Connections conference in Las Vegas! This is a huge release for us and has a bunch of great new features as well as countless minor enhancements and bug fixes all based on customer feedback.

Here are some of the features in the new version of CodeSmith:

CodeSmith Projects (.csp) - This feature makes automating your code generation process really easy and consistent whether you are working from inside of Visual Studio 2005, MSBuild, Windows Explorer, a command line / batch file, or CodeSmith itself.

ActiveSnippets - Imagine Visual Studio 2005 snippets, but with the full power of CodeSmith available to execute any logic or access any complex metadata (including database schema and xml data) to control the output of your snippets.

CodeSmith Maps (.csmap) - This feature will allow you to create dictionary style maps of things like SQL to C# data type mappings.

.netTiers 2.0 - The .netTiers templates have been greatly enhanced and included with CodeSmith 4.0.

Extended Property Management - You can now edit and add new schema extended properties inside of CodeSmith Studio.

Property Persistence - CodeSmith now remembers the property values from the last time you executed a template.

You can view the official press release here:
http://community.codesmithtools.com/blogs/announcements/archive/2006/11/08/CodeSmith40Released.aspx

CodeSmith 4.0 Public Beta Released

ejsmith — Fri, 20 Oct 2006 00:53:00 GMT

CodeSmith 4.0 Beta 2 has been released and is now available to the public. Here are just a few the new capabilities in CodeSmith 4.0:

CodeSmith Projects (.csp) - This feature makes automating your code generation process really easy and consistent whether you are working from inside of Visual Studio 2005, MSBuild, Windows Explorer, a command line / batch file, or CodeSmith itself.
ActiveSnippets - Imagine Visual Studio 2005 snippets, but with the full power of CodeSmith available to execute any logic or access any complex metadata (including database schema and xml data) to control the output of your snippets.
CodeSmith Maps (.csmap) - This feature will allow you to create dictionary style maps of things like SQL to C# data type mappings.
.netTiers 2.0 - The .netTiers templates have been greatly enhanced and included with CodeSmith 4.0.
Extended Property Management - You can now edit and add new schema extended properties inside of CodeSmith Studio.
Property Persistence - CodeSmith now remembers the property values from the last time you executed a template.

You can read the announcement here.

CodeSmith $5,000 cash give away at Dev Connections

ejsmith — Sat, 07 Oct 2006 21:57:00 GMT

CodeSmith will have a booth at Dev Connections in Las Vegas next month and we thought it would be a lot of fun to give away some gambling money! If you are coming to Dev Connections, just make sure to stop by our booth and signup. If you aren't coming to Dev Connections, you better sign up quickly!

In addition to Microsoft having some exciting new announcements for Dev Connections, CodeSmith will have some exciting announcements of our own!

CodeSmith Wins "Best Utility" Award

ejsmith — Fri, 12 May 2006 21:39:00 GMT

CodeSmith has won the "Best Utility" award in the asp.netPRO Readers’ Choice Awards! Thanks to our incredible user base, CodeSmith was able to win this category that included many other great software development products. Thank you to everyone that voted!

HanselMinutes - Code Generation

ejsmith — Wed, 19 Apr 2006 17:18:00 GMT

“Scott talks about CodeSmith as well as other code generators, highlighting their strengths and weaknesses, and offering his usual insight into the issues. Of course, resources abound.”

http://www.hanselminutes.com/default.aspx?showid=20

asp.netPRO Readers’ Choice Awards

ejsmith — Sat, 18 Mar 2006 03:45:00 GMT

If you are a fan of CodeSmith and would like to show your support, please take a moment to place your vote for CodeSmith in the Utility category of the asp.netPRO Readers’ Choice Awards.

Each year, asp.netPRO recognizes outstanding products and vendors in the ASP.NET add-on market. Please take a moment to select your favorite ASP.NET tools. This is your chance to voice your opinions regarding the tools and products you use in your everyday development efforts.