Why Orchard?

Our company wants to convert an old html site to a content management system. We considered between Wordpress and Orchard and picked Orchard. This post strongly influenced us. It is ASP.NET MVC vs php. We are a .net shop. We thought it would be easier to do custom development with Orchard.

Why Webmatrix?

There are a few reason we used Webmatrix:

1. Webmatrix works with Orchard very well. Many Orchard training materials use Webmatrix.
2. Webmatrix is free. We have interns here working on Orchard. We do not have to consume a Visual Studio license.
3. Webmatrix has an excellent story working with both Git and Azure.

Why Git?

We would like to have a version control system. Git is a free and open source distributed version control system. There are several free Git hosts, such as GitHub, Codeplex, and BitBucket. So we picked Git.

Why Azure?

We would like to have website that our entire team can see. Azure web site is an excellent option for us. It is very easy to host Orchard with an Azure website, either with Sql Compact Edition or with Azure SQL Database. Azure also integrates with Git very well.

So how do they work together?

Although it is possible to edit a Azure hosted web site directly with Webmatrix, we hosted out source code in Git because we need a source control system.

It is very easy to work with Git from Webmatrix. One can use git with an existing site or open a site directly from git.

The Sql Server migration solution in Webmatrix is amongst the easiest way to migrate a Sql CE database to Azure SQL Database.

It is also fairly easy to setup an Azure website to pull source code from git directly and build the source code. Each time when we push a changeset to git, Azure is notified and it will automatically pull and build the website. For Orchard, the “built” is actually not more than an xcopy. We just need to embed a .deployment file in our source code. See David Haden’s post for more details.

It is fairly common in applications that we have a list page/details page pairs. When we need to add a new record or edit an existing record, we click on the “Add New” link or the “Edit” link from the list page to go to the details page. When we finish adding or editing, we go back to the list page and expect to see the information updated.

At the first glance, developer can use history.back if no post-back occurred or use response.redirect if a post-back occurred. However, what if the list page has a filter or is at a page number other than the first? How do we return to where we were on the list page?

If we search Bing on “history.back and refresh”, we can see a number of discussions. However, there are no good solutions. It is difficult to have a Javascript only solution because Javascript has the life time of a page. So we need a bit of assistance from the server but we do not want to leave session variables behind because we would leave many session variables behind if the application has many list/detail pairs. The following is one possible solution to the problem:

1. Change links to the detail page on the list page to post-backs . For ASP.NET Web Form, this can be easily done by changing links to LinkButtons.
2. Upon posting back to the list page, capture the current filter criteria and page number. Save the information to a session variable and do a response.redirect to the details page.
3. From the details page, save the filter data from the list page into a hidden control or use ViewState if using ASP.NET Web Form. Remove the session variable.
4. Use post-back in the details page for both “Save” and “Cancel”. Save the filter data into the session again and do a response.redirect to the list page.
5. From the list page, check the existence of a session variable for filter. If it exists, populate the filter and page number and remove it from the session. We have thus returned to where we were on the list page and have left no trace of session variables behind.

Recently, I worked on an old webform project and used the asp.net validation controls. I previously used some features of validation controls, but I encountered more scenarios this time. I found these controls are fairly flexible and can be configured to work in many different ways:

Validator Controls

The most important properties that control the display are Text, ErrorMessage and Display properties.

What is the difference between the Text and ErrorMessage properties. According to MSDN:

If the Text property is set, that text will override the text specified in the ErrorMessage property and appear in the validation control. However, the text specified by the ErrorMessage property always appears in the ValidationSummary control.

One scenario is that you might set Text to “*” and ErrorMessage to the description of the error. You will see an “*” next to the control to validate and a details message in the validation summary section.

The display property takes 3 values:

To show error only in the validation summary section, set the Display property of the validator controls to None.

Validation Summary Controls

The most important properties that control the display are HeaderText, DisplayMode, ShowSummary and ShowMessageBox properties.

If we turn Show Summary off and ShowMessageBox on, the summary will be displayed as a message box. Note that it is necessary to leave EnableClientScript to true to use ShowMessageBox.

Supposing we only want to display a “*” next to each control and a generic message portion, we only need to test Text of each validator control to “*”, ErrorMessage to blank and set the generic message to the HeaderText of the ValidationSummary control.

The ASP.NET validator controls have a good combination of properties to work out all the scenario that I encountered in the project.

We are planning to update our very old web application so that it works better with modern web browsers. I have heard of modern.ie in the past and decided to give it a try. The site has a scanner that can scan a URL entered by a user. Unfortunately, our site requires log-in before getting to the page we want to scan. Fortunately, the site also offers a downloadable scanner that we can use in our local development environment.

I downloaded the scanner. It is packaged as a zip file. I unzipped the package and open the readme file. The tools requires node.js to work. Fortunately, it is fairly easy to setup the tool following the instructions. Firstly, download and install node.js; accept all defaults. The setup will add Node.js to the path. Although Node.js is known for hosting asynchronous web applications, it is actually just a javascript executing environment. Next, I open the command prompt, change to the directory that I unzipped the modern.ie tool and run “node lib/service.js”. A node hosted web server is listening on port 1337 so we just need to open a web browser and points to http://localhost:1337/. We can then enter the URL of a web page to scan. The scanner submits the data to modern.ie and a report is generated. The report is very good and it offers many suggestions and links that helps us to improve the page.

We already have a standard IDE in Visual Studio. We also have a very innovative IDE in WebMatrix that can work with node.js and php in addition to ASP.NET. Why do we need another IDE? Well, let me first talk about the feasibility and then throw in some ideas on what we could do with it.

I have noticed CodeMirror for a long time. Recently, I have also noticed ACE on which the Cloud9 IDE is based. I was impressed that both CodeMirror and ACE supported Typescript soon after it was released. I was also impressed with the Cloud9 IDE that can debug Node.js code. All these projects have benefited from rapid improvement of the Javascript engine in modern browsers and I think that the era for very functional web-based IDEs has arrived.

So what can we do with a cloud-based IDE? Firstly, like what Cloud9 demonstrated, the code lives on the cloud. Secondly, people can collaborate. Thirdly, I think it is possible to improve experience to what we have never seen before, so let me elaborate below.

In Visual Studio, when we want to see how the ASP.NET page looks like, we switch to the designer mode but that is not really close to what we see at runtime. In a cloud-based IDE, we can see the code and how it renders side-by-side, like what JsFiddle has demonstrated. ASP.NET does not execute the code on the website directly; it first compiles the code into the “Temporary ASP.NET Files” directory and executes the assemblies from there. That makes it feasible to edit and run the code at the same time.

Visual Studio has been heavy on ORM (i.e., entity framework) and lighter on code-generation. I have been favoring code-generation over ORM. This is because with ORM we are tweaking a black box to generate desired SQL. With code-generation, we can see exactly what we get and tweak the templates when necessary. Visual Studio has limited runtime information; it gets its information either from parsing the code, or from meta-data stored in XML files. With a cloud-based IDE that is working with running code, it is possible to get richer runtime information and generate code (or scaffold) under much wider scenarios. So how do we map the server-side code to html in the browser? That is where ideas like source map could help, and we have very good tools in querying DOM already.

So I believe all the technical pieces for a cloud-based ASP.NET IDE are available.

I have not done much with my ASP Classic Compiler project for over a year now. The lack of additional work is due to both the economic and the technical reasons. I will try to document the reasons here both for myself and for would–be open source developers.

How the project got started?

I joint my current employer at the end of 2008. The company had a large amount of ASP classic code in its core product. The company was in talk with a major client on customization so that there was a period that the development team had fairly light load. I had a chance to put in a lot of thinking on how to convert the ASP classic code to ASP.NET. One of the ideas was to compile ASP Classic code into .net IL so that they can be executed within the ASP.NET runtime.

I do not have a formal education in computer science; my Ph.D. is in physics. I had been very fond of writing parsers by following the book “Writing Compilers & Interpreters” since 1996. I wrote a VBA like interpreter and a mini web-browser, all in Visual Basic, using the knowledge acquired. Nevertheless, a full-featured compiler is still a major undertaking. I spent several months of my spare-time working on the parser and compiler, and studied the theory behind. I was able to implement all the VBScript 3.0 features and the compiler was able to execute several Microsoft best-practice applications. The code runs about twice as fast as ASP Classic. I had lots of fun experimenting with ideas such as using StringBuilder for string concatenation to drastically improve the speed of rendering. I was awarded Microsoft ASP/ASP.NET MVP in 2010. In 2011, I donated the project to the community by opening the source code after consulting with my employer.

Since then, I faced challenges on two fronts: adding VBScript 5.0 support and solving a large number of compatibility issues in the real-world. After working for a few more months, I found it difficult to sustain the project.

Lesson 1: the economy and business issues

1. Sustainability of an open source project requires business arrangements. I am an employee. Technical activities outside the work are welcome as long as it benefits the employer. However, excess amount of activities not relating to the work could be a loyalty issue.

2. ASP Classic is a shrinking platform. So ASP Classic Compiler would not be a sound investment for most businesses except for Microsoft who has interest in bridging customers from older to newer platform painlessly.

Lesson 2: the technical issues

1. VBScript is a terrible language without a formal specification and test suites. The best “spec” outside of the users’ guide is probably Eric Lippert’s blog. It is not getting much maintenance and it is losing favor. As a result, few other developers with knowledge in compiler would have interest in VBScript.

2. Like the IronPython project and the Rhino projects, I started by writing a compiler. I was very shrilled that my first implementation without much optimization is about twice as fast as VBScript in my benchmark test. However, I had many obstacles on compatibility and adding the debug feature. If I would do it again, I would probably implement it as an interpreter with an incremental compiler. The interpreter would have a smaller start-up overhead. I can then have a background thread compiling the high usage code. The delayed compiling would allow me to employee more sophisticated data-flow analysis and construct larger call-site blocks. The current naïve implementation on DLR results in too many very small call-sites so that type-checking at call-sites is a significant overhead.

In summary, I had lots of fun with the project. I significantly improved my theoretical knowledge of algorithms in general, and of parsers and compilers. I had the honor of being awarded Microsoft ASP.NET/IIS MVP for the past 3 years and enjoyed my private copy of MSDN subscription at home. After a working prototype, the remaining work to take it to a production quality software is more a business problem than else. I am actively contemplating some new forward-looking ideas and wish I have some results to share soon.

I just came back from the Microsoft MVP Summit 2013. I was surprised and excited to learn there are a large number of open source projects from both inside and outside of Microsoft. There is also a strong support for open source frameworks in Visual Studio. I am glad to see that the Microsoft ASP.NET team has done a great job supporting open source and the community is going strong.

Open source projects from the ASP.NET team

Those who interested in the open source projects from the ASP.NET team should first visit http://aspnet.codeplex.com/. This site is a portal to many ASP.NET features that Microsoft has opened the source code.

Next, readers should visit http://aspnetwebstack.codeplex.com/. This site is the home of the latest source code of MVC, Web API and Web Pages.

The following are several open source projects that have been incorporated into Visual Studio:

• http://nuget.codeplex.com/: Nuget is now distributed with Visual Studio 2012.
• https://github.com/SignalR: SignalR is a framework for building long polling application by Damian Edwards and David Fowler. As of ASP.NET and Web Tools 2012.2, SignalR is part of ASP.NET and a Microsoft supported project. Don’t forget to change the namespace if you are using an earlier version.

The following projects are considered experimental:

• http://owin.org/: A specification of standard interface between .NET web servers and web applications.
• http://katanaproject.codeplex.com/: An implementation of OWIN.

Projects from outside of Microsoft

If you think there are no needs for another framework since ASP.NET MVC is already great, you would be surprised to find out some open source alternatives have actually attracted many followers:

• http://nancyfx.org/: Look at what they have built and number of contributors.
• http://mvc.fubu-project.org/: Another MVC framework that has a large number of contributors.
• The OWIN home page (http://owin.org/) has links to other projects.

Open Source Client-Side MVC or MVVM frameworks supported by Microsoft Visual Studio

Microsoft ASP.NET and Web Tools 2012.2 comes with a single page application (SPA) project template that uses KnockoutJS. However, Mads Kristensen also built projects templates for several other highly popular JavaScript SPA libraries and frameworks: Breeze, EmberJS, DurandalJS and Hot Towel. Visit http://www.asp.net/single-page-application/overview/templates for download links. 

Don’t forget to install Mads’ Web Essentials extension for Visual Studio 2012. You will be pleasantly surprised by the number of features that this extension adds to Visual Studio 2012.

Final Notes

I gathered these links for myself and others. If I missed any links, please post a comment or send me a note. I will be glad to update this page.

Like many software developers, my primary computer is a laptop. At home, I like to connect it to an external monitor and use the laptop screen as the secondary monitor. I like to bring the laptop screen as close to my eye as possible. I got an idea when I visited IKEA with my wife yesterday.

IKEA carries a wide variety of legs and boards. I bought a box of four 4” legs for $10 (the website shows $14 but I got mine in the store for $10) and a 14” by 46” pine wood board for $7.50. After a simple assembly, I got my sturdy dual-monitor stand.

I place both my wireless keyboard and mouse partially underneath the stand. That allows me bring the laptop closer. The space underneath the stand can be used as temporary storage space. This simple idea worked pretty well for me.

I imported a 2.5 GB file into our SQL server today. What made this import challenging is SQL Server Import and Export Wizard failed to read the data. This file is a tab delimited file with CR/LF as record separator. The lines are truncated if the remaining line is empty, resulting variable number of columns in each record. Also, there is a memo field that contains CR/LF so that it is not reliable to use CR/LF to break the records.

I had to write my own import utility. The utility does the import in 2 phases: analysis and import phases. In the first phase, the utility reads the entire file to determine the number of columns, the data type of each column, its width and whether it allows null. In order to address the CR/LF problem, the utility uses the following algorithm:

Read a line to determine the number of columns in the line.

Peek the next line and determine the number of columns in the second line. If the sum of the two numbers is less than the expected column count, the second line is a candidate for merging into the first line. Repeat this step.

When determining the column type, I first initialize each column with a narrow type and then widen the column when necessary. I widen the columns in the following order: bool -> integral -> floating ->varchar. Because I am not completely sure that I merged the lines correctly, I relied on the probability rather than the first occurrence to widen the field. This allows me to run the analysis phase in only one pass. The drawback is that I do not have 100% confidence on the schema extracted; data that does not fix the schema would have to be rejected in the import phase. The analysis phase took about 10 minutes on a fairly slow (in today’s standard) dual-core Pentium machine.

In the import phase, I did line merging similar to the analysis phase. The only difference is that, with statistics from the first phase, I was able to determine whether a short line (i.e. a line that has a small number of columns) should merge with the previous line or the next line. The criterion is that the fields split from the merge line have to satisfy the schema. I imported over 8 million records and had to reject only 1 record. I visually inspected the reject record and the data is indeed bad. I used SqlBulkCopy to load the data in 1000 record batch. It took about 1 hour and 30 minutes to import the data over the wan into our SQL server in the cloud.

In conclusion, rather than determining the table schema fully deterministically, I employed a little bit of statistics in the process. I was able to determine the schema faster (with one pass) and higher fidelity (rejecting bogus data rather than accepting bogus data by widening schema). After all, statistics is an important part of machine learning today and it allows us to inject a little bit of intelligence into the ETL process.

I got my Lumia 900 in April, 2012. A few month later, I was disappointed when the news broke out that Lumia 900 would not be able to upgrade to Windows Phone 8. Luckily, AT&T allows early upgrade to Lumia 920 so that I can continue developing software for latest Windows Phone OS without waiting for my two year commitment to end.

Lumia 900 was known for excellent software optimization so that the phone felt very fast even with a single core processor. Now Lumia 920, with its Snapdragon S4 processor, feels blazingly fast. The most obvious improvement is acquiring GPS signal; it is almost instant now when I start an app that needs GPS. 

Apart from being faster and more fluid, I also like the new tile size options that allows me to organize my start screen better. The new phone has a slightly larger display size (4.5” vs. 4.2”) though the physical dimension remains the same. The phone has a larger storage (32GB vs. 16GB) that allows me stored more free Nokia Drive offline maps on my phone so that I can use the GPS without using my monthly data plan.

So much for my first impression, I will tell more story when I start developing for Windows Phone 8.