Looking at ASP.NET MVC 5.1 and Web API 2.1 - Part 4 - Web API Help Pages, BSON, and Global Error Handling

Wednesday, February 26, 2014

This is part 4 of a series covering some of the new features in the ASP.NET MVC 5.1 and Web API 2.1 releases. The last one! If you've read them all, you have earned twelve blog readership points... after you finish this one, of course. Here are the previous posts:

Part 1: Overview and Enums
Part 2: Attribute Routing with Custom Constraints
Part 3: Bootstrap and JavaScript enhancements
Part 4: Web API Help Pages, BSON, and Global Error Handling

The sample project covering the posts in this series is here; other referenced samples are in the ASP.NET sample repository.

As a reminder, Part 1 explained that ASP.NET MVC 5.1 / Web API 2.1 is a NuGet update for the MVC 5 / Web API 2 releases that shipped with Visual Studio 2013. There will be a Visual Studio update that will make them the defaults when you create new projects.

In this post, we'll look at new features in ASP.NET Web API 2.1.

Attribute Routing

We already looked at the updates to Attribute Routing improvements for both ASP.NET Web API and MVC in the second post in this series, I just want to call it out again since this post is overviewing all of the other new features in ASP.NET Web API 2.1 and the Attribute Routing support for custom constraints is one of the top features in the ASP.NET Web API 2.1 release.

As a reminder, custom route constraints make it really easy to create wrap route matching logic in a constraint which can then be placed on ApiControllers or actions like this:

[VersionedRoute("api/Customer", 1)]
public class CustomerVersion1Controller : ApiController
{
    // controller code goes here
}
[VersionedRoute("api/Customer", 2)]
public class CustomerVersion2Controller : ApiController
{
    // controller code goes here
}

In that example, the custom VersionedRoute constraint looks for an api-version header and forwards the request to the correct controller. See the post for more information, including a link to the sample application.

Help Page improvements

Okay, let's dig into some of the cool new features we haven't seen yet. To start with, I'm going to scaffold out a new PersonApiController using the same Person class I've used earlier in this series, shown below:

public class Person
{
    [ScaffoldColumn(false)]
    public int Id { get; set; }
    [UIHint("Enum-radio")]
    public Salutation Salutation { get; set; }
    [Display(Name = "First Name")]
    [MinLength(3, ErrorMessage = "Your {0} must be at least {1} characters long")]
    [MaxLength(100, ErrorMessage = "Your {0} must be no more than {1} characters")]
   public string FirstName { get; set; }
    [Display(Name = "Last Name")]
    [MinLength(3, ErrorMessage = "Your {0} must be at least {1} characters long")]
    [MaxLength(100, ErrorMessage = "Your {0} must be no more than {1} characters")]
    public string LastName { get; set; }
    public int Age { get; set; }
}
//I guess technically these are called honorifics
public enum Salutation : byte
{
    [Display(Name = "Mr.")]   Mr,
    [Display(Name = "Mrs.")]  Mrs,
    [Display(Name = "Ms.")]   Ms,
    [Display(Name = "Dr.")]   Doctor,
    [Display(Name = "Prof.")] Professor,
    Sir, 
    Lady, 
    Lord
}

And we're using the standard Web API scaffolding:

Nothing has really changed for the top level ASP.NET Web API Help Page - you get a generated list of API calls for each API Controller.

What has changed is what you see when you click through on one of the API calls, e.g. the PersonApi GET method. Here's how that looked in ASP.NET Web API 2.1:

It shows sample data in JSON and XML, and you can kind of guess what they are if you've named your model properties well, but there's no information on type, model attributes, validation rules, etc.

Here's how it looks in ASP.NET Web API 2:

The Response Formats section hasn't changed, but now we have a Resource Description area at the top. Let's take a closer look at that:

Here we're clearly displaying both the type and validation rules.

Note that the Salutation type is hyperlinked, since it's using our custom Salutation enum. Clicking through shows the possible values for that enum:

If you've done any work integrating with API's that had minimal or out of date documentation, hopefully the value of the above is really clear. What's great is that this is generated for me at runtime, so it's always up to date with the latest code. If my Web API is in production and I add a new Enum value or change a validation rule, the live documentation on the web site is immediately updated without any work or extra thought on my part as soon as I deploy the code.

Short detour: Filling in Descriptions using C# /// Comments

Now that we've got documentation for our model types, it's clear that we could improve it a bit. The most obvious thing is that there's no provided description. That's easy to add using C# /// comments (aka XML Comments). ASP.NET Web API Help Pages have had support for /// comments documentation for a while, it just hasn't been this obvious.

The ASP.NET Web API Help Pages are implemented in a really clear, open model: it's all implemented in an ASP.NET MVC Area within your existing site. If you're not familiar with ASP.NET MVC Areas, they're a way to allow you to segment your applications into with separate routes, models, views and controllers but still keep them in the same project so it's easier to manage, share resources, etc.

Here's the Help Page Area within the sample project we're working on:

1. In the above screenshot, I've highlighted the \App_Start\HelpPageConfig.cs file because that's where we're going to set up the XML comments. There's a Register method right at the top with the following two lines:

//// Uncomment the following to use the documentation from XML documentation file.
//config.SetDocumentationProvider(new XmlDocumentationProvider(HttpContext.Current.Server.MapPath("~/App_Data/XmlDocument.xml")));

So to use that, we'll uncomment the second line, just as the instructions say.

2. Note that the comments are pointing to an XmlDocument.xml file. We need to check a box in the project settings to generate that XML file as shown below.

That's it!

Once that's done, I'm going to throw the /// comments on the controller methods and model properties and generate XML comments. I used GhostDoc to generate the comments, then cleaned them up and editorialized a bit.

/// <summary>
/// This is an example person class. It artisanally crafted by a
/// bearded, bespeckled craftsman after being lovingly sketched
/// in a leather bound notebook with charcoal pencils.
/// </summary>
public class Person
{
    [ScaffoldColumn(false)]
    public int Id { get; set; }
    /// <summary>
    /// This uses a custom salution enum since there's apparently no ISO standard.
    /// </summary>
    /// <value>
    /// The person's requested salutation.
    /// </value>
    [UIHint("Enum-radio")]
    public Salutation Salutation { get; set; }
    [Display(Name = "First Name")]
    [MinLength(3, ErrorMessage = "Your {0} must be at least {1} characters long")]
    [MaxLength(100, ErrorMessage = "Your {0} must be no more than {1} characters")]
   public string FirstName { get; set; }
    [Display(Name = "Last Name")]
    [MinLength(3, ErrorMessage = "Your {0} must be at least {1} characters long")]
    [MaxLength(100, ErrorMessage = "Your {0} must be no more than {1} characters")]
    public string LastName { get; set; }
    /// <summary>
    /// This is the person's actual or desired age.
    /// </summary>
    /// <value>
    /// The age in years, represented in an integer.
    /// </value>
    public int Age { get; set; }
}

And here's the updated help page with the descriptions:

There are a ton of other features in the HelpPageConfig - you could pull your documentation from a database or CMS, for example. And since it's all implemented in standard ASP.NET MVC, you can modify the views or do whatever else you want. But it's nice to have these new features available out of the box.

BSON

BSON is a binary serialization format that's similar to JSON in that they both store name-value pairs, but it's quite different in how the data is actually stored. BSON serializes data in a binary format, which can offer some performance benefits for encode / decode / traversal. It's been possible to hook up a custom BSON formatter in ASP.NET Web API before; Filip and others have written comprehensive blog posts describing how to do just that. It's even easier now - both for clients and servers - since the BSON formatter is included with ASP.NET Web API.

Important note: BSON isn't designed to be more compact than JSON, in fact it's often bigger (depending on your data structure and content). That's because, unlike JSON, BSON embeds type information in the document. That makes for fast scan and read, but it's holding more data than the equivalent JSON document. That means that it will be faster in some cases, but it may be slower in other cases if your messages are bigger. This shows the value of content negotiation and flexible formatters in ASP.NET Web API - you can easily try out different formatters, both on the client and server side, and use the best one for the job.

I'll look at two examples here.

Testing BSON with a text-heavy model

For the first example, I'll use the Person class we've been using for our previous examples. I'd like to have a lot more people in my database. I grabbed some absolutely silly code I wrote 7 years ago that generates fake surnames (Generate random fake surnames) and added a controller action to slam 500 new people with a first name of Bob but a random last name and age into the database. Then I clicked on it a few times.

Turning on the BSON formatter is just a one line code change:

public static class WebApiConfig
{
    public static void Register(HttpConfiguration config)
    {
        config.Formatters.Add(new BsonMediaTypeFormatter());
        // ...
    }
}

Now whenever a client sends an Accept header for application/bson, they'll get the data in BSON format. For comparison, I'm making two requests in Fiddler. Here's a request with no Accept header specified, so we get JSON:

The content-length there is 118,353 bytes.

Now I'm setting an Accept header with application/bson:

Notice that this BSON request is 134,395 bytes, or 13% larger. I've marked some of the type identifiers in there, but you can see there are a lot more since they're lined up in columns.

Place your bets: think the faster BSON serializer will be faster, despite the larger payload size? Before we answer that, we'll add in a second scenario that replaces our text-heavy Person class with a quite exciting BoringData class that's mostly numeric and binary data:

public class BoringData
{
    public int Id { get; set; }
    public long DataLong { get; set; }
    public byte[] DataBytes { get; set; }
    public DateTime DataDate { get; set; }
}

And here's the test we'll throw at both of these:

private HttpClient _client = new HttpClient();
static void Main(string[] args)
{
    try
    {
        Console.WriteLine("Hit ENTER to begin...");
        RunAsync().Wait();
    }
    finally
    {
        Console.WriteLine("Hit ENTER to exit...");
        Console.ReadLine();
    }
}
private async static Task RunAsync()
{
    using (HttpClient client = new HttpClient())
    {
        await RunTimedTest<BoringData>(client, new JsonMediaTypeFormatter(), "http://localhost:29108/api/BoringDataApi", "application/json");
        await RunTimedTest<BoringData>(client, new BsonMediaTypeFormatter(), "http://localhost:29108/api/BoringDataApi", "application/bson");
        await RunTimedTest<Person>(client, new JsonMediaTypeFormatter(), "http://localhost:29108/api/PersonApi", "application/json");
        await RunTimedTest<Person>(client, new BsonMediaTypeFormatter(), "http://localhost:29108/api/PersonApi", "application/bson");
    }
}
public static async Task RunTimedTest<T>(HttpClient _client, MediaTypeFormatter formatter, string Uri, string mediaHeader)
{
    int iterations = 500;
    _client.DefaultRequestHeaders.Accept.Clear();
    _client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue(mediaHeader));
    MediaTypeFormatter[] formatters = new MediaTypeFormatter[] { formatter };
    var watch = Stopwatch.StartNew();
    for (int i = 0; i < iterations; i++)
    {
        var result = await _client.GetAsync(Uri);
        var value = await result.Content.ReadAsAsync<T[]>(formatters);
    }
    Console.WriteLine("Format: {0,-20} Type: {1,-15}\t Time (s):{2,10:ss\\.fff}", mediaHeader, typeof(T).Name, watch.Elapsed);
}

The BoringDataApi controller's GET method returns lots of data, as you'd expect:

public class BoringDataApiController : ApiController
{
    static Random rng = new Random(Guid.NewGuid().GetHashCode());
    public IEnumerable<BoringData> GetBoringData()
    {
        return GetLotsOfBoringData(100);
    }
    private IEnumerable<BoringData> GetLotsOfBoringData(int quantity)
    {
        byte[] buf1 = new byte[10000];
        byte[] buf2 = new byte[64];

        for (int i = 1; i < quantity; i++)
        {
            rng.NextBytes(buf1);
            rng.NextBytes(buf2);
            yield return new BoringData
            {
                Id  = i,
                DataBytes = buf1,
                DataDate = DateTime.UtcNow,
                DataLong = BitConverter.ToInt64(buf2,0)
            };
        }
    }
}

So, big picture, the test harness will run 500 end to end tests on both controllers, requesting both Person and BoringData as both JSON and BSON. What we're not comparing is the difference between the Person and BoringData responses, we're just looking for some general trends to see if BSON is faster than JSON for a mostly-textual and mostly-binary model. Yes, Kelly Sommers will beat me up for this, and I'm okay with that. My goal is to get some basic guidelines on when BSON works better than JSON.

The real point here is that you won't know how your API or a specific content type will perform until you test it.

So how'd we do?

In this case (and I ran this test many times with the same general result) BSON was a lot faster for mostly binary/numeric data, and a little slower for mostly textual data. In this pretty random example, BSON was 140% faster for the mostly binary case and and 21% slower for the mostly-textual case. That's because both serialize textual data to UTF-8, but BSON includes some additional metadata.

So, very generally speaking, if your service returns a lot of binary / numeric / non-textual data, you should really look at BSON. If you're looking for a silver bullet, you may have to pony up for some silver.

Easier implementations due to BaseJsonMediaTypeFormatter

Yes, that's the most boring heading you'll ever see. But it's hopefully true. The new BaseJsonMediaTypeFormatter has been designed to make it easier to implement new different serialization formats easier, since the the internal JSON formatters have been redesigned to make it easier to extend. I asked Doug, the dev that did most of the work for this BSON update, about his commit message saying recent changes will make it easier to make other formatters like MessagePack happen and he said:

Yes. BaseJsonMediaTypeFormatter introduces a few Json.Net types and concepts. But it also provides solid and reusable async wrappers around easier-to-implement sync methods.

The main thing I've noticed there is the common BaseJsonMediaTypeFormatter. There's not a whole lot of code in the BsonMediaTypeFormatter, since a lot of it's in the common base and in other support classes.

And while I'm mentioning MessagePack, I think it's another great option that's really worth looking at, since (unlike BSON) MessagePack is designed for small message size. There's a MsgPack formatter available now in the WebApiContrib formatters collection, and Filip Woj. wrote a nice blog post overview here: Boost up your ASP.NET Web API with MessagePack.

Global Error Handling

The last feature we'll look at is Global Error Handling. The name's pretty self-descriptive: it lets you register handlers and loggers which will respond to any unhandled exceptions across your entire Web API application.

Global error handling is especially useful in Web API because of the way the parts are so loosely coupled and composable - you've got all kinds of different handlers and filters, wired together with a very configurable system that encourages dependency injection... There's a lot going on.

Note: You can download the PDF of this poster from the ASP.NET site.

That provides you tons of flexibility when you're building HTTP services, but it can make it hard to find out what's wrong when there's a problem. Exception filters help, but as David Matson notes, they don't handle:

Exceptions thrown from controller constructors
Exceptions thrown from message handlers
Exceptions thrown during routing
Exceptions thrown during response content serialization

I recommend David Matson's Web API Global Error Handling wiki article entry in the ASP.NET repository for more information on design and technical implementation. The short excerpt is that you can register one IExceptionHandler and multiple IExceptionLogger instances in your application, and they'll respond to all Web API exceptions.

There's already a pretty clear sample in the Web API samples which shows a GenericTextExceptionHandler (which returns a generic exception message for unhandled exceptions) and an ElmahExceptionLogger (which implements logging using the popular ELMAH logging system). I've been trying to come up with some other use cases, but I think they captured the main ones here - usually if you have an unhandled exception, you want to log it and make sure you return some sort of useful message to your client.

Both of these interfaces are really simple - they both have a single async method (LogAsync and HandleAsync, respectively) which are passed an ExceptionContext and a CancellationToken.

public interface IExceptionLogger
{
    Task LogAsync(ExceptionLoggerContext context, CancellationToken cancellationToken);
}
public interface IExceptionHandler
{
    Task HandleAsync(ExceptionHandlerContext context, CancellationToken cancellationToken);
}

The ExceptionContext includes the exception as well as a lot of other useful context information:

Exception (Exception)
Request (HttpRequestMessage)
RequestContext ()
ControllerContext (HttpControllerContext)
ActionContext (HttpActionContext)
Response (HttpResponseMessage)
CatchBlock (string)
IsTopLevelCatchBlock (bool)

They're registered in your WebApiConfig like this:

public static class WebApiConfig
{
    public static void Register(HttpConfiguration config)
    {
        config.MapHttpAttributeRoutes();
        // There can be multiple exception loggers. (By default, no exception loggers are registered.)
        config.Services.Add(typeof(IExceptionLogger), new ElmahExceptionLogger());
        // There must be exactly one exception handler. (There is a default one that may be replaced.)
        // To make this sample easier to run in a browser, replace the default exception handler with one that sends
        // back text/plain content for all errors.
        config.Services.Replace(typeof(IExceptionHandler), new GenericTextExceptionHandler());
    }
}

There are a few areas to possibly add to, but I'm going to pass on actually implementing them so I can get this series wrapped before ASP.NET Web API 2.2 ships. Maybe an exercise for the reader?

The sample GenericTextExceptionHandler just returns generic text, but since it has access to the exception details and the context, I'd perhaps instead return an error reference code with the response so customers of my API could use them for troubleshooting. Since one cause of API errors is an inability to access the backend database, I might instead send an error message to Runscope to give more context for debugging there as well.
Rather than implement several IExceptionLogger instances, I'd probably use ELMAH's extensibility to register additional log listeners - there are listeners available for files and databases like XML, JSON, SQL Server, PostgreSQL, SQLite, MySQL, and lots of NuGet packages covering Glimpse integration, RavenDb, MongoDb, etc. One other idea is to add in an additional logger to an additional service like Exceptionless.

Damien (damienbod) has a nice overview of Web API Exception handling, complete with a lot of references: Exploring Web API Exception Handling