Pablo M. Cibraro (aka Cibrax)

Most of the Web APIs available out there in the web nowadays use some kind of authentication for identifying client applications. Although they implement authentication in different ways, they can be typically categorized in three main groups, services that use Keys, OAuth or HMAC.

Keys is the first scenario and probably the simplest one. Every client application is identified with a simple and fixed application key. This authentication mechanism is perhaps a bit weak, but the data that the service has to offer is not sensitive at all. The data is available for everyone with a key, and it’s pretty much used for public services such as Google maps or a search for public pictures in Instagram for example. The only purpose of the key is to identify clients and apply different SLA (service level agreements) such as api quotas, availability, etc.

HMAC is typically used for consuming sensitive data that is only consumed by his owner and not shared with anyone else. This kind of authentication is typically used in multitenant applications, where a tenant is the owner of the data. This model fits real well with cloud computing where a vendor such as AWS or Windows Azure use a key for identifying the tenant and provide the right services and private data. No matter which client application is used to consume the services and data, the main purpose of the key is to identify the tenant. Hawk is new specification born in this area to standardize how HMAC authentication.  

OAuth is last one and probably the most complicated one. It was born with the idea of delegating authorization in the web 2.0. The service who owns the data can use OAuth to share that data with other services or applications without compromising the owner credentials.

The analogy given by Eran Hammer Lahav in this post "Explaining OAuth" is very close to what the specification tries to address,

"Many luxury cars today come with a valet key. It is a special key you give the parking attendant and unlike your regular key, will not allow the car to drive more than a mile or two. Some valet keys will not open the trunk, while others will block access to your onboard cell phone address book. Regardless of what restrictions the valet key imposes, the idea is very clever. You give someone limited access to your car with a special key, while using another key to unlock everything else."

This kind of authentication makes a lot of sense in social media services like Twitter, Facebook, Windows Live or Google to name a few, where the service owns some private data like contacts or pictures that can shared with other applications without putting the user credentials into risk.

OAuth assigns a key to every different client application allowed to consume the data, so the access can easily be revoked by disabling the key associated that client application.

One of the features supported by Hawk, an HTTP authentication protocol based on HMAC, is to provide read-only access to a Web API for a short period time.  That’s performed through a token called “bewit” that a Web API can provide to a client. That token is only valid for Http GET calls and it can be used for a limited period of time.

I already implemented this feature in my Hawk port for .NET. A bewit token can be generated as it is shown below,

var credential = new HawkCredential
{
     Id = "dh37fgj492je",
     Key = "werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn",
     Algorithm = "hmacsha256",
     User = "steve"
 };

var bewit = Hawk.GetBewit("localhost", 
  new Uri("http://localhost:8091/Api/HelloWorld"), 
  credential, 
  60000);

The GetBewit method expects the following arguments,

• The host name
• The complete request URI
• The Hawk credentials with information about the key and algorithm to use
• A time-to-live setting in seconds for the token

That token is an string representation that you can add as a additional query string in the Web API call.

new HttpRequestMessage(HttpMethod.Get, 
  "http://localhost:8091/Api/HelloWorld?bewit=" + bewit);

In that way, you can share a link to your Web API with a limited access for a period of time to someone without having to share any security credentials.

On the service side is as simple as configuring the HawkMessageHandler as part of the Web API configuration,

var handler = new HawkMessageHandler((id) =>
{
     return new HawkCredential
     {
           Id = id,
           Key = "werxhqb98rpaxn39848xrunpaw3489ruxnpa98w4rxn",
           Algorithm = "hmacsha256",
           User = "steve"
       };
 });

config.MessageHandlers.Add(handler);

The handler will automatically detect a bewit token in the query string, and it will performed all the required validations.

ASP.NET ships with two built-in mechanisms for doing logging and troubleshooting.  Chasing errors without knowing these two mechanisms might be a daunting task, specially if they happen in the runtime pipeline much before a message gets to a handler or a controller.

The first mechanism is the error policy. You can configure the error policy preferences as part of the configuration object (HttpConfiguration) in the IncludeErrorDetailPolicy property. This is just an enum that instructs Web API about how to deal with exceptions.

The possible values for this enum are,

• Default: It’s uses the customErrors configuration settings if you are using ASP.NET as host or LocalOnly for self-host.
• LocalOnly: Only includes error details for local requests
• Always: Always includes error details
• Never: Never includes error details

When an exception happens, Web API will check the value on this setting for including details about the exception in the response message or not. For example, if Always is enabled, Web API will serialize the exception details as part of the message that you get as response.

The second mechanism is Tracing. Tracing is a service that you can inject as part of the configuration object as well. The default implementation does do anything.

public static void Register(HttpConfiguration config)
{
    config.Services.Replace(typeof(ITraceWriter), new MyTracer());
}

MyTracer is a custom implementation of the ITraceWriter service, which Web API uses for tracing purposes. This is a general tracing mechanism, so Web API will call it for logging everything and not just errors.

public class MyTracer : ITraceWriter
{
    public void Trace(HttpRequestMessage request, string category, TraceLevel level, 
        Action<TraceRecord> traceAction)
    {
        TraceRecord rec = new TraceRecord(request, category, level);
        traceAction(rec);
        WriteTrace(rec);
    }

    protected void WriteTrace(TraceRecord rec)
    {
        var message = string.Format("{0};{1};{2}", 
            rec.Operator, rec.Operation, rec.Message);
        System.Diagnostics.Trace.WriteLine(message, rec.Category);
    }
}

If any of these two work for you, you can still use an Error Filter.  Tugberk has written a blog post about how to integrate ELMAH with an Error Filter in Web API here.

Message Handlers are one of the core components for message processing in Web API. They use an asynchronous model for processing messages, so they receive a request message and returns a Task with the corresponding response.

protected internal abstract Task<HttpResponseMessage> SendAsync(HttpRequestMessage request,
 CancellationToken cancellationToken);

In most cases, a message handler does something and delegates some other work to the rest of the handlers configured in the pipeline. For example, a handler for security checks the Auth Http header, and delegates the call the handlers configured out of the box by Web API, which eventually will call a controller method. The framework also provides a base class to make delegation implicit, DelegatingHandler, which receives the next handler to call as part of the constructor.

The following example shows a message handler implementation for basic authentication,

public class BasicAuthHandler : DelegatingHandler
{
   Func<string, string, IPrincipal> auth;

   public BasicAuthHandler(HttpMessageHandler innerHandler, Func<string, string, IPrincipal> auth)
       : base(innerHandler)
   {
       this.auth = auth;
    }

    protected override System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, 

System.Threading.CancellationToken cancellationToken)
    {
         if (request.Headers.Authorization != null &&
             !string.Equals(request.Headers.Authorization.Scheme, "basic", 

StringComparison.InvariantCultureIgnoreCase))
         {
              return base.SendAsync(request, cancellationToken);
          }

          if (request.Headers.Authorization == null ||
              string.IsNullOrWhiteSpace(request.Headers.Authorization.Scheme))
          {
              return ChallengeResponse(request);
           }

          var authToken = request.Headers.Authorization.Parameter;
          var decodedToken = Encoding.UTF8.GetString(Convert.FromBase64String(authToken));

          var username = decodedToken.Substring(0, decodedToken.IndexOf(":"));
          var password = decodedToken.Substring(decodedToken.IndexOf(":") + 1);

          var principal = this.auth(username, password);
            
          if (principal == null)
          {
              return ToResponse(request, HttpStatusCode.Unauthorized, "Invalid credentials");
           }

          Thread.CurrentPrincipal = principal;
          if (HttpContext.Current != null)
             HttpContext.Current.User = principal;
                        
          return base.SendAsync(request, cancellationToken);
   }

   private static Task<HttpResponseMessage> ChallengeResponse(HttpRequestMessage request)
   {
       var tsc = new TaskCompletionSource<HttpResponseMessage>();

       var response = request.CreateResponse(HttpStatusCode.Unauthorized);
        response.Headers.WwwAuthenticate.Add(new AuthenticationHeaderValue("basic", "realm=localhost"));

       tsc.SetResult(response);

       return tsc.Task;
     }

     private static Task<HttpResponseMessage> ToResponse(HttpRequestMessage request, 

HttpStatusCode code, string message)
     {
        var tsc = new TaskCompletionSource<HttpResponseMessage>();

         var response = request.CreateResponse(code);
         response.ReasonPhrase = message;

         tsc.SetResult(response);

         return tsc.Task;
      }
 }

A good thing about Message Handler is that you can configure them globally or per route. In this case, if you want to enable basic authentication for some routes only, it’s a matter of configuring this handler in the routes you want to have protected.

config.Routes.MapHttpRoute(
   "BasicAuth",
   "MyController",
   new { controller = "MyController" },
   null,
   new BasicAuthHandler(new HttpControllerDispatcher(config), (u, p) => 
   {
      return new GenericPrincipal(new GenericIdentity(u, new string[] {}));
   }));

As you can see, the Inner Handler is a built-in handler provided by Web API, HttpControllerDispatcher, which does all the magic for processing the request and pass it over to the controller action. You can also inject any other dependency as part of the constructor. One thing to consider is that message handlers are singleton if you configure them this way, so make sure to inject the dependencies in the right way for avoiding memory leaks (If you have to use a repository for example, you might want to inject a factory or pass a delegate for resolving the dependencies from a DI container).

A common requirement in SaaS applications is the ability to route requests to different tenants based on a URL routing strategy.

In most cases, a domain prefix is good enough to identify tenants (e.g. mytenant.xxxxx.com). That approach typically relies on CName records for mapping the prefixes or tenants and the domain name to an URL where the application is actually hosted. Many cloud providers support the idea of mapping custom domains to their web hosted services, so this approach with CName works just fine.

An evident problem is that you would require a different CName record per prefix or tenant. If you have to create those records manually, this approach simply does not scale as the number of tenants increase.

An alternative is to use a wildcard CName, and route all the requests that match that wildcard to the hosted application in the cloud. For example, *.xxxxxx.com to your web application url.

Many DNS servers don’t support wildcards in CName such as the ones offered by free for GoDaddy or NameCheap. However, AWS Route 53 supports wildcards and also an API to manage almost everything in the DNS tables.

Configuring AWS Route 53 is relatively easy. Assuming that you already have a AWS account, you need to create first a hosted zone, which represents the association of a domain name with a set of name servers provided by Route 53. Once you specified the domain name (e.g cibrax.me), and the hosted zone is created. Route53 will show you a list of name servers you need to use. If you already own a domain in other place like GoDaddy or NameCheap, you need to go there and update the list of name servers associated to that domain.

Afterwards, you need to create a resource record set, which is basically the CName record containing the wildcard or prefix you want to use. Here, you can specify the CNAme record and the mapped URL. For example, *.cibrax.me goes to www.xxxxxxxx.com.

That’s all from the point of view of DNS configuration. The rest is part of the implementation of your web application.

If you are building a SaaS application in Windows Azure that relies on SQL Azure, it’s probably that you will need to support multiple tenants at database level.

This is short overview of the different approaches you can use for support that scenario,

A different database per tenant

A new database is created and assigned when a tenant is provisioned.

Pros

• Complete isolation between tenants. All the data for a tenant lives in a database only he can access.

Cons

• It’s not cost effective. SQL Azure databases are not cheap, and the minimum size for a database is 1GB.  You might be paying for storage that you don’t really use.
• A different connection pool is required per database.
• Updates must be replicated across all the databases
• You need multiple backup strategies across all the databases

Multiple schemas in a database shared by all the tenants

A single database is shared among all the tenants, but every tenant is assigned to a different schema and database user.

Pros

• You only pay for a single database.
• Data is isolated at database level. If the credentials for one tenant is compromised, the rest of the data for the other tenants is not.

Cons

• You need to replicate all the database objects in every schema, so the number of objects can increase indefinitely.
• Updates must be replicated across all the schemas.
• The connection pool for the database must maintain a different connection per tenant (or set of credentials)
• A different user is required per tenant, which is stored at server level. You have to backup that user independently.

Centralizing the database access with store procedures in a database shared by all the tenants

A single database is shared among all the tenants, but nobody can read the data directly from the tables. All the data operations are performed through store procedures that centralize the access to the tenant data. The store procedures contain some logic to map the database user to an specific tenant.

Pros

• You only pay for a single database.
• You only have a set of objects to maintain and backup.

Cons

• There is no real isolation. All the data for the different tenants is shared in the same tables.
• You can not use traditional ORM like EF code first for consuming the data.
• A different user is required per tenant, which is stored at server level. You have to backup that user independently.

SQL Federations

A single database is shared among all the tenants, but a different federation is used per tenant. A federation in few words, it’s a mechanism for horizontal scaling in SQL Azure, which basically uses the idea of logical partitions to distribute data based on certain criteria.

Pros

• You only have a single database with multiple federations.
• You can use filtering in the connections to pick the right federation, so any ORM could be used to consume the data.

Cons

• There is no real isolation at that database level. The isolation is enforced programmatically with federations.

For simple scenarios of Web API consumption where identity delegation is not required, traditional http authentication schemas such as basic, certificates or digest are the most used nowadays. All these schemas rely on sending the caller credentials or some representation of it in every request message as part of the Authorization header, so they are prone to suffer phishing attacks if they are not correctly secured at transport level with https.

In addition, most client applications typically authenticate two different things, the caller application and the user consuming the API on behalf of that application. For most cases, the schema is simplified by using a single set of username and password for authenticating both, making necessary to store those credentials temporally somewhere in memory.

The true is that you can use two different identities, one for the user running the application, which you might authenticate just once during the first call when the application is initialized, and another identity for the application itself that you use on every call.

Some cloud vendors like Windows Azure or Amazon Web Services have adopted an schema to authenticate the caller application based on a Message Authentication Code (MAC) generated with a symmetric algorithm using a key known by the two parties, the caller and the Web API.

The caller must include a MAC as part of the Authorization header created from different pieces of information in the request message such as the address, the host, and some other headers. The Web API can authenticate the caller by using the key associated to it and validating the attached MAC in the request message. In that way, no credentials are sent as part of the request message, so there is no way an attacker to intercept the message and get access to those credentials.

Anyways, this schema also suffers from some deficiencies that can generate attacks. For example, brute force can be still used to infer the key used for generating the MAC, and impersonate the original caller. This can be mitigated by renewing keys in a relative short period of time. This schema as any other can be complemented with transport security.

Eran Rammer, one of the brains behind OAuth, has recently published an specification of a protocol based on MAC for Http authentication called Hawk. The initial version of the spec is available here. A curious fact is that the specification per se does not exist, and the specification itself is the code that Eran initially wrote using node.js.

In that implementation, you can associate a key to an user, so once the MAC has been verified on the Web API, the user can be inferred from that key. Also a timestamp is used to avoid replay attacks.

As a pet project, I decided to port that code to .NET using ASP.NET Web API, which is available also in github under https://github.com/pcibraro/hawknet

Enjoy!.

If you are dealing with asynchronous work in .NET, you might know that the Task class has become the main driver for wrapping asynchronous calls. Although this class was officially introduced in .NET 4.0, the programming model for consuming tasks was much more simplified in C# 5.0 in .NET 4.5 with the addition of the new async/await keywords. In a nutshell, you can use these keywords to make asynchronous calls as if they were sequential, and avoiding in that way any fork or callback in the code. The compiler takes care of the rest.

I was yesterday writing some code for making multiple asynchronous calls to backend services in parallel. The code looked as follow,

var allResults = new List<Result>();
foreach(var provider in providers)
{
  var results = await provider.GetResults();
  allResults.AddRange(results);
}
return allResults;

You see, I was using the await keyword to make multiple calls in parallel. Something I did not consider was the overhead this code implied after being compiled. I started an interesting discussion with some smart folks in twitter. One of them, Tugberk Ugurlu, had the brilliant idea of actually write some code to make a performance comparison with another approach using Task.WhenAll.

There are two additional methods you can use to wait for the results of multiple calls in parallel, WhenAll and WaitAll.

WhenAll creates a new task and waits for results in that new task, so it does not block the calling thread. WaitAll, on the other hand, blocks the calling thread. This is the code Tugberk initially wrote, and I modified afterwards to also show the results of WaitAll.

 class Program
    {
        private static Func<Stopwatch, Task>[] funcs = new Func<Stopwatch, Task>[] { 
            async (watch) => { watch.Start(); await Task.Delay(1000); 
                Console.WriteLine("1000 one has been completed."); },
            async (watch) => { await Task.Delay(1500); 
                Console.WriteLine("1500 one has been completed."); },
            async (watch) => { await Task.Delay(2000); 
                Console.WriteLine("2000 one has been completed."); watch.Stop(); 
                Console.WriteLine(watch.ElapsedMilliseconds + "ms has been elapsed."); }
        };
        static void Main(string[] args)
        {
            Console.WriteLine("Await in loop work starts...");
            DoWorkAsync().ContinueWith(task =>
            {
                Console.WriteLine("Parallel work starts...");
                DoWorkInParallelAsync().ContinueWith(t =>
                    {
                        Console.WriteLine("WaitAll work starts...");
                        WaitForAll();
                    });
            });
            Console.ReadLine();
        }
        static async Task DoWorkAsync()
        {
            Stopwatch watch = new Stopwatch();
            foreach (var func in funcs)
            {
                await func(watch);
            }
        }
        static async Task DoWorkInParallelAsync()
        {
            Stopwatch watch = new Stopwatch();
            await Task.WhenAll(funcs[0](watch), funcs[1](watch), funcs[2](watch));
        }
        static void WaitForAll()
        {
            Stopwatch watch = new Stopwatch();
            Task.WaitAll(funcs[0](watch), funcs[1](watch), funcs[2](watch));
        }
    }

After running this code, the results were very concluding.

Await in loop work starts...
1000 one has been completed.
1500 one has been completed.
2000 one has been completed.
4532ms has been elapsed.

Parallel work starts...
1000 one has been completed.
1500 one has been completed.
2000 one has been completed.
2007ms has been elapsed.

WaitAll work starts...
1000 one has been completed.
1500 one has been completed.
2000 one has been completed.
2009ms has been elapsed.

The await keyword in a loop does not really make the calls in parallel.

UrlHelper is the class you can use in ASP.NET Web API to automatically infer links from the routing table without hardcoding anything. For example, the following code uses the helper to infer the location url for a new resource,

public HttpResponseMessage Post(User model)
{
  var response = Request.CreateResponse(HttpStatusCode.Created, user);
  var link = Url.Link("DefaultApi", new { id = id, controller = "Users" });
  response.Headers.Location = new Uri(link);
  return response;
}

That code uses a previously defined route “DefaultApi”, which you might configure in the HttpConfiguration object (This is the route generated by default when you create a new Web API project).

The problem with UrlHelper is that it requires from some initialization code before you can invoking it from a unit test (for testing the Post method in this example). If you don’t initialize the HttpConfiguration and Request instances associated to the controller from the unit test, it will fail miserably.

After digging into the ASP.NET Web API source code a little bit, I could figure out what the requirements for using the UrlHelper are. It relies on the routing table configuration, and a few properties you need to add to the HttpRequestMessage. The following code illustrates what’s needed,

var controller = new UserController();
controller.Configuration = new HttpConfiguration();
var route = controller.Configuration.Routes.MapHttpRoute(
         name: "DefaultApi",
         routeTemplate: "api/{controller}/{id}",
         defaults: new { id = RouteParameter.Optional }
);
var routeData = new HttpRouteData(route, 
       new HttpRouteValueDictionary 
       { 
          { "id", "1" },
          { "controller", "Users" } 
       }
);
controller.Request = new HttpRequestMessage(HttpMethod.Post, "http://localhost:9091/");
controller.Request.Properties.Add(HttpPropertyKeys.HttpConfigurationKey, controller.Configuration);
controller.Request.Properties.Add(HttpPropertyKeys.HttpRouteDataKey, routeData);

The HttpRouteData instance should be initialized with the route values you will use in the controller method (“id” and “controller” in this example). Once you have correctly setup all those properties, you shouldn’t have any problem to use the UrlHelper. There is no need to mock anything else.

Enjoy!!.

One of the nice things about the new HttpClient in System.Net.Http is the support for mocking responses or handling requests in a http server hosted in-memory. While the first option is useful for scenarios in which we want to test our client code in isolation (unit tests for example), the second one enables more complete integration testing scenarios that could include some more components in the stack such as model binders or message handlers for example.  

The HttpClient can receive a HttpMessageHandler as argument in one of its constructors.

public class HttpClient : HttpMessageInvoker
{
   public HttpClient();
   public HttpClient(HttpMessageHandler handler);
   public HttpClient(HttpMessageHandler handler, bool disposeHandler);

For the first scenario, you can create a new HttpMessageHandler that fakes the response, which you can use in your unit test. The only requirement is that you somehow inject an HttpClient with this custom handler in the client code.

public class FakeHttpMessageHandler : HttpMessageHandler
{
  HttpResponseMessage response;

  public FakeHttpMessageHandler(HttpResponseMessage response)
  {
       this.response = response;
   }

   protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, 
System.Threading.CancellationToken cancellationToken)
   {
      var tcs = new TaskCompletionSource<HttpResponseMessage>();
            
      tcs.SetResult(response);

      return tcs.Task;
    }
}

In an unit test, you can do something like this.

var fakeResponse = new HttpResponse();
var fakeHandler = new FakeHttpMessageHandler(fakeResponse);
var httpClient = new HttpClient(fakeHandler);

var customerService = new CustomerService(httpClient);

// Do something

// Asserts

CustomerService in this case is the class under test, and the one that receives an HttpClient initialized with our fake handler.

For the second scenario in integration tests, there is a In-Memory host “System.Web.Http.HttpServer” that also derives from HttpMessageHandler and you can use with a HttpClient instance in your test. This has been discussed already in these two great posts from Pedro and Filip.