Kenny Kerr

App Lockdown Supplement

The original draft of my latest security article for MSDN Magazine, App Lockdown, included sections covering web security. They were ultimately removed as they overlapped slightly with other articles already featured by MSDN Magazine. They are however part of the story I wanted to tell and so I include those sections here.

If you enjoyed my article, I encourage you to read these sections on web security to complete the story.

Protecting Web Clients

Classic web technologies have made it very easy to build insecure web-based systems. Writing secure web services and clients takes some works because the web was founded on the idea of openness rather than privacy and security. The original web authentication protocols, such as Basic authentication, are primitive compared to the network authentications protocols used by Windows. In this section I will focus on the security model of web clients and what you need to do to secure your web service clients.

Unlike the previous types of clients we have talked about, web clients really do not have much say over the quality of authentication and privacy. The client needs to be satisfied with the authentication scheme provided by the server as well as the level of privacy offered for protecting data. With native Windows authentication the client is typically in charge of negotiating a satisfactory form of authentication, data integrity and privacy.

Here is an example of a simple anonymous web request using the .NET Framework:

Uri uri = new Uri("http://server/application");
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(uri);
 
using (WebResponse response = request.GetResponse())
{
    // TODO: read response
}

Assuming the server supports anonymous access, the request should succeed. The simplest way to provide a set of credentials to authenticate the client is to create a NetworkCredential object and assign it to HttpWebRequest’s Credentials property. Using this approach, however, is not very safe since you do not know how those credentials will be used. If the server requested Basic authentication then the credentials will be sent over the wire which is clearly not safe. A better approach is to use the CredentialCache class to have a say in negotiating the authentication protocol. Here is a more interesting example:

Uri uri = new Uri("https://server/application");
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(uri);
 
NetworkCredential credentials = new NetworkCredential("principal@authority",
                                                      "password");
 
CredentialCache cache = new CredentialCache();
 
cache.Add(new Uri("https://server/"),
          "Basic",
          credentials);
 
cache.Add(new Uri("http://server/"),
          "Negotiate",
          credentials);
 
request.Credentials = cache;
 
using (WebResponse response = request.GetResponse())
{
    // TODO: read response
}

In this example the web request will use Basic authentication only if SSL is also used, otherwise it will use Windows authentication. The credentials and authentication scheme to use is determined by finding the closest match to the URI prefix in the cache. If the actual URI begins with the https scheme it indicates that SSL will be used. If you simply want to use the client’s current security context then set the HttpWebRequest’s Credentials property to CredentialCache.DefaultCredentials. Assuming the server supports Windows authentication, a network logon session will be created for you on the server. Keep in mind that Basic authentication can be risky even if you employ SSL. If the server is somehow compromised, a bag guy can easily get the client’s cleartext credentials and do unspeakable things masquerading as the client. Windows authentication provides a better solution by proving the client’s identity without ever sending the credentials over the wire.

Despite the benefits of Windows authentication over HTTP, it is not a great way to make friends on the Internet. Few web clients support Windows authentication and few web servers allow it. Even if they do, you still have to deal will the multitude of firewalls that are specifically designed to block anything but simple HTTP requests over well-known ports. One of the benefits of using SSL is that it can provide client authentication. Although not practical for large-scale web applications like amazon.com where SSL is only used for server authentication and privacy, SSL client authentication provides a very portable and secure form of authentication for web clients. Authentication is achieved through the use of client certificates. In a typical SSL handshake, the server proves its identity by presenting the client with its certificate. To authenticate the client with a client certificate, the client simply needs to provide the server with its certificate. This is clearly an oversimplification, but you get the point.

To use a certificate, the client needs to be issued a certificate by a certificate authority. Since the certificate has an associated private key, it needs to be kept safe. This is handled by a certificate store which you can access using the Cryptography API or though Internet Explorer. Here is an example of using a client certificate:

Uri uri = new Uri("https://server/application");
X509Certificate certificate = X509Certificate.CreateFromCertFile(@"C:\client.cer");
 
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(uri);
request.ClientCertificates.Add(certificate);
 
using (WebResponse response = request.GetResponse())
{
    // TODO: read response
}

As you can see, using SSL and client certificates for authentication is quite simple as long as you have the infrastructure set up to support it. Due to the hostile nature of the Internet, it is very likely that a bag guy will attempt to redirect the client to a different server under his control. Therefore proving the identity of the server is important. Server authentication is also managed by a certificate when using SSL. The server presents the client with a certificate that the client can then use to validate the server’s identity. Of course for this to make any sense, a central authority is required. In Windows networking this is achieved by using Kerberos and a Windows domain controller. The web equivalent is through the use of certificate authorities and a chain of trust. The client needs to trust a certificate authority that directly or indirectly issued the web server with its certificate. In this way the client can validate the integrity of the server’s certificate. This is largely an administrative task to configure servers and clients with mutually acceptable certificate authorities. But sometimes all you want is to use SSL for either client authentication or privacy and you do not care about server authentication. This may be acceptable for connecting to a web server on a trusted network or simply in development and testing of your application to avoid the administrative overhead of certificate management. The challenge is that the default behavior for web clients is to validate the server certificate. To provide custom certificate validation a web client can provide an implementation of the ICertificatePolicy interface and assign it to the static ServicePointManager.CertificatePolicy property. Future web requests that use SSL will then call the ICertificatePolicy.CheckValidationResult method to determine whether or not the certificate should be honored. Here is a simple example that will accept any server certificate:

class CustomCertificatePolicy : ICertificatePolicy
{
    public bool CheckValidationResult(ServicePoint servicePoint,
                                      X509Certificate certificate,
                                      WebRequest request,
                                      int errorCode)
    {
        return true;
    }
}

For more control over validation you can query the certificate as well as check the error code to determine the reason for validation failure. The error codes that you can expect are listed in the documentation for the CERT_CHAIN_POLICY_STATUS structure in the Platform SDK.

So far we have only discussed web clients in the context of simple HTTP requests. Web programming has come a long way since the days of simple HTTP GET request and response messages. As a web client developer you need to understand what is involved in securing you clients in the new programmable web. The following is a classic web service client proxy using the .NET Framework:

[WebServiceBinding(Name="SampleServiceSoap",
                   Namespace=SampleService.WebServiceNamespace)]
class SampleService : SoapHttpClientProtocol
{
    public const string WebServiceNamespace = "http://schemas.kennyandkarin.com/sample/";
 
    [SoapDocumentMethod(WebServiceNamespace + "EchoUserName")]
    public string EchoUserName()
    {
        object[] results = Invoke("EchoUserName", new object[0]);
        return (string) results[0];
    }
}

To connect to a Web service using this proxy class, simply create an instance of the SampleService class, set the Url property to the address of the web service endpoint and call the EchoUserName method. This method will block until the SOAP response message is received and return the string contained in the message. Authentication works in the same way as I described for simple HTTP requests. You can use a CredentialCache object to provide a set of credentials to use and set the Credentials property of the SampleService class. Internally the credentials will be passed to the underlying HttpWebRequest object that will actually make the SOAP request over HTTP. That is Web service client programming with the .NET Framework in a nutshell. To take advantage of some of the more modern web security standards you need to turn to Web Services Enhancements 2.0 (WSE) which is an extension to the .NET Framework that provides an implementation of WS-Security. To start using WSE in your client application you need to add a reference to Microsoft.Web.Services2 to your assembly. To take advantage of it for the SampleService class described above, simply change the base class from SoapHttpClientProtocol to the WebServicesClientProtocol class from the Microsoft.Web.Services2 namespace. That’s it! You’ve just written a WSE client. It may not look like anything has changed and indeed the SOAP envelope body will be exactly the same, but if you happen to have a SOAP trace going, you will have noticed that there is now a SOAP envelope header with all kinds of interesting information. If the Web service you are connecting to knows nothing about WSE, it will just be ignored unless the headers are attributed with mustUnderstand=”1”.

The client can provide credentials in the form of security tokens in the Security SOAP header. Here is an example using a WS-Security UsernameToken:

SampleService service = new SampleService();
service.Url = "http://server/Service.asmx";
 
UsernameToken token = new UsernameToken("principal@authority",
                                        "password",
                                        PasswordOption.SendPlainText);

Security securityHeader = service.RequestSoapContext.Security;
securityHeader.Tokens.Add(token);
       
string userName = service.EchoUserName();

Other security token types are available to support certificate authentication as well as Windows authentication using Kerberos security tokens. The UsernameToken is similar to Basic authentication in HTTP. The credentials are sent across the wire in the clear and it even results in the same type of logon session on the server namely a network logon session with network credentials. While experimenting or developing with Web services I often find it useful to use an HTTP trace tool such as the MSSoapT tool from the Microsoft SOAP Toolkit to capture the request and response messages. To do this you need to direct your client code to a different port that the trace tool is listening on. The trace tool will forward the request on to the final destination after capturing the SOAP envelope. This can be a problem for WSE since it has addressing and security capabilities that validate the destination address to ensure that it matches up to what was expected. Of course since WSE supports addressing, it is possible to indicate what the final destination of the message is so that routing is possible. To enable this, simply create a Uri object with the actual address of the Web service and set the Destination property of the AddressingHeaders object for the request:

UriBuilder uri = new UriBuilder("http://server:8080/Service.asmx");
service.Url = uri.ToString();
 
uri.Port = 80;
AddressingHeaders addressingHeaders = service.RequestSoapContext.Addressing;
addressingHeaders.Destination = new EndpointReference(uri.Uri);

Web Services Enhancements 2.0 provides a wealth of powerful functionality to provide fine grained control over the security aspects of your web service programming from authentication to integrity and privacy. For in-depth information on WSE, visit the Web Services Developer Center on MSDN.

Defending Web Servers

There are a number of different options for authenticating web clients. Internet Information Services (IIS) makes it relatively easy to use Windows user accounts to authenticate web clients. Certificates can also be used to authenticate clients when SSL is in use and this is just what IIS provides. Applications can elect to enable anonymous access in IIS and implement their own authentication scheme. One great example of this is ASP.NET Forms authentication. Of course this is only suitable for a web application that doubles as the presentation layer. For Web services you can employ WS-Security to provide more fine-grained control over security capabilities.

To begin to understand how web server security works, it helps to have an understanding of the different security contexts presents at any given time. In Protecting COM Clients I mentioned the notion of an effective token or security context. The effective token is the thread token if one exists, otherwise it is the process token. The .NET Framework exposes the concept of an effective token through the GetCurrent static method of the WindowsIdentity class. The resulting WindowsIdentity object wraps the effective token and provides an elegant interface for querying it. Calling the WindowsIdentity.GetCurrent method from within a web method can tell you a lot about how IIS and ASP.NET manage security contexts. Consider the following simple web method:

[WebMethod]
public string EchoUserName()
{
    WindowsIdentity identity = WindowsIdentity.GetCurrent();
    return identity.Name;
}

What will this return? Well it depends on many things. Let us consider a few options. If the impersonate attribute (system.web/identity/@impersonate) in the web application’s web.config file is set to false it will return the name of the process identity, which defaults to the Network Service account. If the impersonate attribute is set to true and the web client is connecting anonymously it will return the name of the account representing anonymous clients, which is typically SERVER\IUSR_SERVER where SERVER is the host name of the web server. If the web application does not accept anonymous connections it will return the name of the account represented by whatever logon session was established for the client. Getting a feel for how ASP.NET, IIS and Windows combine to provide security for you web applications will take you a long way in understanding web server security in general.

Using Windows user accounts for managing authentication and authorization is extremely convenient and can save a lot of work in development and maintenance. But using Windows user accounts does not always make sense. There are two parts to the problem. The first is that there is a need for a more universally acceptable form of identification. A popular answer to this is X.509 certificates. The second problem is that the mechanics of authentication at the web server and HTTP levels are just not flexible enough to meet the needs of Web service-based applications.

Due to the universal support for SSL in web clients and servers, using SSL to provide data integrity, privacy and authentication is quite simple. The biggest obstacle is issuing certificates to servers and clients that can then be trusted by all through some direct or indirect certificate authority. If the client sent a certificate along with its request, you can retrieve information about the certificate using the HttpClientCertificate object returned by the HttpRequest.ClientCertificate property.

In Protecting Web Clients, I briefly discussed how you can use Web Services Enhancements 2.0 (WSE) to provide WS-Security capabilities to your web clients. WSE also provides an ASP.NET SOAP extension to provide support for processing WS-Security headers. Once the SOAP extension is added to your web application’s web.config file, you can interrogate the Security SOAP header for a request using the Security object returned by the Security property of the RequestSoapContext object. A typical usage would be to enumerate the security tokens in the Security header. Among others there are tokens available for Kerberos tickets as well as X.509 certificates so integrating with your existing Kerberos domain or public key infrastructure (PKI) should be straightforward.

WSE also provides support for signing and sealing all or parts of the SOAP envelope. For in-depth information on WSE as well as WS-Security, visit the Web Services Developer Center on MSDN.

© 2004 Kenny Kerr

Application Lockdown

MSDN Magazine has made a preview of my new security article available online. App Lockdown features in the special security issue of MSDN Magazine coming next month.

App Lockdown: Defend Your Apps and Critical User Info with Defensive Coding Techniques

© 2004 Kenny Kerr

Introduction to MSIL – Part 6 – Common Language Constructs

In parts 1 through 5 of the Introduction to MSIL series we focused on MSIL and the constructs it provides for writing managed code. In the next few sections we are going to examine some common language features that are not intrinsic to instruction-based languages like MSIL. Understanding the intermediate language instructions that are generated for common, programming language statements is critical to acquiring an instinct for the performance characteristics of your code as well as to more easily track down subtle bugs.

In this part we are going to look at some common constructs that are present in many languages. I will be expressing simple examples in C# although the explanations are common to many programming languages that offer the same general constructs. As I am trying to explain a general principle, I may not always present the exact code that a given compiler may produce. The point is to learn more about what compilers generate in general. I encourage you to compare the instructions generated by different compilers using tools like ILDASM.

Programming languages like C and C++ enabled much more rapid development of software compared with classic assembly language programming. A number of rich constructs played a significant part in providing this productivity. Concepts like expressions, selection statements (e.g. if and switch), and iteration statements (e.g. while and for) are what makes these portable languages so powerful. Even more productivity was gained by introducing concepts like objects, but before objects were even considered, expressions and statements provided enormous benefit to the assembly language programmer struggling to build large applications and operating systems. Let’s look at some of these statements.

The if-else Statement

Consider the following simple method that checks that its argument is not null before proceeding.

void Send(string message)
{
    if (null == message)
    {
        throw new ArgumentNullException("message");
    }
 
    /* impl */
}
 
The if statement looks innocent enough. If the expression is true, execution enters the scope block and an exception is thrown. If the expression is false, execution jumps to the first statement following the if statement. There’s enough going on here that I will leave the else clause as an exercise for the reader.

Even if you have never used C# before, this code should be pretty clear (assuming you’re a programmer). So how does the compiler turn this innocent little if statement into something the runtime will understand? As with most programming tasks, there are a number of ways to solve the problem.

In many cases it is necessary for a compiler to create temporary objects. Expressions are a common source of temporary objects. In practice, compilers avoid creating temporaries as much as possible. Assuming a non-optimizing compiler you could imagine the compiler turning the code above into something like this.

bool isNull = null == message;
 
if (isNull)
{
    throw new ArgumentNullException("message");
}

This is not to say that an optimizing compiler would not use temporary objects in this manner. Temporary objects can often help to generate more efficient code depending on the circumstance. Here the result of the expression is first evaluated to a boolean value. This value is then passed to the if statement. This is not all that different from the previous example, but after reading the last 5 parts of this series it should be clear why this is much more appealing to an instruction-based language. It’s all about breaking down statements and expressions into simple instructions that can be executed one by one. Let’s consider one implementation of the Send method.

.method void Send(string message)
{
    .maxstack 2
    
    ldnull
    ldarg message
    ceq
    
    ldc.i4.0
    ceq
    
    brtrue.s _CONTINUE
   
    ldstr "message"
    newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
    throw
   
    _CONTINUE:
   
    /* impl */
   
    ret
}

If you’ve been following along with this series you should be able to understand much of this method declaration. Let’s step through it quickly. The temporary object that the compiler might generate silently is made explicit in MSIL, although it is not named and lives very briefly on the stack. (I guess whether it is actually explicit is debatable.) Can you spot it? First we compare message to null. The ceq instruction pops two values off the stack, compares them, and then pushes 1 onto the stack if they are equal or 0 if they are not (this is the temporary). The code may seem overly complicated. The reason is that MSIL does not have a cneq, or compare-not-equal-to, instruction. So we first compare message to null then compare the result to zero, effectively negating the first comparison. More on this in a moment.

Now that we’ve got a handle on the expression result, the brtrue.s instruction provides the conditional branching you would expect from an if statement. It transfers control to the given target if the value on the stack is non-zero, thereby skipping over the if clause’s logical scope block.

Of course there is more than one way to skin a cat. The example above is very similar to what the C# compiler that I am using generates, although it uses an explicit temporary local variable to store the result of the expression. This implementation seems a bit awkward. Ultimately it does not matter too much as the JIT compiler can probably optimize away any differences. Nevertheless it is a useful exercise to see how we can simplify this implementation. The first thing we can attempt is to reduce the number of comparisons. I mentioned that there is no matching not-equal-to version of the ceq instruction. On the other hand there is a branch-on-false version of the brtrue.s instruction. Quite predictably, it is called brfalse.s. Using the brfalse.s instruction completely removes the need for the second comparison.

Finally, as a C++ programmer you would expect the compiler to use the equality operator, since one of the operands is a System.String type which has an equality operator defined for it, and this is exactly what the C++ compiler does.

.method void Send(string message)
{
    .maxstack 2
   
    ldnull
    ldarg message
    call bool string::op_Equality(string, string)
   
    brfalse.s _CONTINUE
   
    ldstr "message"
    newobj instance void [mscorlib]System.ArgumentNullException::.ctor(string)
    throw
   
    _CONTINUE:
   
    /* impl */
   
    ret
}

Ultimately the JIT compiler optimizes the code as best it can, including inlining methods where appropriate. You should not be surprised if these two implementations result in the same machine code at the end of the day.

The for Statement

Before we delve into the implementation of the for statement, commonly referred to as for loops, lets quickly review how the for statement works. If your background is in Visual Basic or even C# you may not be all that familiar with this extremely useful statement. Even C++ programmers are encouraged to avoid the for statement for common iterations in favor of the safer std::for_each algorithm from the Standard C++ Library. There are however, many interesting applications of the for loop that have nothing to do with iterating over a container from beginning to end.

The following simple pseudo code illustrates the construction of a for statement.

for ( initialization expression ; condition expression ; loop expression )
{
    statements
}

The for statement consists of three semi-colon delimited expressions followed by a scope block. (Don’t get me started on for statements without scope blocks.) You can use any expressions you wish as long as the condition expression results in a value that can be interpreted to mean either true or false. The initialization expression is executed first and exactly once. It is typically used to initialize loop indices for iteration or any other variables required by the for statement. The condition expression is executed next and is executed before each subsequent loop. If the expression evaluates to true, the scope block will be entered. If the expression evaluates to false, control is passed to the first statement following the for statement. The loop expression is executed after every iteration. This can be used to increment loop indices or move cursors or anything else that might be appropriate. Following the loop expression, the condition expression is evaluated again and so on and so forth. For a more complete description, please consult your programming language documentation. Here is a simple example that will write the numbers zero through nine to any trace listeners you have set up.

for (int index = 0; 10 != index; ++index)
{
    Debug.WriteLine(index);
}

We have already spoken about how an if statement might be implemented in MSIL. Lets now make use of that knowledge to deconstruct the for statement into something simpler to interpret by a computer while still using C#.

    int index = 0;
    goto _CONDITION;
 
_LOOP:
 
    ++index;
 
_CONDITION:
 
    if (10 != index)
    {
        // for statements
        Debug.WriteLine(index);
 
        goto _LOOP;
    }

That looks nothing like the for statement! Here I use the infamous goto, more generally referred to as a branch instruction. Branching is inevitable in languages that don’t support selection and iteration statements. It serves no purpose in languages like C++ and C# other than to obfuscate the code. (If you’re a major supporter of the goto statement, please don’t feel the need to share that with me. Feel free to talk about its merits on your own blog.) Code generators however often use goto statements as it is a lot simpler for a code generator to use than to construct the expressions for a for statement for example.

By following the branching in the code example above you should be able to see how we construct the for statement semantics by executing the condition and then jumping up to the loop expression to conditionally loop again. Now let’s consider how we can implement this in MSIL.

    .locals init (int32 index)
    br.s _CONDITION
 
_LOOP:
 
    ldc.i4.1
    ldloc index
    add
    stloc index 
    
_CONDITION:
   
    ldc.i4.s 10
    ldloc index
    beq _CONTINUE
   
    // for statements
    ldloc index
    box int32
    call void [System]System.Diagnostics.Debug::WriteLine(object)
   
    br.s _LOOP
 
_CONTINUE:

After initializing the index local variable we branch to the _CONDITION label. To evaluate the condition I pushed the value 10 onto the stack followed by the index value. The beq, or branch on equal, instruction pops two values off the stack and compares them. If they are equal it transfers control to the _CONTINUE label thereby ending the loop; otherwise control continues through the ‘statements’ in the for loop. To write the index to the debug trace listener I push the index onto the stack, box it and call the static WriteLine method on the System.Diagnostics.Debug reference type from the System assembly. Following this, the br.s instruction is used to transfer control to the loop expression for the next iteration.

Read part 7 now: Casts and Conversions

© 2004 Kenny Kerr

Introduction to MSIL – Part 5 – Exception Handling

In this part of the Introduction to MSIL series I will introduce the constructs that the CLI provides for exception handling.

A try block is used to protect a range of instructions. If an exception is thrown by one of the instructions in the protected block, or by any method called directly or indirectly within the protected block, control is transferred to an appropriate exception handler. A try block is declared with the .try directive.

.try
{
    /* protected code */

    leave.s _CONTINUE
}
<exception handler>

_CONTINUE:

The last instruction in the try block is the leave.s instruction which transfers control to the _CONTINUE label. The leave.s instruction ensures that the evaluation stack is emptied and that the appropriate finally blocks are executed. It follows that if control leaves the protected block by some other means, that an exception was thrown. The exception handler must immediately follow the try block.

The CLI offers four different kinds of handlers that you can employ to construct the error handling semantics that you require for your programming language or application. Let’s examine each one individually.

Catch

The catch exception handler, or catch block, has to be the most well-known form of exception handling, since it is provided directly by C++ as well as C#. A catch block is declared with the catch keyword. The catch clause includes the type of exception that the handler is willing to catch as well as the block of code to transfer control to, given a matching exception object. Catch blocks can also be chained together after a single try block for convenience. The first catch clause with a matching exception type will be chosen as the handler. Consider this example.

.try
{
    ldstr "I'm not a number"
    // ldnull
    // ldstr "123"
    call int32 [mscorlib]System.Int32::Parse(string)

    leave.s _CONTINUE
}
catch [mscorlib]System.ArgumentNullException
{
    callvirt instance string [mscorlib]System.Exception::get_Message()
    call void [mscorlib]System.Console::WriteLine(string)

    leave.s _CONTINUE
}
catch [mscorlib]System.FormatException
{
    callvirt instance string [mscorlib]System.Exception::get_Message()
    call void [mscorlib]System.Console::WriteLine(string)

    leave.s _CONTINUE
}

Here we ask the In32::Parse method to parse "I'm not a number", which predictably throws a FormatException. The first catch handler is never given an opportunity to execute. The FormatException handler dutifully writes a message describing the exception to the console and then calls the leave.s instruction to transfer control to the _CONTINUE label. Where was the exception object? The runtime ensure that a reference to the exception is pushed onto the stack before the handler is invoked. You can experiment by commenting out the ldstr instruction in the try block and un-commenting the ldnull instruction. This will push a null reference onto the stack and result in an ArgumentNullException instance being thrown by the Parse method.

Filter

The filter exception handler is a strange construct for a C++ programmer like me. Instead of matching on an exception type, the filter handler provides a scope block where it can evaluate whether it wants to handle the exception. It indicates to the runtime that it wants to handle the exception by pushing a value of 1 onto the stack. If it decides not to handle the exception it pushes a value of 0 onto the stack. Following the “evaluation” block is the handler block that contains the code that will handle the exception.

.try
{
    // ldstr "I'm not a number"
    ldnull
    // ldstr "123"
    call int32 [mscorlib]System.Int32::Parse(string)

    leave.s _CONTINUE
}
filter
{
    ldstr "filter evaluation\n\t"
    call void [mscorlib]System.Console::Write(string)
   
    callvirt instance string [mscorlib]System.Exception::get_Message()
    call void [mscorlib]System.Console::WriteLine(string)

    ldc.i4.1
    endfilter
}
{
    ldstr "filter handler\n\t"
    call void [mscorlib]System.Console::Write(string)
   
    callvirt instance string [mscorlib]System.Exception::get_Message()
    call void [mscorlib]System.Console::WriteLine(string)

    leave.s _CONTINUE
}

The try block will result in an ArgumentNullException exception because a null reference is pushed onto the stack as an argument to the Int32::Parse method.

The filter is then given a chance to determine whether it wants to handle the exception. In this case it simply writes the error message to the console and then pushes the value 1 onto the stack, using the ldc.i4.1 instruction, to indicate that it wants to handle the exception. The endfilter instruction is called to return from the filter clause.

The filter handler block is then called to handle the exception. Notice that both the evaluation and handler blocks can access the exception in flight by popping it off of the stack. Keep in mind that you can throw any reference type so don’t assume that the object reference you pop off of the stack in the filter handler is an instance of System.Exception. This is not a problem for the catch handler since you will know the type of the exception object.

Finally

The finally exception handler should be recognized by C# programmers and C++ programmers familiar with Structured Exception Handling (SEH). A finally block associated with a try block is always executed, regardless of whether control leaves the protected try block through normal means, using the leave.s instruction, or as a result of an exception, using the throw instruction. It provides a reliable mechanism of ensuring a certain block of code is always run for those languages that do not provide destructors, such as C and C#. Although the C programming language does not target the CLI, SEH is used often enough that the Microsoft C/C++ compiler provides the __finally keyword for the same reason.

.try
{
    /* protected code */
   
    leave.s _CONTINUE
}
finally
{
    /* cleanup code */

    endfinally
}

Fault

The fault exception handler is similar to the finally block except that it is invoked only if it’s associated try block is left as a result of an exception. After the fault handler has been given an opportunity to execute, the exception continues on its way in search or a handler that is willing to catch it.

.try
{
    /* protected code */
   
    leave.s _CONTINUE
}
fault
{
    /* cleanup code */

    endfault
}

With that brief introduction to exception handling out of the way, it is time to change gears a bit and talk about how this all relates to popular programming languages like C# and C++/CLI.

Read part 6 now: Common Language Constructs

© 2004 Kenny Kerr

Introduction to MSIL – Part 4 – Defining Type Members

In Part 3 of the MSIL series, I introduced the basic syntax for defining types. Using the .class directive, you can define reference types and value types. Choosing the type attributes correctly, you can exercise complete control over the definition of your type.

.class abstract Kerr.Sample.Object
{
}

In this installment, we are going to explore how to declare members of a type. To simplify the samples, I may omit the class definitions at times and simply focus on defining the members.

Constructors

Let’s begin with initializers, known as constructors in languages like C++ and C#. CLI supports both type initializers and instance initializers. Type initializers are a very handy thing and address many of the multithreading challenges you encounter when implementing a singleton in native C++. A type initializer is run exactly once for a given type and the runtime guarantees that no methods of the type will be accessible before the type initializer completes.

.method static void .cctor()
{
    .maxstack 1
   
    ldstr ".cctor"
    call void [mscorlib]System.Console::WriteLine(string)

    ret
}

.method public void .ctor()
{
    .maxstack 1
   
    ldarg.0
    call instance void [mscorlib]System.Object::.ctor()
   
    ldstr ".ctor"
    call void [mscorlib]System.Console::WriteLine(string)
   
    ret
}

.cctor and .ctor are known as special method names. A type initializer, which is optional, is defined as a static method named .cctor with no return value. Type initializers are called static constructors in the C++/CLI and C# languages.

An instance initializer, more commonly referred to simply as a constructor, initializes an instance of a type. A constructor is defined as an instance method named .ctor also with not return value. A constructor is called when an instance is created using the newobj instruction. Here is an example.

.locals (class TypeName obj)
newobj void TypeName::.ctor()
stloc obj

When executed, it will result in the following written to the console, assuming the constructors defined above.

.cctor
.ctor

The newobj instruction allocates a new instance of the type and initializes all its fields to the type-equivalent of zero. Then it calls the particular constructor, disambiguated by the signature, ensuring that the first (zero-based) argument refers to the newly created instance. Once the constructor completes, an object reference is pushed onto the stack for access by the caller.

Methods

Although the CLI defines constructors and properties in terms of methods (with some extra metadata for properties), in this section we are going to look at methods in the sense of C++ member functions or C# methods. Of course most of what is said of methods, also applies to instance constructors and property getters and setters. Virtually all of the interesting things that you can do with methods are controlled by method attributes. There are some common scenarios programmers expect so lets cover a few of them.

Static methods are defined using the static attribute. Static methods, as you would expect, are associated with a type but not an instance.

.method static void StaticMethod() { /* impl */ }

Instance methods simply use the instance attribute in place of the static attribute. The IL Assembler assumes instance as the default so you rarely need to specify it explicitly for method declarations.

.method void InstanceMethod() { /* impl */ }

The opposite is true when calling methods. The call instruction assumes a static method unless you specify otherwise. Here is an example of calling both methods.

call void TypeName::StaticMethod()

ldloc obj
call instance void TypeName::InstanceMethod()

Remember to push the object reference pointing to your instance onto the stack before calling the instance method.

Virtual function calls are an important part of object-oriented design and the CLI provides a great deal of flexibility in controlling whether the static or dynamic type of the object will be used to service the call, as well as how this behavior can be overridden in subclasses. When I refer to the static and dynamic type in this context I am referring to it in the C++ sense of the static type known at compile time and the dynamic type determined at runtime. This is generally referred to as polymorphism. There are two aspects to the virtual function support that you need to keep in mind when programming in MSIL. The first is how you declare your instance methods to support virtual function invocation and the second is how you call the method. It should also go without saying that static methods are by definition not virtual.

A method is marked virtual by adding the virtual attribute to the type header. Consider the following example.

.class House
{
    .method public virtual void Buy()
    {
        .maxstack 1
       
        ldstr "House::Buy"
        call void [mscorlib]System.Console::WriteLine(string)
       
        ret
    }

    /* etc */
}

.class TownHouse
    extends House
{
    .method public virtual void Buy()
    {
        .maxstack 1
       
        ldstr "TownHouse::Buy"
        call void [mscorlib]System.Console::WriteLine(string)
       
        ret
    }

    /* etc */
}

The House type has a virtual method called Buy. The TownHouse type extends House and also has a virtual method with the same name. Because of this, TownHouse::Buy is said to override House::Buy. So how do we tell the runtime which method to pick? Obviously if I have a House instance I would like House::Buy to be called, but if I have a TownHouse instance I would like TownHouse::Buy to be called and, being a virtual method, I want this decision to be made at runtime when the actual type is known. So far I have used the call instruction in a number of examples in this series. The call instruction invokes the specified method and will always call the same method regardless of the dynamic type of the object. The callvirt instruction, on the other hand, allows the runtime to determine the specific virtual method implementation to invoke based on the actual type of the object. Consider the following example.

newobj instance void House::.ctor()
stloc house
newobj instance void TownHouse::.ctor()
stloc townHouse

ldloc house
call instance void House::Buy()

ldloc townHouse
call instance void TownHouse::Buy()

ldloc townHouse
call instance void House::Buy()

ldloc townHouse
callvirt instance void House::Buy()

When executed, it will result in the following written to the console.

House::Buy
TownHouse::Buy
House::Buy
TownHouse::Buy

The first call to the Buy method with the house reference invokes the House::Buy implementation since call is only interested in the static, or compile-time, type. The second call to Buy with the townhouse reference invokes the TownHouse::Buy implementation for the same reason. The third call will once again invoke House::Buy despite the fact that the object reference points to a TownHouse. It should now be clear that using the call instruction implies making a compile-time decision on which method to execute. The final method call uses the callvirt instruction to invoke the virtual method House::Buy and since the object reference actually points to a TownHouse, the TownHouse::Buy method will be executed. To be clear, the runtime is not looking at the type of the local variable you declared but rather the type of the object being referenced. We could have stored a reference to a TownHouse in a House local variable and the TownHouse::Buy method would still have been called.

If you want to declare a virtual method but do not want to override an inherited virtual method with the same name, you can use the newslot attribute on the new virtual method in the subclass. If you consider that virtual method invocation by the runtime is not concerned about method names then you should see how this is possible. Just think of newslot as adding a new virtual function pointer to the vtbl for the given type.

CLI virtual methods are very interesting, especially when you consider how and to what extent they are exposed by C++/CLI and C#. This entry is getting long enough so I’ll save that discussion for another day.

Read part 5 now: Exception Handling

© 2004 Kenny Kerr

Raymond Chen defeats MSDN Magazine

After Joel Spolsky reported the defeat of Raymond Chen by MSDN Magazine, Chen made a great comeback. MSDN Magazine Executive Editor, Joshua Trupin, has decided to throw out the whole idea and start a new magazine. Trupin: “I'm bitter about the whole thing. We're tossing it all out and starting TechNet Magazine just to make sure it doesn't happen again.”

Seriously, I was just chatting to Josh on IM. He mentioned that the January 2005 issue will mark their 20th year. They apparently started as a Dr. Bronner's soap label in 1986. Not having grown up in North America, I had no idea what that was. Josh pointed me to this example.

:)

By the way, my latest security article will appear in the November issue of MSDN Magazine.

© 2004 Kenny Kerr

Introduction to MSIL – Part 3 – Defining Types

In this installment of the MSIL series, I describe how types are defined.

Here is a minimal reference type called House. As I write this, we are looking for a house in British Columbia, so this was the first thing that came to mind.

.class Kerr.RealEstate.House
{
    .method public void .ctor()
    {
        .maxstack 1
       
        ldarg.0 // push "this" instance onto the stack
        call instance void [mscorlib]System.Object::.ctor()
       
        ret
    }
}

This is a very simple type. Note that you must declare a constructor for a concrete reference type. Unlike languages like C# and C++, the IL assembler will not generate a constructor for you automatically.

Types are defined using the .class directive followed by a type header. The class keyword is used instead of the more intuitive type for historical reasons. When you read class in MSIL source code, just think type. The type header consists of a number of type attributes followed by the name of the type you are defining. To define the equivalent of a C# static class in MSIL you can write the following.

.class abstract sealed Kerr.RealEstate.MortgageCalculator
{
    /* members */
}

abstract and sealed are the type attributes. An abstract type cannot be instantiated and a sealed type cannot have sub-types. There are attributes to control visibility, such as public and private. There are attributes to control field layout, such as auto and sequential. For a complete list of attributes please consult the CLI specification. Many attributes are applied automatically, which can save you a lot of typing. Fortunately these defaults are quite intuitive so you should become familiar with them quickly. As an example, extending the System.ValueType from the mscorlib assembly defines a value type. Since the CLI requires that value types be sealed, the IL assembler will automatically add this attribute for you.

The name of the type in the example above is Kerr.RealEstate.MortgageCalculator. The CLI does not recognize namespaces as a distinct concept. Rather the full type name is always used. The syntax shown above is only supported by the IL Assembler that ships with version 2.0 of the .NET Framework. If you are working with version 1.x then you need to group your namespace members with a .namespace directive, as shown below. Note that this syntax is also supported in version 2.0.

.namespace Kerr.RealEstate
{
    .class abstract sealed MortgageCalculator
    {
        /* members */
    }
}

Following the type name you have the option of specifying the base type. The extend keyword is used for this purpose. If no base type is specified, the IL assembler will add the extend clause to make the type inherit from the System.Object type from the mscorlib assembly, resulting in a reference type. And finally, the type header can provide a list of interfaces that the type and its descendants will implement and satisfy, becoming the interfaces of the type.

.class Kerr.RealEstate.RoomList
    extends [System.Windows.Forms]System.Windows.Forms.ListView
    implements Kerr.IView
{
    /* members */
}

In this example, the Kerr.RealEstate.RoomList type has System.Windows.Forms.ListView, defined in the System.Windows.Forms assembly, as its base type. Keep in mind that the CLI requires that every user-defined type extend exactly one other type. The RoomList type also implements the Kerr.IView interface type.

With this basic introduction to type definitions, you should now be able to start defining more interesting types. To define an interface, simply use the interface attribute in your type header. If you want a value type, known as a struct in C#, simply extend the System.ValueType type from the mscorlib assembly. Now you should be able to see why the .class directive is perhaps not the best name for it.

.class interface Kerr.IView
{
    /* members */
}

.class Kerr.RealEstate.HouseData
    extends [mscorlib]System.ValueType
{
    /* members */
}

Read part 4 now: Defining Type Members

© 2004 Kenny Kerr

Introduction to MSIL – Part 2 – Using Local Variables

In part 2 of the Introduction to MSIL series, I will be exploring the use of local variables. Without variables, programs would not be very interesting. To illustrate the use of variables, let’s write a simple program to add numbers together.

In an MSIL method, variables are declared using the .locals directive.

.locals init (int32 first,
              int32 second,
              int32 result)

This statement declares three local variables for the current method. In this case they all happen to be of type int32, which is a synonym for the System.Int32 type. init specifies that the variables must be initialized to the default values for their respective types. It is also possible to omit the variable names. In that case you would refer to the variables by their zero-based index in the declaration. Of course, using variable names improves readability.

Before we continue, I want to make sure that it is clear how the stack is used explicitly in MSIL. When you want to pass values to an instruction, those values need to be pushed onto the stack. To read those values, the instruction must pop them off the stack. Similarly when calling a method, you need to push the object reference (if any) onto the stack followed by each argument that you wish to pass to the method. In the process of invoking the method, all the arguments as well as the object reference will be popped off the stack. To push a value onto the stack, use the ldloc instruction indicating the variable that holds the value. To pop a value off the stack, use the stloc instruction specifying the variable you wish to store the value in. Also keep in mind that values (based on value types) are stored directly on the stack, but objects (instances of reference types) are not since the CLI does not allow reference types to be allocated on the stack. Rather references to objects are stored on the stack. This is analogous to a native C++ object allocated on the heap with a pointer to it stored on the stack. Keep the stack in your mind as you read this series on MSIL. It should help you understand why values are continually pushed and popped on and off the stack.

The next step is to get the numbers from the user.

ldstr "First number: "
call void [mscorlib]System.Console::Write(string)
call string [mscorlib]System.Console::ReadLine()
call int32 [mscorlib]System.Int32::Parse(string)
stloc first

As I mentioned in Part 1, the ldstr instruction pushes the string onto the stack and the call instruction invokes the Write method, popping its argument off the stack. The next call instruction invokes the ReadLine method which returns a string. The returned string is pushed onto the stack and since it is already there, we simply call the Int32::Parse method which pops the string off the stack and pushes the int32 equivalent on. Note that I am omitting any error handling for the sake of clarity. The stloc instruction then pops the value off the stack and stores it in the local variable named 'first'. Getting the next number from the user works the same way except that the value is stored in the local variable named 'second'.

Now that we have read the two numbers from standard input, it is time to add them up. The add instruction can be used for this purpose.

ldloc first
ldloc second
add
stloc result

The add instruction pops two values off the stack and calculates the sum. To push the values of the local variables onto the stack, we use the ldloc instruction. When the add instruction completes, it pushes the result onto the stack and the program pops the value off the stack and stores it in a variable named 'result' using the stloc instruction.

The final step is to display the result to the user.

ldstr "{0} + {1} = {2}"

ldloc first
box int32

ldloc second
box int32

ldloc result
box int32

call void [mscorlib]System.Console::WriteLine(string, object, object, object)

We use the WriteLine overload that takes a format string followed by three object arguments. Each argument to the WriteLine method must be pushed onto the stack one by one. Since the numbers are stored as int32 value types, we need to box each value; otherwise the method signature won’t match.

The ldloc instruction pushes each argument onto the stack. The box instruction is then used for each int32 argument. Boxing involves popping the value off the stack, constructing a new object containing a copy of the value and then pushing a reference to the object onto the stack.

Here is the complete program.

.method static void main()
{
    .entrypoint
    .maxstack 4
   
    .locals init (int32 first,
                  int32 second,
                  int32 result)

    ldstr "First number: "
    call void [mscorlib]System.Console::Write(string)
    call string [mscorlib]System.Console::ReadLine()
    call int32 [mscorlib]System.Int32::Parse(string)
    stloc first
   
    ldstr "Second number: "
    call void [mscorlib]System.Console::Write(string)
    call string [mscorlib]System.Console::ReadLine()
    call int32 [mscorlib]System.Int32::Parse(string)
    stloc second

    ldloc first
    ldloc second
    add
    stloc result

    ldstr "{0} + {1} = {2}"
   
    ldloc first
    box int32
   
    ldloc second
    box int32
   
    ldloc result
    box int32
   
    call void [mscorlib]System.Console::WriteLine(string, object, object, object)   

    ret
}

The last thing to notice about this example is that I indicated the method will use at most four stack slots. This is to accommodate the four arguments passed to the WriteLine method at the end of the main method.

Read part 3 now: Defining Types

© 2004 Kenny Kerr

Introduction to MSIL – Part 1 – Hello World

When describing C++/CLI and how it relates to C#, I am often tempted to discuss the Microsoft intermediate language (MSIL) that the Visual C++ and Visual C# compilers generate. The trouble is that most programmers are not familiar with MSIL and programs like ILDASM aren’t that helpful to the newbie because the MSIL they display, although painfully correct, is not very readable. For this reason I decided to post a few entries introducing MSIL at a basic level. Since I doubt anybody reading this will actually go and start writing code in MSIL, I will skim over some of the uninteresting details and focus on things like defining types, writing methods, calling instructions and handling exceptions.

Please let me know if you find this helpful.

Let’s start with a simple main (entry point) function that displays a really unique message. Unlike C#, CLI does not have any requirement that a method must belong to a class. The entry point function also does not have to be called main; however I will use this name in this series for simplicity.

.method static void main()
{
    .entrypoint
    .maxstack 1

    ldstr "Hello world!"
    call void [mscorlib]System.Console::WriteLine(string)

    ret
}

The main method above is called a method definition since both the signature as well as the body of the method is provided. In contrast, when a method signature is provided without a body it is referred to as a method declaration. Method declarations are typically used as call targets (when a method is being called) whereas a method definition provides the actual implementation for a method.

A method definition begins with the .method directive and can be defined at global scope or within a class. The application entry point must be static, meaning an instance is not required to call the method, and that is indicated by the static keyword. Declaring a global method static seems redundant but the ILASM compiler complains if you omit the static keyword in some cases. Think of ‘void main()’ as the signature of the method which, as you would expect, indicates that it does not return a value and takes zero arguments.

The .entrypoint directive signals to the runtime that this method is the entry point for the application. Only one method in the application can have this directive.

The .maxstack directive indicates how many stack slots the method expects to use. For example, adding two numbers together involves pushing both numbers onto the stack and then calling the add instruction which pops both numbers off the stack and pushes the result onto the stack. In that example you will need two stack slots.

The ldstr instruction pushes the string that is passed to the WriteLine method onto the stack. The call instruction invokes the static WriteLine method on the System.Console class from the mscorlib assembly. This is an example of a method declaration. It provides the full signature of the WriteLine method (including the string argument) so that the runtime can determine which overload of the WriteLine method to call.

And finally the ret instruction returns execution to the caller. In the case of the entry point method, this would bring your application to an end.

Read part 2 now: Using Local Variables

© 2004 Kenny Kerr

kennyandkarin.com has moved!

As I mentioned a few weeks back, I moved our website and family blogs to EAServe web hosting. Well I just wanted to report that I have had a great experience. Their support team has been very responsive and they provide a great service including ASP.NET and access to a SQL Server database for just $9.95 (about $13.30 Canadian depending on the weather). They even provide SMTP-based e-mail accounts, though I have been unable to use this since my service provider blocks outgoing TCP traffic on port 25. How annoying.

EAServe even has a link to my C++/CLI article.

:)

© 2004 Kenny Kerr