Contents tagged with .NET
-
Introducing Maestro – A DSL for Actor Based Concurrency
As you may have noticed from my blog, there is a big interest in concurrency. Along with that, I’ve made my way into Actor model based concurrency including such forays into Erlang, the Haskell-Actor package, Mailbox Processing in F#, and even such things as Retlang in C#. The problem with some of the .NET solutions is that they don’t go far enough into memory isolation and channel contracts as to what can and cannot happen. In order to enforce such a behavior, we need some sort of static analysis to indicate that. Of course with static analysis, it could be ignored and might not be as type rich as, say a DSL. And that’s where Maestro comes in.
-
How would the CLR Be Different?
UPDATED: Added improved generics with higher-kinded polymorphism
-
What Is the Future of C# Anyways?
It was often asked during some of my presentations on F# and Functional C# about the future direction of C# and where I think it's going. Last night I was pinged about this with my F# talk at the Philly ALT.NET meeting. The question was asked, why bother learning F#, when eventually I'll get these things for free once they steal it and bring it to C#. Being the language geek that I am, I'm pretty interested in this question as well. Right now, the language itself keeps evolving at a rather quick pace as compared to C++ and Java. And we have many developers that are struggling to keep up with the evolution of the language to a more functional style with LINQ, lambda expressions, lazy evaluation, etc. There are plenty of places to go with the language and a few questions to ask along the way.
An Interview With Anders Hejlsberg
Recently on Software Engineering Radio, Anders Hejlsberg was interviewed about the past, present and future of C# on Episode 97. Of course there are interesting aspects of the history of his involvement with languages such as Tubro Pascal and Delphi and some great commentary on Visual Basic and dynamic languages as well. But, the real core of the discussion was focused around what problems are the ones we need to solve next? And how will C# handle some of these features? Will they be language constructs or built-in to the framework itself?
Let's go through some of the issues discussed.
Concurrency Programming
Concurrency programming is hard. Let's not mince words about it. Once we start getting into multiple processors and multiple cores, this becomes even more of an issue. Are we using the machine effectively? It's important because with the standard locks/mutexes it's literally impossible to have shared memory parallelism with more than two processors without at some point being blocking and serial.
The way things are currently designed in the frameworks and the languages themselves are not designed for concurrency to make it easy. The Erlang guys of course would disagree since they started with that idea from the very start. Since things are sandboxed to a particular thread, they are free to mutate state to their heart's content, and then when they need to talk to another process, they pick up the data and completely copy it over, so there is a penalty for doing so. Joe Armstrong, the creator of Erlang, covered a lot of these things in his Erlang book "Programming Erlang: Software for a Concurrent World ".
Mutable State
Part of the issue concerning concurrency is the idea of mutable state. As far back as I remember, we were always taught in CS classes that you can feel free to mutate state as need be. But, that only really works when you've got a nicely serial application where A calls B calls C calls D and all on the same thread. But, that's a fairly limiting thing idea as we start to scale out to multiple threads, machines and so on. Instead, we need to focus on the mutability and control it in a meaningful way through not only the use of our language constructs, but our design patterns as well.
In the C# world, we have the ability to create opt-in immutability through the use of the readonly keyword. This is really helpful to decide those fields that we don't really need to or want to modify. This also helps the JIT better determine the use of our particular variable. I'm not sure about performance gains, but that's not really the point of it all, anyways. Take the canonical example of the 2D point such as this:
public class Point2D
{
private readonly double x;
private readonly double y;
public Point2D() { }
public Point2D(double x, double y)
{
this.x = x;
this.y = y;
}
public double X { get { return x; } }
public double Y { get { return y; } }
public Point2D Add(Size2D size)
{
return new Point2D(x + size.Height, y + size.Width);
}
}
We've created this class as to not allow for mutable state, instead returning a new object that you are free to work with. This of course is a positive thing. But, can we go further in a language than just this? I think so, and I think Anders does too. Spec# and design by contract can take this just a bit further in this regard. What if I can state that my object, as it is, is immutable? That would certainly help the compiler to optimize. Take for example doing Value Objects in the Domain Driven Design world. How would something like that look? Well, let's follow the Spec# example and mark my class as being immutable, meaning that once I initialize it, I cannot change it for any reason:
[Immutable]
public class Point2D
{
// Class implementation the same
}
This helps make it more transparent to the caller and the callee that what you have cannot be changed. This enforces the behaviors for my member variables in a pretty interesting way. Let's take a look at the actual C# generated in Spec# for the above code. I'll only paste the relevant information about what it did to the properties. I'll only look at the X, but the identical happened for the Y as well.
public double X
{
[Witness(false, 0, "", "0", ""), Witness(false, 0, "", "1", ""), Witness(false, 0, "", "this@ClassLibrary1.Point2D::x", "", Filename=@"D:\Work\SpecSharpSamples\SpecSharpSamples\Class1.ssc", StartLine=20, StartColumn=0x21, EndLine=20, EndColumn=0x22, SourceText="x"), Ensures("::==(double,double){${double,\"return value\"},this@ClassLibrary1.Point2D::x}", Filename=@"D:\Work\SpecSharpSamples\SpecSharpSamples\Class1.ssc", StartLine=20, StartColumn=20, EndLine=20, EndColumn=0x17, SourceText="get")]
get
{
double return value = this.x;
try
{
if (return value != this.x)
{
throw new EnsuresException("Postcondition 'get' violated from method 'ClassLibrary1.Point2D.get_X'");
}
}
catch (ContractMarkerException)
{
throw;
}
double SS$Display Return Local = return value;
return return value;
}
}
What I like about F# and functional programming is the opt-out mutability, which means by default, my classes, lists, structures and so on are immutable by default. So, this makes you think long and hard about any particular mutability you want to introduce into your program. It's not to say that there can be no mutability in your application, but on the other hand, you need to think about it, and isolate it in a meaningful manner. Haskell takes a more hardline stance on the issue, and mutability can only occur in monadic expressions. If you're not aware of what those are, check out F# workflows which are perfectly analogous. But by default, we get code that looks like this and is immutable:
type Point2D = class
val x : double
val y : double
new() = { x = 0.0; y = 0.0 }
new(x, y) =
{
x = x
y = y
}
member this.X
with get() = this.x
member this.Y
with get() = this.y
end
So, as you can see, I'm not having to express the immutability, only the mutability if I so choose. Very important differentiator.
Method Purity
Method purity is another important topic as we talk about concurrent programming and such. What I mean by this is that I'm not going to modify the incoming parameters or cause some side effects, and instead I will produce a new object instead. This has lasting effects if I'm going to be doing things on other threads. Eric Evans talked about this topic briefly in his Domain Driven Design book on Supple Design. The idea is to have side effect free functions as much as you can, and carefully control where you mutate state through intention revealing interfaces and so on.
But, how do you communicate this? Well, Command-Query Separation gets us part of the way there. That's the idea of having the mutation and side effects in your command functions where you return nothing, and then queries which return data but do not modify state. Spec# can enforce this behavior as well. To be able to mark our particular functions as being pure is quite helpful in communicating whether I can expect a change in state. Therefore I know whether I have to manage the mutation in some special way. To communicate something like that in Spec#, all I have to do is something like this:
[Pure]
public Point2D Add(Size2D size)
requires size != null;
{
return new Point2D(x + size.Height, y + size.Width);
}
This becomes part of the method contract and some good documentation as well for your system.
Asynchronous Communication and Messaging
Another piece of interest is messaging and process isolation. The Erlang guys figured out a while ago, that you can have mutation as well as mass concurrency, fail safety and so on with process isolation. Two ideas come to mind from other .NET languages. An important distinction must be made between concurrency memory models between shared-memory and message passing concurrency. Messaging and asynchronous communication are key foundations for concurrent programming.
In F#, there is support for the mailbox processing messaging. This is already popular in Erlang, hence probably where the idea came from. The idea is that a mailbox is a message queue that you can listen to for a message that is relevant to the agent you've defined. This is implemented in the MailboxProcessor class in the Microsoft.FSharp.Control.Mailboxes namespace. Doing a simple receive is pretty simple as shown here:
#light
#nowarn "57"
open Microsoft.FSharp.Control.CommonExtensions
open Microsoft.FSharp.Control.Mailboxes
let incrementor =
new MailboxProcessor<int>(fun inbox ->
let rec loopMessage(n) =
async {
do printfn "n = %d" n
let! message = inbox.Receive()
return! loopMessage(n + message)
}
loopMessage(0))
Robert Pickering has more information about the Erlang style message passing here.
Now, let's come back just a second. Erlang also introduces another concept that Sing# and the Singularity OS took up. It's a concept called the Software Isolated Process (SIP). The idea is to isolate your processes in a little sandbox. Therefore if you load up a bad driver or something like that, the process can die and then spin up another process without having killed the entire system. That's a really key part of Singularity and quite frankly one of the most intriguing. Galen Hunt, the main researcher behind this talked about this on Software Engineering Radio Episode 88. He also talks about it more here on Channel9 and it's well worth looking at. You can also download the source on CodePlex and check it out.
Dynamic C#?
As you can probably note, Anders is pretty much a static typing fan and I'd have to say that I'm also firmly in that camp as well. But, there are elements that are intriguing such as metaprogramming and creating DSLs which are pretty weak in C# as of now. Sure, people are trying to bend C# in all sorts of interesting ways, but it's not a natural fit as the language stands now. So, I think there can be some improvements here in some areas.
Metaprogramming
Metaprogramming is another area that was mentioned as a particularly interesting aspect. As of right now, it's not an easy fit to do this with C#. But once again, F# has many of these features built-in to do such things as quotations to do some metaprogramming because that's what it was created to do, a language built to create other languages. Tomas Petricek is by far one of the authorities on the subject as he has leveraged it in interesting ways to create AJAX applications. You can read about his introduction to metaprogramming here and his AJAX toolkit here. Don Syme has also written a paper about leveraging Meta-programming with F# which you can find here. But I guess I have to ask the question, does C# need this or shouldn't we just use F# for what it's really good at and not shoehorn yet another piece onto the language? Or the same could be said of Ruby and its power with metaprogramming as well, why not use the best language for the job?
Dynamic Dispatch
The idea of dynamic dispatch is an interesting idea as well. This is the idea that you can invoke a method on an object that doesn't exist, and instead, the system figures out where to send it. In Ruby, we have the method_missing concept which allows us to define that behavior when that method that is being invoked is not found. Anders thought it was an intriguing idea and it was something to look at. This might help in the creation of DSLs as well when you can define that behavior even though that method may not exist at all.
In the Language or the Framework?
Another good question though is do these features belong in the language itself or the in the framework? The argument here is that if you somehow put a lot of constraints on the language syntax, then you might prematurely age the language and as a result, decline in usage. Instead, the idea is to focus on the libraries to make these things available. For example, the MailboxProcessor functionality being brought to all languages might not be a bad idea. Those sorts of concepts around process isolation would be more of a framework concept than a language concept. But, it's an interesting debate as to what belongs where. Because at the end of the day, you do need some language differentiation when you use C#, F#, Ruby, Python, C++, etc or else what's the point of having all of them? To that point I've been dismayed that VB.NET and C# have mirrored themselves pretty well and tried to make themselves equal and I wish they would just stop. Let VB find a niche and let C# find its niche.
Conclusion
Well, I hope this little discussion got you thinking as well about the future of C# and the future of the .NET framework as well. What does C# need in order to better express the problems we are trying to solve? And is it language specific or does it belong in the framework? Shouldn't we just use the best language for the job instead of everything by default being in C#? Good questions to answer, so now discuss...
-
Thinking in Concurrently in .NET
In recent posts, you've found that I've been harping on immutability and side effect free functions. There is a general theme emerging from this and some real reasons why I'm pointing it out. One of the things that I'm interested in is concurrent programming on the .NET platform for messaging applications. As we see more and more cores and processors available to us, we need to be cognizant of this fact as we're designing and writing our applications. Most programs we write today are pretty linear in nature, except for say forms applications which use background worker threads to not freeze the user interface. But for the most part, we're not taking full advantage of the CPU and its cycles. We need not only a better way to handle concurrency, but a better way to describe them as well. This is where Pi-calculus comes into the picture... But before we get down that beaten path, let's look at a few options that I chose. Not that these aren't all of them, just a select few I chose to analyze.
Erlang in .NET?
For many people, Erlang is considered to be one of the more interesting languages to come out of the concurrent programming field. This language has received little attention until now when we've hit that slowdown of scaling our processor speed and instead coming into multi-core/multi-processor environments. What's interesting about Erlang is that it's a functional language, much like F#, Haskell, OCaml, etc. But what makes it intriguing as well is that it's not a static typed language like the others, and instead dynamic. Erlang was designed to support distributed, fault-tolerant, non-stop real-time applications. Written by Ericsson in the 1980s, it has been the mainstay of telephone switches ever since. If you're interested in listening to more about it, check out Joe Armstrong's appearance on Software Engineering Radio Episode 89 "Joe Armstrong on Erlang". If you want to dig deeper into Erlang, check out the book "Programming Erlang: Software for a Concurrent World" also by Joe Armstrong, and available on Amazon.
How does that lead us to .NET? Well, it's interesting that someone thought of trying to port the language to .NET on a project called Erlang.NET. This project didn't get too far as I can tell, and for obvious impedance mismatch reasons. First off, there is a bit of a disconnect between .NET processes and Erlang processes and how he wants to tackle them. Erlang processes are cheap to create and tear down, whereas .NET ones tend to be a bit heavy. Also the Garbage Collection runs a bit differently instead of a per process approach, the CLR takes a generational approach. And another thing is that Erlang is a dynamic language running on its own VM, so it would probably sit on top of the DLR in the .NET space. Not saying it's an impossible task, but improbable the way he stated.
Instead, maybe the approach to take with an Erlang-like implementation is to create separate AppDomains since they are relatively cheap to create. This will allow for process isolation and messaging constructs to fit rather nicely. Instead, we get rid of the impedance mismatch by mapping an Erlang process to an AppDomain. Then you can tear down the AppDomain after you are finished or you could restart them in case of a recovery scenario. These are some of the ideas if you truly want to dig any further into the subject. I'll probably cover this in another post later.
So, where does that leave us with Erlang itself? Well, we have the option of integrating Erlang and .NET together through OTP.NET. The original article from where the idea came from is from the ServerSide called "Integrating Java and Erlang". This allows for the use of Erlang to do the computation on the server in a manner that best fits the Erlang style. I find it's a pretty interesting article and maybe when I have a spare second, I'll check it out a bit more. But, in terms of a full port to .NET? Well, I think .NET languages have some lessons to learn from Erlang, as it tackled concurrent programming as the first topic instead of most imperative languages bolting it on after the fact.
MPI.NET
The Message Passing Interface (MPI) approach has been an interesting way of solving mass concurrency for applications. This involves using a standard protocol for passing messages from node to node through the system by the way of a compute cluster. In the Windows world, we have Windows Compute Cluster Server (WCCS) that handles this need. CCS is available now in two separate SKUs, CCS 2003 and CCS 2008 for Server 2008. The Server 2008 CCS is available in CTP on the Microsoft Connect website. See here for more information. You mainly find High Performance Computing with MPI in the automotive, financial, scientific and academic communities where they have racks upon racks of machines.
Behind the scenes, Microsoft implemented the MPICH2 version of the MPI specification. This was then made available to C programmers and is fairly low level. Unfortunately, that leaves most C++ and .NET programmers out in the cold when it comes to taking advantage. Sure, C++ could use the standard libraries, but instead, the Boost libraries were created to support MPI in a way that C++ could really take advantage of.
After this approach was taken, a similar approach was taken for the .NET platform with MPI.NET. The University of Indiana produced a .NET version which looked very similar to the Boost MPI approach but with .NET classes. This allows us to program in any .NET language now against the Windows CCS to take advantage of the massive scalability and scheduling services offered in the SKU. At the end of the day, it's just a thin wrapper over P/Invoking msmpi.dll with generics thrown in as well. Still, it's a nice implementation.
And since it was written for .NET, I can for example do a simple hello world application in F# to take advantage of the MPI. The value being is that most algorithms and heavy lifting you would be doing through there would probably be functional anyways. So, I can use F# to specify more succinctly what types of actions and what data I need. Here is a simple example:
#light
#R "D:\Program Files\MPI.NET\Lib\MPI.dll"
open MPI
let main (args:string[]) =
using(new Environment(ref args))( fun e ->
let commRank = Communicator.world.Rank
let commSize = Communicator.world.Size
match commRank with
| 0 ->
let intValue = ref 0
[1 .. (commSize - 1)] |> List.iter (fun i ->
Communicator.world.Receive(Communicator.anySource, Communicator.anyTag, intValue)
System.Console.WriteLine("Hello from node {0} out of {1}", !intValue, commSize))
| _ -> Communicator.world.Send(commRank,0, 0)
)
main(System.Environment.GetCommandLineArgs())
I'll go into more detail in the future as to what this means and why, but just to whet your appetite about what you can do in this is pretty powerful.
F# to the Rescue with Workflows?
Another topic for discussion is for asynchronous workflows. This is another topic in which F# excels as a language. Async<'a> values are really a way of writing continuation passing explicitly. I'll be covering this more in a subsequent post shortly, but in the mean time, there is good information from Don Syme here and Robert Pickering here.
Below is a quick example of an asynchronous workflow which fetches the HTML from each of the given web sites. I can then run each in parallel and get the results rather easily. What I'll do below is a quick retrieval of HTML by calling the Async methods. Note that these methods don't exactly exist, but F# through its magic, creates that for you.
#light
open System.IO
open System.Net
open Microsoft.FSharp.Control.CommonExtensions
let fetchAsync (url:string) =
async { let request = WebRequest.Create(url)
let! response = request.GetResponseAsync()
let stream = response.GetResponseStream()
let reader = new StreamReader(stream)
let! html = reader.ReadToEndAsync()
return html
}
let urls = ["http://codebetter.com/"; "http://microsoft.com"]
let htmls = Async.Run(Async.Parallel [for url in urls -> fetchAsync url])
print_any htmls
So, as you can see, it's a pretty powerful mechanism for retrieving data asynchronously and then I can run each of these in parallel with parameterized data.
Parallel Extensions for .NET
Another approach I've been looking at is the Parallel Extensions for .NET. The current available version is for the December CTP and is available here. You can read more about it from two MSDN Magazine articles:
What I find interesting is Parallel LINQ or PLINQ for short. The Task Parallel library doesn't interest me as much. LINQ in general is interesting to a functional programmer in that it's a lazy loaded function. The actual execution of your LINQ task is delayed until the first yield in GetEnumerator() has been called. That's definitely taking some lessons from the functional world and pretty powerful. And add on top of that the ability to parallelize your heavy algorithms is a pretty powerful concept. I hope this definitely moves forward.
Conclusion
As you can see, I briefly gave an introduction to each of these following areas that I hope to dive into a bit more in the coming weeks and months. I've only scratched the surface on each and each tackle the concurrency problems in slightly different ways and each has its own use. But I hope I whetted your appetite to look at some of these solutions today.
-
CMAP Code Camp Wrap Up - Dependency Injection and IoC Containers
I really enjoyed speaking at this past weekend's CMAP Code Camp. I hope you all enjoyed my presentation on "Loosen Your Dependencies with Dependency Injection and Inversion of Control Containers". It was a great discussion to have with everyone and I like to learn there as much as I teach.
I also enjoyed teaming up with Scott Allen on his "A Gentle Introduction to Mocking" where we talked about mocks versus stubs, test pattens and mock frameworks such as Rhino Mocks and Moq. Hopefully we'll be doing some more ping-pong sessions in the future.
Once again, I'd like to plug my DC ALT.NET group that I run. Our next meeting is scheduled for April 23rd and the topic will be Continuous Integration with Jay Flowers. We'll announce the location shortly for our group. You can go ahead and sign up for our mailing list here.
Anyhow, here are some resources that can help point you in the right direction. This includes articles, blogs and such that I find will be useful in your journey to better understand these things:
- Articles
- Blogs
- Books
If you note, I my code uses the following products in order to get it to run:
- ASP.NET MVC Preview 2
- xUnit.net
- Castle Windsor
- StructureMap
- Unity Application Block
- Unity Community Contributions
-
xUnit.net RC3 Just Released
Well, Brad Wilson and Jim Newkirk must really be busy lately. After I talked about the release of xUnit.net RC2, just today, Brad announced the release of RC3. As always, you can find the latest bits here. This fixes a number of bugs and adds CruiseControl.NET and ASP.NET MVC Preview 2 support as well in addition to the Resharper 3.1 and TestDriven.NET support. For more information about it, check out Brad's post here. More or less, they are feature complete for version 1.0 and the only that I think really is needed at this point is a decent GUI runner and that's well acknowledged as something they are working on. Visual Studio integration would be nice as well...
If you were in attendance at last night's RockNUG appearance, all tests for my demos were using xUnit.net, so I am actively using it right now and will be for my CMAP Code Camp appearance this weekend. However, I did not show the GUI runner because, well, it's not there yet, and instead, the console runner works just fine, thank you. So, go ahead and pick up the latest bits and give the team feedback!
For my other posts in this series, check them out here:
One last note regarding Brad, he was recently interviewed by Scott Swigart and Sean Campbell over at How Software Is Built and gives some interesting insights in the open source world inside and outside Microsoft and his contributions to it. Very good interview and well worth the time to read.
-
RockNUG IoC Container Presentation Wrapup
I want to thank the fine folks at the Rockville .NET Users Group (RockNUG) and Dean Fiala for giving me the opportunity to speak last night. It was a record crowd last night, so I'm glad that people were interested in Loose Coupling, Design Patterns, Test Driven Development, Behavior Driven Development and Inversion of Control containers. I hope everyone got some good information, and if not interested in using containers, design patterns and such, at least know they exist and have their use. Based on the feedback I've already received, it was warming and why I like presenting at user groups, so that both of us can learn.
Once again, I'd like to plug my DC ALT.NET group that I run. Our next meeting is scheduled for April 23rd and the topic will be Continuous Integration with Jay Flowers. We'll announce the location shortly for our group. You can go ahead and sign up for our mailing list here.
Anyhow, here are some resources that can help point you in the right direction. This includes articles, blogs and such that I find will be useful in your journey to better understand these things:
- Articles
- Blogs
- Books
-
Unity 1.0 Released into the Wild
As Chris Tavares mentioned in his blog, Unity 1.0 has been released a couple of days earlier than the April 7th release date mentioned by Grigori Melnik earlier. Scott Densmore also announced this as well as working on porting the inteception from ObjectBuilder2 which I talked about earlier in some of my Unity and IoC containers posts. Looking forward to that post as we've shared some emails on the subject.
Would You Like To Know More?
For those looking for my posts on the matter, I've covered it quite extensively with the comparison to other IoC containers as well as IoC containers in general:
- IoC and Unity - Configuration Changes for the Better
Covers the latest configuration changes to allow for better constructor injection
- IoC and Unity - The Basics and Inteception
Covers the basics of an IoC container and inteception techniques
- IoC Container, Unity and Breaking Changes Galore
Covers the breaking changes made from the old Unity drop to the new one
- IoC Containers, Unity and ObjectBuilder2 - The Saga Continues
Managing instances and parameter mapping resolution
- IoC and the Unity Application Block Once Again
Setter Injection versus Constructor Injection and PostSharp4Unity
- IoC and the Unity Application Block - Going Deeper
Constructor Injection and comparing Unity with Castle Windsor
- IoC and the Unity Application Block
Covering ObjectBuilder and Unity Application Block
Anyhow, if interesting in more Unity content, check out David Hayden's Unity IoC and MVC screencast as well as others on the subject here.
Speaking of which, I'm going to be a busy man indeed with my upcoming speaking schedule on IoC containers, not necessarily Unity in particular, but all of them, the background and decoupling your applications. Here is the complete schedule for this month:
- RockNUG - April 9th
- CMAP Code Camp - April 12th
- CMAP Architecture SIG - April 15th
- IoC and Unity - Configuration Changes for the Better
-
Covering NUnit 2.4.7
#light
#R @"D:\Program Files\NUnit 2.4.7\bin\nunit.core.dll"
#R @"D:\Program Files\NUnit 2.4.7\bin\nunit.framework.dll"
open NUnit.Framework
let rec fib n =
match n with
| 0 | 1 -> 0
| n -> fib(n-2) + fib(n-1)
| n when n < 0 -> failwith "Cannot be less than zero"
[<TestFixture>]
let FibOfOneShouldReturnZero() =
let fibResult = fib(1)
Assert.AreEqual(fibResult, 0)
-
xUnit.net RC2 Released
public class DatabaseFixture : IDisposable
