Paul Gielens:ThoughtsService : .NET

DEVs and DBAs Unite

p.gielens — Thu, 02 Jul 2009 08:58:00 GMT

Some of my clients have/had challenges embracing object-relational mapping data access API’s. I’ve experienced this with introducing Neo, IBATIS, NHibernate, LLBLGen Pro, LINQ to SQL and Entity Framework in projects for our clients.

As with most things in our field this hasn’t much to do with the technology choices we make. Better collaboration is the key here. I’m glad that Patterns & Practices acknowledges that guidance in this area could help these two groups to leverage each others strengths. I’m confident that enabling a better understanding of both domains will improve the adoption of Microsoft’s data access api’s in the future.

Please give your feedback on the Data Access Guidance wiki.

Entity Framework Tutorial

p.gielens — Tue, 24 Mar 2009 19:29:00 GMT

You don’t know anything about the Entity Framework or didn’t get around installing it on your box and pour the Northwind data onto a screen? This book could give you a great jumpstart into the Entity Framework. There are a lot of Entity Framework books coming your way, so why should you buy a copy of the Entity Framework Tutorial by Joydip Kanjilal? Packt was so kind to send me a copy, but you can always try to get a free copy as a blogger, reviewer or educator. Otherwise I can definitely advise you to get the ebook and use it to inspire your colleagues.

Because that is what this book is great for: convincing co-workers to start thinking and experimenting with the Entity Framework. Many development teams are still handwriting their data access code.

In the first chapter you’ll learn the key concepts and get an overview of the Entity Framework’s architecture. In the second chapter you’ll learn how to get started with the Entity Data Model and bind some data to a GridView data control. From this point on you’re set to build RAD applications!

It continues with getting down to the nitty gritty details in the Entity Data Model. This is not the strongest part of the book. None the less it provides a great way to learn more detailed concepts driving your Entity Framework based solutions. Stored Procedures are up next together with the lower level API’s in the Entity Framework namely Entity Client and Entity SQL. You’ll learn more about executing CRUD operations against your database with the Entity Framework.

The book successfully saves the best for last; LINQ to Entities and the Object Services Layer. I recon 90% of your projects will utilize both API’s. Creating queries, execution and concurrency conflicts are interesting subjects you’ll learn more about. This book only gives a glimpse of these subjects which each have enough going for them to fill a 200+ page book.

I don’t see the need for a chapter on ADO.NET Data Services but it’s included in the book.

Don’t feel like skimming through MSDN and knot together the loose ends? Don’t feel like reading through Julie's excellent 800+ pages? This is a great alternative. I use this book to inspire people about the Entity Framework by letting it floating around the office. For in depth knowledge and to master the Entity Framework’s complexity you’ll be better of with Julie’s Programming Entity Framework.

The PDC Crushed my Old Record

p.gielens — Mon, 03 Nov 2008 23:11:42 GMT

Blogging at the PDC08 crushed my good old refactoring is not free series. In other words, thanks for reading my blog. I hope you enjoyed my somewhat limited live coverage.

What Sets the PDC Apart from Other Conferences

p.gielens — Mon, 03 Nov 2008 23:01:49 GMT

Big thanks goes out to Marjan for sharing these photo's. ps: I'm the guy with his arm under his chin.

While discussing the Entity Framework future with Tim Mallalieu another guru jumped on the couch. Don Box shared his definite experience, without going into much detail, wearing a Microsoft speakers shirt and the bathtub scene at Tech Ed Amsterdam (where Alex Thissen in red assisted). We talked about the future of Olso and language workbenches in specific domain's. Unfortunately I don't have a picture of Chris Anderson who also joined in for a couple of minutes. It was great fun being able to spend time on the couch with passionate Microsofties.

Best Decision to Kill LINQ to SQL

p.gielens — Mon, 03 Nov 2008 22:14:35 GMT

I fully agree with Tim Mallalieu to recommend LINQ to Entities as the data access solution for your application. Sure, for an object-oriented view of the database LINQ to SQL was somewhat useful but in essence this is a scenario that the Entity Framework supports as well. Yes, in some ways the Entity Framework does add a bit of complexity but then again creating a direct mapping with the Entity Framework isn't much more difficult then it would be with LINQ to SQL. Apart from the fact that Microsoft shouldn't have released LINQ to SQL as a product I was quit surprised about the wave of people adopting this piece of technology. A tool for direct mappings misused as an object-relational mapper felt awkward from the start. I hoped for a little while that LINQ to SQL would evolve in a fully fledged object-relational mapper, but the Entity Framework with it's EDM investment had more potential from the start.

We've spend a good amount of time talking with Tim (@PDC) on the future of the Entity Framework. The team is working hard on better POCO and TDD support and to deliver stories that support Domain-Driven Design over the next two releases. Most scenarios Alex and I explained to Tim have been thought through thoroughly and despite certain design choice which are arguable seen in isolation make perfect sense in a broader scope and the vision for the Entity Framework platform.

I'm looking forward to V2 and V3 with input from the dp advisory council... and can't wait to taste mEntity. To bad since I was looking forward to another technology deathmatch with Alex.

Building Textual DSL with Oslo

p.gielens — Thu, 30 Oct 2008 19:51:53 GMT

Chris Anderson, Giovanni Dell-Libera mention that Microsoft folks are working on transformation languages for at least five years now and how textual DSL space became more and more prevalent.

"MGrammer" text to data
"MSchema" Schematizing data
"MGraph" representing data

This talk is about MGrammer. You'll build your won DSL's on top of "M".

This is the basic workflow in through the Intellipad interface to build an MGraph.

MGrammer

Language for creating textual DSL's
Specification will be released under OSP

Lot's of cool demos on stage I wasn't able to capture. Download the CTP today at http://msdn.micosoft.com/oslo and build your own language.

PDC Lost a bit of it's Magic

p.gielens — Thu, 30 Oct 2008 18:03:48 GMT

On the way back to our hotel yesterday a college mentioned the PDC will be held again in 2009, also in November. Earlier on that day it confused me to hear the Olso folks mention they would show more cool stuff next year at PDC. Anyway, I think PDC should be somewhat exclusive and focus on the big picture and future of new Microsoft technology/products. The only question I have at the moment is, why? Why not delay another year, why not keep something for under the belt.

Any clues?

What's new in XAML?

p.gielens — Thu, 30 Oct 2008 16:19:46 GMT

I've spend the large part of this morning threatening my persistently bleeding nose. Occasionally this pops up. Left over from years of cycling. I made it on time for the XAML talk.

XAML lets you say what you want without saying how to do it. This is also referred to as declaring your intent. XAML can be used to create declarative models a simple file format for object-based systems.

Benefits of designing for XAML:

XML reading/writing
Markup==OM
Consistency with .NET
Human readable XML
Easier to tool
Even-based programming model
Ride the XAML wave

Microsoft is investing in XAML in many place. Oslo is a great example with it's importing and exporting XAML capabilities in the repository.

State of XAML in 2008

UI
Workflow
Electronic paper
Runtime: .NET 3.0 & 3.5; silverlight1 & 2
Tools: VS 2008 (WPF + WF designer and XAML editor), expression blend
XAML specifications published

Customers need more language evolution, make it faster, fundamentals of XAML/BAML need continued investment (compilation, debugging, obfuscation, localization, FxCop, VS integration, and more). There also the need to model more domains in XAML.

What's new in XAML 2009

In .NET 4.0 XAML will be easier to author application declaratively and feature a more rich vocabulary.

Better name references (<Label target="firstNameBox">First Name<Label>)
Build in types (Most of the CLR types have an XAML equivalent.)
Support for generic types directly in XAML
Arbitrary dictionary key types
Beyond method names (Button Click="{DelegateCreatingME Foo}" />
Better declarative type offering
Use non-default constructors
Use static factory methods

System.Xaml.dll in .NET 4.0 and will hit the street in the November 2008 CTP.

More Oslo / "M"

p.gielens — Wed, 29 Oct 2008 23:04:40 GMT

Core premise of Oslo is executable models. today we've been looking into the customization features of Quadrant (the graphical tool part of Oslo)

Three main components

Yesterday I blogged mainly about the M modeling language. Today we'll focus on Quadrant the design tool and the Repository services in Oslo.

High level "quadrant" architecture

Architecture for customizing the shell and service

What we've seen:

Quadrant is a flexible tool interacting with diverse data
Quadrant uses repository for both specification and state

Let's focus on the repository for now and the different aspects that play a role.

Insight: It good to store more and more of our applications in the database, and to build our applications on top of SQL.

This means that below our modeling language we need to store certain things in our repository.

What specific features should be added on top of the current SQL Server features.

Oslo SDK Provides

New models: Intellipas, VS.NET languages
- Compile models m.exe, msbuild build tasks
- Deploy models mx.exe
Once in the database, it's just SQL

How to build applications on top of the repository database

After you've expressed a piece of your model in M you can use mx2edmx.exe to generate an .edmx file for use with the Entity Framework. Below a simple example to consume this data.

class Program
{
static void Main(string[] args)
{
     var context = new MicrosoftPDCEntities(..);
     context.AddFriendShips(new Friendship()
     {
       ContextParty = 124,
       ReferenceParty = 328
     });

     context.SaveChanges();
     foreach(var f in context.FriendShips)
     {
       Console.WriteLine("{0} + {1} are friends, "f.ContextParty,
       f.ReferenceParty);
     }
}
}

Repository core services

Deployment
Security
Catalog
Versioning

Deployment

mx.ex packages SQL for deployment to repository nodes
Application models van be used to define applications to be deployed

Security

Security is claim based
- Identity becomes just one of several possible claims
- Claims presented to authorized operations against resources
Repository tables keep track of claims, resources and operations
- Triggers implemented on /t:Repository generated views to check claims
- Views protect against direct access to tables
Domain specific security containers
- Folders to partition data
- Applications decide which data goes into what folder
- Security checks happen on folder boundaries
- Must call the field folder for compiler to find it
- Folder ID must exist in the Item.Folders table

Catalog

SQL Server has a catalog
- List tables views stored procedures etc and adds relationships
Repository has its own catalog
- Adds information about relationship modules, types and extends.
Useful for
- Rich export
- Impact analysis
- Enriched SQL data access code generation
- ..

Versioning

Data change synchronization
- Between nodes using SQL Server replication and occasionally connected systems
- Import/export using SQL server change tracking, eg repository<->file system
Schema evolution
- Extend M type and provide backwards compat for old clients w/ computed values
- SQL server integration services for data migration

In conclusion

Repository is optimized for many reads, few writes
Contains models for Oslo app domain
Can be extended with M
Models can be deployed, secured and versioned

PDC08 Videos

p.gielens — Wed, 29 Oct 2008 16:20:11 GMT

You're not here? No worries see what I've seen at channel9.

Dryhad

p.gielens — Wed, 29 Oct 2008 16:15:11 GMT

Coolest from MS Research (at least from my pov) is DryadLINQ and Dryhad which harasses the power of cluster computing. A very sophisticated query could be executed over 1000s of nodes in the cloud. How cool is that! See this for more information.

Another Day in Paradise

p.gielens — Wed, 29 Oct 2008 15:37:56 GMT

Don't get the me wrong. PDC08 is hard work. I have this constant light headache which is a sign I'm consuming the exact right amount of information my brain is able to process without collapsing. Yesterday we've visited Universal Studio's which was quit an experience. The Simpsons Ride was by far the coolest attraction I've experienced ever in a theme park. Probably because of the amounts of software that was poured into it ;)

Anyway, I'm getting ready for the walk over to the conference center to get a quick breakfast and another keynote. I really hope this keynote addresses more of Oslo and "M". Last night I read through most of the draft specification comfortably in my bath tub and all I can say OMFG. Let's start with some coffee and breakfast to pull me through this day.

The Future of the Entity Framework

p.gielens — Wed, 29 Oct 2008 01:01:05 GMT

The data platform

These are the different scenarios the Entity Framework addresses or should address.

The following layers are the reference architecture used as a starting point for the EF.

Customer segmentations

V1 of EF didn't really address all personas and the team is now working on supporting the following different personas with tools and frameworks.

New stuff

You still need meta data, but you don't need no more entity data model XML. For this you have to decorate your classes with attributes.

New stuff added respecting the different scenarios and customer segmentations.

If you really don't care about the model you can work without it. You still need to describe your meta-data.

The demo shows persistence ignorance, code only no XML.

Model first approach is supported. You can now generate the DDL from the entity data model. Forward only at the moment. So when you update the database from the model it's blow away your existing database.

The demo shows lazy loading. Nice!

It also shows model defined functions (think FullName()). User defined functions can also be declared in the entity data model XML.

Database expressions using the available meta data from the entity data model.

DbExpression expression = context.EntitySet("Categories").Scan();
where c.Property("Name").Equals("Dennis");
select c;
DbDataReader reader = context.ExecuteQuery(expression);

More on M directly from the PDC08

p.gielens — Tue, 28 Oct 2008 22:54:32 GMT

Why M?

We want creating and interacting with Oslo content to be simple and natural. Having a box-and-line design experience is an important enabler. Developers have an emotional band with the text-editor. By not allowing people interacting with text people where not productive. Box and arrows where doomed to fail. This doesn't mean box and arrows aren't useful though. Having a complementary textual experience is equally important.

What is M?

M is a language for defining domain models and textual domain-specific languages (DSLs)
M domain models define schema and query over structure data
M DSLs define projections form unicode to text based structured data

The M language

MSchema, domain-specific data models
MGrammer, domain-specific grammers
MGraph, abstract data model

What M is not

An object-oriented language (no polymorphism, virtual dispatch, is a determined based on structural subtyping, not stipulation)
A data access technology (M domain model compile down to T-SQL. Tool chain support course-grained loading/unloading of schemas and values-not OLTP solution)
A replacement for T-SQL (Far less expansive feature set. tool change supports linking/invoking T-SQL)

Domain modeling constructs

Extents declare storage for values
Types constrain values in a given context
explicit identity enables references
Functions name parameterized expressions

The M framework

M is itself implemented as M DSL (parser exposed using M DSL machinery. Type flow exposed as framework component)
SDK ships with an evaluator (read evaluate print loops - MrEPL) that builds both

There where also a lot of very cool demo's displayed I wasn't able to capture with my typing skills. I'll stick around in this room to capture the future of the Entity Framework.

A Lap Around "Oslo"

p.gielens — Tue, 28 Oct 2008 21:09:00 GMT

Pretty sad actually. I'm sitting here with Alex Thissen and Dennis van der Stelt on my left. All three of us blogging the same stuff ;)

Douglas Purdy and Vijaye Raji are about to lap us around Oslo.

What is a model?

Basically it's a description of given domain.

Drawings (models used to communicate with others; dataflow, use case, ..)
Model-assisted (models used to understand or manipulate code; static structure, sequence, ..)
Model-driven (models executed by the runtime, HTML, CSS, XAML, BPEL)

Model-driven platform

According to Don Box we're making the sort of progression in which we try to solve the impedance mismatch between humans and code COM+, .NET 1.0, webServices, .NET 3.0. Currently we're at the landmark at which we tend to deliver build software which increasingly uses declarative constructs.

It's not only the programming languages and tools focusing on DLS's. Applications often come with textual or graphical DLS to manage and maintain facets of the applications.

Why is this happening?

Transparency (better understanding)
Flexibility (faster changes)
Productivity (more essence, less ceremony)

What is Oslo: The platform for model-driven applications

"M" (the language for authoring models and DSL's)
"Quadrant" (the tool for interacting with models and DSL's, aimed at developers)
Repository (the database for storing and sharing models)

Language, tool and repository

Microsoft recognizes we're living in a model-driven world and is building this technology to bring their products forward. They did the same with OLE back in the days, and are determined to do the same with Oslo. According to Microsoft textual DLS's are the biggest thing to happen in programming looking forward.

Key Oslo concepts

M Example

module Microsoft.Samples
{
type MusicItem
{
    Id: Integer64 = AutoNumber();
    Album : Text;
    Artist : Text;
    Rating : Integer32 where value <= 3;
} where identity Id;

MusicLibrary :MusicItem*;
}

With M-schema to database you can, in two steps (m-compiler and the mx-tool) translate this code to T-SQL and add it to the repository. With a standard data tool such as excel you can view this data.

module Microsoft.Samples
{
MusicLibrary
{
    album = " Four;
    Artist = "Led Zeppelin";
    Rating = 3;
}
}

This code-snippet is enough to write data to the repository and query it using excel. This is called concrete syntax.

With M-grammar you can translate arbitrary uni-code characters into structure data.

"Led Zeppelin" is awesome!

This is a simple way to write textual DSL's using M. The current tooling supports syntax highlighting. In the future the editor will support symantic analysis and intellisense.

We get to see a sneak peek of quadrant consisting of generic viewers you can see as a series of bundled controls. Stock views are provided out of the box which can be used to create your own editors. Quadrant also has a data flow engine which provides the multiple views of data and updates it accordingly. It also has a nice canvas that can pan, zoom, etc to process the information in a visual way. The same underlying system is used to customize the visualizer and language.

We've now seen a custom model, textual-DSL and visual DSL, next up is a custom runtime. This is done by transforming the same textual-DSL syntax as seen above into an XML-DSL. Creating a runtime is as simple as reading the XML file into an ASP.NET page with a datagrid component. Incredible!

This technology will be applied to WPF, the cloud, etc. Microsoft is modeling the Microsoft world.

Oslo value is one model driven architecture applied to Microsoft particular domains, we can use for our clients and their domains. In the CTP there are about 500 models to review and learn from. The tooling currently also supports debugging your own domain-specific languages.

Development domains

Web (ASP.NET with "MWeb"), services (WCF/WF ("Dublin") with "MServices" , entities (Entity Framework with "MEntity") and database (SQL with "MSchema").

What could a domain-specific language for a service look like? Look an shiver...

module Service25
{
service Service
{
    operation Echo(str : text) : Text
    {
      .UriTemplate = "echo/{str}";
      WriteLine { Text = "Message : " + str }
      return str;
    }
}
}

Years ago I dreamt on this blog to be able to write domain-driven design constructs directly into our language. public class Customer {} or public class CustomerRepository {} now becomes entity Customer and repository Customer {}. Guess this is the time to say "we can"!

Two announcements

Public "Oslo" SDK CTP
"M" language specification to be released under open specification promise (OPS)

This means that anyone can implement the M language. Microsoft is going to work hard to have third parties and open source communities to implement this particular language and have a rich ecosystem around it. Microsoft want's this to become as large as XML today is.

How to get ready for Oslo

Use model-driven runtimes (WPF, WF, etc)
use XML or a database to drive applications or services
Embrace model-driven assisted development VSTA, DSL Toolkit, ..)