August 2006 - Posts
Daniel Simmons (MSFT) wrote a couple of blog posts inspired by questions from the ADO.NET Technology Preview forum. Pablo Castro (PM), Daniel Simmons (MSFT), Lance Olson (MSFT), Sergey Melnik (MSFT), Dan Dosen (MSFT) and Zlatko Michailov (MSFT) are providing great information and blazinlgy fast replies, so I suggest you to join the community.
Also see the first part ADO.NET vNext Future Directions of this series of posts on the next-generation data access for the .NET platform.
Syntax sugar for Store Schema (SSDL) is a solution for the ambiguity I mentioned in my ADO.NET Entity Framework Metadata post [discussion]. Both the store SSDL and CSDL schema's are based on the entity data model terms. The team is currently looking at providing syntactic sugar for declaring tables and databases in more direct terms.
Extensibility interfaces allow replacement of certain pieces of the system [discussion]. The team is talking about opening a hook for developers to intercept the update processing.
Specifications for CSDL, SSDL, MSL to provide guidance with the ADO.NET Entity Framework Metadata [discussion]. For now you can look at the .xsd files that are used internally to validate the files at design and runtime.
See C:\Program Files\Microsoft Visual Studio 8\Xml\Schemas\
- CodeGenerationSchema.xsd
- CSDLSchema.xsd
- CSDLSchemaTypes.xsd
- csmsl.xsd
- DMSchemaTypes.xsd
- EDMSchemaTypes.xsd
- SSDLSchema.xsd
- SSDLSchemaTypes.xsd
Disconnected entities would be useful for several scenarios, including multi-tier systems where you need to transfer entities and receive modified entities, and the ASP.NET scenarios where you need to re-construct your state between page hits [discussion]. Another scenario is a rich-client that only occasionally connects to the server. We already did this in Neo with a powerful implementation of the unit of work pattern. One of the features of the Neo ObjectContext is that it does not need a connection to the database. It can even start out connected, with features like lazy-loading enabled, and later on disconnect so that all queries run against the data available in memory only. of course, changes can still be saved to the database.
Integration with ObjectDataSource to provide middle-tier data retrieval and update capabilities [discussion].
Fix the ambiguity!
Store Schema (SSDL) contains the metadata that describes the schema of your datastore. The central concept in the store schema are entities. Entities are instances of Entity Types (e.g., Product) where an instance represents a single record.
Example:
<EntityType Name="Order" Key="OrderID">
Entities are grouped into an Entity Set (i.e., Orders) where an entity set represents a table.
Example:
<EntitySet Name="Orders" EntityType="Self.Order" /> Conceptual EDM schema (CSDL) contains the metadata that describes the conceptual Entity Data Model (EDM) schema. The central concepts in the EDM are entities and relationships. Entities are instances of Entity Types (e.g., SalesOrder, Customer).
Example:
<EntityType Name="SaleOrder" Key="ID">
Entities are grouped into Entity Sets (i.e., SalesOrders is a set of SaleOrder instances).
Example:
<EntitySet Name="SalesOrders" EntityType="Self.SaleOrder" />
Relationships are instances of Relationship Types which are associations among two or more entity types (e.g, SaleOrder PlacedBy Customer).
Example:
<Association Name="SaleOrder_Customer">
Relationships are grouped in Relationship Sets.
Example:
<AssociationSet Name="SaleOrderCustomer"
Association="Self.SaleOrder_Customer">
<End Role="SaleOrder" EntitySet="SalesOrders"/>
<End Role="Customer" EntitySet="Customers" />
</AssociationSet>
Mapping schema (MSL) contains the metadata that describes the mapping between the schema contains in the SDDL file and the conceptual schema contained in the CSDL file. The central concepts in the mapping schema are entity mappings and relationship mappings. Entity mappings are instances of EntityMapping Types. Relationship mappings are instances of AssiciationSetMapping Types. In the mapping schema EDM entity types are mapped against tables and columns in the store schema and EDM relations against foreign key relations.
I hope Microsoft changes the ambiguity between the store schema and EDM schema since they both use the same terminology. A more intuitive approach would be to express these concepts through Entity and Relation types in the store schema and Object and Association types in the EDM. Thoughts?
Ever since I announced the release of the ADO.NET August CTP I have done my best to participate in the community, with the little spare time I have. The key takeaways for this week are:
Spans to pre-fetch data is something the team is exploring but no form plan yet [discussion]. As an application developer it is necessary to have a certain level of control to load the appropriate graph for a certain context. Nearly all object-relational mapping implementers ran into this issue, and solved it using smart spans/prefetch implementations. In Neo we exposed this functionality with a convenience method AddSpans, so you can pass an array of property paths to entities that should be fetched alongside the main entity. For example querying an order with the following span query.AddSpans("OrderLine.Product", "Customer") would return the order and its customer, and its associated order lines and product.
GetObjectByKey as a convenience method [discussion]. Neo as well as NHibernate already use this pattern.
RejectChanges to make the ObjectContext effectively forget all objects and their changes [discussion]. Pablo Castro [ADO.Net Technical Lead] questions whether this is functionality being used enough to push it to be an API in the top-level object. Neo benefits greatly from having this functionality in rich client scenario's and with our nested object context feature (parent/child pattern).
Batch updates to delete or update a number of records in one go [discussion].
Dynamically define entity attributes at runtime is very useful to change and store mapping information without a recompile [discussion]. iBATIS made this possible through flexible result set mapping and a configuration file watcher to reload meta data without having to restart the application. This proved to be very useful in our web applications.
Entity Data Model move to the server is something Microsoft envisions [discussion]. Long-term this is likely to become the foundation for a number of Microsoft's efforts as many aspects of data management will benefit from the higher abstractions that the EDM can provide. A while ago my friend Jimmy came up with the idea to create an abstraction layer on top of different persistent solutions he calls NWorkspace. Who ever said this higher abstraction should be exposed as an API ;) This idea was well ahead of his time.
Properties on a relationship is an upcoming feature to be able to nicely map a Book -> BookAuthor -> Author structure in the logical view against Book -> Author in the conceptual view [discussion].
The ADO.NET Entity Framework consists out of four layers which I will briefly explain in this post.
The Storage Provider consist of the data-access providers that access and communicate with a data source, such as a Microsoft SQL Server database or Oracle (e.g. SqlClient, OracleClient, etc).
The Mapping Layer surfaces the EDM and mapping concepts in the ADO.NET API. The top level classes of this layer are the MappingProvider, MapConnection and MapCommand (these names are subject to change as work progresses). The EDM model provides your application with a conceptual view (client-views) of the the data of a given domain. The mapping provider is given the EDM schema and the mapping information, so it can internally use the mapping infrastructure to translate between the conceptual and logical schema. By using the EDM model and the mapping provider your application no longer uses or sees database-specific constructs; the entire application operates in terms of the higher-level EDM model.
This also means that you can no longer use the native database query language (SQL-92) and have to use Entity SQL instead. Entity SQL is designed to fully leverage the expressivity of the entity data model in which queries can be statically formulated at design time or constructed at runtime.
Object Services is targeted at eliminating the mismatch between data and the application code (impedance mismatch). Regular ADO.NET exposes data from databases as rows and columns, the same data is now exposed as objects (at least we now have a choice). This layer also includes more advanced services often supported by object-relational mapping frameworks such as identity, change tracking, checks for optimistic concurrency and update processing (not a definite list).
LINQ to Entities integrates the LINQ project with the Entity Framework to naturally express queries within the higher-level object-oriented programming language itself (such as C#). The LINQ to Entities layer depends on the object services and the mapping layer. LINQ queries are translated to canonical query trees, which is the same representation that results from parsing an Entity SQL statement and is than handed over to the mapping layer.
Regular SQL doesn’t have the constructs required to deal with the elements available in the Entity Data Model (EDM). For this reason the ADO.NET Entity Framework introduces a new query language called Entity SQL. Entity SQL queries looks pretty much like regular SQL since the structure is the usual SELECT-FROM-WHERE sequence. Entity SQL is designed to leverage the full expressiveness of the entity data model.
Query<SalesPerson> newSalesPeople = aw.GetQuery<SalesPerson>(
"SELECT VALUE sp " +
"FROM AdventureWorks.AdventureWorksDB.SalesPeople AS sp " +
"WHERE sp.HireDate > @date",
new QueryParameter("date", hireDate));
Please note that in this example the query uses the conceptual EDM schema in which the SalesPerson entity maps to the SalesPerson, Employee and Contact table.
Linq to Entities raises the level of abstraction to manipulate data in application code. It allows you to formulate your queries directly within the programming language itself. Say goodbye to string literals that cannot be understood or verified during compilation. The same query now LINQ-ified with compile-time validation:
var newSalesPeople = from p in aw.SalesPeople
where p.HireDate > hireDate
select p;
When to use what? Applications based on the Domain Model pattern would probably consume ADO.NET’s Object Services and use Linq to Entities.
[edit]
Entity SQL is targeted at application scenarios that require a more dynamic mechanism to formulate queries. Entity SQL is optimized to take advantage of and expose concepts in the EDM.
David and Andres speculated on the positioning of the upcoming mapping technologies in .NET.
The problem domain your application targets has a mismatch with the already existing data model. In this case, the ADO.NET Entity Framework is the right choice.
The existing data model matches the problem domain your application targets. You don’t require mapping between these two models. Linq to SQL lets you directly map to your existing or new data model.
Microsoft has announced that its new August ADO.NET CTP is available for download.
The goal of the next version of ADO.NET is to eliminate the impedance mismatch across various data representations in a similar way we tried to do with Neo (The Entity Object Framework)
Features include:
- Object Services
- Entity Data Model (schema)
- Mapping
- Entity SQL
- Linq to Entities
- Linq to DataSet
- Object model generation
- SQL Server 2000-2005 providers
For feedback, asking questions and interact with the team responsible for the next ADO.NET version use the ADO.NET Technology Preview Forum.
In this article we compare the Microsoft Three-Layered Services Application [Microsoft TLSA] architecture against one by Domain Driven-Design architecture [Evans DDD] for organizing domain logic. For this purpose we use the Microsoft .NET Pet Shop 4 application as an example to explain the difference between these two approaches. It then gives some comments on what Microsoft has in store to better support the latter.
This article is aimed at developers and architects who are trying to find better ways to capture the abstraction of the domain in their designs. It will help you if you have some knowledge in building enterprise applications on the .NET platform.
Recently my thinking on the Three-Layered Services Application (MS-TLSA) in comparison to Domain-Driven Design (DDD) was triggered by the following remark:
The choice between MS-TLSA and DDD is a matter of taste.
Perhaps it is. What interests me in the MS-TLSA architecture is the heart of the software: the Business Layer. This is the layer which deals with the most significant complexity in the particular domain to which the application is targeted. Let’s see what the MS-TLSA has to say about this layer.
Large enterprise applications are often structured around the concepts of business processes and business components. These concepts are addressed through a number of components, entities, agents and interfaces in the business layer.
In the MS-TLSA architecture the Business Layer has been split up in Service Interfaces, Business Workflow, Business Components and Business Entities. We’ll focus on this architecture style and the separation of the Business Components and Business Entities for the remains of this article. First let’s take a look at the MS-TLSA definition for the Business Components part.
Business components are the software realization of business concepts. They are the primary units of design, implementation, deployment, maintenance, and management for the life cycle of the business application. Business components encapsulate the business logic, also called business rules. These rules constrain the behavior of a business concept to match the needs of a particular company. For example, the business rule that determines whether a given customer is approved for a line of credit may be encapsulated in the customer business component for small solutions. For larger solutions, it is likely that all credit-related business logic is encapsulated in a separate credit component.
Sounds solid, right? Let’s do the same for the Business Entities.
Business entities are data containers. They encapsulate and hide the details of specific data representation formats. For instance, a business entity may initially encapsulate a recordset obtained from a relational database. Later, that same business entity may be modified to wrap an XML document with minimal impact to the rest of the application.
Business and business workflow components can interact with independent business entity components or they can use a business entity to set their own state and then discard the business entity. Business entities are often used as Data Transfer Objects [Fowler PoEAA]. The data access components will often return business entities instead of database-specific structures. This helps significantly in isolating database-specific details to the data layer.
So in the MS-TLSA approach Business Components and Workflow Components interact with Business Entities (DTO’s [Fowler PoEAA]). This means that the data and behavior that uses it is scattered throughout the software. Let me remind you that one of the fundamental concepts of object-orientation is to build applications out of objects that have both behavior and data. The MS-TLA architecture also violates the object modeling principle of turning information about a real-world entity and the actions performed on it into responsibilities of the object representing the entity. “Good object think” [Nicola et al. SOM] says objects act rather than get acted on, and conduct their own business rather than hold data for functional processing. While qualifying this statement as an object modeling principle and not an absolute law violation, it is a clear warning sign. The Microsoft .NET Pet Shop 4 application (probably already well known to you) is designed to show the best practices for building enterprise, n-tier .NET 2.0 applications. The concrete .NET Pet Shop architecture has its roots in the MS-TLSA architecture which is why we will use it as an example to demonstrate a reference implementation of this style of Business Layer.
In the .NET Pet Shop application customers use the check out web form to place their order. The check out web form creates a new OrderInfo Business Entity and fetches the LineItem Business Entities from the customer’s shopping cart. Next it asks the Order Business Component to process ordering logic which involves processing the customer’s credit card information (step 1), storing the order (step 2), calculating stock quantities (step 3) and storing the actual stock quantity (step 4). Please note that for the sake of clarity I left out the asynchronous strategy for processing the customer’s order. The following diagram captures a small part of the .NET Pet Shop implementation.
MS-TLSA .NET Pet Shop check out process
I hope you directly notice the TakeStock(LineItemInfo[] items) method on both the Inventory Business Component and Inventory Data Access.
public void TakeStock(LineItemInfo[] items) {
// Reduce the stock level in the data store
dal.TakeStock(items);
}
Taking a closer look reveals that the Inventory Business Component TakeStock(LineItemInfo[] items) method just forwards the call to the Inventory Data Access to reduce the actual quantity in stock for an order its line items. The actual inventory logic is executed by the Inventory Data Access.
private const string SQL_TAKE_INVENTORY = "UPDATE Inventory SET Qty = Qty - ";
For this code to work properly it also needs to relate each line item and customer order quantity (step 5) with the order.
MS-TLSA .NET Pet Shop check out process
I think that it’s safe to say that the .NET Pet Shop check out logic is not only scattered over the Business Components and Business Entities implementations but also across the layers. In my experience this type of application often ends up being quite a tangled web of routines without a clear structure.
What if we’d agreed on having a deep model focus and to use the Domain Model Pattern [Fowler PoEAA] for structuring the check out logic of the .NET Pet Shop application? In a nutshell, the Domain Model pattern is an architectural pattern. Its intention is to capture knowledge in an object model that incorporates both behavior and data. It shines when dealing with complex business logic by its ability to solve domain-related problems in a web of interconnected objects, where each object represents some meaningful concept of the domain. For inspiration in designing a good domain model we’ll use the techniques for applying domain driven design as described in [Nilsson ADDDP].
DDD .NET Pet Shop check out process
This model solves the same problems/features as we talked about in the MS-TLSA architecture based .NET Pet Shop application example. In this model we speak of an isolated domain model in that it makes the check out concept explicit (step 1). The customers place their order using the check out web form, nothing different here. As opposed to the MS-TLSA architecture based .NET Pet Shop application where state of the order lines was kept in the shopping cart, now the state lives inside the model. The customer and credit card concepts are made part of the model, which makes it natural to make the CreditCard Domain Object responsible for charging the customers credit card (step 2). Reducing the quantity in stock for a particular product (step 4) is the responsibility of the Product Domain Object. This opens up a whole lot of possibilities in which the model acts as an enabler for new opportunities. Taking stock could cause infringements on the margin of critical safety or be used for calculations to determine a realistic safety margin. We now have full control over the check out logic. The first try could look like this:
public void CheckOut()
{
// Handle credit card logic
customer.CreditCard.Charge(TotalAmount);
// Insert the new order into the system
repository.AddOrder(this);
// Update the inventory stock
foreach(OrderLine line in orderLines)
{
line.Product.TakeStock(line.Quantity);
}
}
I think this design is more maintainable in the long run because of its increased clarity and an implementation that is more true to the abstraction of the domain. Remember when I said earlier that the choice between MS-TLSA and DDD is a matter of taste? We have now come to the conslusion that this statement is wrong. It is a matter of choice how to organize domain logic in your applications. For making the right choice you will need to understand both alternatives equally well. What I find appealing about the model focus is that the resulting model is a great tool for communication amongst the stakeholders. The better the communication is, the better the software will become, both in the short and the long run.
I tried the model approach many times over the years and found problems with it, especially related to the performance overhead with instantiation time in the COM world. Another well known problem is the impedance mismatch between the relational and object-oriented worlds. With .NET we now have a much better toolbox that supports object-orientation. The next version of ADO.NET will enable a higher-level abstraction of data-access programming by introducing an entity model that brings conceptual-level data access concepts directly into the code. It includes a layer (ADO.NET Entity Framework Object Services) that can expose database data as regular .NET objects. This object layer makes an ideal target for LINQ support, allowing developers to formulate queries against a database right from the programming language used to build the domain logic. LINQ provides the run-time infrastructure for translating queries into SQL for execution by the database and then translating the tabular results back into the domain objects. I find it exciting to see that Microsoft is working on great tooling to support DDD. Even today, we can go a pretty long way with the current tools.
[Nilsson ADDDP]
Nilsson, Jimmy. Applying Domain-Driven Design and Patterns with Examples in C# and .NET. Addison-Wesley, 2006.
[Evans DDD]
Evans, Eric. Domain-Driven Design: Tackling the Complexity in the Heart of Software. Addison-Wesley, 2004.
[Fowler PoEAA]
Fowler, Martin. Patterns of Enterprise Application Architecture. Addison-Wesley, 2003.
[Microsoft TLSA]
Three-Layered Services Application
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpatterns/html/ArcThreeLayeredSvcsApp.asp
Microsoft .NET Pet Shop 4.0
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/bdasamppet4X.asp
Microsoft .NET Pet Shop 4: Migrating an ASP.NET 1.1 Application to 2.0
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/bdasamppet4.asp
[Nicola et al. SOM]
Nicola, Jill, Mark Mayfield, and Mike Abney. Streamlined Object Modeling: Patterns, Rules, and Implementations. New Jersey: Prentice Hall, 2002.
Jeremy D. Miller made an interesting
comment on a
post by
Slava Imeshev. Slava breaks down the steps to take when starting a new software project. The sixth and final step is: do not hire people without Computer Sience education. Funny thing is that I discussed the exact same thing with a colleague today. While he has a degree in Music I have a degree in both Mechanical Engineering and Computer Science. I wholeheartedly agree with Jeremy, which was more or less our own conclusion earlier today: the common trait of all strong developers is passion and enthusiasm for software development. It's definitely not about being a CS major or certifications. Today I have also read an article from my sponsor in which the author states: that the modern consultant should shift his focus to business- and process knowledge. Of course the consultant still needs information technology specific knowledge.
Martin Fowler calls this
CustomerAffinity. Martin explains CustomerAffinity as the interest and closeness that the developers have in the business problem that software is addressing, and in the people who live in that business world. So I opt for changing the sixth step: do not hire people who not have customer affinity.
More Posts