Jesus Rodriguez's WebLog

At Tellago, we always try to stay in the frontlines of technology that can enhance our solution development practices. This year we are putting a lot of emphasis on business intelligence and in particular the new set of BI technologies such as Microsoft's PowerPivot, Master Data Services and StreamInsight that are scheduled to be release with SQL Server 2008 R2.

In the last few weeks we have been working closely with different Microsoft field offices to  coordinate a series of customers events that will help to evangelize the SQL Server 2008 R2 BI stack.The first one of those events took place yesterday at the Microsoft office in Tampa FL and we got around 60 Microsoft enterprise customers. Don Demsak (Tellago's CTO) and myself had the pleasure of sharing the stage with Martin Pichardo (Microsoft's Technology specialist) and delivered six sessions that highlighted the fundamental components of Microsoft's BI proposition. Below you can see the agenda

• SQL Server 2008 R2 overview
• Ad-Hoc Reporting and SSRS Enhancements - An introduction to the new SQL Server 2008 R2 Reporting Services features
• Building Self Service Business Intelligence Solutions with PowerPivot
• Building CEP applications using Microsoft's StreamInsight - Complex Event Processing (CEP)
• Introducing SQL Server 2008 R2 Master Data Services
• Real Time Business Intelligence with SQL Server 20008 R2

Thanks to all the folks who attended our presentation. In general the event was a success and we got a lot of positive feedback from a really enthusiastic audience. Kudos to Dan Sandlin and Martin for organizing this event. Our next stop will be Jacksonville FL in April 14th.

By Jesus Rodriguez, Uri Katsir, Tellago, Inc

The code referenced on this post is available at Tellago's DevLabs CodePlex workspace: http://tellago.codeplex.com/releases/view/41246

Introduction

During the last few years, BizTalk Server has established itself as one of the dominant technologies in the integration ecosystem. As part of its current release, BizTalk Server includes a robust publish-subscribe engine along with technologies such as a Business Rules Engine and Business Activity Monitoring that enable important capabilities for integration solutions. Together with its different subsystems, BizTalk Server includes a series of APIs that allow developers to dynamically leverage those capabilities in .NET applications. Currently, the use of these APIs is constrained to .NET environments and, for the most part, only experienced BizTalk developers are able to master those APIs. Having a more flexible and interoperable set of APIs will extend BizTalk capabilities to non .NET technologies such as J2EE or even other Microsoft technologies such as Windows PowerShell or dynamic languages such as IronPython or IronRuby.

This document presents an overview to a RESTful API for BizTalk Server based on the Open Data (OData) Protocol. The fundamental objective of this API is to expose BizTalk resources via HTTP using the Atom Publishing Protocol (AtomPub) or the JavaScript Object Notation (JSON). Using these interfaces, developers should be able to interact with BizTalk Server with a simpler, more interoperable and lightweight mechanism.

Why REST? Why OData?

As explained in the previous section, the purpose of the BizTalk Data Services API is to enable highly interoperable mechanisms to interact with BizTalk Server. We would like to leverage these APIs as a fundamental vehicle to leverage the capabilities of BizTalk Server into heterogeneous environments. In that sense, we've decided to rely on REST as the fundamental architecture style for our API. The use of REST enables us to expose BizTalk Server resources via a stateless, URI-based service enabled API that uses standard HTTP verbs as its fundamental interface.

The use of REST does not impose any constraints on the mechanism used to represent BizTalk resources. We can expose BizTalk native elements such as receive or send ports using XML, XHTML, RSS, OData, etc. Specifically, OData provides an extensible resource representation mechanism based on AtomPub and JSON. OData was initially released as part of WCF Data Services and has gained a great adoption by various Microsoft technologies such as PowerPivot or SQL Server Reporting Services.

BizTalk Data Services

The fundamental premise of the BizTalk Data Services API is to provide the foundations to interact with BizTalk Server using OData. In the initial release, we are targeting four fundamental BizTalk subsystems:

• BizTalk Management Data Service: This RESTful API exposes the functionalities available in the BizTalk Management Console via HTTP-based OData interfaces. Using this API developers can query/search artifacts such as Assemblies, Applications, Orchestrations, Ports, Schemas, Maps, etc.
• Business Rules Data Service: This RESTful API exposes BizTalk's Business Rules as AtomPub/JSON resources. Using this API, developers can create, alter and execute BRE policies using simple HTTP requests.
• BAM Data Service: This RESTful API exposes BAM and allows developers to interact with BizTalk Server BAM using AtomPub and JSON. Using this API developers can deploy, undeploy and query BAM activity models as well as creating, updating and querying BAM activities.
• BizTalk ESB Data Service: This RESTful API is the equivalent of the BizTalk management API for the new capabilities included in the BizTalk ESB toolkit. Using this API developers can interact with ESB artifacts such as Itineraries, Resolvers or Adapter Providers using AtomPub and JSON.

The following figure provides a high level view of the BizTalk Data Services architecture.

Figure: BizTalk Data Services (The current version only includes the BizTalk Management Data Services API)

It's important to notice that the current implementation is focused on the BizTalk Management Data Services API.

BizTalk Management Data Services

The BizTalk Management Data Services API provides a lightweight interface for managing BizTalk artifacts such as Assemblies, Ports, Orchestrations, Schemas, Maps, etc using an HTTP interface that leverages OData as the representation mechanism. The following examples detail some of the available features in the current version of the API[UK1] .

Sample Use Cases

This section includes various samples that illustrate how to interact with the BizTalk Data Services API. Specifically, the samples focus on the querying, navigation and filtering capabilities of the current version of the API.

Basic Querying

The BizTalk Data Services API exposes various BizTalk Server artifacts as collections accessible via HTTP GETs. The following examples how to leverage the API to query BizTalk Server.

Querying BizTalk applications:

http://<WebApp>/BizTalkManagementService.svc/Applications

Querying BizTalk assemblies:

http://<Data Services Web App>/BizTalkManagementService.svc/Assemblies

Querying BizTalk Hosts:

http://<Data Services Web App>/BizTalkManagementService.svc/Hosts

Querying BizTalk pipelines:

http://<Data Services Web App>/BizTalkManagementService.svc/Pipelines

Querying BizTalk receive handlers:

http://<Data Services Web App>/BizTalkManagementService.svc/ReceiveHandlers

Querying BizTalk schemas:

http://<Data Services Web App>/BizTalkManagementService.svc/Schemas

Querying BizTalk maps:

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms

Querying BizTalk SendPorts:

http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts

Querying BizTalk InProcess instances:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessInstances

Querying BizTalk InProcess messages:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages

Querying BizTalk parties:

http://<Data Services Web App>/BizTalkManagementService.svc/Parties

Navigation

The BizTalk Data Services API leverages WCF Data Services to enable navigation across related different BizTalk Server objects. The following examples illustrate how to leverage this capability:

Querying a specific send port:

http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts('SendPort1')

Querying all orchestrations in a specific assembly:

http://<Data Services Web App>/BizTalkManagementService.svc/Assemblies('SampleBTSPrj')/Orchestrations

Retrieving the contents of an in-process message:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages('09665a37-703e-4590-be25-d50f507995c2')/Body/$value

Retrieving the context of an in-process message:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages('09665a37-703e-4590-be25-d50f507995c2')/ContextProperties

Retrieving a XML Schema definition:

http://<Data Services Web App>/BizTalkManagementService.svc/Schemas('MSMQ.MSMQPropertySchema')/XmlContent/$value

Retrieving the generated XSLT of a BizTalk map:

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/XmlContent/$value

Retrieving the source and target schemas of a BizTalk map:

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/SourceSchema

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/TargetSchema

Filtering:

One of the most attractive features of the BizTalk Data Services API is the ability to filter a BizTalk Server collection using OData's query expression. The following samples illustrates how to leverage this feature:

Querying all stopped send ports:

http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts?$filter=Status%20eq%20'Stopped'

Querying all disabled receive locations for a specific receive port:

http://<Data Services Web App>/BizTalkManagementService.svc/ReceivePorts('ReceivePort2')/ReceiveLocations?$filter=Enable%20eq%20false

Real world scenarios

After exploring the capabilities explained in the previous section, you can start thinking about real world scenarios that can benefit from the use of the BizTalk Data Services API. Generally, the use of a RESTful OData-based interface to communicate with BizTalk Server can drastically simplify some of the traditional tasks and challenges of BizTalk applications. The list below summarizes some of the most common scenarios that we have seen in our initial prototypes with customers:

Lightweight management (PowerShell)

Having a RESTful interface to interact with BizTalk Server facilitates the implementation of traditional BizTalk operational tasks with scripting technologies such as Windows PowerShell. Even though the latest release of BizTalk Server includes a PowerShell driver, this one is still dependant on the BizTalk client SDK. However, using a RESTful API will enable the interaction with multiple BizTalk Servers from PowerShell using only HTTP and OData. For instance, an administrator could quickly query the status of all the BizTalk Server Hosts by executing a simple HTTP GET from any Windows workstation without the need of installing any BizTalk libraries or tools.

Lightweight notifications

As explained in previous sections, the BizTalk Data Services uses AtomPub as its fundamental representation mechanism. This capability allows developers or IT administrators to subscribe to specific Atom feeds that describe relevant events. For instance, an IT administrator could configure an Atom reader to receive notifications when a specific receive location gets disabled. Similarly, an information worker can subscribe an Atom reader to feed that exposes messages routed through BizTalk Server that match a predefined criteria (ex: all purchase orders which amount exceeds $10000)

Message tracking and querying

One of the common challenges faced by customers in BizTalk applications is the ability of querying/searching specific tracked or in-process messages. Even though BizTalk Server enables sophisticated message tracking capabilities, it is not completely straightforward to programmatically leverage these from other applications. In general, only experienced BizTalk Server developers know how to effectively use the message tracking APIs and, currently, this is only possible from a .NET development environment. The BizTalk Data Services API exposes both types of messages as OData resources accessible via a simple HTTP GET. In that sense, developers can query specific messages by executing a HTTP GET against the BizTalk Data Services endpoint. This feature enables the integration of the BizTalk message tracking capabilities with other technologies such as reporting tools or even BizTalk Server BAM.

Business Rules Data Services

Not available in the current version...

BAM Data Services

Not available in the current version...

BizTalk ESB Data Service

Not available in the current version...

Conclusion

The BizTalk Data Services API is an experiment intended to bring the capabilities of RESTful/OData-based services to traditional BizTalk Server solutions. The API exposes BizTalk Server artifacts as OData resources available via HTTP. This initial release focuses on the BizTalk Server management capabilities and future releases are intended to target other technologies such as Business Rules, BAM or the BizTalk ESB toolkit.

It's been a few weeks since my last blog post mainly due to the fact that it's been an extremely busy month for us at Tellago. We have been heads down working on several fun projects that we are expecting to share with the dev community in the next few weeks. In any case, today I am really happy to put an end to my blogging hiatus by announcing the availability of Tellago DevLabs on CodePlex. The main purpose of this workspace is to provide a vehicle to share some of the open source projects we have been working on. At Tellago, we are firm believers in open source as a mechanism to make software better and we've decided to maintain a separate CodePlex workspace to publish several open source projects lead by members of Tellago technical staff. We are also expecting that this workspace will improve the interaction between our tech team and the .NET developer community.

Our first release is a project that we have been working on for the last few weeks to implement a RESTful/OData-based API for BizTalk Server. This API intends to provide a REST-based, lightweight and highly interoperable channel to interact with BizTalk Server. More details about this API in a future post...

I hope you find the projects posted on Tellago DevLabs useful and bombard us with feedback.