BizTalk Data Services: Making BizTalk more RESTful with OData
By Jesus Rodriguez, Uri Katsir, Tellago, Inc
The code referenced on this post is available at Tellago's DevLabs CodePlex workspace: http://tellago.codeplex.com/releases/view/41246
During the last few years, BizTalk Server has established itself as one of the dominant technologies in the integration ecosystem. As part of its current release, BizTalk Server includes a robust publish-subscribe engine along with technologies such as a Business Rules Engine and Business Activity Monitoring that enable important capabilities for integration solutions. Together with its different subsystems, BizTalk Server includes a series of APIs that allow developers to dynamically leverage those capabilities in .NET applications. Currently, the use of these APIs is constrained to .NET environments and, for the most part, only experienced BizTalk developers are able to master those APIs. Having a more flexible and interoperable set of APIs will extend BizTalk capabilities to non .NET technologies such as J2EE or even other Microsoft technologies such as Windows PowerShell or dynamic languages such as IronPython or IronRuby.
Why REST? Why OData?
As explained in the previous section, the purpose of the BizTalk Data Services API is to enable highly interoperable mechanisms to interact with BizTalk Server. We would like to leverage these APIs as a fundamental vehicle to leverage the capabilities of BizTalk Server into heterogeneous environments. In that sense, we've decided to rely on REST as the fundamental architecture style for our API. The use of REST enables us to expose BizTalk Server resources via a stateless, URI-based service enabled API that uses standard HTTP verbs as its fundamental interface.
The use of REST does not impose any constraints on the mechanism used to represent BizTalk resources. We can expose BizTalk native elements such as receive or send ports using XML, XHTML, RSS, OData, etc. Specifically, OData provides an extensible resource representation mechanism based on AtomPub and JSON. OData was initially released as part of WCF Data Services and has gained a great adoption by various Microsoft technologies such as PowerPivot or SQL Server Reporting Services.
BizTalk Data Services
The fundamental premise of the BizTalk Data Services API is to provide the foundations to interact with BizTalk Server using OData. In the initial release, we are targeting four fundamental BizTalk subsystems:
- BizTalk Management Data Service: This RESTful API exposes the functionalities available in the BizTalk Management Console via HTTP-based OData interfaces. Using this API developers can query/search artifacts such as Assemblies, Applications, Orchestrations, Ports, Schemas, Maps, etc.
- Business Rules Data Service: This RESTful API exposes BizTalk's Business Rules as AtomPub/JSON resources. Using this API, developers can create, alter and execute BRE policies using simple HTTP requests.
- BAM Data Service: This RESTful API exposes BAM and allows developers to interact with BizTalk Server BAM using AtomPub and JSON. Using this API developers can deploy, undeploy and query BAM activity models as well as creating, updating and querying BAM activities.
- BizTalk ESB Data Service: This RESTful API is the equivalent of the BizTalk management API for the new capabilities included in the BizTalk ESB toolkit. Using this API developers can interact with ESB artifacts such as Itineraries, Resolvers or Adapter Providers using AtomPub and JSON.
The following figure provides a high level view of the BizTalk Data Services architecture.
Figure: BizTalk Data Services (The current version only includes the BizTalk Management Data Services API)
It's important to notice that the current implementation is focused on the BizTalk Management Data Services API.
BizTalk Management Data Services
The BizTalk Management Data Services API provides a lightweight interface for managing BizTalk artifacts such as Assemblies, Ports, Orchestrations, Schemas, Maps, etc using an HTTP interface that leverages OData as the representation mechanism. The following examples detail some of the available features in the current version of the API[UK1] .
Sample Use Cases
This section includes various samples that illustrate how to interact with the BizTalk Data Services API. Specifically, the samples focus on the querying, navigation and filtering capabilities of the current version of the API.
The BizTalk Data Services API exposes various BizTalk Server artifacts as collections accessible via HTTP GETs. The following examples how to leverage the API to query BizTalk Server.
Querying BizTalk applications:
Querying BizTalk assemblies:
http://<Data Services Web App>/BizTalkManagementService.svc/Assemblies
Querying BizTalk Hosts:
http://<Data Services Web App>/BizTalkManagementService.svc/Hosts
Querying BizTalk pipelines:
http://<Data Services Web App>/BizTalkManagementService.svc/Pipelines
Querying BizTalk receive handlers:
http://<Data Services Web App>/BizTalkManagementService.svc/ReceiveHandlers
Querying BizTalk schemas:
http://<Data Services Web App>/BizTalkManagementService.svc/Schemas
Querying BizTalk maps:
http://<Data Services Web App>/BizTalkManagementService.svc/Transforms
Querying BizTalk SendPorts:
http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts
Querying BizTalk InProcess instances:
http://<Data Services Web App>/BizTalkManagementService.svc/InProcessInstances
Querying BizTalk InProcess messages:
http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages
Querying BizTalk parties:
http://<Data Services Web App>/BizTalkManagementService.svc/Parties
The BizTalk Data Services API leverages WCF Data Services to enable navigation across related different BizTalk Server objects. The following examples illustrate how to leverage this capability:
Querying a specific send port:
http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts('SendPort1')
Querying all orchestrations in a specific assembly:
http://<Data Services Web App>/BizTalkManagementService.svc/Assemblies('SampleBTSPrj')/Orchestrations
Retrieving the contents of an in-process message:
http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages('09665a37-703e-4590-be25-d50f507995c2')/Body/$value
Retrieving the context of an in-process message:
http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages('09665a37-703e-4590-be25-d50f507995c2')/ContextProperties
Retrieving a XML Schema definition:
http://<Data Services Web App>/BizTalkManagementService.svc/Schemas('MSMQ.MSMQPropertySchema')/XmlContent/$value
Retrieving the generated XSLT of a BizTalk map:
http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/XmlContent/$value
Retrieving the source and target schemas of a BizTalk map:
http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/SourceSchema
http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/TargetSchema
One of the most attractive features of the BizTalk Data Services API is the ability to filter a BizTalk Server collection using OData's query expression. The following samples illustrates how to leverage this feature:
Querying all stopped send ports:
http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts?$filter=Status%20eq%20'Stopped'
Querying all disabled receive locations for a specific receive port:
http://<Data Services Web App>/BizTalkManagementService.svc/ReceivePorts('ReceivePort2')/ReceiveLocations?$filter=Enable%20eq%20false
Real world scenarios
After exploring the capabilities explained in the previous section, you can start thinking about real world scenarios that can benefit from the use of the BizTalk Data Services API. Generally, the use of a RESTful OData-based interface to communicate with BizTalk Server can drastically simplify some of the traditional tasks and challenges of BizTalk applications. The list below summarizes some of the most common scenarios that we have seen in our initial prototypes with customers:
Lightweight management (PowerShell)
Having a RESTful interface to interact with BizTalk Server facilitates the implementation of traditional BizTalk operational tasks with scripting technologies such as Windows PowerShell. Even though the latest release of BizTalk Server includes a PowerShell driver, this one is still dependant on the BizTalk client SDK. However, using a RESTful API will enable the interaction with multiple BizTalk Servers from PowerShell using only HTTP and OData. For instance, an administrator could quickly query the status of all the BizTalk Server Hosts by executing a simple HTTP GET from any Windows workstation without the need of installing any BizTalk libraries or tools.
As explained in previous sections, the BizTalk Data Services uses AtomPub as its fundamental representation mechanism. This capability allows developers or IT administrators to subscribe to specific Atom feeds that describe relevant events. For instance, an IT administrator could configure an Atom reader to receive notifications when a specific receive location gets disabled. Similarly, an information worker can subscribe an Atom reader to feed that exposes messages routed through BizTalk Server that match a predefined criteria (ex: all purchase orders which amount exceeds $10000)
Message tracking and querying
One of the common challenges faced by customers in BizTalk applications is the ability of querying/searching specific tracked or in-process messages. Even though BizTalk Server enables sophisticated message tracking capabilities, it is not completely straightforward to programmatically leverage these from other applications. In general, only experienced BizTalk Server developers know how to effectively use the message tracking APIs and, currently, this is only possible from a .NET development environment. The BizTalk Data Services API exposes both types of messages as OData resources accessible via a simple HTTP GET. In that sense, developers can query specific messages by executing a HTTP GET against the BizTalk Data Services endpoint. This feature enables the integration of the BizTalk message tracking capabilities with other technologies such as reporting tools or even BizTalk Server BAM.
Business Rules Data Services
Not available in the current version...
BAM Data Services
Not available in the current version...
BizTalk ESB Data Service
Not available in the current version...
The BizTalk Data Services API is an experiment intended to bring the capabilities of RESTful/OData-based services to traditional BizTalk Server solutions. The API exposes BizTalk Server artifacts as OData resources available via HTTP. This initial release focuses on the BizTalk Server management capabilities and future releases are intended to target other technologies such as Business Rules, BAM or the BizTalk ESB toolkit.