BizTalk Data Services: Making BizTalk more RESTful with OData

By Jesus Rodriguez, Uri Katsir, Tellago, Inc

The code referenced on this post is available at Tellago's DevLabs CodePlex workspace: http://tellago.codeplex.com/releases/view/41246

Introduction

During the last few years, BizTalk Server has established itself as one of the dominant technologies in the integration ecosystem. As part of its current release, BizTalk Server includes a robust publish-subscribe engine along with technologies such as a Business Rules Engine and Business Activity Monitoring that enable important capabilities for integration solutions. Together with its different subsystems, BizTalk Server includes a series of APIs that allow developers to dynamically leverage those capabilities in .NET applications. Currently, the use of these APIs is constrained to .NET environments and, for the most part, only experienced BizTalk developers are able to master those APIs. Having a more flexible and interoperable set of APIs will extend BizTalk capabilities to non .NET technologies such as J2EE or even other Microsoft technologies such as Windows PowerShell or dynamic languages such as IronPython or IronRuby.

This document presents an overview to a RESTful API for BizTalk Server based on the Open Data (OData) Protocol. The fundamental objective of this API is to expose BizTalk resources via HTTP using the Atom Publishing Protocol (AtomPub) or the JavaScript Object Notation (JSON). Using these interfaces, developers should be able to interact with BizTalk Server with a simpler, more interoperable and lightweight mechanism.

Why REST? Why OData?

As explained in the previous section, the purpose of the BizTalk Data Services API is to enable highly interoperable mechanisms to interact with BizTalk Server. We would like to leverage these APIs as a fundamental vehicle to leverage the capabilities of BizTalk Server into heterogeneous environments. In that sense, we've decided to rely on REST as the fundamental architecture style for our API. The use of REST enables us to expose BizTalk Server resources via a stateless, URI-based service enabled API that uses standard HTTP verbs as its fundamental interface.

The use of REST does not impose any constraints on the mechanism used to represent BizTalk resources. We can expose BizTalk native elements such as receive or send ports using XML, XHTML, RSS, OData, etc. Specifically, OData provides an extensible resource representation mechanism based on AtomPub and JSON. OData was initially released as part of WCF Data Services and has gained a great adoption by various Microsoft technologies such as PowerPivot or SQL Server Reporting Services.

BizTalk Data Services

The fundamental premise of the BizTalk Data Services API is to provide the foundations to interact with BizTalk Server using OData. In the initial release, we are targeting four fundamental BizTalk subsystems:

  • BizTalk Management Data Service: This RESTful API exposes the functionalities available in the BizTalk Management Console via HTTP-based OData interfaces. Using this API developers can query/search artifacts such as Assemblies, Applications, Orchestrations, Ports, Schemas, Maps, etc.
  • Business Rules Data Service: This RESTful API exposes BizTalk's Business Rules as AtomPub/JSON resources. Using this API, developers can create, alter and execute BRE policies using simple HTTP requests.
  • BAM Data Service: This RESTful API exposes BAM and allows developers to interact with BizTalk Server BAM using AtomPub and JSON. Using this API developers can deploy, undeploy and query BAM activity models as well as creating, updating and querying BAM activities.
  • BizTalk ESB Data Service: This RESTful API is the equivalent of the BizTalk management API for the new capabilities included in the BizTalk ESB toolkit. Using this API developers can interact with ESB artifacts such as Itineraries, Resolvers or Adapter Providers using AtomPub and JSON.

The following figure provides a high level view of the BizTalk Data Services architecture.

BTSDataServices[1] 

Figure: BizTalk Data Services (The current version only includes the BizTalk Management Data Services API)

It's important to notice that the current implementation is focused on the BizTalk Management Data Services API.

BizTalk Management Data Services

The BizTalk Management Data Services API provides a lightweight interface for managing BizTalk artifacts such as Assemblies, Ports, Orchestrations, Schemas, Maps, etc using an HTTP interface that leverages OData as the representation mechanism. The following examples detail some of the available features in the current version of the API[UK1] .

Sample Use Cases

This section includes various samples that illustrate how to interact with the BizTalk Data Services API. Specifically, the samples focus on the querying, navigation and filtering capabilities of the current version of the API.

Basic Querying

The BizTalk Data Services API exposes various BizTalk Server artifacts as collections accessible via HTTP GETs. The following examples how to leverage the API to query BizTalk Server.

Querying BizTalk applications:

http://<WebApp>/BizTalkManagementService.svc/Applications

Querying BizTalk assemblies:

http://<Data Services Web App>/BizTalkManagementService.svc/Assemblies

Querying BizTalk Hosts:

http://<Data Services Web App>/BizTalkManagementService.svc/Hosts

Querying BizTalk pipelines:

http://<Data Services Web App>/BizTalkManagementService.svc/Pipelines

Querying BizTalk receive handlers:

http://<Data Services Web App>/BizTalkManagementService.svc/ReceiveHandlers

Querying BizTalk schemas:

http://<Data Services Web App>/BizTalkManagementService.svc/Schemas

Querying BizTalk maps:

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms

Querying BizTalk SendPorts:

http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts

Querying BizTalk InProcess instances:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessInstances

Querying BizTalk InProcess messages:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages

Querying BizTalk parties:

http://<Data Services Web App>/BizTalkManagementService.svc/Parties

Navigation

The BizTalk Data Services API leverages WCF Data Services to enable navigation across related different BizTalk Server objects. The following examples illustrate how to leverage this capability:

Querying a specific send port:

http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts('SendPort1')

Querying all orchestrations in a specific assembly:

http://<Data Services Web App>/BizTalkManagementService.svc/Assemblies('SampleBTSPrj')/Orchestrations

Retrieving the contents of an in-process message:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages('09665a37-703e-4590-be25-d50f507995c2')/Body/$value

Retrieving the context of an in-process message:

http://<Data Services Web App>/BizTalkManagementService.svc/InProcessMessages('09665a37-703e-4590-be25-d50f507995c2')/ContextProperties

Retrieving a XML Schema definition:

http://<Data Services Web App>/BizTalkManagementService.svc/Schemas('MSMQ.MSMQPropertySchema')/XmlContent/$value

Retrieving the generated XSLT of a BizTalk map:

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/XmlContent/$value

Retrieving the source and target schemas of a BizTalk map:

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/SourceSchema

http://<Data Services Web App>/BizTalkManagementService.svc/Transforms('SampleBTSPrj.SampleMap1')/TargetSchema

Filtering:

One of the most attractive features of the BizTalk Data Services API is the ability to filter a BizTalk Server collection using OData's query expression. The following samples illustrates how to leverage this feature:

Querying all stopped send ports:

http://<Data Services Web App>/BizTalkManagementService.svc/SendPorts?$filter=Status%20eq%20'Stopped'

Querying all disabled receive locations for a specific receive port:

http://<Data Services Web App>/BizTalkManagementService.svc/ReceivePorts('ReceivePort2')/ReceiveLocations?$filter=Enable%20eq%20false

Real world scenarios

After exploring the capabilities explained in the previous section, you can start thinking about real world scenarios that can benefit from the use of the BizTalk Data Services API. Generally, the use of a RESTful OData-based interface to communicate with BizTalk Server can drastically simplify some of the traditional tasks and challenges of BizTalk applications. The list below summarizes some of the most common scenarios that we have seen in our initial prototypes with customers:

Lightweight management (PowerShell)

Having a RESTful interface to interact with BizTalk Server facilitates the implementation of traditional BizTalk operational tasks with scripting technologies such as Windows PowerShell. Even though the latest release of BizTalk Server includes a PowerShell driver, this one is still dependant on the BizTalk client SDK. However, using a RESTful API will enable the interaction with multiple BizTalk Servers from PowerShell using only HTTP and OData. For instance, an administrator could quickly query the status of all the BizTalk Server Hosts by executing a simple HTTP GET from any Windows workstation without the need of installing any BizTalk libraries or tools.

Lightweight notifications

As explained in previous sections, the BizTalk Data Services uses AtomPub as its fundamental representation mechanism. This capability allows developers or IT administrators to subscribe to specific Atom feeds that describe relevant events. For instance, an IT administrator could configure an Atom reader to receive notifications when a specific receive location gets disabled. Similarly, an information worker can subscribe an Atom reader to feed that exposes messages routed through BizTalk Server that match a predefined criteria (ex: all purchase orders which amount exceeds $10000)

Message tracking and querying

One of the common challenges faced by customers in BizTalk applications is the ability of querying/searching specific tracked or in-process messages. Even though BizTalk Server enables sophisticated message tracking capabilities, it is not completely straightforward to programmatically leverage these from other applications. In general, only experienced BizTalk Server developers know how to effectively use the message tracking APIs and, currently, this is only possible from a .NET development environment. The BizTalk Data Services API exposes both types of messages as OData resources accessible via a simple HTTP GET. In that sense, developers can query specific messages by executing a HTTP GET against the BizTalk Data Services endpoint. This feature enables the integration of the BizTalk message tracking capabilities with other technologies such as reporting tools or even BizTalk Server BAM.

Business Rules Data Services

Not available in the current version...

BAM Data Services

Not available in the current version...

BizTalk ESB Data Service

Not available in the current version...

Conclusion

The BizTalk Data Services API is an experiment intended to bring the capabilities of RESTful/OData-based services to traditional BizTalk Server solutions. The API exposes BizTalk Server artifacts as OData resources available via HTTP. This initial release focuses on the BizTalk Server management capabilities and future releases are intended to target other technologies such as Business Rules, BAM or the BizTalk ESB toolkit.

6 Comments

  • Great Work Jesus and Tellago team. It's going to open up opportunities for so many consumers.

  • This is awesome work Jesus! This really opens up BizTalk capabilities to whole new set of developers.

  • Great work Jesus and Tellago! Do you also plan on exposing set operations (enable a send port for example)?
    We've implemented something very similar for a client but as a WCF service.
    Had to ensure the there was a finally{ if(catalog != null) {catalog.Dispose(); } statement wherever BtsCatalogExplorer was used otherwise it seemed to keep connections open.
    Looking forward to how this evolves and to the other services!

    Regards,
    Thiago

  • I think there is tremendous potential here, I love the whole "web 2.0" approach you've presented here. My one overarching concern with your idea (granted: this IS just alpha software and mainly a proof-of-concept, but it is a great concept, so I thought it merits this question) is how do you intend to secure access to these OData services? Security has to be an integral part of your design or else you will not see anyone really embracing this. To be fair, I am not posing this question just to Jesus but to everyone visiting this site: what are some viable ways that we can securely consume these services? The services must use some sort of authentication and authorization mechanism even if encryption is decided as not being a high priority. Please, do not get me wrong: I am not trying to sink this idea, I want to make it fly, but in my mind, this concern must be addressed.

  • Mikael,

    One of my colleagues is already working on adding PUT, POST, DELETE functionalities. Keep an eye on the next release.

    Thanks,

    Jesus Rodriguez

  • Great work. I was testing it and I'm amazed what you accomplished. If it is stable I will push for trying to use it in our testing environment at work.
    Pls continue to develop this wonderful service.

Comments have been disabled for this content.