Christof Claessens' WebLog

BizTalk Server Documentation Update

christof claessens — Tue, 15 Feb 2005 22:07:00 GMT

Get the last BizTalk Server documentation update here. As always the online help remains available at MSDN as well.
Together with this core documentation, following updated documents are also available for download:

the user guide for information workers
the BizTalk Server tutorials
and last but certainly not least: the installation guide

Paul's admin tool moved

christof claessens — Fri, 11 Feb 2005 20:40:00 GMT

Paul Somers (winner of the BTS2004 competition) was so kind to notify me that his BizTalk Admin tool has moved here.

Reading his latest post: the BizTalk Exam certainly tests your BAM skills as well Paul :-)

It looks like we've a lot of good community projects by now and more are coming up. For example: Brian Travis is working on a cool BizTalk Server Message Dashboard; based on my sample project (BizTalk Server Tracking Playground) I made available in June 2004. Hope it will be finished soon as this is cool stuff: his dashboard allows for inspecting tracked messages and context without needing HAT. Even click-through to related messages is supported. More on this as he finishes :-)

Microsoft BizTalk Adapter for WSE 2.0 Available NOW!

christof claessens — Thu, 10 Feb 2005 09:56:00 GMT

We just released the WSE 2.0 adapter for BizTalk Server! In my opinion this is an important milestone: webservices are increasingly fast becoming the number one interoperable transport both inside and between companies. The need for security, trust and policy is crucial to further extend the wealth of scenario's that webservices enable. I find it an exciting idea to know that Web Services within your business processes now have become just a matter of configuration! Get it here: http://www.microsoft.com/downloads/details.aspx?FamilyId=BA643360-E9DC-4FC5-8D60-8E6C5ECA7861&displaylang=en

BizTalk Server 2004 - An update

christof claessens — Thu, 27 Jan 2005 05:06:00 GMT

Recently the BizTalk Server exam has been released. Just do as I did and take it to get yourself certified! For more information, see Scott's blog.

Another important milestone in the life of this great product: SP1 has been released TODAY!! Get it NOW!
I strongly advise everyone to test it in your test environment and bring it in production as soon you can: it's worth it! In addition to that, we got now an adapter migration toolkit: something you might find interesting if you're planning to migrate your integration solutions from BizTalk Server 2002 to 2004.

I'm still alive :-)

christof claessens — Sun, 14 Nov 2004 14:25:00 GMT

Due to several reasons (priority shifts...), it's a while ago I blogged here. To eliminate all doubts: I'm still alive and still open to all BizTalk/XML/SOA related questions and discussions :-)

I'm also glad to announce that tomorrow I'll join Microsoft as a product technical presales. It remains to be seen how many spare time I'll have to dedicate to this blog ;-)

Just one more thing I'd like to share here: if you're into the enterprise space and love the integration solutions Microsoft is providing, make sure to check out MIIS (Microsoft Identity Integration Server) as well!

More GMail accounts for BizTalk Server people

christof claessens — Fri, 01 Oct 2004 11:12:00 GMT

As appearantly everyone is eagerly looking for a gmail account, I will give away a few more. (At least, my previous blog makes me think that :-))

I'll invite to gmail, the first 5 people who can tell me why I implemented my streaming pipeline component using multiple threads instead of just one :-p.

[Yes, that's right :-) You'll have to read the article on custom pipeline processing for this one :-))))))))))) ]

P.S. Make sure you include your name and current emailaddress in your comment as I need these to invite you.

[Update: I've invited the people who responded. If you don't receive your invitiation, please let me know.]

New paper on pipeline components finally published!

christof claessens — Fri, 24 Sep 2004 06:39:00 GMT

Waaw... Finally got my new paper published. Like I already promised a while ago, I was working on a sample and paper, showing how custom pipelinecomponents can be created. Even better: the paper and sample code will show you how to create a *streaming* pipelinecomponent that is able to put data into the msgcontext based upon an XPath expression.

"How's that possible? XPath only works on a DOM Christof - you never can make this streaming!" Yes I can, because of Dare Obasanjo, who recently published a paper and sample code on so called "streaming xpath", which makes up a subset of XPath that can be handled in a streaming, forward-only, way.

For all those funky things: visit the Belgium MSDN site here!

All feedback is welcome, please enjoy this!

Funky BizTalk Server toys!

christof claessens — Tue, 14 Sep 2004 18:28:00 GMT

Microsoft just made the results of the BizTalk Server contest public. I immediately started googling but it appears that Paul Somer's entry is not downloadable yet... (Please drop me a line if I'm wrong.) Neverthless, a few cool cool toys appeared on gotdotnet in the mean time. My selection of these:

BizTalk Server 2004 "deploy with Nant" template
Integrating BizTalk Server with Nant
BizTalk Server 2004 Pipeline Component Wizard
VS.NET addin for quickly building pipeline components
BizTalk AutoDeploy Utility
VS.NET addin for quickly (re)deploying BizTalk Server solutions. No need for Ant
BizTalk Server 2004 Performance Tool
Tool that allows for BizTalk Server tuning.
BizTalk Server 2004 Transactional .NET Adapter
My own adapter that allows for transactional message transmission to any .NET component.
BizTalk Server 2004 Tracking Playground
My own little tracking demo application showing how to make use of the tracking db views and WMI.
BizTalk Server 2004 Management Tool
I'm very curious about this one! There's a GotDotNet workspace but it's empty as for now...
BizTalk Explorer
Doing what VS.NET allows you to do, without VS.NET :-)
BizTalk Server Service Windows
Allows for more flexible service windows.
BizTalk Server POP3 Adapter
Inbound email messages in BizTalk Server
BizTalk Server TIBCO Adapter
Rudimentary sample
BizTalk Server SQL BLOB Adapter
Reading and writing message to binary SQL columns.

Please note: I haven't tested nor downloaded all of these. I do not make any judgement on the quality nor availability of the entries above. My only intention here is to give a short overview of what's currently available.

Have fun!

Reply to Todd's comments on the Transactional .NET Adapter

christof claessens — Mon, 06 Sep 2004 21:06:00 GMT

Todd Sussman posted a few comments/requests on my Transactional .NET Adapter. One of the things I was asked for several times already is whether it would be hard to make it work in a request/response way. Now, to be honest, I did consider this when implementing. A few random thoughts:

Since the adapter is transactional, and since .NET remoting does not support transactions: the code called by the adapter would always run in process in the same appdomain. (I would need to do really funky things to do this otherwise.)
I would advice against doing too much work in the component called by the adapter; remember there's a transaction in progress! Ideally the component would access some queue, database or other transactional backend system. Don't start any actions that take a long time and could risk the transaction to time-out.
Don't start any new threads in your component unless you don't really need the transaction. Any new threads would not retain the transaction context... (Widbey would do a better job here.)
I decided not to promote request reponse too much since I felt that would raise the risk of blocking the worker thread too long. I was convinced that people needing this functionality would better use another transport to correlate response messages with their request. For example: dropping messages from within your .NET component back on a queue which is asynchronously read by the MQSeries or MSMQ adapter...

Was I wrong? Probably... I'd like to hear all comments you have on this. If you feel you need to have request reponse on the transactional .NET adapter in a scenario, please let me know!

Note to myself: a few other enhancements that I could make:

load each custom client assembly in a separate appdomain, this would allow for:
- separate security settings for each assembly
- configurable .NET config file for each assembly
- unloading of the appdomain, would prevent needing to restart the BizTalk Service to release an assembly handle
the request response I just discussed
provide user with multiple interfaces so they can choose to receive an XmlDocument, XmlReader or just the plain bytestream

XML Schema Element reuse vs Type reuse

christof claessens — Thu, 02 Sep 2004 15:48:00 GMT

What my last post demonstrated was actually how element reuse limits your ability to reusing an XML Schema definition. Last few weeks I saw still too many people reusing elements over and over without considering what consequences this can have when later on a schema has to be updated, reused or extended...

Regarding the sample, which was correctly identified as invalid by Martijn, Sachin, Peter and Wilco: indeed the XML Schema spec, paragraph 3.3.3 states that "If ref is present, then all of <complexType>, <simpleType>, <key>, <keyref>, <unique>, nillable, default, fixed, form, block and type must be absent, i.e. only minOccurs, maxOccurs, id are allowed in addition to ref, along with <annotation>."

Unless it is very clear that in each and every situation an element will have the same name and exactly the same structure, an element reference is probably not safe to use. I prefer reusing types, wich allow for much more flexible reuse.

Thanks for the comments - you surprised me :-)

Note: I know of a few people already but if anyone else is planning to use the BizTalk Server 2004 Transactional .NET Adapter I've written in a production environment, please get in contact. I'd like to know how this goes, what you plan to use it for and how any additional needs can be further addressed.

Win a free GMail account by solving this!

christof claessens — Wed, 01 Sep 2004 18:58:00 GMT

Good news :-)

I've a couple of GMail accounts to give away. Let's make this a little contest :-) Since I'm curious how deep the average XML Schema knowledge goes, I'll grant GMail access to the first two people that post a comment explaining what is wrong with the XML Schema below.

Note: XMLSpy will mark this schema as valid while it certainly is not! (Has anyone accidently noticed my aversion against this tool? :-))) )

<?xml version="1.0" encoding="UTF-8"?>

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" attributeFormDefault="unqualified" targetNamespace="myTargetNamespace" xmlns:tns="myTargetNamespace">
   <xs:element name="address" type="tns:addressType"/>

   <xs:element name="customerDetails">
      <xs:complexType>
         <xs:sequence>
            <xs:element name="firstName" type="xs:string"/>
            <xs:element ref="tns:address" nillable="true"/>
         </xs:sequence>
      </xs:complexType>
   </xs:element>

   <xs:complexType name="addressType">
      <xs:sequence>
         <xs:element name="street" type="xs:string"/>
         <xs:element name="number" type="xs:string"/>
      </xs:sequence>
   </xs:complexType>
</xs:schema>

BizTalk Server Contest Entry

christof claessens — Sun, 29 Aug 2004 20:28:00 GMT

Finally submitted my entry to the BizTalk Server contest. Curious?

I created a transactional .NET adapter. This adapter allows you to submit messages to any .NET component that implements a certain interface, in a transactional manner. This means that, for example, if you access SQL Server from within your component, every operation on SQL will be in the same transaction as all message box operations. This guarantees high reliability between the message box and any transactional backend. If the transaction fails, everything will be rollbacked (including all operations on the message box) and the message transmission will be retried.

So, instead of accessing your components using an expression shape inside an orchestration, you finally might think of doing things asynchronously now! By calling .NET component asynchronously using the BizTalk Server 2004 Transactional DOTNET Adapter you can leverage the retries, backup transport, tracking and BAM features as well.

What's in the package?

MSI installation package
.chm documentation files
couple of samples
full documented source code

To show the adapter framework's inner workings, I programmed directly against the adapter framework. (Not using the SDK adapter base classes.)

Now it's up to you guys for using this: please provide me with comments, feedback. Tell me what's good and what isn't. Tell me what or how to improve... But first of all: enjoy it. It's there, it's free, it's for you!

Have fun with the pet project I've spent couple of months with:

The BizTalk Server 2004 Transactional DOTNET Adapter.

Special thanks to my girlfriend for all the patience she had!

About XML Schema's determism requirement...

christof claessens — Tue, 17 Aug 2004 13:07:00 GMT

For those of you who haven't noticed yet: Dare Obasanjo has published a refined version of the work David Orchard did about a year ago. Doing so, Dare explains best practices in designing an extensible xml schema. Writing schemas that are both forward and backward compatible is not easy, believe me. Even an experienced schema author can be tricked by some of the requirements the xml schema spec needs you to comply with.

One of the most import things to be aware of when designing xml schemas is the requirement on the xml content model to be deterministic. Some people like to refer to this as the "Unique Particle Attribution Constraint". For the normative definition of this constraint, I'd like to refer to the W3C. No one could explain this so fuzzy as they can!! MSDN does certainly a better job :-)

Even the very first xml spec itself has recommendations about this in a non normative section. For compatibility reasons with SGML. In addition section 3.2.1 in this specification says: "For compatibility, it is an error if an element in the document can match more than one occurrence of an element type in the content model."

As for the question "why"... even Tim Ewald, recently wondered why exactly xml schema had this requirement in the first place... I can only guess... but it certainly makes writing an xml parser way easier since only one single lookahead symbol is required... (While otherwise some form of backtracking is needed, which could mean an enormous perf hit!)

.NET is a good xml citizen and requires you to comply with this constraint. If you don't, you won't even be able to validate your schema. Good work guys! As close to the standards as possible!

Could someone explain to me then, why it is that "the world's leading product family of XML development tools" - laughing out loud now - doesn't even *support* detection of this constraint!!! .NET, BizTalk Server, Word and InfoPath certainly do a better job! It's not because of Xerces supporting this kind of bad schema's that they have an excuse not to implement at least a check for this critical type of content model requirements! Even worse, I came across this post in their public FAQ. They publicly state that the detection of a non-deterministic model as an error would be wrong! Pfffff... so far for the standards.

Querying the WMI MSBTS_MessageInstance class

christof claessens — Tue, 10 Aug 2004 18:16:00 GMT

As I suppose not everyone reading my blog is reading the newsgroups on a daily basis, here's a highlight. Today, a question came up regarding the querying of the MSBTS_MessageInstance class.

Let's summarise what is possible with this WMI class:

Retreiving all message instances that are currently available in the message box.
Retreiving all message instances that comply with certain conditions and are currently available in the message box.
Saving any of those messages to the file system.

What is not possible with this class?

Saving or retreiving a message that is tracked. Tracked messages are not handled in the same way as messages that are still resided in the message box. If you want to save tracked message instances, you need to use the MSBTS_TrackedMessageInstance WMI class. Please keep in mind, when doing this, that this class does *not* support enumeration. You will need to use the MessageInstanceID in order to create an instance of it!

In addition, remember that it is not possible to use the MSBTS_MessageInstance class to make select queries based upon the content of the message context! Why? Message context is something dynamic and is not compiled into the WMI classes. The message context is accessible on the class as an XML string though... Making selects is only possible using the WMI properties on this class. Like: ServiceName, ServiceInstanceStatus, ServiceInstanceID, ServiceClassId, ServiceClass...

If you'd like to see this class in action using real life code - please check out my pet project: the BizTalk Server 2004 Tracking Playground.

Happy coding!

More on pipeline components

christof claessens — Fri, 30 Jul 2004 20:45:00 GMT

A while ago, I blogged about pipeline components and what you need to pay attention to. Stephen, in reply to this, was wondering (see his comment) how tracking came into play. And... to be honest - I didn't have a clue! Only recently I managed to get the answer.

It turns out that Stephen's question actually was really valuable in helping me understand how exactly things work underneath... So, here's my - somewhat delayed - response and comments to his question.

Assume that your pipeline component screws up and that tracking before the pipeline, is enabled. What happens?

if the message's body stream is seekable:
- it gets tracked
if the message's body stream is *not* seekable:
- it is *not* tracked

In both cases, the message ends up suspended. The body might be in the suspended message if the body stream was seekable, otherwise not. (You should always have the context.)

For example: in the case of HTTP, the stream will not be seekable and the body will not be tracked nor suspended in case of a fatal pipeline component failure. (Note: the client will not get an HTTP either!)

So, the only advise I can give is: *never* screw up in your pipeline component :-)))