Pipeline components, failover and message persistency
BizTalk Server does not have any hard limits regarding the size of messages it is able to receive. To make this possible, all out-of-the-box pipeline components work in a streaming way.
Being able to seek a stream means having the stream in memory. In turn, this obviously means that streams used in pipeline components will nearly never be seekable... (This includes: HTTP, SOAP, FTP, ... you can never go "back" to previously read data.)
Even better: even IF the stream you have in a custom pipeline component is seekable, it would not be a very good idea to make use of it for performance reasons as well...
So far, so good :-)
Now, what happens when your pipeline component fails?
Is the original data still persisted to the message box and marked as suspended?
The answer, unfortunately, is no... Both the adapter and the pipeline component are responsible for retaining any data that has to be suspended. This means that, íf you screw up in a custom component, you might never know what data exactly came in at point of failure...
All together, this is fairly logic. If the pipeline has read part of the non-seekable stream, even the adapter won't be able to 'roll back' the stream's pointer to the beginning! Unless the adapter keeps a complete copy of the message, it won't be able to suspended the original data.
Note: a better design for BizTalk Server would have been that (optionally) you could configure to stream the original data to the message box first. Although this implies a serious perf hit, it could both:
- allow you to retain the very original incoming data, which can be tracked, saved and restored
- allow you to resubmit the message back into Biztalk Server, without any noticeable difference between a "real" submit and a resubmit of a suspended message. (Why would you need this? For example: imagine that your backend fails and you want to provide it again with that data... Imagine an error in your pipeline component or schema: you could fix the schema and just resubmit!)
As for now: the solution to all of this is very easy: don't screw up in custom pipeline components :-))))
Have a nice weekend!