Sylvain Duford's Integration Blog

Que Picasa?

I just discovered this tool from Google: http://picasa.google.com/index.html

It is an awesome and super easy to use tool for managing digital pictures. Highly recommended!!!

Visual Studio Hosted Experience!

Avanade Connected Architectures

Here's an interesting case study on some of our technology.

http://www.datamonitor.com/~a6db66b9040c4a639bb511ec89d181c8~/products/free/Brief/BFTC1124/010bftc1124.pdf

GrokTalks

Many of my fellow RDs got together at TechEd and put together these “GrokTalks”, which are 10 minutes long micro-presentations. Very cool, check it out at: http://www.groktalk.net/blog/ 

EI Challenge #2

EI Challenge #2

The second biggest challenge that we encounter on all EAI projects is testing. Testing something that is essentially a magic black box with messages coming in at one end and out the other is already something of a challenge. When you add to that the fact that you are likely interfacing with multiple other systems, on multiple platforms, and using multiple protocols, you now have a huge challenge on your hands. How can you test that something that came out of SAP, came through BizTalk, got transformed, validated and encrypted, had business rules applied to it, and then was routed to be consumed by an external system actually created the expected results? How do you find out if something wasn’t quite right along the way? Where did it happen? How do you correlate the incorrect results in the target system with the actual transaction that came out of SAP and every other step in between?

Simply put, this requires a lot of planning. Right from the requirements gathering phase, you must identify what will need to be tested and how you will validate the end results. You must also plan your architecture and design to support testing. This usually involves using some kind of transaction ID that you can use to correlate messages from source to destination. This may sound trivial, but a lot of systems do not support such an externally generated ID, and you may not have the option of changing those systems. You may have to resort to the creative use of an unused data field, or some auditing mechanism that stores both your transaction ID and the system’s corresponding ID.

For projects of any significant size you should also plan on developing some tools to help you with the task of correlating messages and validating what happens to them (their state) at each step of the process. You should be capable of injecting and inspecting messages at all stages of the process. You should also design the system so that tracing can be turned on or off at multiple levels of detail to support various levels of testing and troubleshooting. BizTalk already offers some tools like HAT to help you with this task, but HAT doesn’t bring in data from the external processes nor does it provide a mechanism to correlate with that external data.

In summary, testing a complex BPI solution is a challenge, but one that is not insurmountable if you plan for it ahead of time. Just like a solution architect should always wear his/her “security hat”, s/he should also wear a “testing hat” throughout the life of the project, from envisioning to deployment. Failure to properly plan, architect, and design the solution with testing in mind will result in serious difficulties and delays in the latter part of the project, and even throughout the system’s useful life.

Scott Hanselman's 2005 Ultimate Developer and Power Users Tool List

My fellow RD Scott Hanselman's is know for discovering all the neatest little tools and utilities out there. He just updated his famous list:

http://www.hanselman.com/blog/ScottHanselmans2005UltimateDeveloperAndPowerUsersToolList.aspx

Regional Director's Charity Auction

A bunch of my fellow Microsoft Regional Directors have organized another charity auction on eBay.

Please go bid!

http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=5587400881

EI Challenge #1

Identifying the top challenge that we encounter on integration projects is easy: managing the dependencies. Anyone who has worked on non-trivial software projects already knows this: dependencies can quickly and unexpectedly push your project off its rails. Most experienced project managers recognize that and they will appropriately go out of their way to reduce or eliminate all dependencies.

But what do you do for integration projects? Those projects are by their very nature all about dependencies! So you obviously cannot avoid the dependencies, they are everywhere! You have dependencies to all the systems you are interfacing with: you can have dependencies on internal systems, external systems, internal project teams, external organizations, network infrastructure, messaging infrastructure, enterprise directories, document repositories, data warehouses, audit trails, reporting services, etc, etc. Will they need to be modified? Will they provide you with the info you need in a timely fashion? Will they be ready to test when you are? Are they existing systems or in-flight projects? As you can surely see, this exposes any non-trivial integration project to a lot of risk.

So what can you do about it? Since you cannot eliminate those dependencies, you need to manage them proactively and plan for the worst by implementing risk mitigation strategies and contingency plans. As you are estimating your project, you must allow for a lot of extra time to compensate for delays that are out of your control. You must plan for what you will do in case each dependency becomes an issue. You must ensure that you have a point of contact for each dependency with someone that has enough bandwidth and authority to exert the appropriate pressures. You should also ensure that you have the proper level of management support to escalate issues when they become critical. For each dependency, you should investigate possible contingency plans. For example, is there another system or solution that could temporarily replace one that is not ready in time?

I could go on forever, but I think you get the point: you must plan and plan again to mitigate the risk brought by those dependencies! It is really no different then what you would do for any significant software project; it is just that the problem here is an order of magnitude bigger due to the nature of enterprise integration projects. So you must greatly increase your risk planning effort, add some contingencies to both your project plan and your budget, and obtain the right authority and escalation paths. Last but not least, you must constantly communicate so that all parties involved know exactly what is expected of everyone, and what the status of their project is. If you can detect issues early, you are much more likely to be able to deal with them without major disruptions to your project.

So manage your dependencies, or else!

DevTeach 2005

I will be speaking on the subject of BizTalk 2004 and SOA at DevTeach 2005 in June. If you've never heard of this conference, I encourage you to check it out. It's is not only affordable, but it offers an roster of speakers, and it is set in one of the world's best cities.

Check it out: http://www.devteach.com/

Enterprise Integration Challenges

I’ve participated in a lot of Enterprise Application Integration projects and Trading Partner Integration projects during the last 5 years. I’ve recently started looking back at that experience to try and identify what are the top challenges that keep coming back on every such project. I’m not talking necessarily about technical challenges, but also program management challenges. This gave me the idea of starting a series of blog entries to discuss those challenges. Stay tuned for more in the coming days.