WinFS delayed again: corporate politics or incompetency?
Friday, Microsoft brought the news that WinFS, the highly anticipated new filesystem annex object store, is delayed again. It is now said they hope to release a test version in late 2006 and it will not be present in Longhorn server as well.
If WinFS isn't ready for the Longhorn server release in 2007, something serious is holding it back. WinFS is already in development for a long time, and 2007 is more than 2 years away. That's an awful lot of time for a filesystem. When I read articles like the one on news.com, I instantly get al kinds of questions in my head. One I got from reading this particular article was : "Is this the same company that wrote SqlServer 7 almost from scratch in 4 years that is now incapable of producing a filesystem within twice that much time?". Reality gives the answer: yes, apparently it is the same company.
My somewhat paranoid mind however refuses to accept the spin Windows Server Chief Bob Muglia gives to the fact that WinFS will not show up for a long period of time. I mean, somewhere on this planet a guy named Hans Reiser wrote with just a couple of people the Reiser4 filesystem, which sports a large amount of complex functionality and is one of the fastest and most secure filesystems known today. (Read more about ReiserFS by clicking here). How could mr. Reiser succeed while Microsoft with all their power failed? A couple of reasons pop up: 1) Reiser4's functionality is small compared to what WinFS will do. 2) Microsoft hired the wrong people. 3) Microsoft has given WinFS a low priority.
According to what mr. Muglia said, it's clearly option 1. Is this really true? For what we've seen from WinFS so far, I can only conclude: no, it's not. Muglia said:
"This isn't a relational database, this is a brand-new data model, and it satisfies a whole class of applications that frankly have been unsatisfied from a data model perspective since the beginning of history. We've been working on things like this for a long time."
A brand new data-model... Now, I'm not a manager, so I don't know the jargon managers tend to use, but in the world where I live in, the professional software engineering world, a 'brand new data-model' doesn't exist. A couple of decades now a lot of very bright people are doing research on different data-models, database formats and how to handle data. But frankly, the last decade, it's more about the handling of the data than the datastore itself, basicly because everything to be said about storing data has already been said.
So it's all about the handling of the data, i.e.: creating information from structured or unstructured data. Let me dig up an old slashdot posting I made some time ago:click here. The only way WinFS can be something 'brand new' if it's something like that: files are views on data objects in a datastore. Is this something that's new or hard to do? Well... no. You see, Windows works, as every other good OS does, with a virtual filesystem. This means that what the OS API sees as the filesystem is actually a virtualized view on all filesystems mounted. This means that if you want to plug-in a datastore as a filesystem, all you have to do is write a filesystem driver for the virtual filesystem and applications can use your datastore as a filesystem. With an object store at the filesystem core, the virtual filesystem should be extended to offer access to that store as well: not only via the concept of a file, but also via the concept of an 'object'. But the bottom line is: a 'file' is just a view provided by a driver, how it's stored internally is not important, that's the job of the driver and datastore (be it NTFS, FAT32 or SqlServer).
The real problem is in the applications: if they save a file which is actually a collection of objects, and the file arrives as one big object at the gates of the filesystem driver, it can't chop it up in a clever way so it can re-use the individual objects. But this is not the problem of the WinFS team, but the problem of the people writing applications. In other words: deliver the filesystem first, then help application developers migrate their applications to the new filesystem format. This can never be the cause of the delay, as the more you delay the filesystem, the more time it will take to make people convert their application structures to the new filesystem API.
Let me get back to my original question: How could mr. Reiser succeed while Microsoft with all their power failed? With all due respect, but I simply don't buy reason 1) that it is so incredibly complex that it is so incredibly hard to do that it takes such a long time (we're talking 8-10 years) to create this filesystem called WinFS. I also don't think Microsoft hired the wrong people. So we arrive at reason 3: Microsoft has given WinFS a low priority. I actually think, but I'm speculating here, this can be close to the truth. The reason why I think this, is fairly simple: Microsoft is a demand-driven company. It has to make money, so it delivers what people want. A simple, yet, effective strategy, used by a lot of companies around the world.
Unless you've lived under a rock in the last couple of months, it's pretty obvious what the next big thing will be on the desktop: search. Everyone who has seen the Steve Jobs video where he demos the desktop search power of MacOS X 'Tiger', knows it: this will be huge, if not the next big thing which makes an OS a real OS instead of a toy. "At least on the desktop", you say? No, it goes far beyond the desktop. People in corporate networks see the part of the network they're allowed to see as part of their desktop. So a search for some data based on some criteria might look like it's a desktop search but might turn out into a network wide search. The company which can bring that power to the desktop is king.
Microsoft has a problem though: time, or better: Time to Market. It works against them: Google, Yahoo and others have already announced their desktop search engines which will deliver to the Windows desktop what mr. Jobs showed for MacOS X. But not in 2006 or 2007, no, it will be next year, or even: today. So what do you do when you're Microsoft? You continue with this huge project that will deliver a big leap forward in how we work with data, but it will take a couple of years to complete, or you look out (pun intended) your window and buy a small company with a clever guy who can deliver what your customers want on time and what will be delivered is good enough for these customers? The theoretical academic will say: "Go for the huge project!", but a person who knows how business works will say: "Go for the clever guy in the small company!". So Microsoft bought Lookout, and desktop search will come to you in Longhorn client, and because WinFS is delayed, probably also in a server version in Longhorn server (but that's speculation from my part).
In software engineering a famous saying goes: "Nothing is more permanent than a temporary solution.". The temporary solution here is the desktop search functionality that will be included in Longhorn. It's temporary because WinFS will show up eventually in the future, but... will it? Would you invest a lot of money in developing WinFS if you had a temporary solution that is good enough for your customers? Would you invest a lot of money in developing WinFS so that it can replace MS Access and SqlServer desktop edition? Just to replace a search functionality (remember, the user doesn't care how it works) that is good enough, i.e.: it doesn't make a difference in usability?
You do the math.