J e r o e n ' s   w e b l o g

March 2005 - Posts

Search APIs: SOAP or REST?

I've been reading about the SOAP vs. REST-debate for years now and most of it seems quite pointless to me: there are clear cases where SOAP is useful and others where REST is preferable. In fact, I've had this experience recently when writing a tool that aggregated results using the several Websearch APIs. Yahoo! has a REST API and Google a SOAP API. To get results from Google, you basically run wsdl.exe on their GoogleSearch.wsdl-file and before you know it (approximately three lines of code) you've got a reference to a handy array of ResultElement-objects that store all the info you would ever want to have about a searchresult (or well, that they'll give you anyway).

Excellent right? Because in contrast, when you want to do the same thing using the recently released Yahoo! API, you have to concatenate a bunch of parameters to a URL, run the HTTP request (WebRequest/WebResponse for instance) and parse the resulting XML-file. For instance with XmlDocument or some XmlReader. This will probably involve a case/switch-like piece of code unless you choose to do something a little more funky like deserialize it to an object.

In my experience however, if you want to integrate these services in a piece of software, you probably want the results to have a common interface, derive from a baseclass or adhere to some usage pattern. To get this to work with the Google API, you end up wrapping the generated code (you could, ofcourse, ignore the generated code or modify it so you end up with the output you like, but it's less work to just wrap it) and with Yahoo!, you came up with the classes you write results to yourself so you don't have to do anything at all. (Unless you use the code-first-design-later-methodology, which is still disturbingly common nowadays - I use it myself on my fun projects and it always pisses me off afterwards.)

In the end REST and SOAP get you the same amount of work, although the Google code will looke more messy, since you have multiple classes containing the same data (unless you really do discard the generated code, but that will mean a lot more work than wrapping it) and the Yahoo! code you wrote will be very much generic and reusable for other REST APIs.

Still, I see the point of SOAP: in a lot of cases, datatypes are crucial and you have all kinds of legacy systems in the back-end that you want to interface with unambiguously. So everything you write down in the WSDL definition is basically set in stone afterwards. Excellent! But for everything else, REST seems to be the way to go.

MSN Search RSS license destroys usefulness (updated)

In a post discussing the upcoming MSN Spaces API, Dare Obasanjo writes:

After noticing the difference in the media response to the ability to get search results as RSS feeds from MSN Search and the announcement of the Yahoo! Search Developer Network it is clear to me that simply having great functionality and blogging about it isn't enough. To me, getting MSN Search results as RSS feeds gives me at least two things over Yahoo's approach. The first is that developers don't have to register with MSN as they have to with Yahoo! since they need to get application IDs. The second is that since the search results are an RSS feed, they not only can be consumed programmatically but can be consumed by regular users with off the shelf RSS readers. However I saw more buzz about YSDN than about the MSN Search feeds from various corners. I suspect that the lack of "oomph" in the announcement is the cause of this occurence.

Unfortunately however, MSN Search results in RSS format can't be considered a real Web API, since every RSS feed containing search results has a copyright notice containing the following:

These XML results may not be used, reproduced or transmitted in any manner or for any purpose other than rendering MSN Search results within an RSS aggregator for your personal, non-commercial use. Any other use of these results requires express written permission from Microsoft Corporation. By accessing this web page or using these results in any manner whatsoever, you agree to be bound by the foregoing restrictions.

So the advantages the RSS-approach could have over the Yahoo! API are completely nullified by the license. In fact, opening the following link in your non-RSS aggregating browser actually violates the license: this link may only be viewed in an rss aggregator. I would say this is the main reason developers are more excited about the Yahoo! API.

Update: Dare adds:

Mike Torres points out that both Yahoo (see YSDN FAQ) and Google (see Google API FAQ) have similar restrictions in their terms of use. It would be good to see MSN leading the way here. We've already gone one step better by not requiring developers to register and get application IDs. We should be able to the loosen the terms of use as well.

While it's true that according to the linked FAQs I can't commercially use the Yahoo! and Google APIs, I can use them personally for whatever I want, which is something entirely different than only being allowed to view the responses in an RSS-aggregator. For instance, can I write a Google/Yahoo!/MSN Search-combining tool and release it as opensource on Sourceforge? Perhaps Microsoft won't sue me for doing so, but that doesn't mean that creating an application whose sole use infringes on the MSN Search license is legal. So before MSN starts "leading the way", please catch up first.

Posted: Mar 05 2005, 06:46 PM by jvdbos | with 3 comment(s)
Filed under: ,
Yahoo! releases Web API SDK

Wally mentions that Yahoo! has released an SDK to interface with several of their search engines using (what else?) webservices. According to the documentation, Yahoo! will allow 5,000 queries per 24 hours for everything except Local Search, which is limited to 1,000 per 24 hours. This is substantially more than Google's 1,000 for regular search.

Furthermore, Google limits the amount of results per query to 10 while Yahoo! returns up to 50 results per query (again, less for Local Search, but still 20). So effectively and if I didn't miss some really important clause in the license somewhere, you can query Google for 10,000 and Yahoo! for up to 250,000 results per day. If outdoing your competitor's standard limits by a factor of 25 is going to be the norm, I can't wait to see the MSN Search SDK. Although I haven't seen any mention of this anywhere, except for people asking whether it's coming every now and then.

Posted: Mar 01 2005, 04:31 PM by jvdbos | with no comments
Filed under: ,
More Posts