Silverlight's security model prevents making a request to another domain. For instance, if you're serving a page with a Silverlight control from www.silverlight.net, the control can't make a request to maps.google.com, flickr.com, etc; the control can only call back to the silverlight.net domain. That's probably a good thing from a security perspective - it'd just be way too easy for a page to cross security zones and other bad things if a client side networking stack could communicate with any domain it wanted.
But there are some very legitimate uses of cross domain access, such as the (overused) Google map and Flickr mashup example. There are some ways around the Silverlight domain restrictions, though. The easiest way is to use a server side proxy, so the Silverlight control only talks to the site that served it, and the server side code makes cross domain requests.