It turns out that you can easily close a tab with one click on the middle mouse button. On most modern mice, this would be the scroll wheel button. No fuss, no right-clicking, no selecting the tab and going to the other end of the screen for the X button. Just a single middle-click.
This works in Visual Studio 2005, in IE7, Firefox 1.5 and RSSBandit, to go by the applications I've tested it in. Seems to be a common UI convention that I have been entirely ignorant of. It's probably common knowledge for many people, but I only recently discovered it.
When I shared the discovery with my office-mates, it turned out that quite a few of them weren't aware of this trick either, so I figured I'd share it here for all the others who have missed it:
Enjoy.