Sign in
|
Join
ISerializable - Roy Osherove's Blog
Unit Testing, Agile Development, Leadership & .NET - By Roy Osherove
This Blog
Home
About
Syndication
RSS
Atom
Comments RSS
Search
Go
Navigation
Home
Blogs
News
My new book is out!
The Art Of Unit Testing
Buy and read it as I write it.
I work at:
Your ad here
The Art Of Unit Testing Book
Roy's Cool Tools
Subscribe!
Subscribe to ISerializable by Email
About
Hire me
Ask me
On my bookshelf
About me
Ego trip
Roy's Tools
5 Whys - a blog for team leaders
Key.bo - a search engine wiki for keyboard shortcuts
unit testing
ruby styff
All Developer Songs
It's Time for Violence
Que Sera Sera
Articles
3: Oops! Typed Datasets
Are
scalable!
4: Introduction To Regular Expressions
5: Practical Parsing Using Groups in Regular Expressions
6: UI Threading Helper Classes
Make Your App Support Plugins 2 - Dynamic Search (MSDN)
Winforms Data Binding Lessons Learned
Make Your App Support Plugins (MSDN)
1: Introduction to Typed Datasets
2: Typed Datasets Are No Silver Bullet
My articles on MSDNAA
7: Solving VS.NET Debugger Problems
Make your log files searchable using Regex and the XML classes (MSDN)
Introduction to TDD with NUnit
Fun with Unit Tests – Testing abstract classes
New: Creating a generic Site-To-RSS tool
.Net scripting
- the practical way
Simplified Database Unit testing using Enterprise Services
Creating custom test attributes easily with NUnit 2.2.1
Cool tools every .Net Dev should be aware of
Cool Tools every .Net developer should be aware of
New: The case for staged delivery and Agile methodologies
My .Net Deep Dive lectures on video
New: Defensive event publishing in .Net, part 1
Test Feasibility Matrix
Depenedency Breaking Issues
*new* Achieving And Recognizing Testable Software Designs – Part I
Favorite Blogs
The Morning Brew
Martin Fowler
Scott Hanselman
Joel On Software
.NET Weblogs
Microsoft Israel Community
The Runtime
Daniel Moth
Oren Eini
Jimmy Bogard
CodeBetter
Dustin Campbell
Guy Kawasaki
Stephen Toub
Research @ Intel
Udi Dahan
The Typemock Insider
My Projects
Vs.Net Settings import.export Add-in
SchemaHelper - auto-detect & create data relations
Proxy handling using ProxyFactory and ProxyInfo
BackgroundWorker implementation
XtUnit: An Unofficial Unit Testing Extensibility Framework - Add new attributes to NUnit or MbUnit e
Intercetpion Application Block
Extensibility Application Block
The Regulator
VS.Net 2003 registry tweaker
My Tools page
Regular Expressions
RegEx Lib
Expresso
Regex Blogs
Sites : .Net
.Net Tools List
.NetWebLogs Forums
Winforms FAQ
.Net Debugging Resources
.Net WebCasts & Others
.NetWeblogs Archive
MSDN Magazine
Design Patterns in C#
.Net Rocks Radio
.Net Resources
Howto: .Net common tasks
VB.Net blogs on MSDN
.NetSlackers
Sites : Misc
Regular Expression Library
MSR Downloads
Win2k3 Tweak Guide
About Microsoft Interviews
Tech Interview Riddles
Feedster
Amazon Light
C:\Utils
Sites : Unit Testing & XP
NUnitASP
Tips and techniques with NUnit
NUnit
NUnit Addin
XProgramming
MSDN Mag:Simplify Data Layer Unit Testing using Enterprise Services
Tags
.NET
.Net 2.0
.Net Original
.Net Quotations
.NetWeblogs Site
Addin Contest
ADO.Net
Agile
Agile Israel News
Agile Related
altnet
altnetconf
altnetisrael
Architecture
Art Of Unit Testing
ASP.NET
BDD
Blogging
C#
CLR
Community
Community News
Cool Articles
Cool sites
Cool Tools
Extensibility
Family
FeatureFocus
Free book chapters
General Software Development
Interview
Lean
Mobile
MSBuild
NDC09-Video
Off Topic
Open Source
Other
Product Reviews
Project Management
racer
Recommended books
Reflection
Regex
Regular Expressions
review
Security
Sharepoint
Silverlight
SOA
Songs
SQL Server
tdd
Team Agile News
Team System
TechEd 05
Testing Guidelines
TestReview
Threading
Tips & Tricks
Typed Datasets
Typemock
Unit Testing
Visual Studio
web
web services
WebCast
Windows Forms
WinFX
Recent Posts
How to: Move your blog off of weblogs.asp.net (aka ‘This Blog has moved’)
test – ignore
Bounty: 500$ is you can convert my blog to squarespace
Join me for a live webinar on unit testing with Isolator++ this thursday
What’s coming in Test Lint 1.5
Archives
November 2010 (2)
October 2010 (4)
September 2010 (4)
August 2010 (3)
July 2010 (2)
June 2010 (5)
May 2010 (6)
April 2010 (6)
March 2010 (4)
February 2010 (5)
January 2010 (11)
December 2009 (7)
November 2009 (7)
October 2009 (5)
September 2009 (6)
August 2009 (21)
July 2009 (7)
June 2009 (11)
May 2009 (13)
April 2009 (5)
March 2009 (21)
February 2009 (4)
January 2009 (2)
December 2008 (5)
November 2008 (6)
October 2008 (13)
September 2008 (4)
August 2008 (13)
July 2008 (19)
June 2008 (5)
May 2008 (17)
April 2008 (11)
March 2008 (13)
February 2008 (16)
January 2008 (21)
December 2007 (8)
November 2007 (18)
October 2007 (17)
September 2007 (15)
August 2007 (19)
July 2007 (18)
June 2007 (33)
May 2007 (16)
April 2007 (10)
March 2007 (15)
February 2007 (10)
January 2007 (11)
December 2006 (22)
November 2006 (18)
October 2006 (19)
September 2006 (30)
August 2006 (19)
July 2006 (27)
June 2006 (26)
May 2006 (32)
April 2006 (15)
March 2006 (20)
February 2006 (33)
January 2006 (23)
December 2005 (22)
November 2005 (41)
October 2005 (21)
September 2005 (7)
August 2005 (28)
July 2005 (41)
June 2005 (60)
May 2005 (14)
April 2005 (51)
March 2005 (31)
February 2005 (17)
January 2005 (63)
December 2004 (45)
November 2004 (35)
October 2004 (28)
September 2004 (36)
August 2004 (21)
July 2004 (44)
June 2004 (63)
May 2004 (62)
April 2004 (78)
March 2004 (64)
February 2004 (55)
January 2004 (67)
December 2003 (34)
November 2003 (67)
October 2003 (68)
September 2003 (113)
August 2003 (56)
July 2003 (112)
June 2003 (71)
May 2003 (136)
April 2003 (52)
March 2003 (81)
February 2003 (77)
[New article]: Creating a generic Site-To-RSS tool (site scraping)
My blog has moved.
You can view this post at the following address:
http://www.osherove.com/blog/2003/9/28/new-article-creating-a-generic-site-to-rss-tool-site-scrapin.html
Published
Sunday, September 28, 2003 11:48 PM by
RoyOsherove
Filed under:
.Net Original
,
Cool Tools
,
Cool Articles
Comments
Sunday, September 28, 2003 8:15 PM by
Scott Watermasysk
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Hi Roy,
1. DotnetWire already has a feed:
http://dotnetwire.com/newsfeed/rss/
2. I would not recommend hosting a feed the scrapes someone else's site. If a site (such as DotNetWire) wanted to provide its content via RSS, they would. Many of these sites only revenue stream is through page views. Hosting something like this is effectively stealing from them. For your own personal use, I see no harm.
-Scott
Sunday, September 28, 2003 8:26 PM by
Roy Osherove
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Scott: Oops! Ok. I won't host it. But it's still a great excercise! :)
Sunday, September 28, 2003 8:43 PM by
Scott Watermasysk
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Absolutely. For internal/personal use I think it is a great idea. Companies that want to make use of RSS, but don't have the time to create the feeds can probably make a lot of use out of it.
yole used to have a feature in Syndirella which did this for you.
Sunday, September 28, 2003 9:34 PM by
Randy H.
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Roy- don't be down. This is a great idea and good effort, glad to see the work you did here.
Monday, September 29, 2003 9:22 AM by
SBC
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Roy.. good stuff.. you the man..
Monday, September 29, 2003 11:22 AM by
Drew Robbins
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Great article. I need something like this on the backend of PDCBloggers.net to track blogs that do not have RSS feeds.
Tuesday, September 30, 2003 2:52 AM by TheCshark
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Tip: use ToArray instead of GetBuffer, ToArray works when the MemoryStream is closed.
Tuesday, September 30, 2003 9:46 AM by
Roy Osherove
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Thanks for the tip TheCshark!
Friday, October 31, 2003 6:48 AM by
Krisztian Gyuris
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
I tried your sitefeeder site and it is really nice :)
I had a small addition when I clicked on a feed and then tried to look at the help menu, it did not showed up in full only the first item is visible. If I am not watching the one of the feeds the Help menu works ok.
Cris
Monday, April 12, 2004 6:16 AM by
Robin Good
#
re: [New article]: Creating a generic Site-To-RSS tool (site scraping)
Though I highly respect and honour your ethical approach to RSS scraping, let me mention that RSS scraping can ALSO greatly increase a site reach and visibility, including the consequent traffic, market value and even credibility it may gain in cyberspace.
I would think that it is a fair bet approaching RSS scraping by always first contacting the site webmaster/owner and explaining the reasons and interests for doing so. Obviously if you are scraping a site for personal reasons (to keep trac of its changes), you are certainly going out of your way to do something that can be easily done much effectively with dedicated tools, and without a need to have an RSS feed.
(Free and paid service/tools like Copernic Tracker, Watchthatpage offer a simple and very effective way to track any web site for changes of any kind).
Therefore is your intention is syndicating through RSS, please do not restrain yourself on this front, while informing the site owner of your intentions and goals.
Having more re-usable RSS-based content out there can only be a beneficial thing, as the more we are able to convert Internet content to it, the more we will able to brea free of mainstream corporate controlled media and to start really appreciating the value of a growing number of people who will select, aggregate, filter, and combine the best news and content out there in specific categories of interest.
This is the emerging role of the "newsmaster", and the work you have done here in providing an extra route for such professional to increase its abilities, can only be greeted with sincere kudos and thanks from those who can see the above and who are willing to start investing in this direction early in the game.
Many thanks Roy, and keep up the good work!