newsfeed

I have to replace a newsfeed on our site (the current one has become too costly). I am planning to use a spider creator to create spiders that will crawl a number of sites an harvest headlines and links. The spider will write the results to a mysql database.

I need help with several aspects of this project. Once I have the headlines in the database, what is the most efficient way (in terms of server resources) to display them. I have read a little about xml and rss. We are considering converting the data to xml and using a channel to display the feed on the site. Is there a better idea--such as using an asp recordset.

If we put the data into an xml/rss document I could use some ideas about the best way to pull the data into an xml document and then display it. I have never dealt with this technology before but I have a little theoretical knowledge about it.

Thanks for any suggestions

Wallace

If you could get a hold of the xml feeds of other sites, you could create a spider that would crawl though the rss/xml feeds, and add a new entry to your database, when one of them updated.