Turning off the RSS feed seems to have made a vast improvement in the performance of this board.
I googled, and read the wikipedia description. Seems like RSS would be good for a “weather bug” or “breaking headlines”, or stock ticker thing, but what did it (they?) do for the SDMB?
Just like “breaking news,” a messageboard gets new content all the time.
For people who use RSS readers, it was a handy way to get info on new posts right on their desktop. “New post in General Questions - Does a duck’s quack echo?” … something like that. You could set up your reader to grab such new posts from all over. So you’d get a notice there’s a new post to your favorite blog, a new article posted on CNN.com, a new sale item at buy.com, etc. You could pick and choose which items that came through your reader that you wanted to read, and the content was delivered in more “real time” (with slight delays, but fast enough).
Basically having an RSS reader with content from all your favorite sites with RSS capabilities keeps you from having to constantly patrol each site for new content. The content’s delivered to you and you can click to read or not.
Most feeds are also configurable. For example you could tell the SDMB feed to only show GQ and IMHO thread activity and not other forums. For CNN you could tell it you only want Arts & Life articles. Weather is the same - only news for your zip code.
It’s a great idea but unfortunately for this particular board setup the amount of subscribers to the feed and the amount of posting that goes on is just too heavy for the servers to handle. If you noticed, the performance was shit with the new server and as soon as the RSS feed was shut down the performance went up tremendously.
“RSS feed” is really a misnomer, in that it implies the server is pushing content to the clients.
What’s really happening is the client is asking the server repeatedly: “What’s new?” and the server replies either “Nothing”, or “This, and this, and this”. The client then assembles the answers from multiple such queries into something that looks to the human user like each server is sending it the new stuff as it occurs.
Imagine opening the forum contents page in your browser & clicking [refresh page] every 10 seconds. Now imagine 40,000 other Dopers doing the same. That’s what RSS “feed” looks like to the server. And yes, that can bring a server to its knees.
RSS is really a crappy & not very scalable tool, but it’s the only way to get around the security problems associated with true server push, where the Dope could just send you a change when it happens.
Ideally there would be a standard (and people would actually abide by it), that feed clients not be permitted to ping a server faster than, say, once every 5 minutes. A ot more people could get decent service that way.
Some servers do implement crude throttling where if you ping too rapidly you get blacklisted. But even then, until/unless the (usually clueless) user adjusts their client, the server still needs to handle receiving the traffic & deciding to black-hole it. In big installations that can be offloaded to front-end traffic shapers, but that’s well beyond the SDMB budget.
Only if you have a really crappy implementation that hits the database on every query, like vBulletin does. The proper way to do it is write out the XML to a static file every few minutes and let the RSS clients request that.
Thanks for the very informative answers, that explains a lot.