When you access a page here, you access a few records in the database. When you search, you access a lot more records in the database. It all contributes to overall server load. Repeated searches, like if you spell something wrong, really hammer the database hard, since it’s one massive batch of record accesses after another.
Many years ago, we were having all kinds of server problems that were basically the result of server overload. You’d get things like pages timing out whenever you try to read a thread, or posts wouldn’t go through, all sorts of stuff like that. Since searches hammer the database pretty hard, adding a timer between searches was a quick and easy way to drop the load on the servers. It wasn’t enough though, and the problems were only really solved when we moved to bigger and better servers.
The original timer was 5 minutes. After the move to the newer servers, the timer was dropped to 2 minutes. I can only assume that they lowered it instead of getting rid of it completely because even on the new servers, there was still an issue of server capacity vs. server load.
As for why other sites don’t have the problem, that’s a simple tradeoff between user accesses and server capacity. You can run a web site on an off the shelf computer from Walmart, but once a lot of people start accessing it, the web site will quickly slow to a crawl and will become unusable. On the other extreme end of things, you can run a data farm like Google does and you’ll have so much capacity that millions of users can access the site simultaneously and they’ll all work just fine. At every point in between, there’s a certain amount of usage that any site can handle before it runs into problems. Sites that don’t have loading issues have more capacity than they need. Sites with loading issues have less than they need. Sites with occasional loading issues are right on the edge - they have what they need for most cases but not enough for the highest peak loads.
So you might think the solution for us is simple. Move to a hosting site with a greater server capacity, or pay for more capacity if our current hosting site can handle it (many hosting sites are scalable). That’s fine, but how do you pay for it? The SDMB isn’t a public service. We are corporate owned and we’re expected to turn a profit. If we don’t make enough of a profit, then the site is no longer worth maintaining financially, and the SDMB disappears forever. One very important fact is that we are owned by a print media corporation, and print media has been dying due to internet information taking over. You compare us to other sites that are doing well, and yes, we have some issues. But you aren’t comparing us to our industry as a whole. Many message boards have disappeared forever. We’re still around. We’re doing a lot better than they are, because we aren’t gone yet.
As I’ve said earlier, I don’t have access to the server logs and usage statistics, so I’m making a lot of guesses. But we have regularly had complaints about loading issues up until about a year ago. That’s a pretty strong indication to me that we don’t have a huge amount of server overhead available. We could move to a more powerful server, but the risk in that is if we end up costing more than we make in advertising, subscription fees, etc. then one day we’ll just disappear.
With our current setup, we’re able to handle our current usage, and the cost vs. revenue is such that we’re able to keep the lights on. I don’t have any access at all to the financial side of things here, but I suspect that we aren’t rolling in cash so much that we could easily just toss a bunch of money at better servers.
ETA: Searches on google (and tapatalk, I assume) don’t contribute to our server load because they cache their own copies of our posts and perform the searches on their servers instead of ours.
