Okay, I’m going to show off my incredible lack of internet savvy here, but what exactly does the search function do to the server that’s so hard on it? Furthermore (and this is the stupid part), since I don’t really know what a server does: does running searches slow down the board after you’re done running them? And in a more general sense, can anything someone does on the boards affect the server adversely in a permanent way, or would everything run perfectly if everyone but a few people were to suddenly leave the boards? (I hope that all made sense)
OK, when you do something like view a thread, the server uses the nice little number attached to the thread to look it up. These are all stored in the server sorted in a nice, convenient order, so it’s easy to look up. On the other hand, when you do a search (especially one for “any date”, or in “all forums”), the server doesn’t know where to start. It has to look through pretty much everything, and that’s a lot of stuff. While the search is actually going on, this slows everything down considerably.
After the search is over, the worst is essentially past, though it might take a little while (not very long, as I understand) to catch up on other stuff it was doing. This, the techs tell us, is the reason that sometimes, the board will be totally dead, then all of a sudden, it’s flying: Someone’s search just finished.
Nothing you do will have a permanent adverse effect on the boards, beyond normal wear-and-tear. Well, OK, making a new post increases the size of the database, which makes things slightly harder for the server, but that’s what we’re here for, anyway.
To sum up: If you must search, please do it during the off-hours, if possible, and please restrict your search as much as possible. It’ll make things easier on everyone.
Is it possible to put in a feature that will let you search only two or three forums at a time?
I, for one, hardly ever need to find a thread in either of the Comments columns and rarely need something in the About forum. I can even rule out if what I’m looking for is in the Pit or not.
The way it is now, it’s either one or all. Just wondering.
I was about to suggest the same thing, Biggirl. There are lots of times I can narrow it down to maybe IMHO or MPSIMS, or GQ or the Pit, etc, etc. Right now, I have a choice between searching them one at a time, or just searching all the fora. Being selfish and not nearly civic-minded enough, I usually just search them all. Now I realize this isn’t a magic solution, but a few lines of code should be all that’s needed to significantly cut down search times.
Here’s what happens in laymen’s terms:
You can regard the entire SDMB site as a special book where each thread is in a separate page, and each forum is a separate chapter in the book. The book is special in that you can add pages, change their text, and even reorder them so that they’re in chronological order.
The cool thing about this setup: like Chronos mentions, if you want to look at a specific thread it’s easy, you know its page number and you go there. If you want to look at the latest threads in a forum, you look at the last few pages in that chapter. The book even has indices in the back for such things as threads where any user has posted a reply, so you can easily find a user’s postings.
But, searching for some specific words is not that easy. The only way it can be done is to look at each page in sequence and see if the word(s) appears there.
Now all this is being done by some guy/girl called Mysql (let’s stick to guy for simplicity, ok?). If you ask for all threads with the word “foobar”, Mysql starts at the beginning of the book, and scans every page looking for “foobar”. If you ask for all GD threads with the word “foobar”, Mysql only scans the pages in the GD chapter, and if you ask for GD threads with “foobar” updated today, Mysql only scans the last few pages in the GD chapter.
So obviously if you specify a short date range or a not-so popular forum, your search will be quick. And that’s where one of the problems is: most people will search in the most popular forums (IMHO, MPSIMS, GD) that happen to be the thickest chapters in the book. And they don’t specify a date range in fear of missing the one thread they’re looking for. And you can’t blame them, that’s what they’re supposed to do. Excluding low-traffic forums like ATMB doesn’t help that much because their chapters are pretty thin to begin with, you’re still left with the majority of the pages from the big popular chapters.
Another problem: Mysql takes care of everything: adding new postings and threads, making the moderation changes, and doing the searches. That’s a busy guy, and if he’s scanning pages looking for words, he can’t be updating and reordering the pages simultaneously. What’s worse, although Mysql’s pretty good at the updating part (he’s dexterous at cutting and pasting and manipulating pages) and even indexing and looking stuff up in the index, he’s lousy at scanning pages (his eyes are just not that good), but hey he works for free so don’t complain :D.
And because of all this, the problem won’t go away anytime soon. What really needs to be done, is that Mysql should make a different copy of the book and give it to these guys called Unix and grep, now these guys can scan their book lightning fast and tell Mysql which pages have the match and go from there ;).
Hope this helps.
Would there be any way to fix the search function so that it actually looks only at POSTS submitted within the specified time period, rather than THREADS that have been active within that time period?
In case that’s not clear: when I put in my username and select “since yesterday”, I’m only interested in posts from yesterday to today in which my name has been mentioned. However, if somebody mentioned my name a week ago in a thread, if anybody has posted to that thread since yesterday it will turn up in my search results. Can this excruciatingly annoying aspect of the search function be adjusted?
I think that under vB, that’s the way it is, ruadh. It “dates” threads by the last post, and THEN reviews the content. Not the other way 'round, like this: search for all threads wherein ruadh has posted, then limit output to last X days.
And even IF it could do it the other way 'round (which technically might not be impossible, for all I know), it would mean an increasingly heavy search process. So the answer is: ain’t gonna happen.
Yes, there’s plenty of things which are technically possible to do to improve the search engine. Frankly, the built-in vBulletin search engine is the most simpleminded one I’ve ever seen. However, we don’t have much choice: The techs at the Reader are extremely reluctant (for good reasons) to go mucking about with the message board code, and they’re likewise reluctant to upgrade the software as soon as a new version comes out.
What this means is that if we want any new features (like a grep-powered search engine which would recognize regexps), we have to first suggest them to the Jelsoft programmers (makers of vB), hope that they implement them, let a bunch of other schmucks do the beta (and gamma and delta) testing, and then, when we’re reasonably sure that it’ll work, upgrade.
There are some improvements to the Search in vBulletin 2.0, but since UncleBeer already suggested that upgrading the software here is unlikely due the risk, they may not be seen.
There are some vBulletin hacks that also increase the Search engine effectiveness, that ironically may be less risky to implement, since they are less far reaching - and to go back, you just return the one script to its original condition.
Then again, what do I know?
These answers clear it up a lot. Thanks.