It may not lock the site up, but it may lock the database up. Which means that the site server has to wait until the database sever responds before the site server can send the page to your client PC. The effect is the same. You wait.
Microsoft SQL Server has cursorless (loosely speaking) forward-only recordsets that you can explicitly make non-locking: SELECT * FROM MyTable WITH (NOLOCK) WHERE… But if My SQL is not as robust, it may not have such a feature.
You might be misled by caching, either at server, ISP or other levels. It would be better if we could get the word from the horse’s mouth – vB coders AND SQL experts.
If vB does not lock the database for large searches, then we/JDavis/Ed Zotti are barking up the wrong tree and there’s something else going on.
I don’t use caching so that’s definitely not a factor.
And I’m not saying Ed & friends are barking up the wrong tree. There may indeed be something wrong with the database that’s causing it to lock up during certain searches. I’m just saying that’s not how vBulletin is designed.
YES YES YES!
Do a search for “Google-Indexer”. The Google Indexer is an account that is given to Google by sites that are pay walled - often academic journals and the like for Google Scholar. Google gets access, and shows a snippet of the relevant content, but when the person clicks the link they are given a login screen.
You don’t have to coordinate with Google to do this, you just have to cloak your content. There is good cloaking and bad cloaking, which I would be more than happy to elaborate on…
The flip side of alllowing Google in is that everybody who knows your screen name can search for you and anything you might have said or done on the board and collect said information.
Yeah, they can do that here too, but they have sign up and pay money and etc and that has seemed to be too much of an effort for most of your friends and loved ones .and employers and etc … though it has happened; I do get these occasional panicky emails begging me for a solution, usually involving a name change and requests that certain threads be removed from the board. (FTR, we’ll change names but usually what goes here stays here.)
The greater majority of our membership has no wish to live message board life on Google . . . some of them for very good reason indeed. (We tell 'em not to do it and they go ahead and do it and whatchagonna do then?)
If memory serves I also do believe that when Google spiders things get really really slow here., people weren’t happy with that either.
Could we briefly be so democratic as to have a poll about folks’ feelings on that?
You can mitigate the effect the spiders have on the board, and by most accounts their activity constitutes nowhere near the strain the search engine does.
And regarding the whole privacy issue, I really think that’s a foolish reason to resist indexing (I’m not commenting about TubaDiva’s statement - I’m talking about people who think what they post here is somehow safe because Google doesn’t have access). While Google certainly makes it easier to find something interesting that someone has posted on the internet, the fact that Google can’t specifically index these pages is NOT a surefire protection against someone discovering what you’re up to.
Aside from the obvious factor that anyone can pay for an account here and search to their heart’s content (admittedly a somewhat unlikely scenario), there are plenty of ways interested parties can stumble upon information here. First, if you’re worried about employers/spouses reading something, there are other ways for them to see what you’re up to without any help from the internets in general. Logs, history, caches, and the like can be scoured to discover where you’ve been, and it’s not hard to follow behind to take a look see.
Seond, Google of course is not the only search engine out there. You can block the bigguns, but there’s always some little half-assed search engine out there that may be indexing things here. Here’s one I found with about 30 seconds of wandering through a couple links, starting from Google.
And finally, the SDMB has no control over what people post about the content here on other sites (other messageboards, live/deadjournals). If you post something embarrassing here that you don’t want indexed on the web in general, you better hope someone doesn’t link to it somewhere else saying “Hey look at this embarassing thread MrPrivacy posted on the SDMB!”
If anything, I believe disallowing Google access to the site causes more security issues because people mistakenly believe there is some manner of privacy and aren’t as careful about what they post as they should be. If you’re considering posting something ANYWHERE on the internet that would devastate a real life relationship with your boss/spouse/mommy/whatever, don’t do it.
Good advice and info from Slacker there. 
I think a further ‘meeting of the minds’ is in order.
We should have a town hall meeting in IMHO or ATMB (or both) and knock some ideas around.
Who’s in?
You’ve been here since October 2002, and you think your suggestion of a town hall meeting has a chance in hell of coming to fruition? 
Heh.
It’s even in the rules:
Straight Dope Message Board Rules
"Please remember that postings on this board are visible to anyone with access to the Internet, and are retained in a database indefinitely. Please do not post personal revelations or other information that may come back to haunt you. In particular, do not post your telephone number or address. "
Anyone who does believe this place is inviolate and that all material is confidential is truly living in a fool’s paradise.
Privacy concerns are not the only factors in this but a significant one as we had quite a bit of complaint about this issue.
Um, this is a message board community and we appreciate that and appreciate you but do keep in mind that this is not a co-op or a community garden or anything that is owned or operated by the members it serves but rather an asset of a corporate entity and is run as such.
So please, thanks but no.
Hee hee hee.
Not to be nitpicky, but the reason this thread is still in existance is because we can’t search to our heart’s content. (I know, I know, not the point you were making, I just thought it was an odd statement considering what we’re discussing).
It looks like the grapist got ahold of that seach engine.
Given all the pissing, maybe it’s a pee patch.
I keed, I keed
So basically,
“We do not wish to discuss this matter further.” is the message from On High?
Guess it wouldn’t be the first time.
In the first place I’m not particularly high.
(Though I may be after rehearsal tonight when we go out for beer.)
You do not need to put words in my mouth. Neither do I require an interpreter. If what I say is unclear to you please ask and I will endeavor to make it understandable to you.
Most importantly here it needs to be stressed that Dopers are the greatest people in the world. You all pull together for the greater good of the community for things large and small and that is truly wonderful. The main thrust of this thread is yet another example of your willingness to help solve a problem and make things better and that is much appreciated. It’s this spirit that makes this board one of the greatest places going on anywhere.
But it’s not your job to organize a working party. It’s not your problem to solve. It is not a matter of voting or polling.
We do appreciate your input. We always want to hear what you have to say. Please don’t be offended if management doesn’t take you up on your generous offers and clever ideas. Nothing personal, neither.
And for heaven’s sake, don’t be offended by what I just said. No offense intended, ever.
Please note I am only speaking as Administrator of this message board. I do not speak for the Reader unless charged as such. I have not been so tasked in this situation.
In this Discover Magazine map of the blogosphere the SDMB is like the “lonely jocks” - a satellite of the Internet commenting on but not participating in the nucleus, which happens to be where most of the money is to be made.
Oh, baby! Oh, oh, oh, oh! Yes! Yes! YES!
I guess we don’t need to discuss any more solutions with regards to searching because that site, www.omgili.com, has us all indexed!
Search Dopers! Search! I command you!
Just as a point of note, the file robots.txt on this server is used to control what search engines should and should not be able to see. However, it is an entirely voluntary standard and sites, like omgili, do not have to comply.
TubaDiva , if you don’t want omgili to index this site, you got a lot of work on your hands.
Any chance of getting some (semi-)technical follow-up from Jerry or from Ed? Because, like for so many others here, the “8 searches in 4 minutes brought the Boards to a standstill” statement simply does not compute. No modern DB read-locks records, which are being read accessed, from being seen by another thread/process!
I realise that. Of course. Knocking ideas around would not amount to a coup.
The problem that started this thread is still unresolved and hasn’t been explained to satisfaction (can we request that?)
And we have kicked board related ideas around in the past, so it ain’t all that foreign of a thing.
Tuba, I’m glad you took the time to respond here, thanks. Have you heard any more about this search problem and may I ask again why it just came to the front now?