On a board with the mission of fighting ignorance, an easy to use and rapid search function is a fundamental tool because we should be using it frequently.
On a board whose culture emphasizes not resurrecting old threads but creating links to them, searches are going to be even more frequently used.
Then I read this:
8 searches in 4 minutes is ridiculous for a human archivist. For that kind of power searching, you’d need a tool…something like a desktop computer, perhaps.
Seriously – 8 searches in 4 minutes by one person dropped the boards? That flabbergasts me. That doesn’t seem like much at all…and we have hundreds of members and deal with hundreds of complex topics every day? What if two people decide to look something up? It just seems inadequate.
We let Google spider the board for a short while a few years back. Community concern and response put an end to that very quickly but Google did pick up quite a few posts in that small amount of time. This is what you’re seeing through Live Search.
I am also astounded. This just cannot be right. Something is wrong here. This is not wild-ass conjecture, this is a statement based on my experience with web servers. If the SDMB is the only thing on the server, and the server machine itself is a relatively up-to-date machine (i.e., within the last year or so), there is NO WAY a SMALL NUMBER (8 searches in 4 minutes is small, damnit) simple text database searches are going to crash a server. I repeat, something is wrong.
Has anyone considered asking for help? I mean, that’s what they’re there for.
How come after all these years, we’re just learning NOW why the boards crash so frequently? This problem, if it does exist, should have been patched years ago.
Count me as one of the many that find it hard to believe that 8 searches in 4 minutes can cause a meltdown.
Is there something special about the same person searching 8 times or could 8 different people search once each in the four minute span and cause the same problem? If so, this must happen dozens of times a day.
This does not seem logical.
Allowing Google to spider the boards was an accident, not a test. To be honest, I can’t believe that you don’t let Google spider the boards. There is such a wealth of information here, and question answering has become such a huge hit on the web (think Yahoo Answers) that this site has tremendous room for monetization and growth. Let Google spider the boards, more people will come in and give you more money for more servers. Create other “perks” for members. I really can’t believe that searching is supposed to be considered a perk in this day and age.
I respect that you guys are very conservative in changing anything here. On the other hand, a bit of creativity and forward thinking applied to the business model could really be good for this place.
PS - I would consider not being shown advertisements to be the only perk I need. I have a strong opinion that advertising is brainwashing. Unfortunately I sometimes browse from public computers that I cannot install Adblock on.
So here’s your new business model: Don’t show ads to paying members, and open the floodgates to the rest of the Internet.
IF vBulletin and/or SQL lock up the ENTIRE database for a search, and IF the search encompasses ALL (or nearly all) records, it might hold everything up for seconds to minutes. i say that from experience with database programming in the broadest sense.
The only excuse I can see for that kind of software is it is easier to write than alternatives; when it was written, databases were smaller and the total locked time very small, so it could be tolerated.
Locking the entire database ensures that no records will be omitted from the search. It prevents new records from being added or old ones edited, which could screw up the results. Under some circumstances, a database that is constantly changing could throw the search into an infinite loop or cause some records to be read more than once.
There are other conditions that cause file and/or record locking, but they are so brief that no one notices. I assume “add a record” is one of these; it is traditional to lock the file before an append so the file length parameters and pointers are updated correctly. But this only takes a relative handful of machine cycles; searching is open-ended and can take much more time.
I’m not saying this is the problem, but if it is, it’s time for vB/SQL to handle the matter more appropriately for current, multi-user, large-file conditions. There are, indeed, better routines available.
Hey, here’s an idea. Tuba, could Google’s search engine be applied to our data and vB’s disabled? I’m always impressed by Google’s programming, and maybe we could substitute it and have the added advantage that the board’s index is available externally. Is this possible?
Sheesh, if you have some info share it with us. :rolleyes:
IOW, if you know that it doesn’t then you must know why, right?
Besides, Musicat said if.
I like the idea of allowing Google to spider the Dope and archive pages.
Is it possible for that type of Google search by a non-paying member to only offer a tease and then allow paying members to ‘log in’ to get the complete search result?
I don’t mean to be so brief, but it just doesn’t work that way. The reason it doesn’t is because the people who wrote vBulletin and MySQL aren’t idiots. Asking me to explain why is like asking someone why cars don’t have square wheels. It just wouldn’t work that way.
As to how it actually does work, I’m not a database expert, so I can’t expound on that adequately, but I know from using and administering vBulletin messageboards since vB debuted years and years ago that it absolutely doesn’t lock the site up when someone is searching. If you need to see for yourself, find a large vBulletin site like neogaf.com and try it. Or heck, try it here. Open the SDMB in one window and search for something while clicking around in another. Unless the server is choking in general, it won’t be a problem (I just tried it and it worked fine).
Wouldn’t that also allow us to seach the board via google.
ie from google, you could search
[search minutes site:boards.straightdope.com] and pull up this thread.
This way we can tax the google servers instead of ours.
