# Someone please install search engine software on the SDMB! **URL:** \ **Category:** About This Message Board\ **Created:** [April 1, 2007, 6:12am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272 "2007-04-01T06:12:50Z")\ **Posts on this page:** 13\ **Page:** 3
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [April 3, 2007, 2:19am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/41 "2007-04-03T02:19:14Z")
> [@](#): > > Also, my question is not a simple yes or no. You have to know the inner workings of how vBulletin searches to answer it. If there exist inner workings to be known, that right there is evidence that it’s not simple to plug in a different search engine. ---
**Author:** ![Squink](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@Squink](https://boards.straightdope.com/u/Squink)\ **Post date:** [April 3, 2007, 2:20am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/42 "2007-04-03T02:20:19Z")
[QUOTE=Chasing Dreams] I didn’t feel like reading through the technical manuals for vBulletin just to try and figure out what kind of algorithm it is using for search. [/QUOTE] Here’s a [description in the online manual](http://www.vbulletin.com/docs/html/) > [@](#): > > vBulletin supports two types of search indexing. Fulltext searching uses a search index that is constructed by MySQL itself, whereas vBulletin’s own search feature uses its own index… Nothin about slapping in some third party search system. ---
**Author:** ![Chasing\_Dreams](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chasing\_Dreams](https://boards.straightdope.com/u/Chasing_Dreams)\ **Post date:** [April 3, 2007, 2:49am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/43 "2007-04-03T02:49:01Z")
[QUOTE=Squink] Here’s a [description in the online manual](http://www.vbulletin.com/docs/html/) Nothin about slapping in some third party search system. [/QUOTE] From that link, “By default, vBulletin will use its internal indexing feature. The results of this indexing process is stored in two tables, word and postindex. This provides a fast search mechanism but can cause problems on larger forums due to the ever increasing size of these tables. Each unique word is indexed in the word table and each occurrence of the word is indexed in the postindex table. To get around the large amount of space these tables can occupy we implemented MySQL Fulltext Search.” From reading that, it appears that MySQL Fulltext Search is more space efficient that the default search mechanism in vBulletin. Thus, leading to faster searches. However, I don’t know if this is included in the current version the board uses. ---
**Author:** ![Chasing\_Dreams](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chasing\_Dreams](https://boards.straightdope.com/u/Chasing_Dreams)\ **Post date:** [April 3, 2007, 3:38am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/44 "2007-04-03T03:38:50Z")
From the same site as before, check out [Search Engine Friendly Archive.](http://www.vbulletin.com/docs/html/vboptions_group_archive) vBulletin supports archiving of old threads if the site uses “the Apache web server with PHP compiled as a module”. This means that we could use Google to index and search old threads if server power is an issue. Just for the sake of knowing, how many megabytes of data does the SDMB create in a typical day? ---
**Author:** ![Squink](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@Squink](https://boards.straightdope.com/u/Squink)\ **Post date:** [April 3, 2007, 4:05am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/45 "2007-04-03T04:05:42Z")
[QUOTE=Chasing Dreams] it appears that MySQL Fulltext Search is more space efficient that the default search mechanism in vBulletin. Thus, leading to faster searches. [/QUOTE] Um, that’s not at all an obvious conclusion. MySQL’s [Full-Text search](http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html) includes it’s own array of nasty tables, indices and functions. ---
**Author:** ![Polerius](https://avatars.discourse-cdn.com/v4/letter/p/d78d45/32.png) [@Polerius](https://boards.straightdope.com/u/Polerius)\ **Post date:** [April 3, 2007, 5:47am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/46 "2007-04-03T05:47:00Z")
[QUOTE=Una Persson] I run vBulletin and have for 6 years. Unless the newest version has a built-in link to an external search engine, I’m unaware of any such link. Do you have a cite that it does have it as a feature? [/QUOTE] Well, thanks to **Chasing Dreams** , here is a [cite](http://www.vbulletin.com/docs/html/vboptions_group_archive) > [@](#): > > \*\*Forum Archive Enabled \*\*(yes/no) > > The Search-Friendly Archive works only under the Apache web server with PHP compiled as a module. > > It provides a basic structure that search engines can spider to grab all the content on your site. So, if I understand correctly, no special programming or hacking is required. Just set this feature to “yes” and run your favorite search engine on the archive. ---
**Author:** ![Chasing\_Dreams](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chasing\_Dreams](https://boards.straightdope.com/u/Chasing_Dreams)\ **Post date:** [April 3, 2007, 6:32am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/47 "2007-04-03T06:32:13Z")
Hey, Looks like the archive feature is already turned on. I just realized [the archive pages](http://boards.straightdope.com/sdmb/archive/index.php) existed. It is a link the the bottom right of each page. However, there are some issues with the usability of these archived threads. The most important issue is that the archives don’t have a search engine pointed at them. Thus, you can’t search. I believe one of the admins will have to set some parameters in vBulletin so that search engines (ie: Google) have permission to crawl the archive. This should largely mitigate the search performance issues. The second issue is that archived threads are butt ugly. SDMB members won’t want to use something so hideous compared to the normal threads. The ugliness issue can be solved too with a bit of creativity. Maybe there is a different style that the admins can use. However, the more elegant solution would be to put a link on a archived thread that points to the unarchived version. Example, the thread “Ack! It was in my kitchen!!!” can be found in the archive under: [http://boards.straightdope.com/sdmb/archive/index.php/t-415028.html](http://boards.straightdope.com/sdmb/archive/index.php/t-415028.html) The unarchived thread is located at: > **[Ack! It was in my kitchen!!!!](https://boards.straightdope.com/sdmb/showthread.php?t=415028)** > > I just killed a baby Red-Bellied Watersnake…in my kitchen! What was it doing in my kitchen? A snake. In my kitchen. I just can’t get over this. How did it get in my apartment? Was it heading for my dog’s waterbowl? Well it scared the shit out... You can see that if you know the URL of the archived thread, you also know the URL of the unarchived one. The key is the thread number, in this case it is 415028. QED ---
**Author:** ![Chasing\_Dreams](https://avatars.discourse-cdn.com/v4/letter/c/258eb7/32.png) [@Chasing\_Dreams](https://boards.straightdope.com/u/Chasing_Dreams)\ **Post date:** [April 3, 2007, 6:42am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/48 "2007-04-03T06:42:08Z")
Based on the number of threads in existence at the moment, 324 827, at about 100kb of data on average per thread, I estimate we have about 32-33 gigabytes of user content on the SDMB. Can any admins/mods confirm this? This is a sizable amount of data but nothing stupendously large. ---
**Author:** ![Garfield226](https://avatars.discourse-cdn.com/v4/letter/g/9e8a1a/32.png) [@Garfield226](https://boards.straightdope.com/u/Garfield226)\ **Post date:** [April 3, 2007, 6:43am UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/49 "2007-04-03T06:43:39Z")
IIRC Google indexing was done once, and has since been disallowed. I think it was right around when they added ads. ---
**Author:** ![Una\_Persson](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/una_persson/32/346_2.png) [@Una\_Persson](https://boards.straightdope.com/u/Una_Persson)\ **Post date:** [April 3, 2007, 1:29pm UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/50 "2007-04-03T13:29:17Z")
[QUOTE=Polerius] Well, thanks to **Chasing Dreams** , here is a [cite](http://www.vbulletin.com/docs/html/vboptions_group_archive) So, if I understand correctly, no special programming or hacking is required. Just set this feature to “yes” and run your favorite search engine on the archive. [/QUOTE] Good grief. You do know that the archive is not the same as the database, right? We haven’t been talking about the archive, we’ve been talking about the database. The whole point of the OP was the slow searches for the database. The archive has its own set of issues which people have been complaining about in this forum for quite some time, as threads have been getting moved into it from the database. ---
**Author:** ![Squink](https://avatars.discourse-cdn.com/v4/letter/s/b5e925/32.png) [@Squink](https://boards.straightdope.com/u/Squink)\ **Post date:** [April 3, 2007, 2:05pm UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/51 "2007-04-03T14:05:07Z")
[QUOTE=Una Persson] Good grief. [/QUOTE] Indeed. Reading around in a manual for a few minutes is no substitute for actually knowing what it is you’re talking about. ---
**Author:** ![Polerius](https://avatars.discourse-cdn.com/v4/letter/p/d78d45/32.png) [@Polerius](https://boards.straightdope.com/u/Polerius)\ **Post date:** [April 3, 2007, 3:48pm UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/52 "2007-04-03T15:48:00Z")
[QUOTE=Una Persson] Good grief. You do know that the archive is not the same as the database, right? [/QUOTE] They _aren’t the same_? I didn’t know that! You’re a fucking genius! Thanks for that enlightening piece of information. You know, instead of giving an exasperated and prissy answer, you could have just said “The Search-Friendly Archive does not work as adverstised”, or something like that. ---
**Author:** ![TubaDiva](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tubadiva/32/507_2.png) [@TubaDiva](https://boards.straightdope.com/u/TubaDiva)\ **Post date:** [April 3, 2007, 4:39pm UTC](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272/53 "2007-04-03T16:39:43Z")
Whoa! Let’s not go to Fist City here. Whatever personal opinions people may hold on this issue, I sincerely doubt there will be any software changes to the current server. That being said we’re always interested to hear what people have to say, especially people with some understanding of the hardware and software issues of the message board. So please bring us your ideas and certainly any interesting software you find that could enhance the message board experience for all of us. Thanks. [Previous page](https://boards.straightdope.com/t/someone-please-install-search-engine-software-on-the-sdmb/398272.md?page=2)