# SDMB Archived On Another Site **URL:** \ **Category:** About This Message Board\ **Created:** [April 12, 2002, 3:34am UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162 "2002-04-12T03:34:21Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [April 14, 2002, 6:57pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/22 "2002-04-14T18:57:03Z")
> [@](#): > > \*Originally posted by OpalCat \* > \*\*Hey! Don’t put me on the reasons to prohibit list! \*\* [list=“1”][li]Sorry, Opal![/li][li]Sorry, Opal![/li][li]Hi, Opal![/list][/li] Would I get off the hook if I let you share Tuba’s chocolate? 🙂 ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [April 14, 2002, 7:22pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/23 "2002-04-14T19:22:49Z")
I guess I just can’t shut up on this topic, so here are some rambling comments/observations. [generic admin/tech hat on] First, from [boardreader.com](http://boardreader.com)’s [submit site FAQ:](http://www.boardreader.com/submitsite.asp) > [@](#): > > By submitting your website information, you agree to allow us to include your > site in our search or list it in our promotional materials. Therefore, if **Tuba** didn’t know about it, some non-official SDMB person must have submitted it. However, it looks like [boardreader.com](http://boardreader.com) does not verify WHO submits WHAT before spidering, so anyone could have done it. (Maybe a little unethical, or maybe someone thought they were doing a good thing as a favor to the mods.) I just submitted my own Board to them, and before I had time to open a new window and run a search to test, they had already spidered the site! But the initial spidering was only topic headers. This explains why, checking my server stats, I found only 9 accesses from boardreader today. That about matches the number of forums, so I imagine they read each forum list once, but went no further. I haven’t checked to see if they dipped into the forum archives yet. It looks like boardreader indexes deeper than just headers for other sites, though, so my guess is they go back and do it again more thoroughly later. I certainly hope that their spidering schedule is very low-key and intelligent. The last thing any admin wants to see is 20% of their stats full of robots on a regular basis. Maybe this is configurable on request? [generic admin/tech hat off] [user hat on] **From a user’s point of view,** it is a _terrific feature_ to be able to search multiple forums/boards at once! I may get flak from this statement, but BBSes and Usenet are dead. I support a bunch of clients, mostly computer newbies, and they have never heard of Usenet. I think this is the long-term trend, as web forums take their place. Conventional search engines index Usenet, but not all spider the boards, altho I see no reason why they can’t. [user hat off] I hope there is something useful in here. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [April 14, 2002, 7:36pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/24 "2002-04-14T19:36:11Z")
> [@](#): > > But the initial spidering was only topic headers. Correction: The inital spidering was only topic headers and user names. Still, they managed to do it on one sweep. Doesn’t sound like much, but there’s a lot of poorly-written software out there that would have taken 2 \* 10[sup]23[/sup] requests to do the same thing. ---
**Author:** ![tasmisr](https://avatars.discourse-cdn.com/v4/letter/t/d07c76/32.png) [@tasmisr](https://boards.straightdope.com/u/tasmisr)\ **Post date:** [May 31, 2002, 10:38am UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/25 "2002-05-31T10:38:23Z")
boardreader is a nice service… but , Musicat, you’re sayign there is 2\*10(23) requests everytime ??! did you notice any slowdown in site performance as a bandwidth or system resources utilization? I wonder if they just hit only thread pages at a time… ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [May 31, 2002, 11:57pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/26 "2002-05-31T23:57:31Z")
No, **Musicat** is saying that some poorly designed programs would take a great many requests (I presume that 2\*10[sup]23[/sup] was an exaggeration), but that Boardreader appears (to **Musicat** , at least) to be a well-designed program which does not cause too many problems. Any updates on that, **Musicat**? Have they finished their sweep of your site yet, and if so, how much of a hit was it? ---
**Author:** ![TubaDiva](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/tubadiva/32/507_2.png) [@TubaDiva](https://boards.straightdope.com/u/TubaDiva)\ **Post date:** [June 1, 2002, 3:45am UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/27 "2002-06-01T03:45:38Z")
Haven’t heard back from the Reader (they’ve been frying other fish, I do believe), I’ll ask 'em again. And this week I’ll go look at the site to see what we can retrieve, I’d like to do that if we can. No promises, though. your humble TubaDiva Administrator ---
**Author:** ![erislover](https://avatars.discourse-cdn.com/v4/letter/e/71e660/32.png) [@erislover](https://boards.straightdope.com/u/erislover)\ **Post date:** [June 1, 2002, 5:37am UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/28 "2002-06-01T05:37:19Z")
How are you guys finding old posts? When I search all I get are links back to this board, complete with missing posts (if that makes any sense). ---
**Author:** ![tasmisr](https://avatars.discourse-cdn.com/v4/letter/t/d07c76/32.png) [@tasmisr](https://boards.straightdope.com/u/tasmisr)\ **Post date:** [June 1, 2002, 2:31pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/29 "2002-06-01T14:31:41Z")
If the thread was lost on staightdope, it should be archived on [boardreader.com](http://boardreader.com). ---
**Author:** ![Humble\_Servant](https://avatars.discourse-cdn.com/v4/letter/h/47e85d/32.png) [@Humble\_Servant](https://boards.straightdope.com/u/Humble_Servant)\ **Post date:** [June 1, 2002, 3:57pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/30 "2002-06-01T15:57:37Z")
Maybe we don’t need to rescue all 987 LOTR threads, but I just printed out several threads I’ve thought about since the crash. Remember all of the threads on the “Who Wrote the Bible” Staff Reports? They’re all there. So are all of the CCC threads. Even if we can’t jsutify recapturing MPSIMS, are be we can make a case for the column-related stuff. This is a good thing. ---
**Author:** ![erislover](https://avatars.discourse-cdn.com/v4/letter/e/71e660/32.png) [@erislover](https://boards.straightdope.com/u/erislover)\ **Post date:** [June 1, 2002, 5:13pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/31 "2002-06-01T17:13:36Z")
I understand that it should be archived there. But how do I access their archives is the question. When I perform the search all the search results point back here which obviously does no good. ---
**Author:** ![Humble\_Servant](https://avatars.discourse-cdn.com/v4/letter/h/47e85d/32.png) [@Humble\_Servant](https://boards.straightdope.com/u/Humble_Servant)\ **Post date:** [June 1, 2002, 8:19pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/32 "2002-06-01T20:19:50Z")
**eris** : Here’s how I found the “Who wrote the Bible threads.” I logged onto [boardreader.com](http://boardreader.com). At the top of the page is a search box. I typed in: Dex Euty Who Wrote The Bible Part. I didn’t specify SDMB or any special search. The result is a screen of links to about 10 or so threads, all with the titles I remember seeing in CSR. When clicked on, the text of the old threads is there (minus links and some coding). I’m gonna try a link direct to one of the archived threads. [Who Wrote the Bible Part I](http://www.boardreader.com/scripts/texis.exe/viewthread?query=Dex+Euty+Who+Wrote+the+Bible+Part&ttype=research&postid=3ced7bcc90). Works in preview. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [June 2, 2002, 1:55pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/33 "2002-06-02T13:55:16Z")
> [@](#): > > _Originally posted by Chronos \* > \*\*No, **Musicat** is saying that some poorly designed programs would take a great many requests (I presume that 2_10[sup]23[/sup] was an exaggeration), but that Boardreader appears (to **Musicat** , at least) to be a well-designed program which does not cause too many problems. > > Any updates on that, **Musicat**? Have they finished their sweep of your site yet, and if so, how much of a hit was it? \*\* Right you are, Sir. 2\*10[sup]23[/sup] was intended to be an exaggeration and should have been followed by the ubiquitous smiley. 😉 My initial enthusiasm might have been premature. Searching for random data (message headers only) from my own Board on Boardreader turned up NOTHING! Perhaps they deleted it from their spidering list? I didn’t get any notices from them. I’ll try submitting it again and see what happens. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [June 2, 2002, 2:28pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/34 "2002-06-02T14:28:42Z")
Uh, **Tasmisr,** please read my lengthy post carefully, as I think it answers your questions. And you might not be aware that the SDMB sends out an email message to all those who posted in a thread whenever a new post happens. So emailing me about your post isn’t really needed; your email just shows up alongside the automated SDMB ones. I submitted my site once again to Boardreader and have configured a stats watch to keep an eye on their activity, assuming it comes from [boardreader.com](http://boardreader.com). We’ll see what happens over the next few days. ---
**Author:** ![femtosecond](https://avatars.discourse-cdn.com/v4/letter/f/f08c70/32.png) [@femtosecond](https://boards.straightdope.com/u/femtosecond)\ **Post date:** [June 2, 2002, 5:01pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/35 "2002-06-02T17:01:07Z")
(I didn’t want to struggle with our board’s long-link-text-truncating feature, so my ‘links’ are just blue. Is there a way to avoid it?) > [@](#): > > _Originally posted by Humble Servant_ > **eris: Here’s how I found the “Who wrote the Bible threads.” I logged onto [boardreader.com](http://boardreader.com). At the top of the page is a search box. I typed in: Dex Euty Who Wrote The Bible Part. I didn’t specify SDMB or any special search. The result is a screen of links to about 10 or so threads, all with the titles I remember seeing in CSR. When clicked on, the text of the old threads is there (minus links and some coding).** I repeated your search query. Clicking on “**Who wrote the Bible? (Part 1)**” (link [http://www.boardreader.com/scripts/texis.exe/search/+txDde9Ymw/redirect.html?query=Dex+Euty+Who+Wrote+The+Bible+Part&ttype=research&postid=3ced7bcc120&posturl=http%3A//boards.straightdope.com/sdmb/showthread.php%3Fthreadid%3D108842%26highlight%3DDex%2BEuty%2BWho%2BWrote%2BThe%2BBible%2BPart](http://www.boardreader.com/scripts/texis.exe/search/+txDde9Ymw/redirect.html?query=Dex+Euty+Who+Wrote+The+Bible+Part&ttype=research&postid=3ced7bcc120&posturl=http%3A//boards.straightdope.com/sdmb/showthread.php%3Fthreadid%3D108842%26highlight%3DDex%2BEuty%2BWho%2BWrote%2BThe%2BBible%2BPart)) redirects to [http://www.boardreader.com/scripts/texis.exe/viewthread?query=Dex+Euty+Who+Wrote+The+Bible+Part&ttype=research&postid=3ced7bcc90](http://www.boardreader.com/scripts/texis.exe/viewthread?query=Dex+Euty+Who+Wrote+The+Bible+Part&ttype=research&postid=3ced7bcc90) and it indeed displays their cached version. Clicking on “ **Who wrote the “Who Wrote the Bible” column -OR- Where’s Cecil?** ”, however and for instance, redirects to [http://boards.straightdope.com/sdmb/showthread.php?threadid=108842&highlight=Dex+Euty+Who+Wrote+The+Bible+Part](http://boards.straightdope.com/sdmb/showthread.php?threadid=108842&highlight=Dex+Euty+Who+Wrote+The+Bible+Part), our very own site. So they giving you a link doesn’t necessarily mean they will redirect you to their cache when you click on it, and you have no way of finding out beforehand. Strange. Maybe they don’t keep all of the threads they indexed in their cache for us to read? But… taking their postID out of above non-caching link and feeding them something like [http://www.boardreader.com/scripts/texis.exe/viewthread?postid=3ced7bcc120](http://www.boardreader.com/scripts/texis.exe/viewthread?postid=3ced7bcc120) makes them display their version. So they _do_ have it cached. Did I mention that’s strange? I don’t see an easy way of getting a list of what they actually have, short of asking them directly. BTW, on christmas 2001 I did a snapshot of some of our forum display pages. I did read only 10 pages or few days worth of MPSIMS, for instance, but my GQ,CCC+Mailbag lists start at 26.12.2001 and go back beyond 07.12 (the point at which the posts start missing). looks like this: threadid, replies, views, date, thread title, you get it > [@](#): > > 106247 O 1 7 256 12-26-2001 09:13 AM \>“I work for a prominent insurance company in Bloomington, Illinois.”\< Flypsyde NotMrKnowItAll > 106408 O 1 5 104 12-26-2001 09:10 AM \>Why does outer space look like night time?\< lefty23 scr4 > 106153 O 1 24 678 12-26-2001 08:47 AM \>Why do dogs hump people’s legs?\< Tsubaki Jeff Olsen > 106089 O 1 8 247 12-26-2001 08:37 AM \>Learning to fall safely\< AskNott Diver > 106451 O 1 0 5 12-26-2001 08:36 AM \>Peronists in modern Argentina\< Ringo Ringo > 106421 O 1 3 109 12-26-2001 08:33 AM \>Amazing LED flashlights\< astro Attrayant Any use for that? ---
**Author:** ![tasmisr](https://avatars.discourse-cdn.com/v4/letter/t/d07c76/32.png) [@tasmisr](https://boards.straightdope.com/u/tasmisr)\ **Post date:** [June 2, 2002, 8:26pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/36 "2002-06-02T20:26:19Z")
thanks Musicat, sorry for email if this is annoying 🙂 femtosecond, I think they have all the threads cashed in thier version, but it displays only if the board is NOT available on the original site… ---
**Author:** ![spurdon](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@spurdon](https://boards.straightdope.com/u/spurdon)\ **Post date:** [June 2, 2002, 9:46pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/37 "2002-06-02T21:46:40Z")
Hello All, I’m one of the founders of [Boardreader.com](http://Boardreader.com). Nice long thread here. Great community action on a lot of differnet subjects. Keep it up! ‘Musicat’ from [doorbell.net](http://doorbell.net) contacted me today to submit his site and to tell me about this thread and offered me to post. There is a lot of questions I can see here. Let me try and answer some. [Boardreader.com](http://Boardreader.com) is a search engine designed and developed to index and keep track of dynamic web pages (ever changing forums, comments, etc.). . Tasmisr is correct with his posted comment: “I think they have all the threads cashed in thier version, but it displays only if the board is NOT available on the original site…” We ALWAYS try our best to connect to sites but the problem with crawling message boards is that the forum file path or the cgi path always are changing. New forums come up and old forums drop off creating a very dynamic page to index and connect to. We are always trying new ways to keep on top of these boards. Sometimes you get the archive because the time to connect is to long (connection time out) or sometimes the forum has been removed or whatever. We are going to be launching a new layout providing better displays of threads. Keep a look out. Comments are always welcome. I hope this helps and thank you to those of you who made many nice comments. Best, Scott ---
**Author:** ![Polycarp](https://avatars.discourse-cdn.com/v4/letter/p/82dd89/32.png) [@Polycarp](https://boards.straightdope.com/u/Polycarp)\ **Post date:** [June 3, 2002, 2:59am UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/38 "2002-06-03T02:59:04Z")
I would have this question for **jdavis** and **spurdon**. Jerry, would it be a major problem to add into the present database structure some or all of the missing material from the period between the last complete backup and when the board vandalism/crash when the intervening material was lost, if it were provided in a compatible format? Scott, would it be possible to provide that material in a format compatible with VBB 2.x? **Admin.folks** , if there is a charge for Scott supplying the material, would the Reader consider using member contributions (without any strings attached) to subsidize that recovery? ---
**Author:** ![spurdon](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@spurdon](https://boards.straightdope.com/u/spurdon)\ **Post date:** [June 3, 2002, 3:07am UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/39 "2002-06-03T03:07:37Z")
Polycarp, I think we could provide this data if we knew what exactly you guys wanted. I’m thinking we could just give you a xml feed for you to pick up on a ftp. I’m not sure if it would fit right in. No cost of course. Let me know what I can do… scott ---
**Author:** ![Nenya\_Elizabeth](https://avatars.discourse-cdn.com/v4/letter/n/ea5d25/32.png) [@Nenya\_Elizabeth](https://boards.straightdope.com/u/Nenya_Elizabeth)\ **Post date:** [June 3, 2002, 4:10am UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/40 "2002-06-03T04:10:44Z")
Wow, stuff from the Great Whiteout? Cool beans! 😃 🙂 –Nenya ---
**Author:** ![femtosecond](https://avatars.discourse-cdn.com/v4/letter/f/f08c70/32.png) [@femtosecond](https://boards.straightdope.com/u/femtosecond)\ **Post date:** [June 3, 2002, 12:44pm UTC](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162/41 "2002-06-03T12:44:58Z")
> [@](#): > > femtosecond, I think they have all the threads cashed in thier version, but it displays only if the board is NOT available on the original site… The other archiving site does something completely different but very similar to that by redirecting to the original files if they aren’t available _at their_ site. They know best if they are. But for someone to know whether a page is available at the _original_ site, he would have to acces that page there each time it is requested, to make sure. **spurdon** , you say every time someone clicks on one of your search results, BoardReader _does_ look up at the original board if the page is available? Well, obviously it does, because I just did a query in the half hour when our board is shut down for maintenance, and BoardReader redirected to its archive. Afterwards it redirected to the original site again. That’s convenient, because it reliably prevents people from getting displayed “page not available” errors, but maybe consider adding to your search results two rows of little buttons directly leading to your archive and the original pages respectively. Available or not, just to give the link for completeness’ sake, if people want (or need, see below) to follow them directly. What’s your timeout on deciding the page isn’t available on the original board? Because this one sometimes (or, sadly, more than sometimes) needs a few minutes to eventually spit out a thread. 🙂 And, like **erislover** ’s problem shows, some threads _seem_ available at our board, but have posts missing. Make sure your software doesn’t overwrite its complete copy with our incomplete one, thinking it’s more recent. :eek: Also, some threads may appear to have different content, because some threadIDs that completely went missing have been re-assigned by now. [Previous page](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162.md?page=1) [Next page](https://boards.straightdope.com/t/sdmb-archived-on-another-site/103162.md?page=3)