Another idea for generating CASH to use in the for the maintenance of the SDMB (yawn)

[Janx]

This thread here was opened in IMHO, it goes into the age old question of what can be done about the reliability of the boards, speed etc.
**Coldfire ** wrote…

Bottom line: there’s just way too much traffic. Whenever we upgrade the hardware (granted, we don’t do that often), it speeds up. For about 2 weeks, and by then, demand has caught up with supply again.

The thread goes on to reference this thread here
where ** TubaDiva ** wrote…

As for the Reader’s attitude, you have to look at it in the bigger picture. We are not their main chance here and never have been; the tail does not wag the dog. They have stuck to their core business over the years, that is, publishing.

-AND

Because this is the property of a corporate owner, we have not had the flexibility to let you PBS us, nor have we been able to pass the hat by the usual and accepted online ways of allowing volunteer payment programs. The accounting and tax headaches would make us more trouble than we are worth to the Reader and I hope you can see that.

I can see this. Donations are a pain in the ass. The Reader is a publisher, not really into the whole cyber business deal. So here is my 2 cents worth that would give money to the company, hopefully to reinvest into the boards. It smacks of passing the hat, but it is within the realm of publishing, with all the same politics, legal hassles and use of distribution channels that The Reader is (presumably) accustomed to.

The SDMB is (IMHO) a freaking gigantic storehouse of knowledge, information, and fun. This constitutes a massive on line reference library with lots of “in print” references on almost any subject, as well as linked internet references (that will go sour over time), packed with hard core usable reference information, LOTS of very informed opinions and LOTS of outright lunacy that make it a perfect snap shot of the era in which we live.

One day it will be gone. And with it, everything I mentioned will be gone as well. My grand children will never know it. To bad.

Why not sell it all on CD? Is this plausible? Obviously you couldn’t sell the package formatted as it sits in the vBulletin format, but I bet it wouldn’t take much to copy all the existing HTML and package it with some sort of emulator/reader that has a generic HTML search engine. Then announce it on the boards.
Let it be known that the proceeds go right back into upgrading/maintaining the system. Then sell yearly updates.

The reader then would not be to far away from it’s publishing mission, and could realize a way for the boards to be self supporting, in a commercial manner, that would not put undue strain on the accounting department. It would give me a way to preserve all this knowledge no matter what happens to the boards or The Reader and make a “donation” of sorts, doing my part to preserve the area that has come to be my biggest anchor into American Society (this is important to me because while I am an American I live in South America). I have thought of doing just this. Just get web spider, set it to the dope and let her rip. But first I have NO IDEA how much disk space is required, and I’m sure this would constitute some violation of the user agreement. Also, who would I pay for the resources I was abusing. So I’m asking you to sell me a back up of the SDMB and yearly updates. Wadda say?

Any thoughts on this?

[JeffB]

A couple of thoughts:

[ul][li]The contents of the SDMB are stored in a database (MySQL, if I remember correctly), not in HTML format. That might lead to database licensing issues.[/li][li] vBulletin is the front-end for accessing that database. That might lead to software licensing issues.[/li][li]I don’t know what the current size of the database is, but I’m sure where talking several DVDs or lots of CDs to hold the entire thing.[/ul][/li]
It doesn’t seem like a practical idea.

[Shalmanese]

Conversion from database to any common file format is a trivial task and neccesarily so. There wouldn’t be any licensing or technical issues. I think a DVD would be about the right size to hold the boards as is. Other fairly big boards top out at around 5 - 6 gig and I doubt this one would be that large. However, I think the main problem is just the setting up of an online store and properly accounting for revenue. Believe me, any idea you have thought of has already been thought of innumerable amount of times before.

[Una_Persson]

There are no licensing issues with MySQL, PHP, or vBulletin in the use put forward. The Reader has the right to access the data contained within the SDMB database in any way they see fit, and MySQL and PHP are open-source products anyhow. Writing queries to do custom dumping of the data to HTML is straightforward, but not trivial - there is a bit of busy work to do, if you want to present it in anything close to the same look and feel as current (and avoid “reverse engineering” legal issues with vBulletin).

I have thought of doing just this. Just get web spider, set it to the dope and let her rip. But first I have NO IDEA how much disk space is required, and I’m sure this would constitute some violation of the user agreement. Also, who would I pay for the resources I was abusing. So I’m asking you to sell me a back up of the SDMB and yearly updates. Wadda say?

I would say if you sic a web spider on the SDMB to try and parse through the 4,000,000+ posts, the Staff would probably let something else “rip”…

[Janx]

** JeffB ** wrote…

• The contents of the SDMB are stored in a database (MySQL, if I remember correctly), not in HTML format. That might lead to database licensing issues.

Not really, you would only need to write a program that pulls all the info out of its respective DB and re-formats it someplace else, perhaps in plain text or html, after all, isn’t that what Vbulletin is doing, and isn’t that what I am doing when I save a thread to my HD? I don’t see how this would be very difficult, the info is there, it only needs to be re-formatted in a more/less universal and copyright safe manner.

** Shalmanese ** wrote…

However, I think the main problem is just the setting up of an online store and properly accounting for revenue.

There is an on line store Cecil’s House O’ Values You can buy all if Cecil’s books, Tee shirts, coffee mugs. Adding one more product would be no big deal.

And while I’m sure all things have been thought of, I have never seen this Idea proposed. If it was thought of already – oh well. I would like to know why it wouldn’t fly, because I would be first in line.

[Janx]

** Una Persson ** wrote…

There are no licensing issues with MySQL, PHP, or vBulletin in the use put forward. The Reader has the right to access the data contained within the SDMB database in any way they see fit, and MySQL and PHP are open-source products anyhow.

Welp, there ya go, I know it would not be trivial – but the labor involved would be factored into the price as well as the costs involved, just like producing a book, or a shirt, or a mug for that matter.

** Una Persson ** also wrote…

I would say if you sic a web spider on the SDMB to try and parse through the 4,000,000+ posts, the Staff would probably let something else “rip”…

Exactly. This also pretends that I know what I’m talking about and that I live in a manner that would permit me to do such things. On the first account, not really. Copying the content of a web site to your HD for viewing off line Vs. Copying the entire Los Angeles public library for your personal archives are entirely different things, this much I can appreciate. On the second point, I have far to much respect for the board, the people involved and it’s name sake to EVER attempt a stunt like that. I would pay them for a copy of the content though.

[dantheman]

The biggest problem with putting all the data on a medium such as a CD or DVD - never mind the difficulties in just putting them there - is the database itself is always growing, and posts made today reference posts made in the past. This is a continuing process.

So let’s say you buy a DVD that (somehow) contained a certain portion of the entire database. What if you come across a link that goes nowhere? A lot of posts have been lost or intentionally deleted over the years.

[wolfstu]

dantheman, I think the idea is that you buy a ‘static’ product: You get the whole SDMB as of <Date>.

As far as posts referring to other posts, if you get a copy of the board as of 7 Nov 2003, any and all posts on the disk would refer to posts on the disk, and not to posts made later, since those posts don’t exist yet.

True, as soon as they make the disk, it’s out of date. But the point is to be a reference, not to replace the experience of posting to the board.

What if you come across a link that goes nowhere? A lot of posts have been lost or intentionally deleted over the years.

Right, but those links are just as dead if you’re using the boards. And such a disk could have mitigated the damage caused by the Winter of our Missed Content. The posts would still exist, and… could a copy of one of these disks have been used to restore the posts lost between Dec 7 2002 (the beginning of the post-loss window) and the time the CD was made?

If large numbers of people actually had one of these, it could save load on the server by reducing the number of searches people would need to do of the database in Chicago. They can use their own hamsters, right at home.

A similar product is available from Transport Canada: The Canadian Aviation Regulations on CD-ROM. Single price: C$25. One year’s subscription to the quarterly updates: C$90.

I’m not sure how big the CARs are compared to the SDMB, but on paper they fill 7 3-inch binders, and that’s just one language. The CD version has them in two.

In the case of the CARs, they mail you a CD, and then mail you a new one with the updates every 3 months. (In Canada, Air Regs amendments are written to come into effect at specified dates, basically every three months.)

In the case of the SDMB, they could issue updates every 3 months, or just once a year, or whatever. And members wishing to buy the product could either subscribe for a year (CARs subscribers save C$10 this way over the price or the four issues individually) or just buy every fourth quarterly edition, so that they still get updates, but their database is a year old by the time they get a new one.

The question of wheter it would be worth it for the Reader to set up the system required to do this is one I’m not qualified to answer. On the other hand, they could just do it once, and make some number of copies, and consider making a new one once those have sold out.

I would definitely be interested in such a product. Even if it was just a one-time thing, this would be Solid Gold as far as reading material for entertainment goes.

• Read it whenever you’re at a computer, whether there’s internet or not.
• Read it when the boards are down.
• 4,000,000 posts. Will you ever fail to find new and exciting things?
• It’s a veritable encyclopaedia of stuff, and contains answers to countless General Questions.
• Re-read old threads with ease.
• Search for stuff without guilt.
• Relive the joy of Bomb Bay Doors’ birth, the Horror of Blimps, the long soap opera of newbies and bannings and promotions to Mod, and much much more - all without taxing the Reader’s Server.
• Contribute profits to the Reader, in thanks for the Board, and hopefully to remind them of the value and profitability of The Straight Dope.

I’d be very interested in one. And if the Reader ever decided to shut the Board down, I’d consider it one of my finest investments. There are millions of very valuable posts here, and their eventual loss would be a tragedy.

I think I’d buy one just to put in a concrete box for the alien archeologists to find after we’re all gone.

[Janx]

** dantheman ** wrote…

The biggest problem with putting all the data on a medium such as a CD or DVD - never mind the difficulties in just putting them there

It’s my opinion and that of others that “putting all the data on a medium” is not such a big deal.

AND

…the database itself is always growing, and posts made today reference posts made in the past. This is a continuing process.

That’s the point, I would like a back up of this process. One day it will be gone. I want a copy of it before it goes. I would pay money for that. That money could help support the board.

AND

So let’s say you buy a DVD that (somehow) contained a certain portion of the entire database. What if you come across a link that goes nowhere? A lot of posts have been lost or intentionally deleted over the years.

I don’t get it. If you had a copy of the entire board from say Mar,1999 Mar 2004, why could it reference parts of the board that don’t exist? If a post was deleted, (Intentionally or not), a message of some sort says so, just as it does now. Now if you mean that you run the thing straight from CD (or something) and the referenced THREAD is not on that CD, all you need to do is look at the last 7 digits of the refrenced thread, that’s the post number. All you do is pull out the CD (or DVD or what ever) that has that post number on it. If the referenced item is a link outside the SDMB, you point your browser to it and cross your fingers, if the link is dead, oh well - you still have years worth of other very useful and entertaining content. Further, updates would be offered at regular intervals (6 months? A year?). It is ment to be an archive of the SDMB not a companion to the SDMB (is this what you think I am talking about?). Not all references on the boards are strictly hyperlinks, (please note that in the OP, I specifically mention that after a time hyperlinks would go “sour”), there are jillions of references to books and authors and subjects, very well informed debates, and lots of fun stuff like MPSIMS and amateur, first hand, social commentary (the pit). Not to mention GQ. I think this would be cool to have in about 100 years, if for no other reason than to have a view of what was going on in the brains of the people of the era, from slang, to clothing trends, to what we think of the current, as well as past wars. How to fix a broken gate, or replace defective car part etc. etc. Its all here. Currently, I pay for phone time and a dial up net connect, I would rather not if I don’t have to. So when I get into a debate about what the difference is between a vegetable and a fruit, weather men have actually walked on the moon or how is that a ducks quack doesn’t echo and no one knows why, I wont have to. I have a gigantic refrence library that (if it doesnt have the answer to whatever it can normally point the way) right here on my PC. I still think its a cracking idea, and I still can’t see why it is not viable.

[Janx]

Thank you ** wolfstu **. You said (with far more eloquence) *exactly * what I was trying to say – and more. That was a simul-post, had I seen what you were writing, I would have just said Mee TO!

[dantheman]

*Originally posted by wolfstu *
Right, but those links are just as dead if you’re using the boards. And such a disk could have mitigated the damage caused by the Winter of our Missed Content. The posts would still exist, and… could a copy of one of these disks have been used to restore the posts lost between Dec 7 2002 (the beginning of the post-loss window) and the time the CD was made?

On the boards, you can search for the correct thread. I’m not saying the thread or post is gone - I’m saying the link might no longer be valid when the various indexes are updated.

If you’re talking about also adding a search engine to the CD, now we’re talking about even more labor.

(And regarding the lost data - I would think they do some kind of backup already, so when the board is down for a short while - a few hours or so - it can be brought back up. Maybe the Winter outage was just too big to be accomodated by a backup. This is just a guess, though.)

If large numbers of people actually had one of these, it could save load on the server by reducing the number of searches people would need to do of the database in Chicago. They can use their own hamsters, right at home.

In theory, sure. In practice, of course, only those who bought the thing would do that. Since it’s all available online for free, why would they do this?

• Read it whenever you’re at a computer, whether there’s internet or not.
• Read it when the boards are down.
• 4,000,000 posts. Will you ever fail to find new and exciting things?
• It’s a veritable encyclopaedia of stuff, and contains answers to countless General Questions.
• Re-read old threads with ease.
• Search for stuff without guilt.
• Relive the joy of Bomb Bay Doors’ birth, the Horror of Blimps, the long soap opera of newbies and bannings and promotions to Mod, and much much more - all without taxing the Reader’s Server.
• Contribute profits to the Reader, in thanks for the Board, and hopefully to remind them of the value and profitability of The Straight Dope.

How much strain do you think is caused by people reading old stuff? It’s much more likely to be caused by people visiting current threads, whether by viewing new posts, by going forum to forum, or by doing vanity searches. Do you really think there are a lot of people searching the events you’ve mentioned?

And bear in mind that searching for a thread from, say, two years ago puts just as much strain on the server as searching for one from yesterday. The server doesn’t care how old it is.

I’d be very interested in one. And if the Reader ever decided to shut the Board down, I’d consider it one of my finest investments. There are millions of very valuable posts here, and their eventual loss would be a tragedy.

If the board was going to be shut down permanently, I would think it’d be a great idea. We don’t know that’ll be the case anytime soon.

[dantheman]

Originally posted by Janx *
** That’s the point, I would like a back up of this process. One day it will be gone. I want a copy of it before it goes. I would pay money for that. That money could help support the board.*

And I would bet the farm that if and when the CR decides to jettison this board, they’ll let everyone know well in advance. And at that point, it would make perfect sense to try to make the thing available on CD, if possible. Heck, do it then and you’d be guaranteed to have the entirety of the database, more or less. Do it now and you’ll have only up to now.

I don’t get it. If you had a copy of the entire board from say Mar,1999 Mar 2004, why could it reference parts of the board that don’t exist? If a post was deleted, (Intentionally or not), a message of some sort says so, just as it does now.**

Actually, there’s no message for the regular user when a post has been deleted - there’s just a gap in the post ID.

Why would it reference parts that don’t exist? Because sometimes people link to threads that are simply no longer there, threads that were swallowed up the last time the server went kablooey.

It’s also worth noting that since there are so many linked threads, clicking on them on a CD would do nothing. You couldn’t cut and paste them, of course, unless you were online at the time. Like I said in my last post, you’d have to incorporate a search engine onto the CD so people could find everything all right.

Now if you mean that you run the thing straight from CD (or something) and the referenced THREAD is not on that CD, all you need to do is look at the last 7 digits of the refrenced thread, that’s the post number. All you do is pull out the CD (or DVD or what ever) that has that post number on it. If the referenced item is a link outside the SDMB, you point your browser to it and cross your fingers, if the link is dead, oh well - you still have years worth of other very useful and entertaining content. Further, updates would be offered at regular intervals (6 months? A year?). It is ment to be an archive of the SDMB not a companion to the SDMB (is this what you think I am talking about?).

No. And I said before, it’s difficult to archive a database that is constantly changing. But I guess the point is lost on me. All you would be doing is moving all that data to a CD so people could search the database at their leisure and not tax the server. But as I said in my last post, I really don’t think that it’s people who are searching through the entire database that’s taxing the server; rather, it’s people searching posts from the past few days. In other to solve that, you’d have to update the CDs at least weekly.

Not all references on the boards are strictly hyperlinks, (please note that in the OP, I specifically mention that after a time hyperlinks would go “sour”), there are jillions of references to books and authors and subjects, very well informed debates, and lots of fun stuff like MPSIMS and amateur, first hand, social commentary (the pit). Not to mention GQ. I think this would be cool to have in about 100 years, if for no other reason than to have a view of what was going on in the brains of the people of the era, from slang, to clothing trends, to what we think of the current, as well as past wars. **

Well, here’s the thing. I know there are people who’d like to have the SDMB info at their fingertips. Do they want GQ? Sure. GD? Sure. CCC and CSR? Sure. CS? Maybe. The Pit? Maybe.

MPSIMS, though? They might not mind it, but I bet they wouldn’t want to pay for it. So then you’d have to either do the whole database (in which case people will pointedly not buy the thing if so much “fluff” is on it) or parse particular fora, which would be a real pain in the ass. (And it’s worth mentioning that MPSIMS is by far the most-populated forum.)

Also - and shame on me for not mentioning it - there’s always Boardreader.

All you have to do to search the SDMB archives is go to www.boardreader.com and type a search string +SDMB. It’ll grab the threads for you without pushing the CR server to the limit. (For example, I could search “blimps +SDMB” and get Scylla’s thread.)

I think a CD would be kind of neat, but it seems it would be a very intensive process that might be more trouble than it’s worth.

However, should the CR decide to shut this down, I would hope they’d give the idea a try at that point. (If nothing else, they wouldn’t have to do updates.)

[Janx]

** dantheman ** wrote…

Since it’s all available online for free, why would they do this?

Because while the info is free, the phone line is not, at least in my case.

AND

Do you really think there are a lot of people searching the events you’ve mentioned?

I sure as hell am. I hunt down the most entertaining threads, or the ones on obscure subjects and print them out, then I read them during my commute. Far cheaper then any magazine printed in English and far more entertaining - this would be a real boon to have on my lap top during those 12 hour flights between São Paulo and Los Angeles as well.

AND

I’m saying the link might no longer be valid when the various indexes are updated.

How could this be. I should think whoever gets the contract would de bug the product before it is released. All the threads on the board are currently referenced by a post number (take a look at your address window), when you do the DB dump, just respect this system and all should be well.

[Janx]

** dantheman ** wrote…

Like I said in my last post, you’d have to incorporate a search engine onto the CD so people could find everything all right.

I think you might want to take a closer look at the OP.

**Janx ** wrote…

package it with some sort of emulator/reader that has a generic HTML search engine.

[dantheman]

I’ll need a tech to confirm or refute that. I know it’s been talked about in here before, though.

Janx, there are nearly 40,000 members here. Do you really think more of them search on old threads than search on the most recent ones? That’s just not very likely. For one thing, a lot of people click the “View New Posts” link to get their new posts.

As far as the phone line goes, are you suggesting that if you had these CDs you wouldn’t need access to the Internet? You’d give it up? Because then, of course, you’d be paying only for the CD and not the privilege of accessing the net.

By paying to access the Internet, you get plenty of bang for the buck - or at least the opportunity for such. I can visit hundreds and hundreds of sites (most of which will have the same info I’d get here) for one price. Your CD would have a finite amount of info - much less than the Internet - for several prices (the first CD and each update).

[dantheman]

*Originally posted by Janx *
I think you might want to take a closer look at the OP.

Eh? You said it, so I can’t? Wha?

Well, I’d need to hear from a tech on this, but I doubt it’s as simple as you may perceive it to be.

[Janx]

And I said before, it’s difficult to archive a database that is constantly changing. But I guess the point is lost on me. All you would be doing is moving all that data to a CD so people could search the database at their leisure and not tax the server.

No, I mean dump the DB to a simple format that can be built upon, not really added to, also I don’t care if the server is taxed or not, except that it makes the board a drag, the purpose of the project is to drum up cash so the boards can be upgraded in order to not be a drag, with out turning the SDMB into an on line PBS, or creating a accounting nightmare or a bureaucratic pain in the neck. (again, you may want to give the OP another look).

[Janx]

** danthe man** wrote…
** dantheman ** wrote…

Eh? You said it, so I can’t? Wha?

No, not at all. I was under the impression that you did not realize I had said that, in fact it would be essential. Umm…since we seem to be posting simultaneously here, please ignore the next reference to not reading the OP and take it for what it is worth. The purpose of the idea is to support the upgrade of the boards. Sorry about the attitude. I guess you did read it.

[Una_Persson]

A vBulletin board can be put into a different or HTML format without any major barriers, but there is a lot of busy-work involved. Especially if you want the look and feel to be anywhere like the current SDMB. To see some of what I mean by that, click on “view source” on this page, and see just how much formatting goes into a typical thread.

I really doubt that a “standard” HTML search utility is going to work that well on more than 1 GB of content. This is one of the many reasons the Board is in a database right now. If you think about it, a fulltext HTML search CGI would take forever on a multiple-CD program, so you’d end up using some sort of local database structure anyhow.

There are several technical barriers to actually doing it. Several people in the past have asserted that it is “easy” to organize it in the same look and feel and overall functionality offline, yet when I have challenged them to post actual fully working code, or to try a small sample conversion of a vBulletin database with only 10,000 or so posts, no one has stepped forward to do it. Not because it can’t be done, but because once you get into the details, there are a lot of t’s to dot and i’s to cross. Or the other way around.

[dantheman]

*Originally posted by Janx *
**No, I mean dump the DB to a simple format that can be built upon, not really added to, also I don’t care if the server is taxed or not, except that it makes the board a drag, the purpose of the project is to drum up cash so the boards can be upgraded in order to not be a drag, with out turning the SDMB into an on line PBS, or creating a accounting nightmare or a bureaucratic pain in the neck. (again, you may want to give the OP another look). **

You now say you want to dump the DB to a simple format that can be built on. Earlier in this thread you were talking about several CDs. I presume you now mean that every time an update was released, the subscriber would download it, appending it to their existing at-home database? Ok. Doesn’t sound very practical to me on a large scale, since those who have dial-up aren’t going to want to download a huge file for each update. And then there’s the point I’ve been making - why bother paying for it, even to have as an archive - if the entire DB exists online, for free? You’ve stated you don’t care about the server issue, so what would be the point?

No matter how simple you want it to be, it’s not going to be that simple. The CR would probably need to hire someone to do it; the only guy working on the SD behind the scenes is Jerry, and this isn’t the only thing he does for the CR. So you’d have to pay someone to help (even temporarily). Then you’d have to spend many man-hours trying to get the data packaged in an acceptable format (how much data to save? 1999 to present, all in one shot? 1999-2000?). Then you’d have to actually create all those CDs.

And for what? Would there really be enough people buying all those CDs to justify the cost? I’d be surprised if that were the case.

I wish some more tech people would chime in. I have no clue about this stuff, and I could be way off the mark. It just seems to me that this project would cost more to do than it would profit the CR.

(Whoops, I see Una posted. Well, I defer to her.)