I tried searching all sorts of ways for the earliest posts I remember making, way back in 1999. I can’t find anything I posted earlier than 2000. I know there’s a limit to how many results a search can return, so I tried limiting the search to only one forum at a time, and older than 1 year. I got screens with as few as 2 hits, but cannot find any of my very earliest posts.
In some cases those oldest threads may have been deleted or otherwise lost. But whenever I want to see my oldest posts I will fix the search criteria to go by “Last Posting Date” but “in Ascending Order.” This gets me as close as I can to the start of my posting career. You may have to use a few old Dopers’ names or some specific keywords to make it go all the way back before your own posting began, but at least the Ascending order thing bypasses the limits that going from “now” backwards imposes.
Those earliest posts have been offloaded. We expect to return them to view in some sort of archival form in the future.
With over 6 million posts in the system we are bumping up against the limits of what the server will hold. If we continued to retain those older posts you wouldn’t be able to make new ones. Something’s got to give.
How is 6 million posts a problem? Even at 1K per post that is ‘only’ 6 gigabytes. Is it search speed constraints that are the problem or something else? Maybe there is enough expertise within the membership to help fix these problems. What version of MySQL are you using?
That’s a pretty good estimate, actually. My vBulletin board is approaching 1.2 million posts and the database takes up a little over 1GB of space at the moment.
As to why 6GB is a problem, you have to remember what hard drive sizes were like back when this server was purchased.
Hate to contradict an Administrator, but I see no signs that any vintage threads have been offloaded. My March '99 first-ever post (not counting AOL of course) is still right here where I put it. (I’m the “Guest” who used to use “Designated Optional signature at Bottom of Post” as a sig line)
The only threads viewable from 1999 are those posted in ATMB. Oh, and perhaps CoCC. All the rest are archived and unavailable at this time, as Jenny said.
So how far in the future? How searchable will this “some sort of archival form” be? Because for Pete’s sake, you are using the slow server schtick to take away even the simplest things that were undoubtably promised to the membership:
(my bold)
As Bippy said, this is not a huge amount of info in the world of databases. I find it hard to believe offload or die are the only choices available.
But main questions: how long gone? how searchable when back?
I’m a little curious because a quick look at bigboard’s statsgraph makes me think whatever you’ve taken offline since January will be replaced within a month or so. Are we in for more pruning?
Could you stick them on some kind of read-only drive or something? What do I know.
How many posts were ‘offloaded’ and when can we expect to see them back ? Are you going to continue culling posts ?
I know you’ve been saying that this might happen for a while but was there an official announcement ? I guess those threads weren’t visited very much but still it seems a bit of a shame.
I guess the hold up is some sort of search mechanism for them, if you don’t want google to be able to search them. Just say to hell with it and archive everything older than 3 years to a static, read-only, indexed by google site. That way you’ll still provide some value-add for members (they can search three years worth of threads) and have the bonus of bringing newbies from google hits. That’s a relatively easy (and server friendly) solution.
I never understood that. This messageboard is in no way private. Anyone can come by and read to their heart’s content. So they can’t search? So what? If they’re determined to find something they’ll just keep reading until they find it. Plus, you can sort any forum by thread starter and peruse someone’s threads all you like. I found my list of MPSIMS threads within 2 minutes.
Letting Google index the site would provide a reliable search engine, bring new people to the boards, and take a load off the server. The only downside is the fact that searching the “massive” database here is the only thing they really have to offer potential subscribers.