Can't we get it straight?

[Marley23]

Copying Jerry’s announcement:

@jdavis

The message boards are going to be taken down tonight (Friday, June 14th) starting at 1 AM CDT. The master database is going to be taken out of service to be thoroughly checked and possibly repaired. While that is occurring the message board will be unavailable. Expected downtime is 4 to 6 hours so the message board will not be available again until some time on Friday morning.

This is the first step in a series of steps we are taking to address the current functionality issues. While tonight the master database is being worked on there are multiple replicated slave databases used by the system as well. They will also need to be checked and possibly repaired. Those checks can occur while the message board remains online and will occur from Friday night throughout the weekend. While any of the slave databases are offline there will be some degradation in performance.

In tandem to this work on the underlying vBulletin databases we’ll be rebuilding the various internal vBulletin caches and indices. Due to the size of the database this process can take quite a while and I do not have an estimate yet on when that work will be completed. It will be in the range of days though not hours.

At this point we have not determined the root cause of the functionality problems so it’s unknown if tonight’s upcoming database work will lead to some resolutions of the issues. It is a first step in a process that will eventually lead to a resolution.

Finally, waiting in the wings is a new front end web server to replace what is currently being used. Right now the front end server is on 3 year old hardware. The new server is a current operating system build and new software stack that should provide performance benefits to what we see today. Testing is ongoing. It will be installed sometime in the next few weeks. As for the back end database we believe it has more than enough horsepower. The SDMB database shares resources on a 64 core 5 server database cluster running with SSD storage technology. The SDMB database usage averages about 4 cores of utilization under normal load and can peak up to 11 cores so there is of headroom on the back end.