hey all - so I come home from my BALUG meeting, only to find out that the server is completely down.
Obviously I curse several times, panic a bit... then look at the issue. This time the whole thing was down... (unlike earlier at 4pm where the main site was up but the forums was down)... I couldn't do crap from the console... I was getting disk I/O errors... and it wasn't even responding to the shutdown command.
I had to forcefully bring the server down, and when it tried to boot up... I got 'invalid boot disk... insert a bood disk to continue...' (or something like that). This is when I realized how bad it was
Luckily my brother helped me out... as he's a much better sys admin than I am.
Long story short, the spare server that I was planning on building with slackware (haven't been able to get the ISOs yet), already had RH9 shrike and all the packages we needed. It was as simple as pulling the hard drive from the bum server, making it a secondary drive and pulling whatever data we could from it.
I had recent backups for the forums, but it was even luckier that we were able to get mysql dumps off of the drive... which means the data should be as recent as when the whole thing went down (hopefully!).
Anyway... the site is now running on the spare server. I still plan on building out a slackware server to move the site to, but know at least we know that it probably wasn't a RH9 issue, rather than a disk issue.
I'm TERRIBLY SORRY for the issues we've been having recently. I hope that you all bear with me through this.