Well That’s One Way To Solve The Problem

This morning I discovered, thanks to a few IM pages, that my TTZ Media server was down. That’s not a good thing at all. The TTZ Media server is one of the most important servers on the rack and I can’t afford to have it down at all.

Normally when a server goes down, it means some of the services, mostly Apache, have overloaded and a remote reboot of the server is all that is needed to fix it. However, this time I couldn’t access the server at all - it couldn’t be pinged. That meant the server has lost power somehow. The only way to fix that is to physically go to the data center and power it back on. Or you can you ask the techs at the data center to do it, which is what I did.

However, the tech couldn’t find the media server from among the 10 servers in the cage. I don’t see how this is possible as I’m sure I had labeled each server. So I message the tech telling him at which slot the TTZ Media server was at. While trying to find the slot the tech accidentally (his claim) hit the off switch on the power bar that powers all the servers. So instead of having one server down, we now have ten down.

The tech, realizing what he has done (again, his claim), immediately turn the power bar back on. All the servers were restarted within a few minutes, including the TTZ Media server. It’s not the most elegant solution to the problem, but it worked.