DudleyGarrett
09-17-2007, 11:50 AM
So I'm on this "urgent rotation." There are three "urgent teams" so two out of every six weeks, I have alarms sent to my BlackBerry (PDA/phone) regarding any downed servers, networks, or systems in the data center. Mind you, there are no procedures or policies that establish what the "urgent rotation" does. We're to respond -- restart services, reboot servers, troublshoot networking equipment, etc. There are four people on my "urgent team." Whoever gets to the issue first is the one to handle it (lack of policy and procedure here). Keep in mind, there is no extra compensation for this duty and not everyone on the team has to do it.
I'm a network guy, so if anything happens with the network, regardless of my "urgent rotation" status, I have to fix it. My everyday workload has nothing to do with servers and frankly, I'm a little lost on how these crappy homegrown applications function. My head and hands are full with our complex networking environment. Since I'm the only network guy on my "urgent team," I by default get any issue that even remotely looks like it could be network related. That's fine. It annoys me but I don't complain.
Beginning Friday afternoon I began getting pages every couple of hours. There was a problem with the MONITORING system, which is under the direct charge of another team member, who is also on my "urgent team." It's his baby. He set everything up and declared that we should not use anything else. And it was sending about five false alarms every two hours. I sent five e-mails since Friday night asking if something could be done. No response from anyone. The monitoring system was not fixed until this morning.
My wife said, "I can't take another night of that thing beeping every 2 hours. Either sleep downstairs or turn it off." Done. It's off. Naturally, all night last night, there was a system that was really down. It's another one of our crappy homegrown applications that needs an IISRESET every three or four days. My BlackBerry was in silent mode. The other three people didn't respond, either. The boss asks if we are receiving alerts. I responded that I was, but I put my BlackBerry in phone-only mode due to the monitoring system issues and subsequent e-mails failling on deaf ears, so I didn't get audible alerts. His response? "That's not an option." Mine to him: "Expecting people to get beeped over 250 times during a single weekend, causing spousal distress and generally household disharmony, when the problem is correctable by another team member is also not an option -- especially when e-mails to that team member are ignored. It's your choice. Either we get genuine alerts or we turn off the beeping." No response. He's all pissy.
Fuck him. This "urgent rotation" has been a complete mess and source of confusion since he decided to establish it a couple years ago. Out of the 11 or 12 people who are on the rotation, there are about 6 that actually have their PDAs configured to audibly alert. He ignores suggestions to improve it. I'm done with it. I'm on it until next Monday. Guess what's going to stay in Phone Only mode?
I'm a network guy, so if anything happens with the network, regardless of my "urgent rotation" status, I have to fix it. My everyday workload has nothing to do with servers and frankly, I'm a little lost on how these crappy homegrown applications function. My head and hands are full with our complex networking environment. Since I'm the only network guy on my "urgent team," I by default get any issue that even remotely looks like it could be network related. That's fine. It annoys me but I don't complain.
Beginning Friday afternoon I began getting pages every couple of hours. There was a problem with the MONITORING system, which is under the direct charge of another team member, who is also on my "urgent team." It's his baby. He set everything up and declared that we should not use anything else. And it was sending about five false alarms every two hours. I sent five e-mails since Friday night asking if something could be done. No response from anyone. The monitoring system was not fixed until this morning.
My wife said, "I can't take another night of that thing beeping every 2 hours. Either sleep downstairs or turn it off." Done. It's off. Naturally, all night last night, there was a system that was really down. It's another one of our crappy homegrown applications that needs an IISRESET every three or four days. My BlackBerry was in silent mode. The other three people didn't respond, either. The boss asks if we are receiving alerts. I responded that I was, but I put my BlackBerry in phone-only mode due to the monitoring system issues and subsequent e-mails failling on deaf ears, so I didn't get audible alerts. His response? "That's not an option." Mine to him: "Expecting people to get beeped over 250 times during a single weekend, causing spousal distress and generally household disharmony, when the problem is correctable by another team member is also not an option -- especially when e-mails to that team member are ignored. It's your choice. Either we get genuine alerts or we turn off the beeping." No response. He's all pissy.
Fuck him. This "urgent rotation" has been a complete mess and source of confusion since he decided to establish it a couple years ago. Out of the 11 or 12 people who are on the rotation, there are about 6 that actually have their PDAs configured to audibly alert. He ignores suggestions to improve it. I'm done with it. I'm on it until next Monday. Guess what's going to stay in Phone Only mode?