Topic: Failure = Good

[TechPro]

This morning, when I arrived at the University where I work, it was discovered that one of the buildings on the campus appeared to be missing (well, "missing" from a network communication perspective) because the network couldn't communicate with any of the devices in that building.  This meant devices in that building that I'm responsible for couldn't function.  Turns out that during the night, a utility power breaker had tripped, the line automatically switched to generator power and the generator ran as it was supposed to ... until the fuel was exhausted.  Then the UPS devices kept things running until they were drained ... and at 5:00 AM today that building lost all network communication.  Naturally, once it was understood what happened, it didn't take long to get things back up and running, and my equipment resumed normal functions.

The result?  People responsible for providing power and people responsible for the network equipment are now conferencing to determine why the problem happened (and what to do about it), and how problem notification communications failed, resulting in a service outage.

None of us like it when something doesn't work, especially if it's something you rely on.  We also don't care for it very much when we're trying to do or create something and as far as you know it should work but isn't turning out the way you expected or not at all.  It can be very frustrating.

There was an outage.  Should I be upset with them because there was an outage?  No, certainly not.  The outage was not expected and probably couldn't be predicted.  Now if they didn't take any steps to make sure an issue doesn't happen again, then I'd be justified in being upset, or at the least quite disappointed.

Have you had an instance where your Internet provided failed (even briefly) to provide you with working service?  Were you unhappy with them?  Did you voice (mildly or with great "articulation") how you felt about it?  Is it possible you were not justified in your feelings about it?

Next time you experience "downtime" due to matters not in your control, keep in mind that the failure may be a good thing (especially if your provider wants to stay in business), and because there was a failure, you're likely to get better and more reliable service in the future.

Disclaimer:  Failure happens.  Not all providers are equal, not all try enough to keep it from happening again. Some providers are chronic for not making things better.  We don't need to hash about who they are.

[Kaiaatzl]

Not all failures are equal either.  Some kinds of failures are unnacceptable even if they result in major improvements.

[Foil]

The internet in our building at work goes out multiple times per day (the ISP's fault).  I haven't said anything, myself, but I know others have voiced their displeasure.

I'm hoping this eventually drives a change in provider. 

[-<WillyP>-]

Sounds like the real failure was that switching over to generator power didn't sound the alarms, or the alarms weren't ringing in the right places. Excuse me my phone is ringing.

False alarm, phone spam.

Anyway, I am still a little confused by the title, do you mean the failure was good, probably not, I think you mean the failure will result in something good, most likely a better failure handling system in the future.

Mostly my tech failures are of my own doing. For example since I let Prepare4descent.net expire, my email at that domain no longer works and I have used that at dozens of places that can no longer notify me of replies many of which I don't care... But one was one of my own sites. It never took of and I sorta let it slide but the other day I noticed it had thousands of spam comments, which I didn't know because it was sending out notices to a defunct e-mail addy.... oops...

I have Comcast for my ISP. Everyone loves to bash Comcast, but seriously, I personally can only complain about the high rates they charge. I have never lost service when I didn't also lose power. Whenever I think damn slow today, I look over and see my wife watching a movie streaming from Netflicks, my son watching a video on You-Tube or playing Minecraft, and my other son playing online on the PS3. Oh, well. I guess it's not that slow after all.

[CrazyEnzo03]

No, no they don't give better service afterward.

[VANGUARD]

I don't know how fitting this story goes, but I'll say it anyway.

I still remember the day, not too long ago year or more now, that when I arrived at home with my mom and sister, our internet went down. No one could connect.
Next day, I tried to get on, no luck. I later discover that my mom and sister were online. hm... 
I tried again, nothing.
That day or another, with my smartphone, I went into a forum (one I went to fairly frequently) asking for help.
Now, some may say, I know computers. When it comes to routers and modens, wi-fi and Ethernet cables, um, not so much. I know wifi is wireless, and ethernet is a cable that looks a bit like a phone cord, just a wider end and thick cable. I know SSID and password, but really, I am not a tech when it comes to fixing those issues.

On my smartphone, I tried my best to explain what happened and this guy tried to help. someone I kind of knew, and overall a good person.
He wanted me to copy+paste such information. I said, I can't connect.
I decided the only way to get help is go into my moms room and connect via Ethernet. So I started in with the copy+Paste, etc, and as a result after page after page in the forum, being lopped around with trying to solve this, he declared publicly in that topic that he won't help me anymore and basically held true to that.

Frustrated, I took this to Geek Squad. Free of charge, under warranty. Upsetting part, two weeks later, get it back, 0 battery, and them saying, "they couldn't do anything because they need windows. you only have linux."

This information was on the sheet before I gave it to the guy. He had no issues seeing "Linux OS" on it.

Then I thought I'd take matters into my own hands and unscrew the bottom and replace the wi-fi. Screw stripped, badly.

So there I sat, with a laptop with no wi-fi, no chance of getting it fixed. I loved it a lot, but mistake or not, I just gave it to someone else.


edit: failure=good, oh yeah. so where is the good? I suppose it's helping that girl out that I gave it to. We stopped talked since January, but I heard it came in handy when they did some DJ stuff. So, that's good. Sorry if this didn't quite fit.

[-<WillyP>-]

WiFi can be a royal pain to get connected. Mainly because I don't do it often enough and I forget what I finally did that worked, I have to go through it all every time something happens. Recently I had to remove viruses from my wife's lappy and I got lucky with the WiFi but then I had to get it to talk to the printer which is connected to my desktop. It's like pulling teeth. Then my kid's machine got corrupted and wouldn't boot. The Windows disk wouldn't fix it, and I could not format it. So I loaded Ubuntu, but couldn't get some of their games working. grrr... Then the windows disk wouldn't boot, so I took the drive out and connected it to mine and formatted it. Then I had to go through loading Windows, getting them Firefox and Thunderbird, anti virus, updating drivers, etc... And then getting it to talk to the printer. I need to start taking notes I guess. Or get a job repairing computers then I'd be doing it often enough to remember it.

[Matthew]

I would disagree that this is a good failure. There should've been alerts for such events set up, and that it took a total failure of power to the entire building for somebody to notice is... not good. It's good that they are actually conferencing to determine what went wrong, but it seems like somebody should've noticed before you. Lack of network connectivity is a rather indirect way of discovering a power outage. Surely something, somewhere, should've notified somebody. Either the utility company should've been alerted that their breaker had tripped, or the generator backup should've triggered an alert to campus staff, or as a last line the UPS system or one of the servers connected to it should've alerted the campus staff.

My provider (and I think most providers for non-business clients) don't generally seem to improve their record after a failure unless it gets to the point where you're bugging them constantly (like when the interconnect outside our house was faulty and kept going out every time the temperature changed too rapidly).

[Scyphi]

Actually, speaking as a student from said university, how soon it'd be noticed would depend on the building, and the time of day.

But that said, you're right, by the time Techpro found out about it, somebody else should've known about it already.

Thus I feel compelled to point out that Techpro did not say that he was the FIRST to find out, just that's when HE found out about it. More likely they just hadn't figured out what the problem was by the time Techpro found out, which actually isn't that uncommon, and for something like this, they were probably thinking the problem was more serious than it actually was. Or something. Granted, I'm speculating without knowing the whole story, but that's generally how it works on campus.

I should point out that one cannot predict every possible outcome until they happen sometimes. This is one of those cases.

[TechPro]

To clarify, there were/are persons who are tasked with monitoring the status of the systems but they failed to notice the issue.  Thus the generator and UPS units did their best to keep things running... But those persons continued to fail to notice.  The problem was automatically detected and automatically reported to the consoles and displays they monitor.  What they were doing instead of their job, became a 'hot' topic.

[Matthew]

To clarify, there were/are persons who are tasked with monitoring the status of the systems but they failed to notice the issue.  Thus the generator and UPS units did their best to keep things running... But those persons continued to fail to notice.  The problem was automatically detected and automatically reported to the consoles and displays they monitor.  What they were doing instead of their job, became a 'hot' topic.

*Ding ding ding* And we have a winna!