I think everyone in our business noticed this news article last week -” Keys to the City – San Francisco Cisco WAN. ” It’s about the fired rogue IT employee that held the City of San Francisco’s network hostage because he was the only one with the root passwords to the wide area network. He had set up the city’s network with a protocol in their Cisco equipment that erased the configurations if anybody tried to access the equipment without the right password. In this case, if the network configuration was erased, the computer services for the entire city would have been down for a minimum of weeks, if not months while the entire rest of the IT staff reconfigured the whole network. Although there was a happy ending to the story, basically the employer (the city) was screwed!
Does this seem like an anomaly? Think again. I see this situation a lot here in Central Florida. It’s usually not as blatant nor on such a massive scale, but let me give you an example.
I met with an IT manager last month who proudly took us for a tour of his facility. His company had about 100 employees and the network was built and maintained by him and one other full-time staff member. I won’t name the specific brand names of the equipment here. The new IP telephone system was an Asterik-type system that loaded onto an appliance server and could use generic SIP endpoint phone sets from several manufacturers. He pointed out the low-cost managed Layer 2 POE switches as well as an excellent firewall that was self-configured on a generic server appliance. He was using a Linux-based email server because he read about “problems” with Microsoft Exchange 2007. The application servers were for the most part a combination of Linux and Microsoft Servers on generic hardware.
Among the IT manager’s upcoming projects was the implementation of a document management system. He indicated that he had recently been invited to a vendor-sponsored presentation for a well-known commercial document management system but was put-off by the pricing associated with the end-to-end solution and was looking for other solutions that he could build into his network.
As we toured the facility he commented that when he was hired, the company had no network to speak of. He built the entire network from scratch, using all of the best and cost-effective products he could find and deploy. In other words, the network was a conglomeration of point solutions that very few people would ever have the capacity to support or trouble shoot. Did I mention that the documentation for this network was at least a year old because the IT staff was busy servicing the day to day needs of the company and didn’t have time to formally document any changes?
One of my Account Managers had brought me in to meet this IT manager because he was apparently looking for some backup assistance and support for this network. I could tell that he was in dire, if unspoken need of a technical resource that could back him up for emergency support as well as when he wanted to take a two week vacation. When asked how many people I had on staff that could support the technology he had deployed, the bottom line was that we couldn’t touch this company’s support requirements for any amount of money. The guy had built a great network, don’t get me wrong. But the company in general, has a big problem at hand when the IT manager decides to move on or gets hit by the proverbial bus.
I see this “IT Hero” syndrome every month as I am called on to consult with businesses. He is the lone wolf that ends up being responsible for the company network. They don’t want to give up control over “their” systems. These employees pose a real risk to their company if they leave the job or are unable to work – or if they just want to strike back for some reason. Nobody else has the knowledge to work around them.
The solution at an enterprise-sized company with a big staff is to cross train the IT department – make it so no one person controls and solely understands the network. If you spread responsibility for your systems over several employees and adopt change management processes and policies, the harm any one person can inflict is limited.
Building an IT team without relying on an IT Hero requires business discipline and a substantial budget. But in the SMB world, businesses usually don’t have the luxury of a big IT staff. This is an issue that we can help with at i-Tech Support. Our Total Technology Support program provides SMB businesses with the benefit of a team of IT staff without the cost of a large in-house department. For many of our clients, we act as an extension of their in-house network services department.
You’ll know you are on the right track when you can send your lead IT employee on vacation without asking for their cell phone number and hotel in case something “comes up”.