on training your system administrators

John Strand has a great, quick blog post over at pauldotcom.com on tapping and training your system administrators as a security asset.

There are actually quite a few similarities and strengths that system (and network) administrators have that parallel or complement security professionals. Disclaimer: I *am* biased since I am a sys/network administrator as my major function in my day job. (And yes, looking to get deeper into security in a more full-time role.) Here are a few of my notes I’d throw down.

1. I’ve said it many times before, but I believe if you truly want the real story about security in an organization, you get as low down into the trenches as you can get. This usually means your IT staff. And often means your desktop support staff, admins, and even app coders; but most probably admins. (I knock the desktop guys [I used to and still pretend to be one] only because they tend to be evaluated by their customer service. They don’t get rewarded for properly configuring a host-based firewall, but they do get rewarded for disabling it and allowing Joe Blow to get back to work.) Incidentally, if you want to know the health of a company, I’d also say your admins are a good source; they touch and know a lot!

2. Administrators are used to being at the end of the shaft, even more so than desktop specialists or coders. More often than not, the admins are the glue keeping all the disparate parts working together and covering for other IT sections that aren’t really up to snuff. They also tend to absorb things like poor applications or bad business process decisions that impact IT. They likely, along with desktop dudes, to see firsthand the people who violate policy. Bottomlines usually end with the administrators, and they’re used to getting it from management and from other IT persons. If anyone in the company is used to putting on the brakes and talking policy and standards and getting people to line up with them, it is your admins.

3. If you want to roll out a security initiative or project of any type, more than likely you’ll need an admin to give you access, gather logs, set up servers, configure the network for your visibility, provide documentation and diagrams, or be next to you during an incident response. Basically, get to know them and get on their side (and they on yours). They’re also the one who will make or break your policies, even more so than desktop dudes, especially if they need to understand and follow the steps on, say, how to harden a server. (Many desktop workers aspire to be systems workers, so their is also a tendency to look to the admins for leadership, formally or informally; compare paygrades if you doubt this.)

4. They also want to make sure things are done right. Admins are usually time-sensitive and risk-averse, and they don’t want things to break, they don’t want to be blamed when things break that they didn’t cause, and they want to troubleshoot intelligently. Even the worst admins tend to have the beginnings of these habits. It’s just a result of a ball rolling downhill.

5. That “A” in CIA is a shared role with the admins. It is also their duty to monitor and maintain availability for the masses.

6. Everyone learns about least privilege and separate of duties, but out here in the real world, business IT is run by admins with godlike access. That’s just how it is. If you think otherwise, you’re not thinking of all the ways they can end up pwning you. This means they really absolutely need to have a mind for security if you expect to get anywhere. If the admins don’t do it, then you’re stuck with a top-down approach which doesn’t always work. Even if bottom-up approaches get mired in budget constraints and buy-in, you can still do a lot to combat insecurity by having security-minded admins. Think about code. To secure code, you need to bake it into the creation with skilled coders and low-level policy. Same thing with systems and admins.

7. It is my really quick, knee-jerk opinion that every real IT security pro needs some practical, hands-on, systems or network or desktop administrator experience. This helps immensely on various levels. This isn’t always true, which is why I always keep this opinion short!