Via McKeay, I read a list of 10 things your auditor isn’t telling you, compiled by David Shackleford. Utter, terrible truths! So much so, that I had to yoink them and add comments.

If you read nothing else in this post, read my comments on #6. In fact, I’ll quote myself here: “This is where pen-tests can trump audits. A pen-test can say WRONG, but an audit is trying to say CORRECT, and it often can’t.”

1. I am actually just following a checklist.
A subjective checklist. An incomplete checklist. A checklist I can’t intelligently talk about because I don’t get it, nor can I really give you anything beyond obsurd vagueness if you ask me how to meet those checklist bullets! Oh, Dave covers some of those coming up! 🙂

2. I do not understand the technology I am auditing.
Also, too many varied ways of using varied technologies in various environments. Either you follow the checklist in #1, or you have to have a very large swath of knowledge. We’re just not close to being at the latter, yet. Kudos to any teams of auditors who have a nice cross-selection of skills that the lead can use to fill such gaps!

3. The well-dressed, experienced greyhairs came in and sold this deal, but I graduated from college 8 months ago and went through ( E&Y || IBM || Deloitte ) auditing bootcamp.
Possibly good if the guy is smurt, but honestly experience in a working environment does go a long way to “getting it,” both with technology and the how’s and why’s of business.

4. Most firms are really incentivized to help you pass.
In addition to Dave’s comments, I would say no one wants to lose business because your client only wanted a passing score. They *will* shop around to pass a weak audit rather than actually work up to passing any audit. Sad, but security will continue to be an economic function.

5. Show me a viable set of compensating controls, and I’m liable to pass you.
Just say no! Then again, combine #4 with #1 and you get #5. Don’t lose the business, but cover your ass so you’re not passing obviously wrong things. The one thing I dislike about this situation is if the controls are there, but just not really used except when the auditor is around, i.e. that AV/IPS management console full of alerts that no one ever looks at.

6. Auditing standards suck.
I’m not sure how this can get better, mostly because of what I said in #2 about varied technologies used in varied ways. *CAN* you have an easily understood Ubuntu Server build checklist? Doubtful, especially when you have no context as to what that Ubuntu Server should be doing. This is where pen-tests can trump audits. A pen-test can say WRONG, but an audit is trying to say CORRECT, and it often can’t. Yes, we can get better, but this is a Big Deal. And we all know the reaction when they see NIST docs for the first time. “Oh, just follows the recommends at NIST [and keep some Tums on hand.]”

7. Compliance regulations suck.

8. You can’t have it “your” way.
Combine this with #1, #2, and #3, and your auditor may WORSEN your security. But it is true, the audit’s real effectiveness is going to be rooted in the auditor and somewhat in the client technical staff (who may be able to pass off an auditor as being inexperienced). <--Of course, those staff that can do that probably need to be recruited into security/auditing!! 9. I know more than you.
Dave’s comments remind me why I think the trend on-going is to have in-house auditing/security. The biggest things stopping that will be a solid workforce and the Blame Game when a breach does occur. You can’t have someone blitz in for a week or two and be effective with anything but a checklist. You can’t expect a firm’s auditor to give you MSSP-like/consultant-like hours without either being gouged or limiting how many other paying clients he can handle. And you can’t always expect a client sticks to what they say, especially if they have no real security analysts whose job is to maintain such secure practices.

10. Covering my ass is my major goal.
Dave mentions the audit firm pestering to get answers/details to make sound decisions. Given #1, #2, #3, #4, and the ego-part of Dave’s comments in #9, this leads down the road of eliciting a response you want and then client wants, even if it is false. “Yes, fine, we have a log management product and sure we …watch…it…” can be written down as “Check!” even if it’s not true. “Honey! You let Billy track mud all over the living room!” “But dear, I asked if he had taken off his shoes and he said yes!” “Right, but did you actually CHECK that he was doing it?” “Wait, blame Billy, he lied about it!”

bonus: I know you probably don’t like me.
Really, we techs should like auditors. Tech/Sec managers should like their auditors. If you’re doing a good job, they legitimize it. If you’re doing a bad job because you can’t get budget, they’ll justify it. But if you’re being subpar and you know it (or don’t know it), yes, you dislike your auditors because they look at things you suck at and are asking for details that you don’t have. In that case, you need to look at them as being helpful to improve what you’re doing, not trying to expose you for a hack in front of your boss. If I’m driving stick horribly and someone gives me a tip, it’s just that…helpful!

This sounds cynical of me, but it’s likely because I’m too close to all of this to really appreciate it sometimes. Even my most cynical days are liked by some people because there is a deep thirst for security knowledge beyond sec geek circles. They just don’t like all the work we remind them needs done. No magic buttons… 🙂