I liked this post by Curphey in relation to the SourceFire IPO. In fact, I like it because of how it portrays IDS/IPS and the typical installation.
[1:20:17 AM] XXXX-XXXX says: I’ve never been at a company where i’ve heard them say they were happy with their sourcefire deployment or for that matter… convinced me they were glad they made the purchase
[1:21:58 AM] XXXX-XXXX says: The security departments gets this new toy, they quickly figure out they dont have the time to babysit it (or configure it properly) then they outsource the monitoring
[1:23:02 AM] XXXX-XXXX says: once the monitoring company gets it.. they detune it as much as possible.
[1:24:44 AM] XXXX-XXXX says: What I see happening is “what do you mean this IPS might stop legit traffic? well lets just run it in IDS mode then”
[1:24:52 AM] XXXX-XXXX says: and after talking to XXXX-XXXX sales engineers
[1:25:02 AM] XXXX-XXXX says: 90% of XXXX-XXXX deployments are in IDS mode only
[1:25:40 AM] XXXX-XXXX says: Less then 5% of XXXX-XXXX deployments take advantage of the SSL decryption and analyze features.
While we have a larger and larger IT force doing things like desktop support and making sure the business world still works in the digital world, there is still a huge shortage of the type of geeks who “get it” and can make a difference with truly technical things. This is why the dashboard IDS/IPS has been superficially successful because it doesn’t require deep technical knowledge to get and click through alerts. But the knowledge of what those alerts means is pretty damn spotty and if the IDS/IPS doesn’t support tools to drilldown into the mucky darkness of the real technical trenches, that solution is overall just superficial.
But how do you know your out-sourcer is decent with security? Really, we shouldn’t move to make security a commodity that is driven by checklists and statistics without understanding. We need more skilled professionals, even if that means they have an inflated salary for a while and later take a small dip.
[10:15:40 AM] XXXX-XXXX says: Hey, I’m so glad you guys took over our security monitoring! We had no clue what was going on with the IDS/IPS after the installation techs left. You guys have helped us pass important compliance initiatives and haven’t impacted our business at all!
[10:18:23 AM] SecMonTech04 says: No problem! Looks like we came in just in time too! You had 12,476 alerts in the last month alone, but we’ve totally taken care of you! Just look how much you needed us!
[10:19:49 AM] XXXX-XXXX says: Sweet mother of all that is good and pure, that’s a lot! Whew! By the way, is that the number of alerts after you’ve tuned the monitoring?
[10:20:45 AM] SecMonTech04 says: Uh, yes.
[10:22:27 AM] XXXX-XXXX says: What did you all tune out?
[10:23:33 AM] SecMonTech04 says: Um, we ignore ARP alerts because it’s really just too noisy.
[10:24:12 AM] XXXX-XXXX says: That’s it?
[10:24:56 AM] SecMonTech04 says: I believe so…
[10:26:43 AM] XXXX-XXXX says: This is kind of odd. How many of those alerts are important enough to warrant further investigation or worry and wouldn’t ever be tuned out by anyone?
[10:29:42 AM] SecMonTech04 says: Looks like about 3…maybe 6 if I am paranoid.
[10:30:31 AM] XXXX-XXXX says: That’s it?
[10:31:21 AM] SecMonTech04 says: Oh, and we’re not really monitoring much on incoming port 80 because there’s too many application level attacks that we don’t want to give you a false sense of security about if we said we protected port 80.
[10:32:22 AM] XXXX-XXXX says: Huh? Why the hell not??
[10:34:45 AM] SecMonTech04 says: By the way, did you read the latest alerts from the anti-virus companies? The Internet is falling apart and is being overrun by hooligans and criminals. You better be glad you have us!
[10:37:32 AM] XXXX-XXXX says: Hold on a minute, back up. You’re not tuning anything out and not monitoring what might be one of our most important incoming ports. Are you actually blocking any attacks at all?
[10:39:12 AM] SecMonTech04 says: No, we’re operating in IDS-only mode. We don’t want to risk negatively impacting your business and cause you to distrust and dislike us.
[10:44:41 AM] XXXX-XXXX says: Oh god, I need some Tums…
[10:49:40 AM] XXXX-XXXX says: You realize we will need to start blocking some things?
[10:51:40 AM] SecMonTech04 says: Tell you what, we will turn in blocking (IPS mode) for all incoming ports between 55000 and 58000. Will that be enough?
[10:53:11 AM] XXXX-XXXX says: Whew, I think that will be ok…glad you guys are the experts.
[10:55:54 AM] SecMonTech04 says: Actually, we hire not only the inept techs you let go because you outsourced security, but we also employ interns who just click “ok” to every alert that comes in. They don’t really know what this means either.
[10:56:30 AM] XXXX-XXXX says: …I’ll assume you meant to type that in another window.
[10:59:10 AM] SecMonTech04 says: Oops, yes I did, sorry.
You hit the nail right on the head; we need more specialists.
When I needed to move to Chicago two years ago I started looking for jobs. I thought I’d be a shoe-in but alas, everyone was looking for the Exchange-SQL-Checkpoint-Oracle-Linux-Unix-and-all-the-Windows-versions guy. Sucks to be me I guess.
We’re also seeing a lot of people we hire that have credentials but not the proper experience (paper MCSE’s basically).
The care and feeding of an IDS is no small task. Add the pressure of blocking legitimate traffic and you have your hands full. Its for that very reason that I always suggest that part of the feature set you are looking for in an IPS is the ability to put it inline and in simulation or test mode. While in that mode it needs to log all actions it would normally take so you can tune the device with actual traffic.
But then again, we’re back to the need for someone who actually ‘gets it’ and can accurately tune the IPS without turning it into just another bump on the network.