Joel Snyder over on Opus1 has a couple white papers posted about evaluating IPS solutions. Granted, these are dated 11/2007, but they read well enough to stand valid still. The first paper lists 6 steps to selecting the right IPS (pdf).. The second paper lists 7 key requirements for IPS vendors (pdf).

I don’t have much to add to the first paper as it is pretty complete. The second paper has a few things I’d mention.

1. I still prefer calling an IDS/IPS just an IDS. Unless specifically configured (and you have the confidence in the device) to actually prevent attacks, they all work as an IDS instead. And this is good so no managers start thinking all attacks are being prevented even though 90% of the IPS device is working as an IDS device. It’s an expectations thing.

2. In the performance item (#1), I’d just briefly mention along with failopen capabilities, that the device should do so as seamlessly as possible, especially during an upgrade of the device/software. I don’t like patches/upgrades being disincentivized by downtime and off-hours work. That just leads to admins dragging ass. Same with power cycling the device if it isn’t very stable…

3. Item 2 in this paper should be read along with item #2 in the first paper; both deal with what sort of detection the IPS will be doing (rate, signature, anomaly, behavior…). Keep in mind that many IPS offerings doing all of them ends up doing all of them sort of watered down. If you already have netflow analysis efforts, you might value that the least.

4. Item #7 asks for some limited firewall capabilities. While noble to include, I don’t want to confuse network gurus in thinking they should be mucking heavily in these ACLs and IPS rules just because this is the closest device to the source traffic. In IDS/IPS shouldn’t be heavily leaned on for such duties, and thus arguably shouldn’t even begin to be leaned on.

5. I’d add item #8 to the mix and say that enterprise IPS should give the operators the ability to be informed and capture enough evidence in an alert to make an informed decision. No data = fail. 1 packet = fail. And so on. This should be part of the evaluation of the IPS and not something you take as truth just because a sales guy says so.

6. Additionally, the alerts an IPS gives should not only be clear and precise on the problem, but signatures should be viewable by analysts to compare why something was triggered. Bonus points if you have capability to craft new signatures, either fully new or using an existing one as a template.