notes on 5 secrets to building a great security team

Via the Infosecnews mailing list, I’ve read a CSOOnline article on “5 secrets to building a great security team”. Sounds fun, despite being geared around more of a C-level managerial perspective in a larger organization where “security” encompasses brand protection, organizational risk, and other things beyond just digital security.

1. Rethink everything. – Pretty much a safe, vague item, but a good one. There’s no right answers, and it really helps to sometimes sit back, figure out what is working and what isn’t working (not what’s broken, but just what isn’t awesome), and try something new.

2. Formalize underserved functions. – This item focuses entirely on diaster recovery / business continuity sorts of efforts. While not necessarily part of “security” in a traditional sense, it does deal with organizational risk, operational resiliency, and personal safety; things that “security” often has in its vision statements as well. I don’t mean to downplay these efforts, they’re just a different slice of the security pie than what typically gets my juices flowing.

3. Demand proven business skills. – Essentially, this talks about the value of an MBA and, more importantly, being able to understand and talk with the business, and its leaders, in their language. It’s hard to disagree with this as being a useful skill when you’re not 100% in the trenches every day.

4. Create a communications czar for security. – This sounds interesting, and I’m not sure I’ve heard of something like this before, but it certainly makes sense. I got the impression part of this role was to ease the changing (i.e. HR issues with sweeping changes) of how security works at Caterpillar, but the details really show someone who acts as internal PR for security, and probably as trainer and support. Security can definitely use some people people.

5. Nurture dissent. You know, I could leave this entire article and forget about it in minutes if but for this bullet point. Security (privacy, risk…) is a constantly debatable topic entirely because of its nature; always being at ends with evolving threats, but also it’s balancing act of security vs usability/convenience. Keeping this as an important, specific item allows a leader to always be able to illicit the most knowledge from his team members, rather than all of them just nodding and agreeing to whatever and letting the leader walk off their own cliff edge. It also really helps support the first item.

a top source for digital security news

It’s true, the blogsphere (blogosphere?) for security news is smaller and a bit more watered-down these days. At least stuff that is interesting enough to link to. I’ve also found my own time for such reading to be smaller than usual lately. Normally I don’t plug sources, but I admit when I have a moment to catch up on 2 weeks’ of news, I typically start with the Infosecnews mailing list emails that build up. Older posts can also be perused at’s archive.

Part of the reason for this plug is to cover my butt a bit and share the love. Sometimes when I make a mention of news elsewhere I may forget to say how I got pointed over there. When I can, I try to share that bit, but more often than not my forgotten sources are Twitter (thank you lack of search due to shortened URLs) and ISN.

harvard business review touches on the cloud

The Harvard Business Review out on the newsstands right now has an article about what CEOs need to know about The Cloud. I’d link to it, but it’s behind a registration wall, and it’s not really worth but a skim for those who’ve heard the term before.

My first reaction is, “Gosh, looks like we lost the battle on what the definition of ‘cloud’ is.” Basically, anything that runs on a different system that you consume is the cloud. Web, email, files, whatever. Fine, I get it and I’m fine with that. Oh well!

I felt the article had a decent review of the definition of the cloud as well as the quick benefits. However, I did want to mention one I think didn’t get enough face time.

The things offered by the cloud are not so amazing that internal IT teams couldn’t do them. Sort of. That’s what the article says, and mentions that internal IT is stretched. I agree, and I’d say that at least these third-party app providers (cloud providers, SAAS…) can afford to have such a laser-focus that they can do a really good job with what they provide. I fully think that needed to be explicitly stated as a tangible benefit. Your internal CRM is cool, but that CRM is going to be the better. I thought this just needed to be really highlighted as a point.

On the flip side, I felt the review of the risks/costs of cloud were really glossed over too lightly.

Specifically I didn’t like the lack of mention on how cloud applications (like docs in the cloud or email, or CRM..) are not going to every be nearly as customizable as your own internally managed apps and products. My company is currently in the midst of reviewing the replacement of our internal Microsoft Exchange infrastructure with something like GMail/Apps. But I can guarantee that many of the little pieces in our mail and calendaring settings and tweaks and processes are just not going to be possible in someone else’s product.

You get a laser-focused tool, but you get the same tool as everyone else gets, without any special sauce of your own.

Which actually brings up an entirely related point: What do you have when you and all of your competitors all use the same platform? You lose any ability to say you have better technology or better channels or better processes. You’re all going to be doing things the same way. (Some may even wonder if there might be a conflict of interest when Big Fish in an industry with bigger wallets influence a cloud provider’s tool to tailor to what they want…)

Some might praise this as a way to compete directly and solely on people and business product. I can buy that, but I’m not sure that’s something many businesses think about when going onto cloud services. In that way, the benefit is not to do work better than your competitors, it’s simply to cut costs of infrastructure.

I also felt like there was a glossed-over cost of control over your own data. At the end of the day, if you move to another provider or go back in-house or want to provide and have assurances that your data is protected and private and recoverable, you’re working under blind faith with your cloud provider. At least with internal infrastructure, you have unlimited ability to audit, test, and verify. (At the cost of an unlimited ability to cut corners, lie, cheat, and be negligent/ignorant.)

Essentially, we had this cycle where we moved from mainframes and time/cycle clocks and sharing to decentralized PCs, and we’re moving back to centralized computing. Yes, it will go back again, and repeat and probably for the same reasons: customization and control.

keeping your rockstars (and other employees) happy

I’m pretty sensitive to worker happiness; hey, it’s an INFP thing for the most part. And I liked this article on Gigaom: 5 ways to keep your rockstars happy.

First, some sub-points to mention.

a. “…even-keeled bosses who made time for one-on-one meetings…” – I like 1-on-1 meetings with my boss, if only to make sure I get face time, share my challenges and accomplishments, and just chat a bit. And this is coming from an introvert who hates “chatting.” If I ever move up to mgmt, 1-on-1s will be a staple. It also helps to foster less boss-employee relations and lets people be more formal and at ease with each other. Well, at least that’s always been my boss-employee relationship with strong 1-on-1 comm.

b. “…[help] people puzzle through problems by asking questions, not dictating answers” – How do you best get someone to go what you want? Let them come to the conclusion you want on their own; just guide them. This also allows for better accountability (ownership) and pride and confidence. I also believe this helps foster better innovation, especially if you let them run with an idea that you might not initially think it better. This can be hard for me at times, as I (and I can’t say this without sound like a douche) for most of my life in school and work and personal am usually right in my judgements (see? I can’t say that without being an ass). It’s sometimes difficult to let someone else’s idea be the torch, especially if I don’t immediately believe it’s the best option. But sometimes allowing that helps, especially when a manager is a little further apart from the trenches. I think ultimately puzzling through problems can lead to more acceptance of innovation and even mistakes.

c. “…who took an interest in employees’ lives and careers.” – This isn’t quite as important to me, which isn’t surprising since I keep to myself, am reserved, adore my privacy, and keep some things separated rather well (work vs online persona, e.g.). But I still do like when a boss knows some of my interests and hobbies, and vice versa, and they also have an interest in my career. That 1-on-1 point up above supports this.

So now the real points of the article!

1. Create a culture of education – Agreed, but managers can’t just shove learning down people’s throats. I don’t necessarily want to learn about how department 12 does their job, or take classes on Juniper routers when I’d rather have Cisco classes. Education has to match up with the desires, interests, and goals of each employee. But it needs to be made available and gently guided as well. This should also include actual useful training on technologies available, especially in IT and security. One of my beliefs is that IT ops learns the most when shit hits the fan and we’re in troubleshooting mode. That’s a reactive way to learn, which can be partially fixed with proactive learning and encouragement and value. And truly, the top 10% of your team will be interested in learning. I’m positive there’s a direct relationship there.

2. Provide regular, consistent feedback – Consistent is probably key here. No one likes to have a moving target or a friend (manager) who waffles constantly and flip flops more than a pendulum. The points in the article are excellent, though.

3. Set time aside for weekly 1:1 meetings – Oops, I covered this already. I personally don’t think you can manage people very well and keep them happy without this. Unless you’re a stupid or tyrannical manager that you people don’t actually want to talk to. Then I’d say skip this. 🙂 This also has a security aspect to it, as I strongly believe that the first line of defense against insider attacks and disgruntled employees is the managerial relationship.

4. Manage the grunt work properly – I hadn’t actually thought much on this, but I do like this idea. Managers should also know what tasks employees consider “grunt work,” and manage accordingly. Right now we’re looking to hire a more “juniorish” person on our operations team. This is a great opportunity for each of us to get a “grunt” task off our plates and onto someone less seasoned into their career.

5. Publicly acknowledge good work – Again, as an introvert, this isn’t usually a big deal to me. But it does get noticed when other people are praised and I get left off the list (which happens often when you’re infrastructure ops or security). I do, however, care what my immediate boss thinks of me, just like a parent-child relationship in the early years. And I do understand the more public acknowledgement serves the careers of both me and my manager, so I’ll totally ‘get it.’ If my CEO or other C-level knows my name and can greet me, that alone is a cool feeling, and it helps if there is some public or at least manager-level-back-room acknowledgement going on. For instance, thinking back to high school, some of the best praise and good feelings I got from my teachers was completely indirect, where I would find out that one or two would talk about me in their own circles (for better or worse). That sort of knowledge in their interest means a ton.

This can easily get a bit cheesy however, and I’ve seen awful examples of this in my career. For instance, don’t have a mandatory rotating award; it devalues the spontaneity of it, especially when someone undeserving gets it. When that happens, the entire purpose is destroyed, if not worse. Second, if you have a reward/recognition program internally where employees can send “good jobs” perks to others, don’t go making them mandatory or otherwise so devalued that friends just bounce them back and forth in a you-pat-my-back-I’ll-pat-your-back way. This also somewhat benefits more social people than those reserved introverts that probably only give praise when it truly is heartfelt.

how a cso can make life harder for an attacker

Really diggin’ an article by Drazen Drazic where he goes over 14 things a CSO (read: IT security) can do to make an attacker’s life harder. What’s nice is this list goes beyond the typical (yet effective!) suggestions like just patch systems. In normal fashion, I’ll summarize and react to the bullet points.

1. Avoid password re-use for admins. Duh!

2. Run something that detects new hardware on your network. – Oh my god, absolutely! This isn’t usually as easy to do as it sounds and doesn’t easily fall under the “can-we-buy-a-box-or-tool-turn-it-on-and-it’ll-protect-us-this-way” category that CSOs too often fall into. But the value in this is really phenomenal. Know what’s normal on your network and know when something strange and new pops up.

3. Monitor your internal network to detect weird behavior and unexpected requests. – This bullet point is like a 3-punch combo and should be printed out and taped to walls. I love this: “Your Network Admins…should be allowed and supported with time and resources to monitor logs of the systems they manage.” And this: “Outsourced perimeter management providers don’t care. Their SLA’s claim that they do, but they don’t…” This item also mentions monitoring traffic, which is also invaluable.

I should break here and say two of these items are orgasmically valuable, but they’re also not things that CSOs like. You don’t just buy a box or tool to do it. You don’t just hire more staff (you need staff who know their shit). You don’t just make a project, task someone to scope it out, and then start, progress, and end it with a stamp of success. There’s no end to it. That takes effort to justify to businesses and accounting. Oh wait, that’s basically the point of a CSO. If you want security, you have to spend the manhours and you have to make it an intrinsic cultural goal.

4. Monitor external DNS to detect new website/hostname exposed on Internet by your company. – Whoa, this is new, and I’m not even sure how to interpret this. I think this gets down to knowing what has been published by your domain team to our external DNS and/or what has been exposed by your firewall/perimeter team. You don’t ever want to not know that your dev/test server has had its balls hanging out in the digital breeze of the Internets on accident. For websites, this might be an indication that your web server is hosting sites you didn’t know about, perhaps.

5. Let your System/Network Admins use their magic. – I completely agree that you need to let your talented admins leverage their talent. But there’s a few gotchas. First, not all admins give a rip about security or know jack shit about it or what questions they should be answering. You really need your security folks to also be admin folks, or vice versa. Second, scripting and rolling your own stuff is fine, but that usually has drawbacks such as easy and useful reporting, performance, scalability, feature creep, and limited support outside the people who built the internal tools. Keep in mind that not every system/network admin has the chops (or desire) to dive deep into scripting or even real coding.

I should also break here and say that I still think it is valuable to let talented staff do their thing, even if it means if they leave, their thing is going to go to waste. If you bring in a painter to paint your house, he’ll use his tools and equipment and experience and preferences to do his work. If he leaves halfway into the job, you won’t expect the replacement painter to adopt the exact same project plan, preferences, and tools as your last guy. You let them do the job they do in the way they do it, even if it means starting over.

6. Win small fights – one at a time – Even down-in-the-trenches guys like me need to adhere to this. We can’t get our way on everything, but we do need to make progress whenever we can, so we pick our battles and win the ones we can while noting future challenges we can tackle later.

7. Save the money to hire people with skills instead of getting magic boxes that do little or nothing. – It all comes back to people. Enterprise, especially IT, tends to hate this (or at least be bad at swallowing this pill). At least in my experience.

8. Use open source. – I can agree and disagree with this item, but really the point still gets back to letting staff use their talents, and I agree a talented staffer can probably be more valuable wielding small, more surgical open source tools than unwieldy big-box suites or tools that suck away time and don’t give quite as much value back. Honestly, I think blending tools/appliances from the traditional commercial space along with open source/DIY tools is a solid way to go.

9. Go to real hacking conferences. – Absolutely. This is the “training” security-minded talent yearns for.

10. As a CSO, you MUST be involved with all “critical” projects. – This is a bit political for my taste, but I agree, ultimately. Even from an operations standpoint, it sucks goat balls to be surprised at the final hour of a major project with tasks and requirements you need to meet for their project to work. Security is even lower on that totem pole of information-sharing and inclusion… Ideally, if you run an absolutely tight ship with regards to many of the above bullet points and beyond, I’d almost hope that security is so tight, anything new needs to go through security or at least be noticed by security in quick order. I like to think of security like good ol’ bumper bowling for the kids, where security are the bumper pads placed into the lane gutters that keep the ball rolling toward the pins. If security is tight, people aren’t going to accidentally find themselves throwing gutter balls and upsetting the order of things.

11. Rub shoulders with those in the trenches. – Absolutely, for the most part. I’ve always said if you want to know a company’s security posture, you just have to ask the admins and desktop support persons. They know the score more than any manager or C-level.

12. It takes time. – Yup!

13. Find a blend of talented people for various roles. – I absolutely love this item as well. There really isn’t a security person around who can talk toe-to-toe with the Unix team, the Windows team, the networking team, the virtual team, the web dev team, the software team, the mobile team, and then desktop team at the same time. Assuming the “security guy” can answer every single question is setting him up for failure and loss of credibility. Find the security allies in every team and tap them.

14. Dedicate time to your security technologies. – Just like having talented staff, it can’t be said enough how time investment is important. The article mentions WAF and IDS, and that’s completely true in all cases. You can’t just stand up a WAF and expect it to do magic; you have to get it up, tune it, adjust it, work with devs as they make changes, tighten it up right to the point of breaking shit but not quite breaking it, and then test it, tune it, validate it, etc. That’s not a project, that’s a job.

the passing on of steve jobs

This is just a personal placeholder to note that Steve Jobs passed away 2 days ago on 10/5/11. The reality has been that, God willing, he was going to pass away before me no matter what (as will many other luminaries from my lifetime, as I get older), but it’s still a sad time to see such a successful influencer no longer with us. Even as I dislike most of the Apple products in principle, there’s no denying the success and vision and inspiration of Steve Jobs.

Just as an illustration of how news travels in this new age, I happened to be playing World of Warcraft at the time, and saw mention in my guild about the event. I windowed out to check, saw no updates, so went back into game for a bit, not really sure if that was yet another joke or not. About 15 minutes later, I checked again and saw the actual update on CNN and then noticed my Twitter feed start to light up. I still feel pretty connected, even if I haven’t watched national/local news television spots in 7+ years, don’t read the traditional newspaper, and really don’t listen to local radio but for 15 minutes in the mornings. I’m certainly not a young’in, but at least I’ve got my toes in the technology of today (even if I abhor MySpace Facebook). Granted, if there’s anything I miss, though, it’ll be some local issue…

the vendor beating and lessons in operations mgmt

Via Securosis, I got pointed to an excellent article from an EMC VP talking about vendor beating and some hard lessons in IT. While clearly the point is geared around beating up your vendor for poor reasons and how EMC will work with you, the more powerful points in the article revolve around management of an organization’s IT situation, which includes keeping up with technology, forging relationships, knowing users, keeping the team current, and otherwise just not letting your team be the low group on the totem pole who gets the fingers pointed at them, regardless of reality. Things like the slow creep of poorly planned and poorly grown IT operations. Or the slow obsolescence of systems and even people.

graham on ways to do real damage to the nyse

Rob Graham (ErrataSec) has a fun post about Anonymous threats against the NYSE. In it:

The NYSE runs a completely separate network. Well, lots of people say this, like the operators of the power grid, and it’s rarely true. But it’s true in the case of the NYSE: I doubt hackers will find a way from the Internet into the NYSE private network.

But, there are lots of things on the NYSE private network, such as terminals on the desks of traders among the members of the NYSE. If a hacker could get physical access to one of those terminals, he could do a lot of damage.

If that doesn’t scream traditional espionage/infiltration… Which would be quite the interesting attack, and one I’m sure has been on their minds for decades: insiders who either have hacking skills or facilitate access to those who do. Can Anonymous do that? Probably not, but I’d also wonder why they would want to. It’s not like they’ll drive off the fat cats and then sit back and live happily on themselves; such an event would have dramatic effects on their own lives, in not good ways. Then again, that might fit into someone’s anarchist viewpoint…

htc phones log information and don’t secure it

AndroidPolice (via full-disclosure) have detailed an issue with recent HTC phones (I own one). HTC has new tools that allow for a wide range of logging. These logging features (and resultant logs) are horribly secured, leaving pretty much any app able to harvest this information.

Things like this underscore three small points.

1. There’s been recent hand-wringing about pessimism in security. But it’s things like this, either a priori, or just by being more security-conscious and exposing these things, that really reveals why we are a bit less cheerful. Is it pessimistic just because I know about shoddy code and a vulnerability like this, and likewise would I be more optimistic if I wallowed in ignorance? It’s like not liking strippers as much because you’ve seen them in the back room with their make-up off and holes in their underwear.

2. The lack of initial response from HTC, but then subsequent response and offering of a patch when things go public illustrates the challenge security has, especially when we’re talking things that are so ubiquitous as a cell phone (ok, smartphone) and in use not just in IT circles, but in consumerland. The fact that crap like this even happens is enough to cause an extra drink or two a night. I really believe there are far more people than I’m comfortable with thinking about who will bend and/or break rules and do as little as possible as long as they have a decent chance of not being exposed; part of what I’ll always call the Security Gamble.

3. Why is there this loggingi n the first place? I can only think of three reasonable things. First, compliance with law enforcement initiatives. Second, marketing to gain more information on users and use that for revenue generation. Third, support for when things go wrong, or to improve the product after crashes and such. I firmly believe in the first item, shrug at the second, and sort of doubt the third as being way too proactive for most orgs.

I also think this continues to illustrate why smartphones just can’t last forever and how unmanagable and unscalable they are as technological devices. Keeping up with apps and the underlying security and usefulness and minimizing the frustration is just not going to get better. Sure, they’re smaller (handheld) and it’s easier (cheaper) to buy apps and have them auto-install, but that’s only successful for today because those are improvements over just 2 pieces of the desktop/laptop experience. There is still the quagmire of user garbage that accumulates on these devices that causes just as much frustration with them as any previous computing device.