Security may be the hottest topic in IT. But it's also one of the least understood.
So BriefingDirect assembled a panel this week to examine the need for IT security to run more like a data-driven science, rather than a mysterious art form.
Rigorously applying data and metrics to security can dramatically improve IT results and reduce overall risk to the business. By employing and applying more metrics and standards to security, the protection of IT becomes better, and the known threats can become evaluated uniformly.
Standards like Information Security Management Maturity Model (ISM3) are helping to not only gain greater visibility, but also allowing IT leaders to scale security best practices repeatably and reliably.
With standards and greater reliance on data, security practitioners can understand better what they are up against, perhaps gaining close to real-time responses. They can know what's working -- or is not working -- both inside and outside of their organization.
The security metrics panel and sponsored podcast discussion are coming to you from The Open Group’s Enterprise Architecture Practitioners Conference in Seattle on Feb. 2, 2010. The goal is to determine the strategic imperatives for security metrics, and to discuss how to use them to change the outcomes in terms of IT’s value to the business.
Our panel consists of a security executive from The Open Group, as well as two experts on security who are presenting at the consortium's Security Practitioners Conference: Jim Hietala, Vice President for Security at The Open Group; Adam Shostack, co-author of The New School of Information Security, and Vicente Aceituno, director of the ISM3 Consortium. The discussion is moderated by Dana Gardner, principal analyst at Interarbor Solutions.
Here are some excerpts:
Hietala: We think there's a contribution to make from The Open Group, in terms of developing the ISM3 standard and getting it out there more widely. [Being a data-dListen to the podcast. Find it on iTunes/iPod and Podcast.com. Read a full transcript or download a copy. Sponsor: The Open Group.riven security organization means] using information to make decisions, as opposed to what vendors are pitching at you, or your gut reaction. It's getting a little more scientific about gathering data on the kinds of attacks you're seeing and the kinds of threats that you face, and using that data to inform the decisions around the right set of controls to put in place to effectively secure the organization.
A presentation we had today from an analyst firm talked about people being all over the map [on security practices]. I wouldn’t say there's a lot of rigor and standardization around the kinds of data that’s being collected to inform decisions, but there is some of that work going on in very large organizations. There, you typically see a little more mature metrics program. In smaller organizations, not so much. It's a little all over the map.
... The important outputs of a good metrics program can be that it gives you a different way to talk to your senior management about the progress that you're making against the business objectives and security objectives.
That’s been an area of enormous disconnect. Security professionals have tended to talk about viruses, worms, relatively technical things, but haven't been able to show a trend to senior management that justifies the kind of spending they have been doing and the kind of spending they need to do in the future. Business language around some of that is needed in this area.
Shostack: We have an opportunity to be a heck of a lot more effective than we have been. We can say, "This control that we all thought was a really good idea -- well, everyone is doing it, and it's not having the impact that we would like." So, we can reassess how we're getting real, where we're putting our dollars.
The big change we've seen is that people have started to talk about the problems that they are having, as a result of laws passed in California and elsewhere that require them to say, "We made a mistake with data that we hold about you," and to tell their customers.
We've seen that a lot of the things we feared would happen haven't come to pass. We used to say that your company would go out of business and your customers would all flee. It's not happening that way. So, we're getting an opportunity today to share data in a way that’s never been possible before.
Aceituno: The top priority should be to make sure that the things you measure are things thatare contributing positivity to the value that you're bringing to business as a information security management (ISM) practitioner. That’s the focus. Are you measuring things that are actually bringing value or are you measuring things that are fancy or look good?
Because metrics are all about controlling what you do and being able to manage the outputs that you produce and that contribute value to the business ... you can use metrics to manage internal factors.
I don’t think it brings a bigger return on investment (ROI) to collect metrics on external things that you can't control. It’s like hearing the news. What can you do about it? You're not the government or you're not directly involved. It's only the internal metrics that really make sense.
Basically, we link business goals, business objectives, and security objectives in a way that’s never been done before, because we are painfully detailed when we express the outcomes that you are supposed to get from your ISM system. That will make it far easier for practitioners to actually measure the things that matter.
Business value approach
Shostack: Vicente’s point about measuring the things you can control is critical. Oftentimes in security, we don’t like to admit that we've made mistakes and we conceal some of the issues that are happening. A metrics initiative gives you the opportunity to get out there and talk about what's going on, not in a finger pointing way, which has happened so often in the past, but in an objective and numerically centered way. That gives us opportunity to improve.
Hietala: There's some taxonomy work to be done. One of the real issues in security is that when I say "threat," do other people have the same understanding? Risk management is rife with different terms that mean different things to different people. So getting a common taxonomy is something that makes sense.
The kinds of metrics we're collecting can be all over the map, but generally they're the things thatwould guide the right kind of decision making within an IT security organization around the question, "Are we doing the right things?"
Today, Vicente used an example of looking at vulnerabilities that are found in web applications. A critical metric was how long those vulnerabilities are out there before they get fixed by different lines of business, by different parts of the business, looking at how the organization is responding to that. We're trying to drive that metric toward the vulnerabilities being open for less time and getting fixed quicker.
Shostack: We've seen over the last few years that those security programs that succeed are the ones that talk to the business needs and talk to the executive suite in language that the executives understand.We've seen over the last few years that those security programs that succeed are the ones that talk to the business needs and talk to the executive suite in language that the executives understand.
The real success here and the real step with ISM3 is that it gives people a prescriptive way to get started on building those metrics.
You can pick it up and look at it and say, "Okay, I'm going to measure these things. I'm going to trend on them." And, I'm going to report on them."
As we get toward a place, where more people are talking about those things, we'll start to see an expectation that security is a little bit different. There is a risk environment that's very outside of people's control, but this gives people a way to get a handle on it.
Aceituno: The main task of the ISM3 Consortium so far was to manage the ISM3 standard. I'm very happy to say that The Open Group and ISM3 Consortium reached an agreement and, with this agreement, The Open Group will be managing ISM3 from here on in. We'll be devoting our time to other things, like teaching and consulting services in Spain, which is our main market. I can't think of anything better than for ISM3 to be managed from The Open Group.
Hietala: You have metrics and control approaches in various areas and you can pick a starting point. You can come at this top-down, if you're trying to implement a big program. Or, you come at it bottoms-up and pick a niche, where you know you are not doing well and want to establish some rigor around what you are doing. You can do a smaller implementation and get some benefit out of it. It's approachable either way.
