Should you bet your risk assessment strategy on a single list?by Steve Ragan - Mar 1 2010, 15:00
Two well-known security vendors have both recently and independently released classification lists that aim to help organizations deal with risk assessment and patch management. Considering that assessment is a major part of security, these types of tools offer some value, but they can be abused as well. So, should an organization rely on a list alone for assessment metrics?
The fascination with using lists as a measurement of risk gained solid traction in October of 2008, when Microsoft launched their Exploitability Index (EI).
Microsoft created the Exploitability Index to help IT departments prioritize patches and updates. What the EI does is give IT a bookie-like betting platform on the odds of exploit code appearing for any single vulnerability that was patched. Using three levels, the most critical being an EI level listing of one (1), Microsoft essentially places bets on what patch will see the most targeted and consistent types of attacks in any given month.
For a while now, pundits and security vendors have placed a good deal of weight on the EI listing, and in return IT shops are prioritizing their patching and risk assessments based on those views. It is no surprise that the vulnerabilities listed with an EI level of one (1) are in fact either already under attack, or shortly after the patch release were targeted for exploitation. This gives serious credibility to using lists.
One vendor, nCircle, recently released their own version of Microsoft’s EI, called the Patch Priority Index (PPI). The PPI is a monthly ranking of the highest risk vulnerabilities from vendors like Microsoft and Adobe. [See it in action here.] According to nCircle, the PPI – which is available free to any IT administrator online – is based on nCircle’s Risk Score.
The Risk Score is a culmination of math and data collected by the firm over the years as they performed security assessments. In short, the scoring used in the PPI is based on the overall risk factor in having the vulnerability present on a system, a measurement of the skill needed to exploit the vulnerability, and the amount of time that has passed since the vulnerability was publically disclosed.
“Security operations professionals understand that risks often aren't evaluated and fixed inside a 30 day window,” said Andrew Storms, Director of Security Operations. “The nCircle PPI helps prioritize risk reduction decisions by helping evaluate new patches within the context of the bigger security picture.”
That statement is telling, and it has raised a few eyebrows as well. More than one discussion last week, shortly after PPI was announced last Tuesday, mentioned the statement in passing and referenced that there is more to risk assessment that patches and vulnerabilities. One comment to us in passing even went so far as to point out that it appears nCircle is conflating patches and vulnerabilities. They are very different things.
This leads to the second security vendor who released something similar to Microsoft’s EI, Rapid7. Rapid7 shipped Exploit Exposure (EE) on the same day nCircle released PPI. EE offers risk assessment across several attack vectors, including Web applications, operating systems, databases, and more. EE details risk by looking at exploit ranking data from Metasploit, as well as the Exploit Database. Rapid7 has released Exploit Exposure in both the commercial and community editions of their NeXpose product, and existing users are asked to update to the newest release to see it. [NeXpose Community Edition is here.]
"When it comes to vulnerability management and reducing the risk of an attack through exploits, detailed information is critical and organizations greatly benefit from data gathered by the security community," said Mike Tuchen, president and CEO of Rapid7, in a statement.
Like the comment from nCircle, this statement is telling too. It attempts to highlight the flaws in a general index based on patches alone. However, does the inclusion of ongoing intelligence mean that Exploit Exposure is a more stable list if compared to Microsoft’s EI and nCircle’s PPI?
Consider the case of a ZeroDay Microsoft vulnerability. By definition, there is no Microsoft patch available for this zero-day and so this vulnerability won't show up on any list that prioritizes patches. At the same time, a ZeroDay has an exploit.
So does this mean that nCircle is erroneously reassuring customers with their PPI that they don't need to patch a ZeroDay vulnerability with an exploit?
On the other hand, Rapid7's EE prioritizes its list based on vulnerabilities. They tell you which vulnerabilities have a real-world exploit. Then, because the EE is included with NeXpose, advice is offered on how to remediate the vulnerabilities, including patching your systems if a patch is available, adding firewall rules, and so on. Yet, does this really make Exploit Exposure a better offering?
The answer, when looking at it from a risk perspective, to both questions is no. No, nCircle isn’t needlessly assuring customers with their PPI that they won’t need to patch a ZeroDay flaw. Likewise, Exploit Exposure can’t be claimed as superior simply because it has more intelligence and options packed into it, though Rapid7 will surely disagree with us on this; nCircle too for that matter.
No serious security program would ever center itself on an application or list alone. Much like the EI listing from Microsoft, both offerings from nCircle and Rapid7 hold a good deal of value and information. Considering all that went into developing them, they are highly impressive. However, at the end of the day, they're still lists, and a vendor’s perspective on what is more important to an organization’s security.
The real risk assessment comes from ensuring that security and the business goals for the organization are equal and share a firm partnership. From that point risk assessment becomes a mission of discovering what assets are the most important in keeping the business profitable and productive while protecting them.