Most security products fail to perform report says

by Steve Ragan - Nov 16 2009, 17:00

According a new ICSA Labs report, co-authored by the Verizon Business Data Breach Investigations Report research team, the majority of security products tested by ICSA Labs fail to perform as intended and generally require two or more cycles of testing to earn a coveted ICSA Labs certification.

The ICSA Labs report, which has a solid bit of history as well as certification information, says the number one reason for why a product fails certification more often than not is that it simply doesn’t perform as intended. In fact, only 4-percent of products tested attained certification during the first testing cycle.

However, 82-percent of products resubmitted for testing eventually earn ICSA Labs certification. Once a vendor earns certification, products are required to undergo ongoing testing to maintain certification.

The failure of a product to completely and accurately log data was the second most common reason security products do not perform as intended. Incomplete or inaccurate logging of who did what and when accounted for 58-percent of initial failures.

Across seven product categories, core product functionality accounted for 78-percent of initial test failures -- for example, an anti-Virus product failing to prevent infection or an IPS (intrusion prevention system) product failing to filter malicious traffic. Product categories studied were anti-Virus, network firewall, Web application firewall, network IPS, IPSec VPN, SSL VPN, and custom testing.

Moreover, the findings suggest that some vendors and enterprise users consider logging a nuisance. That is, it’s merely a “box to check.” Logging is a particular challenge for firewalls. Almost every network firewall (97-percent) or Web application firewall (80-percent) tested experienced at least one logging problem.

“Our goal is to help vendors develop more secure products. The question I ask vendors is: ‘Who would you rather have find an issue in your product -- ICSA Labs in a safe testing environment or a criminal in the real world?’” said George Japak, managing director, ICSA Labs and a co-author of the report. 

Also worth noting is the percent of security products that had an inherent security problem itself, which came in at number three on the ICSA Labs list with 44-percent. ICSA said that the inherent security problems included “…vulnerabilities that compromise the confidentiality or integrity of the system and random behavior that affects product availability.”

“When a product fails, we encourage vendors to view that as an opportunity to improve the product before it goes to market. In addition to benefiting the security industry, this open exchange of information can greatly benefit enterprises by providing them more reliable and available information to make educated product purchasing and use decisions,” Japak added.

The full report is online here.

 

Around the Web