The recent “State of the Net” report, issued by consumer advocate Consumer Reports, offered up some impressive figures and advice. However, the advice and testing methods are once again getting a sharp poke in the eye by some of the bigger security vendors and professionals alike. The issue most have is simple: Consumer Reports has skewed testing methods at best, and offers some well meaning, if uninformed, advice to the masses.
Experts and vendors have a different view on the reviews by Consumer Reports. (IMG:Consumer Reports)
Most adults, and even some of the younger crowd, are familiar with the reputation and scope of Consumer Reports (CR). Entire marketing campaigns are based on the CR scores earned by household brands, including car manufactures, television manufactures, DVD players, etc. In fact, if a product is recognized, and has the status of being a “household name,” then you can be sure that CR issued a review or report on it at some time in the last five to ten years, if not recently.
However, with that said, there are some household name security vendors that are taking issue with the testing and reviewing methods that CR served up in its September issue, which focused on “The State of the Net.”
A blog post by Symantec's David Cole took considerable issue with the review CR performed, as well as its methods for testing both in the past and present. The general issue is simple: should a company that tests cars and televisions be qualified to offer technology advice and perform software reviews in a selective and granular vertical market such as computer or internet security?
The first issue that Cole, as well as other security reporters and experts such as Larry Seltzer at eWeek, took notice of is the software versions tested. Cole explained it best when he asked:
“First off, what would you think of a publication that released a review for this year’s line-up of luxury sedans a month before next year’s line-up was released?”
The reasoning for the question is simple: the CR report covered most of the popular and known vendors, but focused on their 2008 releases, which, according to many, is the first failure. Why is it the first epic fail in the eyes of some? Because most of the versions tested were outdated by the time the report went to press in early August.
Security companies are predictable and early summer will be the time many of them start their PR campaigns and beta testing for the newest versions of their respective software offerings. Then, by early or late autumn (August to October), the final product is shipped to market for mass consumption by businesses and consumers alike, as well as a new round of PR buzz. Consumer Reports tested the 2008 versions just before the 2009 versions were released in some products. CR even admits this failure when pointing out that one product, Avira Personal Edition Classic 7, was replaced by a new version.
Another issue that Cole had with the review was the use of “Instant Off” for security features. When CR rated anti-virus protection, it looked for a feature that would allow an easy way to “...disable [or] pause protection when you’re installing software.” In short, the ability to kill security features that prevent the installation or downloading of applications of software.
“Translation: Accidentally installing applications that contain security threats is the #4 top online blunder, but make sure you hit that “easy disable” button on your security product so you can install malware. Your security software should be checking the applications that you attempt to install. Period. Pausing AV protection to install software is dubious advice to consumers, especially given the rise in threats, like the Storm Trojan that use social engineering tactics to convince users to install malware,” Cole said.
Larry Seltzer adds another interesting observation dealing with conflicting reports from eMarketer, which provided the stats and data used in the Consumer Reports story. The CR report offers that one in 94 households have suffered monetary losses due to Phishing attacks in the last two years; however, another eMarketer report (one not cited by CR) claims the actual risk of victimization by Phishing is lower than expected.
“I'm confused; is it a big threat or not,” Seltzer commented.
Cole and Seltzer, both respected in their fields of work, have very valid points. Consumer Reports has not issued comments on testing methods, or on the commentary online surrounding its September issue. Questions directed to it by The Tech Herald have gone unanswered as of time of posting.
The largest complaint is not just how CR performed its reviews and testing, but includes some of the facts and data that it used in its reporting. Numbers can be twisted to suit the needs of any article, as has been proven in the past with various examples of statistical data. However, an issue that I have is related to another of CR's “7 Blunders,” directly related to Apple's Safari Web browser.
CR reported that Blunder number four, as mentioned, deals with downloading free applications, and calls them a risk. The very next blunder, number five, is entitled: “Thinking your Mac shields you from all risks.” While the title is true, as a Macintosh will not and cannot protect you from all threats, the suggestion that users dump the native Safari browser due to lack of Phishing protection harkens back to the PayPal-induced FUD from earlier this year.
“According to this year’s State of the Net survey, Mac users fall prey to phishing scams at about the same rate as Windows users, yet far fewer of them protect themselves with an anti-phishing toolbar. To make matters worse, the browser of choice for most Mac users, Apple’s Safari, has no phishing protection. We think it should,” reads CR's blunder number five.
Like PayPal, CR suggests using Firefox or Opera until, “Apple beefs up Safari.” This is poor advice, as all it does is spread fear to the average consumer, and does little to mitigate the problems or offer advice.
Yes, its suggests other browsers, which is a small measure of mitigation advice. But what if the Apple user does not want to switch? This is where CR earns the ultimate failing grade.
Case in point, there are security options for the Mac, none of which earns a mention by CR, which offer more than simple virus-related protection. These security options include operating system tweaks, common sense, and even AV-related software packages. Aside from the software and OS tweaks, there is also OpenDNS to consider, which will offer security and Phishing protection no matter which OS you use (Mac security offerings, as well as OpenDNS, will offer levels of Phishing protection).
Another issue is the false sense of security that the “anti-Phishing” measures in the mentioned browsers will offer the average user. While true insofar as they will flag most Phishing sites on contact, there is the consideration that Phishing scams are ever changing, and new Phishing methods and sites are often left unblocked for a few days, if not longer.
The only real Phishing protection consumers have is their own logic. If you think a site is safe because Opera or Firefox did not mark it as forged or risky, then you have already compromised your personal security, and no software or magazine review will protect you.
Add your comment (no registration required)
page: 1
Martha CollierSep 9th, 2008 - 05:27:33
Any site that uses obnoxious IntelliText ads loses any credibility to speak to computer users. I'm outa here.
Report this comment
Advertising
Martha CollierSep 9th, 2008 - 05:27:33
Any site that uses obnoxious IntelliText ads loses any credibility to speak to computer users. I'm outa here.
Report this comment