Research: Security certificate warnings are not workingby Steve Ragan - Jul 28 2009, 20:45
Research: Security certificate warnings are not working. (IMG:J.Anderson)
In a paper that will be presented August 14 at the Usenix Security Symposium in Montreal, researchers from Carnegie Mellon University will discuss the various security certificate warnings used by popular browsers. The warnings are there for protection and security, but the study and research shows that more often than not they are ignored, stripping them of their security value.
The researchers, Joshua Sunshine, Serge Egelman, Hazim Almuhimedi, Neha Atri, and Lorrie Faith Cranor of Carnegie Mellon University, conducted an online survey of over 400 Internet users to examine their reactions to and understanding of current SSL warnings. To follow that research, they then took 100 people into the lab and studied how they use the Web and their reaction to various security certificate warnings.
“Web users are shown an invalid certificate warning when their browser cannot validate the identity of the websites they are visiting. While these warnings often appear in benign situations, they can also signal a man-in-the-middle attack,” the paper says.
The research started with an online survey. Over 400 people were asked to look at screenshots from three different browsers and asked questions about each one. They were shown warnings related to expired certificates, unknown issuer, and mismatched domains. The browsers used in the study, Firefox versions 2.x and 3.x, as well as Internet Explorer 7, each deal with warnings in a different way. The study focused on various topics such as were the security warnings understood, would the users continue to the website regardless of the warning and ignore it, and so on. Another interesting aspect to the paper was the role of risk perceptions.
Sticking to the topical data, the results of the online survey showed that for the most part, despite their level of Internet savvy, most users did not understand the current warnings. As a result, the warnings were ignored by the majority.
“We found that respondents' abilities to correctly explain each warning was a predictor of behavior, though not in the way we expected: respondents who understood the domain mismatch warnings were less likely to proceed whereas we observed the opposite effect for the expired certificate warnings,” the paper explained.
When it comes to risk perceptions, it came as no surprise to see responses such as “I use a Mac…”, “Since I use FreeBSD…”, or “On my Linux box…” as reasons for ignoring warnings and visiting the site regardless. There has always been a false sense of security from those with a greater expertise of operating systems and Internet usage. Yet, based on the study, even those experts miss the point of the security warnings.
As the paper says, “This indicates that either our metric for expertise needs to be improved, or that regardless of technical skills, many people are unable to distinguish between the various SSL warnings.”
While the online survey results were eye opening, the real tests would come from real-word use. So the lab results were what give this research a gold star if you will. The researchers put a good deal of effort into the report, going so far as to keep the 100 participants off guard so to speak.
“We tried to ensure that participants were not primed to think about security. The study was presented not as a security study, but as a ‘usability of information sources study.’ Our recruitment postings solicited people who were ‘CMU faculty staff or students’ and had ‘used online banking in the last year.’ However, we also required that participants have ‘purchased an item online in the last year’ and ‘used a search engine’ to avoid focusing potential participants on the banking tasks,” the researchers explained.
Since the largest fact remains that security certificates are misunderstood, the lab test focused on real-world usage by presenting participants with an online banking application and the CMU library catalog. During the test, participants were shown various certificate warnings, including ones created by the CMU researchers. The results, which will be discussed on August 14, were almost the same as the ones from the online survey. The users still ignored the warnings, placing themselves and any information they might submit at risk. The only plus was that the warnings designed by the researchers faired slightly better than the ones used today.
The paper itself is a well thought-out bit of research. Out of respect for the CMU team's work and presentation, we will not link to the paper here. It will be available on August 12 from the USENIX website. If you are interested in this topic, it’s a great read. (Thanks to Joshua Sunshine for sharing the paper with us.)