Windows vs Linux security report card redux

morpheusv6 · 09-06-2007, 03:07 AM

source: http://blogs.zdnet.com/security/?p=268&tag=nl.e622

Orlando, Florida — Jeff Jones has expanded his project to count security flaws (publicly reported and fixed) in the major workstation operating systems and his latest numbers show Windows Vista has by far the best security profile when compared to the major Linux distributions.
Jeff Jones, security strategy director in Microsoft’s Trustworthy Computing group, led a TechEd 2007 discussion on the metrics and techniques used to keep track of vulnerabilities and offered a glimpse at his upcoming report card that compares flaws found/fixed during Vista’s first six months on the market against Windows XP, Red Hat Enterprise Linux 4 WS (full), Ubuntu 6.06 LTS (full), Novell SUSE Linux Enteprise Desktop 10 (full) and Mac OS X 10.4 (Tiger).
Here’s a chart from Jones with the results, which will be revealed in full in a few weeks:

Jones uses data from several public databases and vendor security bulletins to track “days of risk” and actual flaws being reported and patched to determine which workstation OS could be considered safer.
[ SEE: 90-day report card: Windows Vista fared better than competitors ]

He explained the difficulties — and dangers — associated with trying to get an accurate picture of the flaw landscape because of the different ways that vendors release flaw information in advisories and suggested that the NIST’s NVD (National Vulnerability Database) does the best job of aggregating flaw information across the board. Still, he warned against using the NVD as a foolproof database because it’s “only accurate for certain things.”
Jones also discussed some problems with rating the severity of reported flaws since all vendors use different rating systems. Some vendors, like Apple, offer no rating whatsoever, putting the counting/rating game into a bit of a subjective twist.
During a Q&A session, Jones provided a clue as to why Microsoft does not use the CVSS (Common Vulnerability Scoring System) to rate flaws in its bulletins, describing the methodology as confusing.
He made it clear he was expressing his personal opinion (not Microsoft’s official take on CVSS) before picking apart what he perceives as weaknesses in the system currently being used by Cisco, Oracle and several big-name vulnerability research firms.
“I don’t agree with how CVSS works,” Jones said. “I believe a rating system should provide practical usefulness for making decisions and CVSS doesn’t do that in all cases,” he added.
Specifically, Jones pointed out that the middle-range scores offered by CVSS can be interpreted differently. “I think a CVSS 10.0 is probably a 10.0 and a 2.0 or 3.0 is probably a low-risk issue. But, everywhere in the middle, it becomes much less definitive and confusing,” he added.

90-day report card: Windows Vista fared better than competitors

Ninety days after the release of Microsoft's Windows Vista to business customers, the new operating system has a much better security vulnerability profile than its predecessor and several other modern workstation operating systems including Red Hat, Ubuntu, Novell and Apple products.
That's according to Jeff Jones, security strategy director in Microsoft's Trustworthy Computing group.

Jones has published a 90-day report card (.pdf), stacking up flaws reported and fixed in Vista against vulnerabilities covering during the first 90 days of Windows XP, Red Hat Enterprise Linux 4 WS, Ubuntu 6.06 LTS, Novell SUSE Linux Enteprise Desktop 10 and Mac OS X 10.4 (Tiger).
During the period under review, Jones said Microsoft shipped a solitary security bulletin affecting Vista users — MS07-010, which covered a remotely exploitable hole in the Microsoft Malware Engine. He also called attention to four other reported Vista bugs that remain unpatched, one carring a "high risk" rating.
By comparison, during the first 90 days after Windows XP shipped, Jones research showed that Microsoft patched a total of 14 vulnerabilities, 8 rated critical. "At the end of the 90 day period, a total of 4 publicly disclosed [Windows XP] vulnerabilities did not yet have a patch available from Microsoft," Jones said.
Regarding Red Hat Enterprise Linux 4 Workstation (rhel4ws), Jones said the open-source vendor fixed a total of 181 vulnerabilities, 58 rated "high severity" by the U.S. governments National Vulnerability Database. He acknowledged that many of these bugs covered components that Red Hat ships and supports as Red Hat Enterprise Linux 4 WS, noting that it might be construed as "unfair" to count those.
However, even with RHEL4WS reduced component set, Jones said:
The reduced rhel4ws set of components had 86 vulnerabilities already publicly disclosed prior to general availability. Patches available on the first day of ship addressed 34 of these.

During the first 90 days, Red Hat fixed 137 vulnerabilities affecting the reduced rhel4ws set of components. 40 of those addressed were High severity.
At the end of the 90 day period, a total of 64 publicly disclosed vulnerabilities in the reduced set of components did not yet have a patch from Red Hat.

In the first 90 days after Apple's Mac OS X v10 shipped, Jones showed that Windows Vista fared much better, arguing that the data does not support Apple's marketing stance that the Mac OS X does not have the same security issues that face other operating systems.
Specifically, Jones reported that:

Mac OS X v10.4 had 10 vulnerabilities already publicly disclosed prior to the April 29, 2005 ship date and Apple provided fixes for 4 of these during the first 90 days after ship. Four of the vulnerabilities were High severity.
During the first 90 days, Apple fixed a total of 20 vulnerabilities affecting Mac OS X v10.4, of which 8 were rated High severity in the NVD.
At the end of the 90 day period, there Mac OS X v10.4 still had 17 publicly disclosed vulnerabilities that did not yet have a patch from Apple.

He also provided comparable numbers for Ubuntu 6.06 LTS and Novell's SUSE Linux Enterprise Desktop 10 (SLED10) to show that Vista's security vulnerability profile was noticeably better.