tags: all,

apache, apachecon, apacheweek, bryce, cve, fedora, fudcon, geocaching, gps, ha, jabber, metrics, microsoft, nashville, north carolina, oscon, red hat summit, security, trips

CVSS v2 Base Score 6.5 (AV:N/AC:L/Au:S/C:P/I:P/A:P)

This is really a moderate severity flaw because you need a remote attacker who has the ability to start/stop/control ZoneMinder, and you really should protect your ZoneMinder installation so you don't allow arbitrary people to control your security system. (Although I think at least one distributor package of ZoneMinder doesn't protect it by default, and you can find a few unprotected ZoneMinder consoles using a web search).

I discovered this because when we went on holiday early in April I forgot to turn down the heating in the house. Our heating system is controlled by computer and you can change the settings locally by talking to a Jabber heating bot (Figure 1). But remotely over the internet it's pretty locked down and the only thing we can access is the installation of ZoneMinder. So without remote shell access, and with an hour to spare at Heathrow waiting for the connecting flight to Phoenix, I figured the easiest way to correct the temperature was to find a security flaw in ZoneMinder and exploit it. The fallback plan was to explain to our house-minder how to change it locally, but that didn't seem as much fun.

So I downloaded ZoneMinder and took a look at the source. ZoneMinder is a mixture of C and PHP, and a few years ago I found a buffer overflow in one of the C CGI scripts, but as I use Red Hat Enterprise Linux exploiting any new buffer overflow with my ZoneMinder compiled as PIE definately wouldn't be feasible with just an hours work. My PHP and Apache were up to date too. So I focussed on the PHP scripts.

A quick grep of the PHP scripts packaged with ZoneMinder found a few cases where the arguments passed to PHP exec() were not escaped. One of them was really straightforward to exploit, and with a carefully crafted URL (and if you have authorization to a ZoneMinder installation) you can run arbitrary shell code as the Apache httpd user. So with the help of an inserted semicolon and one reverse shell I had the ability to remotely turn down the heating, and was happy.

I notified the ZoneMinder author and the various vendors shortly after and updates were released today (a patch is also available)

Figure 1: Local heating control

My full risk report was published yesterday in Red Hat Magazine and reveals the state of security since the release of Red Hat Enterprise Linux 4 including metrics, key vulnerabilities, and the most common ways users were affected by security issues.

"Red Hat knew about 49% of the security vulnerabilities that we fixed in advance of them being publicly disclosed. For those issues, the average notice was 21 calendar days, although the median was much lower, with half the private issues having advance notice of 8 days or less."

First thing to do was to find the original source of the advisory, as co-ordination centres and research firms are known to often play the Telephone game, with advisory texts mangled beyond recognition. Following the links led to the actual advisory on the HP site. This describes the vulnerability as follows:

But then they give the CVE name for the flaw, CVE-2007-6388, which is a known public flaw fixed last month in various Apache versions from the ASF and in updates from various vendors that ship Apache (including Red Hat).

This flaw is a cross-site scripting flaw in the mod_status module. Note that the server-status page is not enabled by default and it is best practice to not make this publicly available. I wrote mod_status over 12 years ago and so I know that this flaw is exactly how the ASF describes it; it definitely can't let a remote attacker execute arbitrary code on your Apache HTTP server, under any circumstances.

I fired off a quick email to a couple of contacts in the HP security team and they confirmed that the flaw they fixed is just the cross-site scripting flaw, not a remote code flaw. The CVSS ratings they give in their advisory are consistent with it being a cross-site scripting flaw too.

So happy with a false alarm we cancelled our Critical Action Plan and I went off and had a nice weekend practicing taking panoramas without a tripod ready for an upcoming holiday. My first attempt came out better than I expected:

Using our public tool, for every Red Hat product and service, for 2007 we issued 306 advisories to fix 404 vulnerabilities. Of those 404 vulnerabilities 41 were critical (on the scale used by Microsoft and Red Hat).

Most people are not going to be using every Red Hat product, so taking just Enterprise Linux product you find 348 vulnerabilities, of which 27 were critical. A given user is going to only be vulnerable to the issues that affect the products and packages they have installed. Using the scripts on our pages you can figure it out for your own circumstances. But as an example, the default installation of Red Hat Enterprise Linux 4 AS had 172 vulnerabilities of which 4 were critical.

The Secunia report does actually make it clear you can't use their vulnerability count as a method of comparing platforms, in part due to the differences in methodology of the vendors, but I'm sure this won't stop some press from jumping to conclusions if they don't read the actual report.

I've asked Secunia how they got to their number of vulnerabilities, but in the meantime, a raw count of vulnerabilities is only a small part of the overall risk exposure in using a product. I've got some more reports that go into this in more detail for two years of Enterprise Linux 4 and Enterprise Linux 5.0 to 5.1.

Update: Coverage of this: ZDNet

Update: Secunia told me that they treat each advisory separately; so for example yesterday we issued updates for some moderate severity issues in the Apache Web server, but we did separate advisories for each affected product: Red Hat Enterprise Linux 2.1, 3, 4, 5, Red Hat Application Stack v1, v2. So in this case the same Apache vulnerability would be counted 6 times.

Between releases there are lots of changes made to improve security and I've not listed everything; just a high-level overview of the things I think are most interesting that help mitigate security risk. We could go into much more detail, breaking out the number of daemons covered by the SELinux default policy, the number of binaries compiled PIE, and so on.

	Fedora Core						Fedora		Red Hat Enterprise Linux
	1	2	3	4	5	6	7	8	3	4	5
	2003Nov	2004May	2004Nov	2005Jun	2006Mar	2006Oct	2007May	2007Nov	2003Oct	2005Feb	2007Mar
Firewall by default	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
Signed updates required by default	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y	Y
NX emulation using segment limits by default	Y	Y	Y	Y	Y	Y	Y	Y	Y2	Y	Y
Support for Position Independent Executables (PIE)	Y	Y	Y	Y	Y	Y	Y	Y	Y2	Y	Y
Address Randomization (ASLR) for Stack/mmap by default3	Y	Y	Y	Y	Y	Y	Y	Y	Y2	Y	Y
ASLR for vDSO (if vDSO enabled)3	no vDSO	Y	Y	Y	Y	Y	Y	Y	no vDSO	Y	Y
Restricted access to kernel memory by default		Y	Y	Y	Y	Y	Y	Y		Y	Y
NX for supported processors/kernels by default		Y1	Y	Y	Y	Y	Y	Y	Y2	Y	Y
Support for SELinux		Y	Y	Y	Y	Y	Y	Y		Y	Y
SELinux enabled with targeted policy by default			Y	Y	Y	Y	Y	Y		Y	Y
glibc heap/memory checks by default			Y	Y	Y	Y	Y	Y		Y	Y
Support for FORTIFY_SOURCE, used on selected packages			Y	Y	Y	Y	Y	Y		Y	Y
All packages compiled using FORTIFY_SOURCE				Y	Y	Y	Y	Y			Y
Support for ELF Data Hardening				Y	Y	Y	Y	Y		Y	Y
All packages compiled with stack smashing protection					Y	Y	Y	Y			Y
SELinux Executable Memory Protection						Y	Y	Y			Y
glibc pointer encryption by default						Y	Y	Y			Y
FORTIFY_SOURCE extensions including C++ coverage								Y

¹ Since June 2004, ² Since September 2004, ³ Selected Architectures

The graph below shows the total number of security updates issued for Red Hat Enterprise Linux 5 Server up to and including the 5.1 release, broken down by severity. I've split it into two columns, one for the packages you'd get if you did a default install, and the other if you installed every single package (which is unlikely as it would involve a bit of manual effort to select every one). So, for a given installation, the number of packages and vulnerabilities will be somewhere between the two extremes.

So for all packages, from release up to and including 5.1, we shipped 94 updates to address 218 vulnerabilities. 7 advisories were rated critical, 36 were important, and the remaining 51 were moderate and low.

For a default install, from release up to and including 5.1, we shipped 60 updates to address 135 vulnerabilities. 7 advisories were rated critical, 26 were important, and the remaining 27 were moderate and low.

• These figures include ten updates we released on the day we shipped 5.0. This was because we froze package updates some months before releasing the product. Only one of those updates was rated critical, an update to Firefox.
• The six other critical updates were:
  1. Three more updates to Firefox (May, July, October) where a malicious web site could potentially run arbitrary code as the user running Firefox. Given the nature of the flaws, ExecShield protections in RHEL5 should make exploiting these memory flaws harder.
  2. An update to the Kerberos telnet deamon (April) A remote attacker who can access the telnet port of a target machine could log in as root without requiring a password. None of the standard protection mechanisms help prevent exploitation of this issue, however the krb5 telnet daemon is not enabled by default in Enterprise Linux 5 and the default firewall rules block remote access to the telnet port. This flaw did not affect the more common telnet daemon distributed in the telnet-server package.
  3. An update to Samba (May) where a remote attacker could cause a heap overflow. In addition to ExecShield making this harder to exploit, the impact of any sucessful exploit would be reduced as Samba is constrained by an SELinux targeted policy (enabled by default).
  4. An update to the PCRE library (November). This was labelled critical because the Konqueror web browser uses PCRE to handle regular expressions in JavaScript, and therefore a user browsing a malicious site in Konqueror could trigger this issue. (Konqueror is not part of a default install, but I've left this issue as critical in the results).

• Updates to correct all of these critical issues were available via Red Hat Network within a day of the issues being public.

Red Hat Enterprise Linux 5 shipped with a number of security technologies designed to make it harder to exploit vulnerabilities and in some cases block exploits for certain flaw types completely. For the period of this study there were two flaws blocked that would otherwise have required critical updates:

1. A stack buffer overflow flaw in the RPC library in Kerberos. This flaw was blocked by FORTIFY_SOURCE which removed the possibility of remote code execution. We still issued an update, as a remote attacker could trigger this flaw and cause Kerberos to crash.
2. Another flaw in Kerberos, this time due to the free of an invalid pointer. This flaw was blocked by glibc, although a remote attacker could still cause a crash, so we issued an update.

This data is interesting to get a feel for the risk of running Enterprise Linux 5 Server, but isn't really useful for comparisons with other versions or distributions -- for example, a default install of Red Hat Enterprise 4AS did not include Firefox. You can get the results I presented above for yourself by using our public security measurement data and tools, and run your own metrics for any given Red Hat product, package set, timescales, and severities.

Common Platform Enumeration (CPE) is a naming scheme designed to combat these inconsistencies, and is part of the 'making security measurable' initiative from Mitre. From today we're supporting CPE in our Security Response Team metrics: we publish a mapping of Red Hat advisories to both CVE and CPE platforms (updated daily) and you can use CPE to filter the metrics. Some examples of CPE names:

cpe://redhat:enterprise_linux:5:server/firefox -- the Firefox browser package on Red Hat Enterprise Linux 5 server.
cpe://redhat:enterprise_linux:3 -- Red Hat Enterprise Linux 3
cpe://redhat/xpdf -- the xpdf package in any Red Hat product.
cpe://redhat:rhel_application_stack:1 -- Red Hat Application Stack version 1

User reports a security vulnerability (this includes things later found not to be vulnerabilities)	47 (30%)
User is confused because they visited a site "powered by Apache" (happens a lot when some phishing or spam points to a site that is taken down and replaced with the default Apache httpd page)	39 (25%)
User asks a general product support question	38 (25%)
User asks a question about old security vulnerabilities	21 (14%)
User reports being compromised, although non-ASF software was at fault (For example through PHP, CGI, other web applications)	9 (6%)

That last one is worth restating: in the last 12 months no one who contacted the ASF security team reported a compromise that was found to be caused by ASF software.

The National Vulnerability Database provides a public severity rating for all CVE named vulnerabilities, "Low" "Medium" and "High", which they generate automatically based on the CVSS score their analysts calculate for each issue. I've been interested for some time to see how well those map to the severity ratings that Red Hat give to issues. We use the same ratings and methodology as Microsoft and others use, assigning "Critical" for things that have the ability to be remotely exploited automatically through "Important", "Moderate", to "Low".

Given a thundery Sunday afternoon I took the last 12 months of all possible vulnerabilities affecting Red Hat Enterprise Linux 4 (from 126 advisories across all components) from my metrics page and compared to NVD using their provided XML data files. The result broke down like this:

Red Hat	13% Crit 24% Important 39% Moderate 24% Low
NVD	30% High 20% Moderate 50% Low

So that looked okay on the surface; but the diagram above implies that all the issues Red Hat rated as Critical got mapped in NVD to High. But that's not actually the case, and when you look at the breakdown you get this result: (in number of vulnerabilities)

NVD: High

23 Critical 24 Important 35 Moderate 8 Low

NVD: Moderate

9 Crit 18 Important 22 Moderate 12 Low

NVD: Low

7 C 32 Important 62 Moderate 51 Low

That shows nearly half of the issues that NVD rated as High actually only affected Red Hat with Moderate or Low severity. Given our policy is to fix the things that are Critical and Important the fastest (and we have a pretty impressive record for fixing critical issues), it's no wonder that recent vulnerability studies that use the NVD mapping when analysing Red Hat vulnerabilities have some significant data errors.

I wasn't actually surprised that there are so many differences: my hypothesis is that many of the errors are due to the nature of how vulnerabilities affect open source software. Take for example the Apache HTTP server. Lots of companies ship Apache in their products, but all ship different versions with different defaults on different operating systems for different architecture compiled with different compilers using different compiler options. Many Apache vulnerabilities over the years have affected different platforms in significantly different ways. We've seen an Apache vulnerability that leads to arbitrary code execution on older FreeBSD, that causes a denial of service on Windows, but that was unexploitable on Linux for example. But it has a single CVE identifier.

So if you're using a version of the Apache web server you got with your Red Hat Enterprise Linux distribution then you need to rely on Red Hat to tell you how the issue affects the version they gave you -- in the same way you rely on them to give you an update to correct the issue.

I did also spot a few instances where the CVSS score for a given vulnerability was not correctly coded. CVSS version 2 was released last week and once NVD is based on the new version I'll redo this analysis and spend more time submitting corrections to any obvious mistakes.

But in summary: for multi-vendor software the severity rating for a given vulnerability may very well be different for each vendors version. This is a level of detail that vulnerability databases such as NVD don't currently capture; so you need to be careful if you are relying on the accuracy of third party severity ratings.