From time to time I publish metrics on vulnerabilities that affect Red Hat Enterprise Linux. One of the more interesting metrics looks at how far in advance we know about the vulnerabilities we fix, and from where we get that information. This post is abstracted from the upcoming "4 years of Enterprise Linux 4" risk report
For every fixed vulnerability across every package and every severity in Enterprise Linux 4 AS in the first 4 years of its life, we determined if the flaw was something we knew about a day or more in advance of it being publicly disclosed, and how we found out about the flaw.
For vulnerabilities which are already public when we first hear about them we still track the source as it's a useful internal indicator on where the security response team should focus their efforts.
So from this data, Red Hat knew about 51% of the security vulnerabilities that we fixed at least a day in advance of them being publicly disclosed. For those issues, the average notice was 21 calendar days, although the median was much lower, with half the private issues having advance notice of 9 days or less.
Red Hat Enterprise Linux 5.3 was released today, around 8 months since the release of 5.2 in May 2008. So let's use this opportunity to take a quick look back over the vulnerabilities and security updates we've made in that time, specifically for Red Hat Enterprise Linux 5 Server.
The chart below shows the total number of security updates issued for Red Hat Enterprise Linux 5 Server as if you installed 5.2, up to and including the 5.3 release, broken down by severity. I've split it into two columns, one for the packages you'd get if you did a default install, and the other if you installed every single package (which is unlikely as it would involve a bit of manual effort to select every one). So, for a given installation, the number of packages and vulnerabilities will probably be somewhere between the two.
So for a default install, from release of 5.2 up to and including 5.3, we shipped 45 advisories to address 127 vulnerabilities. 7 advisories were rated critical, 21 were important, and the remaining 17 were moderate and low.
For all packages, from release of 5.2 to and including 5.3, we shipped 61 advisories
to address 181 vulnerabilities. 7 advisories were rated critical, 28 were
important, and the remaining 26 were moderate and low.
The 7 critical advisories were for just 3 different packages:
- Five updates to Firefox (July, July, September, November, December) where a malicious web site could potentially run arbitrary code as the user running Firefox. Given the nature of the flaws, ExecShield protections in RHEL5 should make exploiting these memory flaws harder.
- An update to Samba (May), where a remote attacker who can connect and send a print request to a Samba server could cause a heap overflow. The Red Hat Security Response Team believes it would be hard to remotely exploit this issue to execute arbitrary code due to the default enabled SELinux targeted policy and the default enabled SELinux memory protection tests. We are not aware of any public exploit for this issue.
- An update to OpenSSH (August), provided to mitigate an intrusion into certain Red Hat computer systems. The attacker was able to sign a small number of tampered packages but they were not distributed on the Red Hat Network. We classified this update as critical to ensure any tampered packages would be replaced with official packages.
Although not of critical severity, also of interest during this period were the spoofing attacks on DNS servers. We provided an update to BIND (July) adding source port randomization to help mitigate these attacks.
Updates to correct all of these critical vulnerabilities (as well as migitate the BIND issue) were available via Red Hat Network either the same day, or one calendar day after the issues were public.
In fact for Red Hat Enterprise Linux 5 since release and to date, every critical vulnerability has had an update available to address it available from the Red Hat Network either the same day or the next calendar day after the issue was public.
To compare this with the last updates we need to take into account that the time between each update is different. So looking at a default installation and calculating the number of advisories per month gives the following chart:
Red Hat Enterprise Linux 5 shipped with a number of security technologies designed to make it harder to exploit vulnerabilities and in some cases block exploits for certain flaw types completely. For 5.2 to 5.3 there were two flaws blocked that would otherwise have required updates:
- A double-free flaw in unzip. The glibc pointer checking limited the exploitability of this issue to just a crash of unzip, a client application, which does not have security implications. No security update was needed.
- Two format string flaws in c++filt. The format string protection caused these issues to have no security implications. No security update was needed.
This data is interesting to get a feel for the risk of running Enterprise Linux 5 Server, but isn't really useful for comparisons with other versions, distributions, or operating systems -- for example, a default install of Red Hat Enterprise Linux 4AS did not include Firefox, but 5 Server does. You can use our public security measurement data and tools, and run your own custom metrics for any given Red Hat product, package set, timescales, and severity range of interest.
See also:5.1 to 5.2 risk report
We always have a moose theme for Christmas. Tracy bought some small cute wood moose decorations from John Lewis online this year for the tree. The John Lewis Glasgow store had a bit of a moose theme too, with Christmas displays made up of much larger wood versions of the decorative moose. The floor manager said they'd usually get thrown away or sold to staff at the end of the event.
A moose is for life, not just for Christmas. Tracy managed to arrange to buy some from the store to save them being thrown away. So we are now the proud owners of 6 large and 6 medium sized wooden moose.
Perhaps we can find some way to integrate them into the decorations for our wedding next year.
Secunia collect some very interesting information about the patch state of Windows systems. Their results from 20,000 machines published yesterday were that over 98% of PCs were insecure, having at least one out-of-date application installed.
Actually this isn't surprising and is exactly what I'd expect; it's all down to third party applications.
Let's say you're browsing the web. It's more than likely that at some point you'll want to view some PDF files, watch some Flash content, or play a Java game. Those tasks are all dealt with by third party applications, although to the end user it's all part of the browser experience. Since your system is only as secure as its weakest link, you need to manage security updates for those third party applications just as carefully as you manage security updates for the rest of your system. That's why Adobe Reader, Java, Flash, and all the myriad of other applications you've installed in order to make your system useful have their own update mechanisms. Some applications on Windows will 'phone home' when they are run and check to see if they need to be updated, others deploy services that sit in the background looking for updates from time to time, others even check every time your system starts. Many don't get automated updates at all.
How do you deal with all that risk? I believe it's possible by providing an OS distribution which includes all the bits you'll likely need to make a useful computing environment, thereby taking away that update uncertainty. Red Hat ship several PDF viewers in our distributions for example, but we also ship (in an Extras channel) Adobe Reader. Our Security Response Team are monitoring for security issues in everything we ship, all the third party applications, and providing a single point of contact, a single notification system, and a single way to get the updates.
If Microsoft knew that say 25% of all their users installed Firefox, wouldn't they be better bundling it and providing their centralised automated updates for it, to reduce their customers overall risk? They do already bundle some third party applications, although it's been with mixed success as we found 3 years ago when they didn't provide security fixes for bundled Flash (ZDNet coverage).
This is, in part, why you've not seen me respond recently to the Vista security reports which compare vulnerability counts. In these reports they use a cut-down minimal Red Hat Enterprise Linux installation in order to make it look more like Windows for the comparisons. But this is completely backwards -- the fact that we're including and fixing the flaws using a common process in so much third party software is actually helping reduce the risk and protect real customers. For example we could easily cut our vulnerability count by shipping only one PDF viewer instead of four. But if we know that these other viewers are going to get installed by the customer anyway all we've done is to hide the vulnerability count elsewhere, and you've made the customers overall risk increase.
So it may seem counter-intuitive but we should ship as much third party applications (that we know people use) as we can, because a single managed security update and notification process will decrease a users overall risk. The fewer third party applications that users have to get from elsewhere and install and manage for themselves the better in my opinion.
Red Hat Enterprise Linux 5.2 was released last week, around 6 months since the release of 5.1 in November 2007. So let's use this opportunity to take a quick look back over the vulnerabilities and security updates we've made in that time, specifically for Red Hat Enterprise Linux 5 Server.
The graph below shows the total number of security updates issued for Red Hat Enterprise Linux 5 Server starting at 5.1 up to and including the 5.2 release, broken down by severity. I've split it into two columns, one for the packages you'd get if you did a default install, and the other if you installed every single package (which is unlikely as it would involve a bit of manual effort to select every one). So, for a given installation, the number of packages and vulnerabilities will probably be somewhere between the two.
So for a default install, from release of 5.1 up to and including 5.2, we shipped 46 updates to address 119 vulnerabilities. 8 advisories were rated critical, 24 were important, and the remaining 14 were moderate and low.
For all packages, from release of 5.1 to and including 5.2, we shipped 62 updates to address 179 vulnerabilities. 9 advisories were rated critical, 29 were important, and the remaining 24 were moderate and low.
The nine critical updates were in five different packages:
- Four updates to Firefox (November, February, March, April) where a malicious web site could potentially run arbitrary code as the user running Firefox. Given the nature of the flaws, ExecShield protections in RHEL5 should make exploiting these memory flaws harder.
- An update to the GnuTLS library (May), where
a remote attacker who can connect to a server making use of GnuTLS could
cause a buffer overflow. In Red Hat Enterprise Linux 5, the CUPS print
server uses GnuTLS.
- An update to MIT Kerberos (March),
where a remote attacker who can conect to the krb5kdc or kadmind
services could cause a buffer overflow.
- An update to OpenPegasus
(January), where
a remote attacker who can connect to OpenPegasus could cause a buffer overflow.
The Red Hat Security Response Team believes that it would be hard to remotely
exploit this issue to execute arbitrary code, due to the default SELinux
targeted policy, and the default SELinux memory protection tests.
- Two updates to Samba (November, December) where a remote attacker who can connect to the Samba port could cause buffer overflows. In addition to ExecShield making this harder to exploit, the impact of any sucessful exploit would be reduced as Samba is constrained by an SELinux targeted policy (enabled by default).
Updates to correct all of these critical issues were available via Red Hat Network either the same day, or one calendar day after the issues were public.
To get a better idea of risk we need to look not only at the vulnerabilities but also the exploits written for those vulnerabilities. A proof of concept exploit exists publicly for one of the Samba flaws, CVE-2007-6015, but we are not aware of public exploits for any other of those critical vulnerabilities. Also of high risk was an important "zero-day" exploit affecting the Linux kernel where a local unprivileged user could gain root privileges. Red Hat Enterprise Linux 5.1 was affected and a fix was available two calendar days after public disclosure.
Red Hat Enterprise Linux 5 shipped with a number of security technologies designed to make it harder to exploit vulnerabilities and in some cases block exploits for certain flaw types completely. For the period of this study there were two flaws blocked that would otherwise have required updates:
- A double-free flaw in CUPS. The glibc pointer checking limited the exploitability of this issue to just a crash of CUPS and not the ability to execute arbitrary code. code execution. We still issued an update, as a remote attacker could trigger this flaw and cause CUPS to crash.
- An uninitialized pointer free flaw in unzip, caught by the glibc pointer checking. As exploitation of this flaw results in just a crash of a user application, no updates were needed.
This data is interesting to get a feel for the risk of running Enterprise Linux 5 Server, but isn't really useful for comparisons with other versions or distributions -- for example, a default install of Red Hat Enterprise 4AS did not include Firefox. You can get the results I presented above for yourself by using our public security measurement data and tools, and run your own custom metrics for any given Red Hat product, package set, timescales, and severities.
See also 5.0 to 5.1 risk report
ZoneMinder is an amazing Linux video camera security and surveillance application I use as part of my home automation system. ZoneMinder prior to version 1.23.3 contains unescaped PHP exec() calls which can allow an authorised remote user the ability to run arbitrary code as the Apache httpd user (CVE-2008-1381)
CVSS v2 Base Score 6.5 (AV:N/AC:L/Au:S/C:P/I:P/A:P)
This is really a moderate severity flaw because you need a remote attacker who has the ability to start/stop/control ZoneMinder, and you really should protect your ZoneMinder installation so you don't allow arbitrary people to control your security system. (Although I think at least one distributor package of ZoneMinder doesn't protect it by default, and you can find a few unprotected ZoneMinder consoles using a web search).
I discovered this because when we went on holiday early in April I forgot to turn down the heating in the house. Our heating system is controlled by computer and you can change the settings locally by talking to a Jabber heating bot (Figure 1). But remotely over the internet it's pretty locked down and the only thing we can access is the installation of ZoneMinder. So without remote shell access, and with an hour to spare at Heathrow waiting for the connecting flight to Phoenix, I figured the easiest way to correct the temperature was to find a security flaw in ZoneMinder and exploit it. The fallback plan was to explain to our house-minder how to change it locally, but that didn't seem as much fun.
So I downloaded ZoneMinder and took a look at the source. ZoneMinder is a mixture of C and PHP, and a few years ago I found a buffer overflow in one of the C CGI scripts, but as I use Red Hat Enterprise Linux exploiting any new buffer overflow with my ZoneMinder compiled as PIE definately wouldn't be feasible with just an hours work. My PHP and Apache were up to date too. So I focussed on the PHP scripts.
A quick grep of the PHP scripts packaged with ZoneMinder found a few cases where the arguments passed to PHP exec() were not escaped. One of them was really straightforward to exploit, and with a carefully crafted URL (and if you have authorization to a ZoneMinder installation) you can run arbitrary shell code as the Apache httpd user. So with the help of an inserted semicolon and one reverse shell I had the ability to remotely turn down the heating, and was happy.
I notified the ZoneMinder author and the various vendors shortly after and updates were released today (a patch is also available)
Figure 1: Local heating control
So if you're wondering why I've not bloged in a while it's because we're just back from holiday, the first in a few years. It was pretty eventful; I got engaged to Tracy at Shoshone Point at the Grand Canyon, we saw Spamalot in Vegas, and went to see Rocco Deluca play live in LA (Tracy even managed to get a photo with her favourite actor Kiefer Sutherland who turned up to watch).
I'm out on holiday soon to Arizona, so we've been looking for ways to geocode the photos we'll be taking and get a record of our route. I use a Mio A701 phone which has built-in GPS, and this time we'll be using Tom Tom in the USA rather than Mapopolis. The problem with Tom Tom Navigator is that it doesn't keep a track log, and there doesn't seem to be any plugins to allow it to do so. So here is the solution I've been experimenting with over the weekend.
On the PocketPC:
- Make sure the GPS Intermediate Driver is enabled, on the MIO there is a built-in "GPS Settings" utility where I have it set to COM4 and "Manage GPS automatically"
- Use the GPS2Blue utility. Make sure it's set to GPS on COM4, 4800 baud, with logging only of GGA/GLL/RMC/VTG NMEA, and select 'Log processed raw data...'. You don't need to enable the "2blue" bit, we're just using it to write the tracklog.
- Make sure your camera has a date and time that is close to the one being shown by GPS2Blue from the satellites
- Start TomTom. Make sure it's also set to COM4, 4800 baud. This will work because the GPS Intermediate Driver is opened by GPS2Blue. You can't start TomTom first, but you can exit GPS2Blue and leave TomTom running.
- After finishing you end up with a NMEA track log with an hour of logging taking up about 1.6Mb. Transfer it to your Fedora machine.
On my Fedora machine:
- Use gpsbabel to convert the NMEA
track log and clean it up a bit. I used:
gpsbabel -i nmea -f GPS_2008-03-03_122630.log -x discard,hdop=10,sat=5 -o gpx -F out.gpx
- Use gps2photo.pl to
add the geocoding to your images. This script looks at the time and date the
photo was taken and tries to match it up to an entry in the tracklog, so you
may need to play with the timeoffset to
deal with timezone differences. Although we have snow, being in the UK in the Winter has
it's advantages as we're UTC+0, so I just used:
gpsPhoto.pl --geoinfo=osm --dir ./ --gpsfile out.gpx --timeoffset 0 \ --city=auto --sublocation=auto --state auto --country auto --kml out.kml
The exif metadata inside each jpeg now contains the approximate co-ordinates of where you were when you took the photo along with a guess of the location (city, country, etc). You can load out.kml into GoogleEarth to see the tracklog and photos on a map. If you've allowed Flickr to read the location data from exif then uploading a geotagged photo will automatically place it on a map. (Make sure you consider the consequences before enabling that option or you may end up unintentionally leaking information like the location of your friends houses or parties you've been to). Here's a quick pic taken in the snow today to test it out:
| output from exiftool:
GPS Position : 55 deg 46' 58.21" N, 4 deg 0' 5.50" W City : Motherwell Province-State : Scotland Country-Primary Location Name : United Kingdom |
It sometimes seems like the Security Response Team at Red Hat are pushing security updates every day, but actually a default installation of Enterprise Linux 4 AS was vulnerable to only 7 critical security issues in the first three years since release. But to get a picture of the risk you need to do more than count vulnerabilities.
My full risk report was published yesterday in Red Hat Magazine and reveals the state of security since the release of Red Hat Enterprise Linux 4 including metrics, key vulnerabilities, and the most common ways users were affected by security issues.
"Red Hat knew about 49% of the security vulnerabilities that we fixed in advance of them being publicly disclosed. For those issues, the average notice was 21 calendar days, although the median was much lower, with half the private issues having advance notice of 8 days or less."
Last Friday, just as I was finishing work for the day, an email appeared in my mailbox from the UK CPNI announcing a public remote code execution flaw in Apache on HP-UX. As Chair of the Apache Software Foundation Security Team I knew there were no outstanding remote code execution flaws in Apache HTTP server (in fact we've not had a remote code execution flaw for many years) so I was expecting to invoke the Red Hat Critical Action Plan which would have meant a rather long weekend for me, my team, and various development and quality engineering staff.
First thing to do was to find the original source of the advisory, as co-ordination centres and research firms are known to often play the Telephone game, with advisory texts mangled beyond recognition. Following the links led to the actual advisory on the HP site. This describes the vulnerability as follows:
But then they give the CVE name for the flaw, CVE-2007-6388, which is a known public flaw fixed last month in various Apache versions from the ASF and in updates from various vendors that ship Apache (including Red Hat).
This flaw is a cross-site scripting flaw in the mod_status module. Note that the server-status page is not enabled by default and it is best practice to not make this publicly available. I wrote mod_status over 12 years ago and so I know that this flaw is exactly how the ASF describes it; it definitely can't let a remote attacker execute arbitrary code on your Apache HTTP server, under any circumstances.
I fired off a quick email to a couple of contacts in the HP security team and they confirmed that the flaw they fixed is just the cross-site scripting flaw, not a remote code flaw. The CVSS ratings they give in their advisory are consistent with it being a cross-site scripting flaw too.
So happy with a false alarm we cancelled our Critical Action Plan and I went off and had a nice weekend practicing taking panoramas without a tripod ready for an upcoming holiday. My first attempt came out better than I expected:
