Security Patch Management Tools
On April 13, 2004—a day that’s become known to some as “Patch Tuesday”—Microsoft released a collection of four security bulletins that patched an aggregate total of 21 vulnerabilities. Among the items patched was a vulnerability in the Local Security Authority Subsystem Service (LSASS, which runs in Task Manager’s Processes view as LSASS.EXE). About three weeks later, during the weekend of May 1, several variants of the now-infamous Sasser worm, which exploited this very vulnerability, started to appear. To its credit, Microsoft immediately posted warnings and a repair tool, but also reported more than a million and a half downloads of the tool by May 3. Other virus-monitoring services and security sites estimate that somewhere between half a million and a million systems have been infected with Sasser so far.
What happened? Simply put, though Microsoft released the patch with its highest severity rating—“Critical”—on April 13, many systems or network administrators hadn’t yet gotten around to installing update 835732 (documented in Security Bulletin MS04-011) by the time the Sasser worm started spreading. Antivirus and security experts have observed that the gap between identification of vulnerabilities (which often occurs before a patch is released, if not at the same time) and exploits that use them to attack or infect systems is growing smaller all the time. Certainly, the speed of Sasser’s spread–and the many infections it caused–should make the point that patching systems is indeed a critical activity, as other similar infestations like the MS Blaster virus, MyDoom, Netsky, Welchia, Beagle and so many others have already shown, albeit with a longer lead time between vulnerability reports and vulnerability exploits.
That’s what makes security patch management tools so important for organizations with more than a handful of systems to manage. Although many smaller outfits and individuals can use the Windows Update service and install updates one system at a time, needs for testing, system image controls and orderly deployment require bigger outfits to plan ahead and automate the patch or update process as much as possible. Modern patch or update management tools not only collect and archive updates or patches for easy identification and access, they also permit authorized administrators to schedule and stage deployment and installation of such items as best fits organizational needs. At the same time, however, it’s also important to use Group Policy Objects (GPOs) or other security controls within the organization to block individual access to Windows Update (or other similar patch or update delivery services)—assuming that organizations want to control where such updates come from and when they’re applied. This is necessary in some organizations. In others, desktop updates may not be so rigorously controlled, and only servers or other important production machines should be subject to such restraints.
In fact, security patch management tools fall under the more general heading of update management tools. This is a heading under which systems administrators can find many options to assist them with the process of identifying, prioritizing, testing, deploying and otherwise managing security patches, fixes, updates and so forth. (See “Top Security Patch Management Tools” for pointers to some leading solutions.) One key issue to which all readers must be sensitive is that of unintended side effects from patches—not only because of possible consequences, but also to explain why it’s not always a good idea to install every security patch as soon as it appears. Sometimes, though a patch may block or disable potential security exploits or vulnerabilities, it may also prevent other applications from working properly or cause other problems in complex software environments that may have nothing obvious to do with the security problem being addressed.
This explains why prioritizing and testing patches is an important part of deciding whether to deploy them or not. On the one hand, priority analysis may indicate that a patch isn’t needed because it would affect services or behaviors that aren’t used on systems and therefore don’t need patching. Careful analysis is required here, since vulnerabilities in seldom- or never-used system capabilities remain a favorite way for savvy malefactors to attack and compromise systems. On the other hand, testing patches in a working model of a production environment is equally important, since it will indicate if the patch affects system behaviors, services or code negatively. Sometimes, companies or organizations must perform hurry-up risk assessments when they learn that a patch that fixes a real security threat also hampers or knocks out important system services or behaviors. They may have to decide if the costs of a security breach exceed the costs associated with loss or harm to other key services and capabilities, or vice versa, before they can decide whether to patch or not. Organizations that can’t apply patches for such reasons can take other steps to mitigate vulnerabilities or block access to vulnerable systems. Deciding what to do and taking care of necessary access controls or system alterations also takes time, costs money and factors into overall risk analysis as well.
There are some companies that offer patch management services to customers—most notably BigFix. This business model lets customers turn over patch monitoring, relevance assessments and some testing to responsible third parties. Customers simply subscribe to the service, describe their systems and configurations and depend on the service to send them patches for further testing and deployment. This probably appeals most to organizations that may not have sufficient staff or resources to do the entire job themselves, but BigFix boasts some enterprise-class customers, so this offering seems worthy of further consideration and investigation.
Other key activities in the patch management process include:
- Maintaining a database of current, applicable patches and updates (the entire collection of current patches or updates potentially applicable to all systems).
- Associating appropriate or necessary patches with specific systems (the subset of the collection that applies to any specific system).
- Creating patch packages and defining necessary delivery and installation mechanisms. (This is generally automated inside most patch or update management systems, but requires administrators to create scripts or other forms of instruction.) This can (and often should) include performing pre-installation system backups to make sure clean rollback is possible.
- Testing systems post-installation to make sure patches are installed properly and systems are working properly. (This is usually automated as part of the post-installation cleanup.)
- Providing documentation for installation procedures, configuration changes and new software components. (Support for this capability varies from tool to tool and may require additional work.)
- As an optional activity, providing a rollback mechanism to reverse applied changes, should updates or patches need removal later. (Some systems support this automatically, some don’t.)
Whether patching or updating systems is manual or automated through software of some kind or another, conventional wisdom regarding such activities still applies. It’s essential to test patches or updates in non-production environments first, to check for unwanted or unforeseen side effects. It’s also highly recommended that any system about to be patched or updated be backed up first to make sure that it’s possible to return to a known-good working configuration should something go wrong during the patching or updating process.