Patching – an often ignored, yet vital component of operating a predictable, manageable storage and IT environment - is particularly important for regulated businesses, such as biotechnology and research organizations, running validated systems.
For those who don't know, patching is the process of applying updates to a device or a system. Patches are referred to by several terms including security updates, hot fixes, service packs, flash updates and others. They are issued for several purposes, such as stability, feature update, and scheduled update, or more critically, to mitigate exposure to security vulnerabilities. What all patches share, regardless of their name and purpose, is the need to resolve a known issue.
Patching is tedious, time-consuming, and for the most part, a completely manual process. Compounding the issue, patches are rarely planned or budgeted; rather it is assumed to be a standard responsibility of the IT organization. Unfortunately, the majority of devices, operating systems and applications in the computing infrastructure will more than likely require patching at some point in time, some on a regular basis. The time investment to stay current with all patches precludes the expectation that it be a standard unplanned task.
Now is the time to take a proactive approach to managing patches. Although it might be disheartening to realize that such a huge requirement has basically been overlooked, there is a solution. Moreover, what is reassuring is the fact that the process itself is relatively simple.
Identify categories
Getting started may be the hardest part, considering the amount of devices and infrastructure components that may need patching. Start small and identify categories of the infrastructure that may require patching, both hardware and software. Include items such as firmware, operating systems, applications, network devices, storage devices, and peripherals. From these categories, perform an inventory of your environment, expanding each category to identify entities that may require patching. For example, networking device entities may include firewalls, routers, switches, etc; operating systems may include Windows and Unix systems. Prioritize the categories and the entities based on your organization's threshold for exposure to risk for each of these categories. Risk should be measured based on the significance of the risk - if the vulnerability occurred, how bad would it be, as well as on the likelihood of risk - how likely is it that the vulnerability will occur.
Create a supportability matrix
For each prioritized entity, gather pertinent information to build a supportability matrix to facilitate the patching effort. The supportability matrix will define criteria on how and when the entity should be patched based on several factors, such as the vendors' preferred patching method; supportability with other software versions and service packs; testing criteria to verify entity stability; and contact information, both for the vendors as well as people internally who will assume responsibility for the entity. Any known interactions with other entities must also be included; for instance, does one software patch negatively impact another application? Once the supportability matrix is complete, develop test scripts that can be executed against the entity to verify stability, leveraging the criteria defined above.
Test three times
Once the scripts have been developed, the testing phase can begin. It is preferable to test on an image or copy of an entity rather than on one in production. Use the Business Continuance Volume (BCV) capability provided by many leading storage vendors if available. Best practices suggest that testing is performed three times: before the patch has been applied to verify that the entity is stable to begin with; after the patch has been applied to verify that the patch didn't affect stability; and then once again after the patch has been rolled back to verify that if the patch needs to be rolled back, it can be without negatively impacting the entity.
Following successful testing, notify the groups who will be impacted by the patch, and after that, the patch can be rolled out successfully.
