Changes, Incidents and Unintended Consequences Index
The changes that are made to the IT environment – whether authorized or not – can result in incidents. These incidents can quickly spiral out of control and cause a major impact on the company. This is common knowledge in IT, but often the evidence around this relationship is not reliable.
Jason Druebert [ITSM] has designed an approach to get a feel for incidents in organizations, calling it the 'Unintended Consequences Index.'
On this chart you plot on the same axis your total number of major incidents per month (going back six months to a year) and the total number of change records per month for the same period.
Nine times out of ten, the graph looks something like this:
Unintended Consequences Index by Jason Druebert
You can take this as a metric and track the number of incidents related to changes. More often, the graphical outcome has a similar shape.
Jason Druebert* has presented a graph like the one above, and people were taken aback. "I have even been accused (half-jokingly) of doctoring the data to make a point."
Jason Druebert has "found that this relationship holds true even for companies that have good change Management by ITIL standards—regular CAB meetings, minimal unauthorized changes, and a well-defined approval processes.
So, if a company is doing change control 'by the book,' why would they see this relationship?"
Below is his list of the most common reasons:
- Evaluation of changes is based on procedural correctness as opposed to potential risk and impact.
- Information needed to properly evaluate the potential impact of a change is unavailable or inaccurate.
- Inadequately tested changes and Releases; often because of a lack of suitable non-production testing environments or a perceived lack of time.
What to do About Changes
This graph shows what many IT professionals usually have a hunch about—changes often result in unintended consequences.
The next question is what to do about changes (desired or not) in your IT environment?
Proactive – Proactively identify undesired changes and difference, before they turn into environment incidents. As part of the proactive problem management process, discover the critical and non-critical changes before they turn into incidents.
Share the Information – When you discover the changes, Tier I personnel should use the detailed information to identify the specific change area that could be the trigger for an incident. They can hand off appropriate detailed information to relevant Tier II and Tier III specialists that can use it to decide how the changes will have an impact on the environment.
Validate Release Transition – With application deployment and software deployment, say from Test Acceptance to Production, changes are implemented into your IT environment. Too often it is 'Release and Pray'. What can you do to ensure that the changes are transitioned correctly? You need to assure integrity in environments, by providing visibility deep into the release content to see even the most minute changes and thus ensure that the release is transitioned as expected.
Major Incident Reviews – In Incident Management, once a major Incident has been resolved, what are you doing to keep similar incidents from occurring again? You need to be able to sit down in a your group, and closely examine what happened. This means , know what were the critical changes that initiated the incident. Otherwise don't be surprised if history repeats itself. This should also be part of the problem management process.
Your first step should be to keeping track of all the changes in your organization, and analyzing them for criticality. Jason Druebert notes "When there is discussion of plan, do, check, act, we all nod our heads in agreement. In most cases, however, an organization's quality circle is heavily weighted to the "plan, do" side and flat on the 'check, act' side."
Many IT groups are in recovery mode, where they are maintaining their current environments based on limited budgets. Since departments are strained for resources, IT professionals need to maintain their complex organization even if staffing is an issue. The variety of applications and technologies comprising IT environments have thousands of individual configuration parameters. It used to be that you could just ask the IT manager about the system – the IP address, software installed, or patches – and he could answer off the top of his head. IT administrators aren't like that today. Today's level of complexity, makes it very hard to maintain a sense of control. Yet you are still being held accountable for thousands and thousands of configuration parameters.
The ability to implement timely changes for business needs without negative impact is the goal of the ITIL process. To succeed in this mission, you need to start by figuring out what changes are in your environment and gauging them for criticality and impact to IT.
To learn how you can gain visibility and control over the vast configuration parameters in your IT environment, see the key capabilities for Evolven's Change Management and Configuration Management solution.
* Jason Druebert is a consultant with BT Professional Services. Jason has extensive experience in ITSM, IT operations, and project management.