Hitting the Bullseye of Success During a Change
Why measure change?
In 2021, Forbes reported that despite aligned leadership, 70% of digital transformation projects fall short of their goals. McKinsey found that only 16% of employees believe their company’s digital reforms have enhanced productivity and are long-term sustainable. So many of our initiatives are not as successful as we hope they will be. Why don't our initiatives hit the bullseye more frequently?
Prosci research shows a strong correlation between measuring performance and meeting and exceeding project objectives. So, one of the keys to boosting the success of our change initiatives, is ensuring we are measuring what happens before, during, and after our changes. If you don't know exactly what you are aiming at and how far you are from your target, it is very difficult to hit it. You could count on luck or charm …but that's not a reliable way to win.
We often prioritize initiatives that bring value to the business and demonstrate our value. In fact, many of us work at organizations where a fundamental goal is to measure and communicate the value that we deliver to our company. Demonstrating our value is important for several reasons:
Building trust: When we demonstrate value to customers, partners, and stakeholders, we build trust and credibility. By delivering on our promises and providing high-quality products or services, we show that we are reliable and worthy of their trust.
Building loyalty: When our internal customers perceive value in our offerings, they are more likely to become loyal to us. By consistently meeting or exceeding their expectations, we can create positive long-term relationships with our business counterparts.
Positive reputation: Demonstrating our value can help build a positive reputation for us within the company. When stakeholders see that we consistently deliver value and uphold our commitments, it enhances our image and can attract new opportunities.
Employee engagement: When employees see that their work contributes to providing value to customers and the company as a whole, it can increase their engagement and motivation. This can lead to higher productivity and job satisfaction.
Differentiating yourself: In a competitive market, you can set yourself apart from your competitors by helping the company do what you do better, faster, and more efficiently. It shows why customers should choose your products or services over others.
In addition to helping our organization and company, measuring change management can actually help you address many of the challenges you’ll face as a change manager. For example, if you know that the communications you sent are not being opened and people are not demonstrating the change you need them to, you can adjust your strategy to manage your stakeholders more effectively toward success. If you conducted a training with a knowledge check, and the results say people don't understand the material, you may need additional training sessions on that topic.
The bottom line is if you want to deliver better changes, you need to measure what you’re doing right, show people, and convince them of the value. Doing so will make your job easier, please stakeholders, improve employee experience, and make you a better change manager.
What are the steps to measure change?
Set goals and objectives: identifying the project’s objectives (what the project will achieve) and the organizational benefits (what the organization will gain).
Establish metrics and KPIs or OKRs
Collect pre-change data
Implement the change and collect data
Analyze, learn, and adjust
Do it again
To measure success, we first need to establish a common definition of success. There are two parts to that:
What does success look like from a Change Management perspective?
What does success look like from a business perspective?
A deep understanding of “success” from both perspectives and connecting the dots between the two will allow us to deliver truly impactful value.
In Change Management we assess tactical effectiveness and the effectiveness of individual change. Tactical effectiveness measures how well our activities are driving the change and should be measured after each activity. For example, if one of the change management activities were a training delivery, the Change Practitioner could conduct a training evaluation. The results of the evaluation would yield a metric of tactical effectiveness, and a possible corrective action if the results weren’t positive could be to update the training approach for the following session. Individual change is often simplified to mean “adoption” but it really includes lots of potential measures of success as well as the perception of the change. Some of these are outlined in more detail below.
So, as a first step, you should engage your key stakeholders - including your primary sponsor, senior leaders of the impacted groups, subject matter experts, and the project manager - in co-creating a shared definition of success for your project. This effort may require you to identify and facilitate any misalignment that exists among the key stakeholders. In addition, a definition of success that extends beyond technical implementation to include measuring adoption and satisfaction of the change by impacted individuals and groups is imperative.
The next step is to set some KPIs or metrics on how you will measure progress toward those goals. There are some suggestions below on what to measure and methods to arrive at a trustworthy understanding of those metrics. It's a good idea to limit the number of change success metrics for the project to a few simple, trackable metrics. You don't need more data, you need better (i.e. more accurate) data.
It is essential to determine what “people side” metrics you will measure early in the project. This is because you may need to collect data prior to, during, and after the change in order to demonstrate a change. You also may need to prepare interview questions, focus groups, or surveys. It is also a good idea to think about how to quantify qualitative data whenever possible and get input from your key stakeholders in preparing how you will collect the data.
Then implement the change, collect your next set of data and analyze it. Use the data to determine what actions you need to take next and document the results. Then do it again.
Ways to define your change metrics
There are lots of methods to decide on which metrics will give you an accurate and full picture of success. If you are interested, there are lots of articles out there with suggestions and methods. I'm going to give you just a few suggestions here that have been recommended by experts in the field and by me. Every initiative is unique and your definition of success for both the Change Management perspective and the business perspective will also be unique. As always, use your best judgement and contact me if you would like help.
I suggest defining success from 3 different perspective levels: organizational level, individual level, and OCM performance/Team level (this is also aligned with what Prosci suggests). At the organizational level, think about what success means for the company. What would the business define as success? At the individual level, we will be measuring how we want to define success for each impacted individual as they move through the change process. We recommend when deciding on individual metrics, a change manager focuses mainly on two outside-in outputs: awareness: measuring if employees understand the change, and preparedness: measuring if employees have the knowledge and ability to make the change and sustain it. At the OCM performance level, we will measure how well the team did at planning and executing the change.
There are also dimensions of the change to be measured: Execution, Accomplishment, and Satisfaction. These are different from the levels discussed above. They describe the quality of the metric rather than the perspective. Execution is the shallowest dimension and involves measuring whether you created a plan and how well you were able to adhere to the plan. Accomplishment is the dimension that asks, “Did we accomplish all the benefits we wanted?”. This dimension includes ROI, benefits realization, cost benefits, revenue generation, or time savings. The third dimension, Satisfaction, captures how people feel about the change. This could include how well individuals were prepared and how satisfied they are with the solution. It can also include how satisfied stakeholders are with the way we managed the change itself - our change management competency.
You can choose to measure all three dimensions across all three levels, but depending on the needs of your initiative, that may not be necessary. It may be more appropriate to choose a single metric at the organization and OCM performance level and maybe 3 relating to individual change. Below are some examples of metrics that may give you valuable insight into how successful your change is divided into Individual, Project, and Organizational benefits.
How to measure what matters
There are whole books written about this subject that I highly recommend. However, I'll try to give a brief summary of some of the high points that might be most relevant to those of you managing changes in a business and/or IT environment.
There are many ways to measure the impact of Change Management: user adoption percentage, utilization, speed of adoption, perception surveys, training delivery, user competency, behaviors, etc. It can easily be overwhelming to decide what works best in your situation. It is also difficult to pinpoint a desired set of metrics without having a clear picture of the type of change and organizational context.
You need a consistent strategy to collect data systematically throughout your change initiative. Consider the types of data that are available, both quantitative and qualitative, and which will give you credible insights into what you are trying to achieve. Identify sources of data and determine if you will need to develop surveys, focus groups, interviews, reports or financial records. Usually more than one method is needed to be convincing. Creating a step-by-step procedure to consistently collect data prior to the change and throughout the change is the key to understanding if the change is successful and lasting.
Using a balanced scorecard instead of focusing only on financial or technical benefits allows us to recognize the value of all performance drivers, even ones that may seem difficult to measure. Incorporating multiple perspectives and data sets allows us to get a more wholistic picture of how the change is progressing and benefiting us in expected and unexpected ways. Quantify qualitative data whenever possible. Measuring people and their emotions is difficult to quantify. You can score qualitative data by counting how many similar responses you get in a survey or interview, for example, or by giving people a Likert scale to rate their satisfaction.
Consider leading and lagging indicators to understand whether the change is going well throughout the initiative. Leading and lagging indicators are types of measurements used to assess business performance. Leading indicators provide early warning signals and help predict future events, while lagging indicators focus on historical performance.
Usually, to measure an increase in something (opinion, satisfaction, competency, use) you need to measure before the change to understand what you baseline is. In measuring change readiness this also applies. Measure if the business is prepared for the change before communicating it out. Then execute the change and measure how people feel afterwards about whether they were actually ready and what they need to keep moving forward in terms of reinforcement or additional training. To determine the extent of the impact, compare the post-change results against the baseline or pre-change measurements.
Here are some more suggestions:
Track communications analytics whenever possible to make data-driven decisions about how to make adjustments and improve your support to stakeholders.
Use employee surveys or focus groups to gather feedback early and feed results back into change strategy and plans.
Use training participation and competency checks to measure how effective the training is and make improvements.
Measure adoption rates and time-to-adoption
Measure stakeholder satisfaction with both the solution and the change process.
Measuring resistance can also be difficult as some resistance is passive and invisible (ignoring, not participating, willful ignorance). Try quantifying it by using adoption metrics and issue tracking: monitor and categorize issues raised by employees related to the change. Then track how many of these issues or concerns are actively addressed and resolved. Then measure whether adoption changes in response.
You must also measure the change management process itself to ensure it’s helping meet targets and objectives. Did you create a plan, how well did you adhere to the plan, did you update the plan and communicate changes to all internal stakeholders. Gather feedback from your project sponsor and others on the team to determine how well your change management looks from the inside. Also measure how stakeholders felt about how you managed the change:
Did they feel prepared when the change happened?
Was there anything they wished they had known?
Did they feel heard and engaged?
Did they feel the change went smoothly?
One metric is whether we realized all the benefits we wanted to when compared to the cost of initiating the change. This is typically called Return on Investment and can be calculated. Usually this needs to be calculated after the change has taken place and may need to be calculated over time to understand how ROI might increase exponentially. How to calculate ROI:
Start by identifying the financial benefits or gains resulting from the change initiative. This includes increased revenue, cost savings, productivity improvements, reduced operational expenses, improved customer satisfaction, or any other measurable financial impact.
Assign monetary values to these benefits and subtract the total change cost from the total financial benefits.
Divide the net financial benefit by the total investment costs and multiply the result by 100 to express the ROI as a percentage.
Sustainability is the ability to maintain the change over time. This is also sometimes difficult to measure. To measure if the change stuck, you may need to pull reports year after year and compare the results or conduct surveys over long stretches of time. For example, sometimes over time people find workarounds or ways to go back the the previous ways of doing things. How can you find out if this is happening? Sometimes you will want to conduct follow up focus groups or observe people to see if they are still demonstrating the desired behavior. You may need to look at financial records to see if the cost savings/revenue generation is still benefiting the company. If not, you need to reiterate and reinforce the change.
Ensuring data accuracy
To ensure data accuracy, you can follow some of these general suggestions:
Use reliable sources: Gather data from reputable sources that are known for providing accurate and reliable information.
Verify data: Cross-check the data with multiple sources to ensure its accuracy. Check the data you've gathered against the data you gathered at different times during the change. Check the data against other stakeholder groups to identify anomalies. Compare the information from different sources and look for any discrepancies.
Validate data: Use various methods to validate the data, such as conducting surveys, performing experiments, or consulting experts. If there is data available from other similar changes or other similar initiatives, check it against that data as a benchmark. This helps to ensure that the data is accurate and reliable.
Document data sources and processes: Keep a record of the sources of data and the processes used to collect, validate, and maintain it. This documentation helps to ensure transparency and accountability.
Holding teams accountable
A major component of measuring change is assessing if people are doing the job of planning and executing to success. Managers should expect and actively review the insights from change metrics. They should be curious about the context, so they understand how to better support their teams to implement the change and to help improve future endeavors.
If teams are collecting data consistently, we should be able to use that data to drive our decision making and action. Asking your team to show you the data and demonstrate why it supports a recommended action is another way to hold ourselves accountable.
Conclusion
In summary, as Leaders, we should endeavor to be measured by outcomes in everything we do. That includes OCM; we measure and monitor the progress of our change initiatives at different levels and by different dimensions to get a clear full picture of how successful we are and what benefits and value we bring to the company. It benefits everyone to plan, measure, learn, and adjust. If you aren't measuring, you're guessing. In order to hit a target, you need to know what you are aiming at and how far away you are. Measuring how far we are from success and using data to help us make informed decisions about how to get there is a surefire treasure map to winning.