Measuring employee engagement the right way is something companies often struggle with. This article answers the questions whether you want to measure it yourself or work with a survey provider. In addition, we dive into the two most common employee engagement scales and discuss their characteristics. After reading this article, you will measure employee engagement the right way.
Table of content
Why measuring engagement is difficult
How to measure employee engagement
General tips for engagement surveys
Measuring employee engagement: Utrecht Work Engagement Scale
Measuring employee engagement: Gallup Workplace Audit
Measuring ROI on employee engagement
Why measuring engagement is difficult
In this article, we will explore a lot of challenges regarding the measurement of engagement. We will start with the biggest.
The biggest challenge to measuring employee engagement is its lack of a unified definition. Try answering the following questions:
- Is engagement about feeling happy at work?
- Is it about being absorbed with what you do?
- Is it about being energized in your work?
- Is it about having work that is meaningful to you?
- Or is it all of the above?
This question is hard to answer because different organizations use different definitions of engagement. Some define it as employee happiness, some take a more academical approach (i.e. absorption, vigor, and dedication).
One of the first prominent researches to write about this was William Kahn from Boston University. He wrote about the conditions people needed to personally engage in their work. According to Kahn:
“I define personal engagement as the harnessing of organization members’ selves to their work roles; in engagement, people employ and express themselves physically, cognitively, and emotionally during role performances.”
According to Kahn, engagement happens when employees’ personal selves are aligned with what they do in their work. While a bit abstract, Kahn’s definition has been the basis for further research and the measurement methods that we will dive in next.
How to measure employee engagement
So what are the different ways we can measure engagement? There are three ways you can measure employee engagement in an organization.
- Having engagement measured by an employee engagement survey provider;
- Measuring employee engagement yourself;
- A hybrid approach in which annual engagement is measured by the survey provider while pulse engagement is measured by the company throughout the year.
Before we dive into the technicalities of how to measure engagement, let’s discuss these three options one by one.
Having engagement measured by a survey provider
This is the approach that is traditionally used by a lot of companies. A provider measures engagement every one or two years. The provider takes care of the logistics and the software. The survey’s results are reported to the company on a high level.
Example of well-known providers include Gallup and Mercer
Although this outsources a lot of the difficult work, there are a number of cons.
- First of all, this approach is costly. The cost can vary between low five figures to the low six figures depending on the number of employees.
- There is also little room to change questions. This is mostly a pro but can be a con.
It can be valuable to measure company-specific elements in the survey. Examples include how a recent merger was perceived, or employees’ perception about company leadership. Including these elements can prove insightful. However, asking a good question in a questionnaire is difficult and there are a lot of ways to do it wrong.
- Reporting is done in standard formats. Again, this can be an advantage as it removes all the hassle of reporting on key findings yourself. However, there are limited possibilities for slide-and-dice reporting and engagement data is not integrated with other key data such as employee turnover or financial data.
- The most important drawback is, however, the unavailability of data. The engagement survey provider is usually hesitant, or even downright unwilling to share the data. If data is shared, it’s usually done on an aggregated level, which greatly decreases the value of the data for analysis.
This prevents the organization from doing its own data analysis and including engagement data in HR dashboards. If your organization hasn’t started with people analytics, this may not seem a big problem. However, if you ever want to start, you need access to the data if you want to analyze your historical data.
- A final limitation is the operationalization of engagement. Different companies define and measure engagement in different ways. Engagement can be a powerful predictor for many HR and business outcomes – but only if it’s measured the right way. This is where a lot of survey providers go wrong. This phenomenon strongly devaluates the value of the concept of engagement into a potential fad.
So although working with an external provider brings in a lot of know-how and enables you to get basic insights (much) quicker, the use and impact of the engagement data is usually limited
Measuring engagement yourself
After our previous discussion, you can imagine that measuring staff engagement yourself would be beneficial. The advantages:
- You own your data
- You can add relevant questions on the antecedents of engagement and other HR outcomes.
- You can include engagement data in tactical and strategic reporting
- You can easily analyze data
- You can use the results of previous years and your experience in data analytics to improve the questionnaire and tailor it to the organization
However, there are a number of drawbacks that shouldn’t be underestimated. The disadvantages:
- The process of designing an employee engagement survey takes a long time
- Unfamiliarity with questionnaire software can slow the process down
- There’s a big risk of designing questions that don’t measure what you want to measure. The internal employee survey ‘owners’ are rarely psychometrics specialists and expertise in this area is hard to find. With questionnaires, you only have one shot to do it right.
- Question selection is often more of a political process instead of a science-based process. Every department can bring in a few questions.
All of these reasons taken together, make the choice almost impossible. However, there’s hope in the tips below the next section!
The hybrid approach
A lot of companies that are starting with analytics are using hybrid approaches. I’ve heard from companies that use questionnaires for specific employee segments on top of the annual survey. They use that data for specific analyses, for example, to assess sales performance, or to analyze reasons for absenteeism in a specific department.
Another approach that more and more companies are using are pulse surveys. After sending one or a few questions to a select group of employees, you can get continuous insight on what the organization is saying, doing, and thinking.
Pulse surveys are all the buzz in the last couple of years. Despite the limited amount of questions that you can ask, there are some indications that they could help in measuring engagement in a valid and reliable way. However, the reliability and validity of these tools are a real concern. The previous link provides the first valid 3-item questionnaire that can be used to measure engagement (first published in October 2017).
General tips for engagement surveys
To simplify things, here are some tips that you should keep in mind, regardless of who is doing the survey.
- Always work with a good psychometrician. Let him or her vet your survey.
- Check the reliability and predictive validity of your provider’s survey. Let your psychometrician do this.
- If you decide to go on your own: use pre-validated engagement surveys that have proven to make a positive impact on the business and lead to positive HR outcomes.
- Make sure to always own your data and to have it accessible.
- Don’t promise anonymity to employees but promise confidentiality. You need individual data to do proper analytics which means that the data is not anonymous.
- Always combine your staff engagement survey with other employee data that you want to make tangible. The engagement survey is usually the only time in the year you can quantify a lot of qualitative information.
Measuring employee engagement metrics
The big question then is: how do you measure staff engagement? Coming up with the questions yourself is inadvisable. Why would you reinvent the wheel that researches have studied for the past 20 years?
Utrecht Work Engagement Scale (UWES)
I’m always a big advocate of sticking to science-based methods. The best-researched conceptualization of engagement is by Wilmar Schaufeli and colleagues. They define employee engagement as a “positive, fulfilling, work-related state of mind that is characterized by vigor, dedication, and absorption”.
- Vigor is characterized by high levels of energy and mental resilience
- Dedication is characterized by feelings of a sense of significance, enthusiasm, inspiration, pride, and challenge
- Absorption is characterized by being fully concentrated and deeply engrossed in one’s work, whereby time passes quickly and one has difficulties with detaching oneself. This is similar to the concept of flow.
Employee work engagement, as measured by Schaufeli and colleagues, has been proven to lead to multiple HR outcomes like in-role performance, extra-role performance (i.e. employees going the extra mile), turnover intention, and organizational commitment. In addition, the interventions to increase engagement are very well-documented.
The business impact of work engagement is documented to a lesser degree. It has been shown to correlate with innovation, superior service quality, and fewer errors, occupational injuries and accidents (Schaufeli, 2012).
This form of engagement is measured using the UWES, or Utrecht Work Engagement Scale. The most common version being the UWES-9, a scale with 9 questions, which is translated into many languages. For those with academic access, the full list can be found on page 14 of the document.
In 2017, the UWES-3 was published, a scale with 3 questions with very similar predictive results. This shorter version would be very useful for pulse surveys, for example.
Gallup Workplace Audit
An alternative way to measure engagement is the Gallup Workplace Audit (GWA). Gallup has long been one of the most dominant publishers on engagement and their global reports on the effects of engagement show impressive numbers.
According to a meta-study by Harter, Schmidt & Hayes (2002), correlations between engagement were highest for customer satisfaction–loyalty (.33) and employee turnover (.30), followed by safety (.32), productivity (.25), and profitability (.17).
The GWA is a 12-question list, which includes items about expectations, resources, and opportunities. The full list can be found in this document on page 6. We can’t include the questions in this article due to copyright.
Measuring ROI on employee engagement
A key theme in HR analytics has always been measuring the benefit of employee engagement.
The best-known example of an ROI calculation is the case of Best Buy. According to this Harvard Business Review article, a 0.1%. increase in engagement leads to a return of $100,000 per shop for the electronics retailer.
When we want to look into measuring the ROI on employee engagement, we need to combine engagement data with internal financial data. We published an article on linking engagement with business performance earlier. You can use this as a guide.
Proving the ROI on employee engagement takes a long time. We would like to turn this around and start with a business question. Only if engagement is one of the key drivers of a desired behavior or strategic business goal, you should analyze it.
This is where HR can improve on. Oftentimes engagement is measured for engagement’s sake. The purpose is to get the number up every single year regardless of its impact. Focusing on the impact of engagement and the other concepts that are measured in the survey, will generate a better ROI.
Engagement is often wrongly seen as the one number in HR to rule them all
Engagement is just one of the concepts that’s measured in an engagement survey. Let’s consider an example. An organization surveys its employees and measures three key constructs:
- Employee engagement;
- Organizational commitment;
- Job satisfaction.
Each construct is measured through a number of questions, also called items.
Say the company has a problem retaining its future leaders. All three constructs are relevant and should contribute to retention. Using analytics, you can determine which of these concepts are most strongly related.
If organizational commitment only leads to a small increase of retention but job satisfaction to a much higher increase, it makes sense to focus on the latter.
The next step would be to estimate the cost of improving these constructs. For each construct, there are a number of proven ways to increase it, which can be found in the academic literature. Estimating the cost and return using Human Resource Costing will help to estimate the ROI.
The individual questions that make up the construct can also guide your interventions. To give another example, say engagement has the highest impact on employee turnover. The average engagement score for an organization is 3.6/5, or 72%. Three of the nine questionnaire items are:
- At my work, I feel bursting with energy (avg. score: 4.3/5)
- I find the work that I do full of meaning and purpose (avg. score: 2.8/5)
- I am immersed in my work (avg. score: 4/5)
As you can imagine, the easiest way to improve the average score is to increase item 2. So the challenge is how to make sure that people feel their work has meaning and purpose. This is a well-defined task compared to ‘improve engagement’.
In the end, you want to retain people as best you can. Simply focusing on the one number to rule them all might not give the optimum return. This is important to keep in mind when measuring employee engagement.
Measuring employee engagement is a challenging task. Coming up with good questions, preparing the questionnaire, and ensuring a high response rate are all critical parts for taking a successful measurement.
The main challenge in questionnaire research is that you only get one shot. If you make a mistake in a question, there’s a good chance it will be invalid. This may render a big part of the full questionnaire useless.
One of the key take-home messages is therefore not to design questions yourself unless you’re a psychometrics expert. Designing questions takes a lot of nuances and requires extensive testing before they can be put to real use. When you do this without expert knowledge you run the risk of creating a survey that doesn’t measure what you want to measure. As a result, the survey often cannot be used for analytics nor can it be used to calculate an ROI.
The best tactic is still to stand on the shoulders of giants. Using tested surveys helps to ensure that the engagement you measured will have an impact on critical organizational outcomes.