Are Gallup’s Engagement Numbers fake?

At least they are heavily cited

According to Gallup’s recent worldwide study in 2017 only 15% of all employees in the world are engaged. Since years Gallup shares the dramatic news, around 8 out of 10 employees are either not engaged or (even worse) actively disengaged. Taken this seriously, you are probably not engaged. Most of your colleagues, friends, and relatives are not engaged. Most employees you meet at stores, airports, hospitals, and universities are not engaged. Even the majority of Gallup’s workforce might not be engaged, maybe.

Disengagement sells. Gallup did a great job over the last few years to gain executives’ attention built on news, which probably are misleading, dare I say it, fake. This seems to be another version of the here-is-the-problem-but-luckily-we-have-the-solution-business-model.

I have spent hours reading reports, news and even articles published in scientific (peer-reviewed) journals written by Gallup fellows and have not yet found any definition of what “engagement” or “active disengagement” means in statistical terms. The statistical or scientific basis of what have become one of the most cited studies in management has never been disclosed. It has been kept as a secret. Why is that? Obviously 100s or 1,000s of keynote-speakers, consultants, executives, and bloggers sharing the dramatic output delivered by Gallup do not exactly know what they are talking about.

What we do know is the following: Since years Gallup is using its Q12-Questionnaire, which consists of 12 questions. Those 12 items did never change substantially, for reasons of comparability of course. Speaking of these questions, they reflect a kind of outdated and hierarchical view on leadership, anyway. For instance, one question goes like this: “I know what is expected of me at work”. Respondents then have the choice to select one option out of five ranking from 5 (strongly agree) to 1 (strongly disagree). The same applies for the remaining 11 items such as “I have a best friend at work” or “This last year, I have had opportunities at work to learn and grow”. What initially had started as a tool for workplace auditing (Gallup Workplace Audit) miraculously transformed into a tool supposed to measure employee engagement without having substantially changed its content. But, working conditions – as measured by the Q12 – and employee engagement are two different concepts even though there might be a connection between the two. Even Gallup reports a correlation of limited magnitude only between what the Q12 measures and business outcomes on business unit level. I guess employee engagement sells better than workplace conditions.

I have spent years in the employee survey business and have been involved in dozens of global surveys using similar items like Gallup does, which by the way, I would never do so again for multiple reasons even though I have published two books on that particular practice of satisfaction and engagement surveys (but this is a different story). Once you ask employees to evaluate their working conditions or estimate their satisfaction with whatever it is at work, you more or less receive a bell curve with a slight tendency towards the positive end.

It is quite astonishing how constant and stable these results turn out when asking a big enough sample. One might separate different responses as shown in the following picture:

There is always a minority of employees considering themselves as being totally satisfied. Working conditions are seen as perfect. There is a majority in the middle with a slight tendency towards the positive end of the continuum. To those people working conditions are fine but could be improved. And, of course, there is always a minority of employees, which is kind of pissed-off. They evaluate their working conditions as bad and expect significant changes.

While the Q12 measures employees’ view on their working conditions Gallup strangely translates results into what they name employee engagement.

Now it comes the critical question: Where does Gallup make the cut between the so called engaged, the not engaged and the actively disengaged ones? What is Gallup’s definition? We simply do not know. Wherever Gallup is doing the cut, probability is high that a picture is drawn, which is more dramatic than reality actually is. Even though Gallup has never been clear about the cut, they offer a definition of how employees in the different groups behave. Gallup describes “not engaged” employees (the middle segment) as follows: “Employees are essentially ‘checked out’. They’re sleep-walking through their workday, putting time – but not energy – or passion into their work”.

This is even more confusing, since there is literally no reason to draw such extreme conclusions or characterizations based on the content of the Q12-questions (working conditions). And again, Gallup refuses to explain, where this typology actually comes from. It definitely sounds more like arbitrary horoscope than as insights based on serious social science.

Gallup is an institute of top reputation. Its founder, George Gallup, has been a leading pioneer in its field, really a great man and a role model for generations of social scientists to come. What surprises me though is the fact that, Gallup is keeping a secret around its most cited outcomes. What is going on here?

So, why is this bothering me? I am a social scientist and expect clarity as all good social scientist do. Published scientific results must be replicable so that others could repeat studies in the same manner. Empirical methods and outcomes not being communicated in a clear way are useless in the eyes of the scientific community.

In times of fake news we have to be careful, as we all have learned in the last few months and years. Whenever I find myself listening to a keynote even of reputable speakers, chances are high to again hear about Gallup’s disengagement drama. Then, my first thoughts are always: Fake, fake, fake. I cannot hear it anymore. Should I stand up and ask the speaker: “Hey, have you read the report? Could you tell, how these numbers you’ve just presented have been calculated?”. I better don’t, but silently I continue questioning myself whether or not I should seriously take home the remaining parts of the keynote.

Above all this, what bothers me most is the way Gallup dares to slap billions of employees all around the globe. Gallup draws a picture of the global workforce, which probably is worse than it actually is. By doing so, Gallup consciously hurts global economy, while filling its own pockets. Executives who believe in the Gallup drama treat their people accordingly. Considering employees as being lazy, dumb “not engaged” or “actively disengaged” (as Gallup puts it) might lead to management practices that will not move companies forward.

In certain ways, Gallup actively prepares the ground for mistrust and control, which is the opposite of what many companies and employees desperately need. We all could imagine what results delivered by Gallup do to a regular executive. He/she might think: “I never thought how dramatic things are”. According to the hindsight bias he/she might add: “But somehow I felt it in advance”. What might be the consequences? More trust? More autonomy and self-determination? Probably not.