Survey Misstep#3 - Treating All Variables as Equal

In our previous “Survey Missteps” discussion, we spoke about the challenges with using averages for reporting and the value of effective segmentation. If you missed those articles, they can be found here:

Survey Misstep #1 – Using Averages

Survey Misstep #2 – Underutilizing Segmentation

Another common misstep we see is companies treating all variables (survey questions) the same, which can create focus on the wrong areas. A very common approach in many organizational surveys is to review the results and try to improve every item with a “low” score. While perhaps a valiant effort, this process treats every variable as though it deserves equal attention, which is inaccurate.

To illustrate, let us use the following simple example:

Company ABC conducts an employee survey with four questions. The employees are required to answer each on a scale of 1 to 7, with 1 being “Completely Disagree,” 7 being “Completely Agree,” and 4 being “Neutral.” All 100 of ABC’s employees answer the survey, the randomly generated results of which (using frequency distributions) are as follows:

Question (n=100)	High (6 or 7)	Medium (3, 4, or 5)	Low (1 or 2)
I believe my total compensation is fair.	50%	30%	20%
I believe the organization communicates effectively.	55%	35%	10%
I am happy with my parking space.	30%	40%	30%
I am happy with the performance of the MN Vikings.	10%	70%	20%

The first inclination of many organizations when reviewing this data would be to look down the list of questions, find those that had low scores and try to determine how to improve them. In this case, we can see that employees are less enthused about their parking spaces and the performance of the MN Vikings. Obviously, more resources need to be focused on finding better parking options and ways to improve the performance of the MN Vikings!

While this example is obviously exaggerated (and perhaps facetious), it is done so to make the point that just because a variable’s score is low does not mean its improvement would have a material impact on the desired outcome, such as employee engagement. Even if employees are unhappy with the performance of the MN Vikings (or their parking spaces), is it really something we need to improve? In many cases, the answer is “no.” Treating all variables as if they have the same impact on the dependent variable creates an environment where the improvement of all things is equally important, spreading resources thin and eliminating focus. We see this issue (in less obvious ways) in nearly every survey engagement we encounter, whether with customers or employees. It is generally ineffective at improving employee engagement (or other desirable organizational outcomes), inefficiently allocates resources, and causes confusion with organizational messaging.

The question then becomes, “If some variables are more important than others, can we determine which variables are most important?” The answer is, “yes, we can!”

Please contact us to learn more about how your organization can use statistical analysis to determine those variables that have the greatest impact on their desired outcomes!