Survey Misstep #2 - Underutilizing Segmentation

In our last “Survey Missteps” discussion, we spoke about the importance of using frequency distributions for reporting survey results as opposed to averages. If you missed that article, it can be found here:

Survey Misstep #1 – Using Averages

Another misstep we frequently see is more of a missed opportunity…that of underutilizing segmentation for reporting purposes. Some organizations choose not to use reporting segmentation; while others do, but perhaps not as effectively as they should. Segmentation can be an extremely valuable tool for analysis as it allows companies to focus on key parts of the organization, allowing the creation of specific action plans for improvement.

To illustrate, let us use the following simple example:

Company ABC conducts an employee survey with one question: “I love my job.” The employees are required to answer on a scale of 1 to 7, with 1 being “Completely Disagree,” 7 being “Completely Agree,” and 4 being “Neutral.” All 100 of ABC’s employees answer the survey, the randomly generated results of which (using frequency distributions) are as follows:

I love my job (n=100):

	Total (n=100)	Dept. 1 (n=38)	Dept. 2 (n=41)	Dept. 3 (n=21)
High (6 or 7)	33 (33%)	13 (34%)	15 (37%)	5 (24%)
Medium (3, 4, or 5)	45 (45%)	17 (45%)	22 (54%)	6 (29%)
Low (1 or 2)	22 (22%)	8 (21%)	4 (10%)	10 (48%)

Segmentation by Department in this example provides a great deal of value beyond the data for the total sample size. While Departments 1 and 2 are relatively similar to the Total values, Department 3 differs significantly. Not only is the percentage of “Highs” lower, but the percentage of “Lows” is much greater, potentially indicating an issue in Department 3 that might not have been discovered were it not for the segmentation of data among departments.

While the difference in samples sizes would also need to be considered, on the surface, this example provides a good illustration as to how the omission of segmentation can also unknowingly exclude valuable insights, such as the ability to target more specific areas for improvement (Department 3 in this case). One of the most fun parts of survey data analysis can be the search for meaningful segmentation. While department segmentation might provide some insights in this case, other segments (such as job title or office location) might offer even more golden nuggets. We must also be careful not to “over segment.” While segmenting data can provide meaningful information, over segmenting can lead to small sample sizes and an inclination to create an abundance of action plans targeting many different segments, stretching resources thin and causing organizational confusion. In addition, small sample sizes can prevent the meaningful use of statistical models for further analysis, something we will discuss in one of our future articles. The key is finding the “segmentation sweet spot” that provides insight without removing the potential for further analysis.

Please join us in our next discussion as we continue to examine some of the ways you can improve your employee and customer surveying techniques!