When tools promise to measure and develop personality traits, including the famous DISC models of workplace behavior, we owe it to our learners and leaders to look at the science. Is the tool reliable? Does it measure what it claims to? How well does it work where we use it?
To answer these questions, this series of three articles will draw on the publisher’s own evidence (primarily the Everything DiSC Manual, Chapter 4), along with outside perspectives and comparisons with other tools to assess the utility of Everything DiSC fairly and accurately. While this series will raise some concerns based on that evaluation, the intention is not to pick apart a tool that many L&D professionals use and facilitate. Instead, we hope to provide information that empowers people who use these tools – whether they are facilitating with Everything DiSC or another assessment – to understand what the tool measures well, where it can be useful in an organization, and where it falls short.
Only with this type of nuanced cost-benefit analysis can we make responsible decisions about which tools to use and how to use them.
This second article will dive into the psychometric reliability and validity evidence for Everything DiSC. While many popular articles bring up these topics, they often mischaracterize the measurement quality of Everything DiSC or fail to acknowledge that just because a tool has limitations doesn’t mean that it has no value in a development setting. So, let’s take an unbiased look at the science – starting with the good news.
Relevance to Organizational Learning and Development
Everything DiSC is popular in L&D because it can be useful for developing self-awareness, communication skills, teamwork, and conflict resolution. It is not intended to be used as a clinical or deeply reflective instrument for measuring traits of personality. Everything Disc was designed with organizational behavior in mind. It targets behavioral preferences that can be observed, discussed, and changed by most people in a workplace setting. It has benefits (discussed below) and some limitations related to this design goal. For example, while it may lack the depth to assess certain aspects of personality, its primary focus on observable behaviors has the benefits of being practical, relatable, and actionable.
Let’s take a look at some of the psychometric properties that support Everything DiSC as a learning tool.
Construct Validity: Does the Everything DiSC Model Actually Measure Up?
One important reason that construct validity matters for workplace L&D assessments is that it can tell us if the assessment actually measures the concepts it purports to. For example, if we administer the Everything DiSC assessment to individuals, we want to be sure that it measures the four constructs of Dominance, Influence, Steadiness, and Conscientiousness (DiSC) in a manner consistent with its conceptual foundations. As mentioned in the introduction to this series, Everything DiSC purports to measure eight scales (Di, i, iS, S, SC, C, CD, and D) that form a circumplex model with twelve resultant styles. Construct validity evidence can help us see if that is the case.
Everything DiSC Manual, Chapter 4 provides multiple sources of construct validity evidence for the assessment, all based on samples representative of workplace populations (e.g., N = 752 for many of the following analyses).
Positive and Negative Correlations Between Adjacent and Opposite Scales
The circumplex model of DiSC theory predicts that because adjacent scales are similar in behavioral expression, they will correlate moderately and positively with one another (e.g., Di and i should correlate about .50) and that this correlation will be smaller than the internal consistency of each scale (both .90 in one sample reported). Theory also predicts that opposite scales will correlate negatively and substantially (e.g., i and C should correlate about -.70). The data from 752 participants showed this pattern: all neighboring scales correlated moderately and positively with each other and all opposite scales correlated negatively and strongly with each other (see Table 4.4 in manual). The authors state, “The correlations among all eight scales conform to the proposed circumplex model. That is, moderate positive correlations are observed among adjacent scales and strong negative correlations are observed between opposite scales.” (p. 4.17)
A more stringent test of this pattern looks at whether the correlations follow the predicted ordered pattern of r₁ > r₂ > r₃ > r₄ where r₄ is the correlation with the opposite scale. Table 4.5 shows the expected pattern of correlations between scales and actual medians of these correlations across multiple samples are very close to the predicted magnitudes (e.g. r₁ actual median=.45 vs. predicted magnitude=.42; r₄ actual median =-.69 vs. predicted magnitude=-.73; see Table 4.6). Not only can this strength and order of correlations help facilitate discussion about why some styles work well together and others do not “click,” but this test also provides supportive evidence that the scales form a circumplex.
Multidimensional Scaling (MDS) of Everything DiSC Scales
MDS is a way of visually representing the distance (or lack thereof) between scales. The closer two scales are in the resulting diagram, the more strongly positively correlated they are. The farther apart the scales are, the less they correlate. The circumplex model of DiSC would predict that when plotted using MDS, the eight scales will roughly fall into a circle, spaced at equal distances. Figure 4.2 reveals that this is the case: look at how close Di is to D and i, how far away i is from C, etc. Notice that while the scales are not perfectly spaced (it is very difficult to get data to fit this perfectly), the distances between scales are approximately equal, and scales that should be next to each other in the circumplex are closer to each other than they are to scales that are not adjacent in the model. Two fit statistics provided by the analysis tell the story: stress = .01326 and RSQ = .99825. Remember that lower stress values are better (ideal is 0) and higher RSQ values are better (ideal is 1). The authors summarize: “This analysis adds strong support for the two-dimensional DiSC model and the ability of the Everything DiSC assessment to measure that model.” (p. 4.23)
Factor Analysis of Everything DiSC Scales
When the authors conduct a principal components analysis with varimax rotation on the eight scales, they find two factors with eigenvalues greater than 1 (the first two are 3.10 and 2.95; all remaining eigenvalues drop well below 1: 0.60, 0.38, and so on down to .04; see Table 4.7). In other words, the underlying structure of the scales is two dimensions. (These dimensions are often interpreted as Assertiveness vs. Receptiveness or some other paired traits that make sense in the context of a circumplex model.) This is consistent with the predicted structure of the Everything DiSC model rather than suggesting three or more hidden factors.
How is this information meaningful for using Everything DiSC in organizations? Primarily, it means that when participants complete the assessment, they can have confidence that it measures the four behavior preferences it claims to. Further, when participants plot their dot on the DiSC map, they can be fairly certain that their responses to the questions about how they tend to react to challenges (D/CD), people (i/iS), pace (S/SC), or accuracy (C) are accurately represented in the relative distance of the dot to each of the four quadrants. The circumplex model allows us to create intuitive tools like the DiSC map to facilitate discussion about where individuals prioritize their time and attention in meetings, during feedback conversations, or in their leadership development. More to come in Article 3!
