|About ACR||Editorial Board||Guide for Authors||View Articles||Submit Manuscript|
You are not permitted to access the full text of articles.
If you have any questions about permissions,
please contact the Society.
ํ์๋์ ๋ ผ๋ฌธ ์ด์ฉ ๊ถํ์ด ์์ต๋๋ค.
๊ถํ ๊ด๋ จ ๋ฌธ์๋ ํํ๋ก ๋ถํ ๋๋ฆฝ๋๋ค.
|[ Invited Scholarly Essay ]|
|Asian Communication Research - Vol. 20, No. 2, pp. 76-83|
|ISSN: 1738-2084 (Print) 2765-3390 (Online)|
|Print publication date 30 Aug 2023|
|Received 18 Apr 2023 Revised 21 Apr 2023 Accepted 02 May 2023|
|Too Many? Too Few? Just Right? Construct Proliferation and Need for a Construct Dump|
Franklin J. Boster
|Department of Communication, Michigan State University, East Lansing, MI 48824|
|Correspondence to Franklin J. Boster Department of Communication, 404 Wilson Rd., Michigan State University, East Lansing, MI 48824 Email: email@example.com|
Copyright ⓒ 2023 by the Korean Society for Journalism and Communication Studies
Franklin J. Boster (Ph.D., Michigan State University, 1978) is presently Professor Emeritus in the Department of Communication at Michigan State University. He is the recipient of the Distinguished Faculty Award at Michigan State University (2003), the Faculty Impact Award from the College of Communication Arts and Sciences (2009), the Outstanding Achievement Award for Teaching (Arizona State University, College of Public Programs, 1981), the Charles H. Woolbert Award for Scholarship of Exceptional Originality and Influence (National Communication Association, 1989), the B. Aubrey Fisher Mentorship Award (International Communication Association, 2005), and the John E. Hunter Meta-Analysis Award (International Communication Association, 1998). In 2016 he was named Fellow of the International Communication Association. His research interests focus on social influence. He resides in Mason, Michigan with his wife of 37 years (Linda), two dogs, three cats, and two other humans.
|Keywordsoperationalism, validity, constructs, unidimensionality
“Operationalism.” The term has a nice, scientific sound to it. It contributes to the jargon that helps us present ourselves as scientifically legitimate, perhaps providing respite against doubts some have that what we do actually does have scientific legitimacy. I suspect, but cannot confirm, that it appeals particularly to those with physics envy.
At times, people have asked me how I “operationalized” some construct around which my research focused. I now give a simple answer, “I didn’t.” Typically, the person asking the question looks perplexed. Because I once used the term in articles I wrote or to which I contributed, I understand the question. And, although I could provide an answer satisfactory to them, I finally stopped doing so and went with the “I didn’t.”
My reason involves my coming to terms with Percy Bridgman (1927)The Logic of Modern Physics, the volume in which he coined the term “operationalism,” and certainly one of the more boring volumes I have read. I read it for a paper that I wrote in an undergraduate philosophy of science class. My reflections on the book led me to believe that what he had in mind by the term did not capture how we in communication now use it. Moreover, it led me to reflect on measurement in communication science. Some comments on operationalism follow, and after that, reflections on constructs and their measurement in communication science.
An example can facilitate understanding Bridgman’s notion of operationalism. Consider distance, a critical variable for a physicist like Bridgman. Let us ask two questions of it: “How do we conceive of it?” and “How do we measure it?” To the former we might answer, “The quantity of space separating two objects.” To the latter we might answer, “We measure the space separating two objects A and B by placing some measuring instrument, call it Rod S, on A and concatenate it until we reach B. The number of Rod Ss between A and B equals the distance between A and B.” But note that these two answers correspond perfectly, we correctly characterize them as isomorphic. Put differently, the operations involved in measuring distance and our way of conceiving distance do not differ.
S. S. Stevens introduced psychologists to the term (e.g., 1935a, 1935b), from whence it filtered on to communication scientists. Despite some disagreements with Bridgman, he used the term (he termed it “Operationism”) in much same manner. And, as a psychophysicist, he would have found the variables with which he worked more amenable to Operationism than, say, those in social psychology. Others, such as psychophysicist and psychometrician Warren Torgerson, played a role in diffusing the idea as well, albeit under the phrase, fundamental measurement (Torgerson, 1958, p. 22).
Generally, we do not operationalize (or operationize) the constructs on which our discipline focuses. Put differently, what we mean by a construct, its conceptual definition, rarely displays an isomorphism with the way in which we measure it (for a counterexample think of “group size”).
A measure has validity to the extent that it measures what it purports to measure, and nothing else. If we did or could operationalize our constructs, we would have no need to validate our measures of them, because no gap between conceptual definition and measure would exist. The procedure by which we operationalize would insure the equivalence of the measure and the conceptual meaning of the construct. When colleagues have asked the question as to how I have operationalized my construct(s), they want to find out how I measured it (them). And, they wish to do so, so that they might ponder on the quality of the measure(s). I can have confidence in that claim because after I tell them what measure I employed I am then asked about the reliability coefficient and the evidence that I have amassed pertinent to the measure.
Subsequently, I shall argue that some of the constructs employed in communication research do not meet the definition of a construct, and that we should (both a scientific and hence moral imperative) commit them to a place I refer to as the “Construct Dump.” If we could operationalize, thus circumventing the issue of the validity of our measures, the only crime for which we would sentence a construct to the Construct Dump would involve its empirical failure to enter into predictable relationships with other constructs. But, given how we measure our constructs, we must invoke other criteria. I now turn to this issue.
By measurement I shall mean the procedures we employ to assign numbers to the values of a variable. And, given the definition of “validity” provided previously, we must acknowledge that we have a gap between the conceptual meaning of a construct and the procedures we use to measure it. For this reason, we have the task of demonstrating that our measures have high validity.
When we use the term “construct” we refer to a dimension on which people, or sometimes other units of analysis (e.g., groups, messages, etc.), differ. On occasion, we may succumb to the error of believing that a dimension of individual difference exists because we can imagine it, failing to realize that constructs reside, not necessarily in our theoretical or conceptual thinking, but in the cognition, affect, and action of those we study. We can make two errors: (1) treating a concept as a construct when it does not have the status of a dimension of individual difference, and (2) failing to identify or acknowledge a construct that does exist. Correcting the former error produces refuse for the construct dump. Correcting the latter error results in a panoply that can inform our understanding of human communication. For the most part we have employed statistical techniques like factor analysis, both the exploratory and confirmatory models, to help us recognize and avoid these errors. Our efforts have not always resulted in success. Consider some examples.
Example 1: Machiavellianism. Christie and Geis’s 1970 volume, Studies in Machiavellianism, introduced the social scientific community to the construct “Machiavellianism,” the development of scales to measure it, and a series of studies providing evidence of the validation of those measures. By Machiavellianism scholars mean the willingness to exhibit callous disregard for others and to exploit and manipulate them in order to get one’s way. Christie and Geis (1970) thought of Machiavellianism as a construct, a single dimension of individual difference.
If Machiavellianism forms a single dimension of individual difference, and Christie and Geis’s commonly employed Mach IV scale provides an acceptable measure of it, then factoring these items would yield a unidimensional solution. As Hunter et al. (1982) showed long ago, however, it does not. Rather, their confirmatory factor analysis (and exploratory factor analysis as well) generated a radical multidimensional solution with four factors: deceit, flattery, immorality, and cynicism emerging, and seven of the 20 items exhibiting a lack of content validity as they fit into none of these four factors nor any other factor.
One might object that the Mach IV might exhibit second order unidimensionality (Hunter & Gerbing, 1982). That is, treating the four factors as four items and factoring them would produce a unidimensional solution, and the four factors, treated as items, would exhibit parallelism, i.e., they would correlate with other constructs in the same way (sign and magnitude). Meeting these conditions would warrant treating the Mach IV as a single dimension. But, the Hunter et al.’s (1982) data showed that the Mach IV failed to exhibit second order unidimensionality as well (see Table 2, p. 1300). Particularly, correlations across the four factors differed on constructs such as self-esteem, locus of control, and religiosity, among others.
These results indicate that subject responses to the Mach IV do not form a single dimension of individual difference. They also indicate the possible existence of four constructs: deceit, flattery, immorality, and cynicism, each of which could exert a powerful impact on various dimensions of human communication behavior. And, the modest reliability of each of these four factors (unreported) indicated that they would benefit from increased attention in the form of constructing additional indicators so as to increase the reliability of each of them.
I have a bibliography of more than 100 articles published prior to 1980 that measured Machiavellianism and assessed its relationship with other variables. Because the Machiavellianism Scale used in these studies does not assess a single dimension of individual difference, none of the reported results inform us about human behavior. Circumstances have not changed. Readers can satisfy themselves with a Google search, and discover that investigators continue to study Machiavellianism and continue to use the Mach IV Scale which they then examine for its association with other variables.
Consider for a moment the implications of that correlation. For example, a correlation between the Machiavellianism Scale and, say, dogmatism (Rokeach, 1960) results from the weighted (by number of indicators) mean of the correlations of dogmatism with the various dimensions contained in the Machiavellianism Scale. Dogmatism might correlate substantially with none of the dimensions, one of them, two of them, three of them, or all four of them. Hunter et al. (1982) report that summed scores on the Mach IV scale correlation .41 with dogmatism. But, that results masks important findings. Specifically, dogmatism correlates -.06 with deceit, .29 with flattery, .04 with immorality, but .84 with cynicism. The only way to reach substantively meaningful conclusions would require examining the correlation with each of the four dimensions separately.
One would likely conclude, and conclude correctly, that most people who think about Machiavellianism as a construct and who measure it with the Mach IV Scale have not read Hunter et al. (1982). A citation search indicates relatively few citations. And, I find that state of affairs perfectly understandable. To read everything written about Machiavellianism prior to doing a study focusing on it might take so long as to make the study obsolete prior to conducting it.
But, I fear the problem goes deeper than that, and that many scholars fail to see the implications of what constitutes a construct for their own research. And, we need go no further than Christie and Geis themselves to see the fear realized. They knew in 1970 that their Mach IV scale did not fit a unidimensional solution, as they report multidimensional factor analytic solutions in Appendix A of their book! Parroting a phrase I have heard or read on numerous occasions, they appear to think of Machiavellianism as a “multi-dimensional construct,” failing to realize that the phrase forms an oxymoron.
Of course, the possibility always exists that Machiavellianism does form a dimension of individual difference, but that the Mach IV or Mach V or other existing measures fail to include the indicators that would assess it. The burden of proof, however, lies with those who endorse such a hypothesis. We must retain our skepticism that Machiavellianism exists and relegate it to the construct dump, resurrecting it only if/when someone produces indictors which tap such a dimension of individual difference.
Example 2: Opinion Leadership. Katz and Lazarsfeld emphasized the concept of opinion leadership in their classic 1955 volume, Personal Influence. And, although Katz later marked the concept for retirement (Katz & Fialkoff, 2017), it remains one to which scholars refer frequently. Both Katz and Lazarsfeld (1955) and Rogers (2003) refer to opinion leaders as having three attributes that distinguish them from non-opinion leaders. Specifically, they characterize opinion leaders as well connected, persuasive, and experts in a domain (mavens). That characterization raises the issue of whether opinion leadership forms a single dimension of individual difference or three dimensions of individual difference (or perhaps neither).
Subsequent factor analytic work (Boster et al., 2011) developed indicators of connection, persuasiveness, and mavenness. They found their data consistent with the predicted three factor solution. Moreover, they found that the internal consistency test of the second order unidimensional model failed, and that the parallelism test of second order unidimensionality failed as well. They also showed that commonly employed measures of opinion leadership (e.g., Flynn et al., 1996) correlated substantially only with the maven scale, so that studies using such measures to identify opinion leaders produce results limited to the expertise of the person exhibiting influence.
This set of three studies and the research program that followed it have implications for the ontology of opinion leadership. Katz and Lazarsfeld’s (1955) and Rogers’s (2003) conceptual thinking, asserts that an opinion leader has high connectivity, high persuasiveness, and high mavenness. Suppose for a moment that we falsely think of these three variables as dichotomies. We can then imagine a 2 × 2 × 2 matrix in which opinion leaders fall into one cell of the matrix, namely the high/high/high cell. To say that opinion leaders, and only opinion leaders, had exceptional influence (the dependent variable, say) would imply a three-way non-additive effect in which those in the high/high/high cell exerted exceptional influence, and those in the other seven cells exerted less, and approximately equal, influence. Conceptually and empirically, however, we gain little by employing the opinion leader concept. Instead, we can profitably rely on the three constructs: connectivity, persuasiveness, and mavenness, to contribute to our theoretical thinking. Affirming Katz and Fialkoff ’s (2017) position we can retire opinion leadership to an assisted living facility in the construct dump.
Example 3: Communication Anxiety. In the first two examples I suggested eliminating concepts that fail to meet the criteria that define a construct. In this example I examine a concept that likely meets the criteria that define a construct, but that needs consolidation.
Communication scholars have long focused on communication anxiety, in its various forms and labels, and continue to do so. In 1959, Clevenger responded to the need to summarize and integrate this burgeoning literature, terming it “stage fright.” Among his conclusions he noted that three ways of measuring the concept had emerged: self-report, observers’ reports, and physiological measures. He also observed modest correlations among these three methods. Currently, self-report measures dominate, but these self-report measures appear in many forms and under many labels. Consider some of them: communication apprehension (McCroskey, 2005), unwillingness to communicate (Burgoon, 1976), social anxiety (Leary, 1983), shyness (Zimbardo, 1977), extraversion (Eysenck, 1973), and need for affiliation (McClelland, 1985).
The manner in which scholars conceptualize these ideas, and the manner in which they measure them, suggests three differences among them. Fear, or anxiety, associated with unpleasant consequences that result from interacting with others would form one point of frequent emphasis, a skills deficit another, and preferring (substantially) one’s own company to that of others the third. Although sorting out causal order would prove a daunting task, we can expect these three attributes to correlate very highly because we can anticipate some causal order among them. For example, feeling anxious about talking with others could lead to a skills deficit and, ultimately, to a preference for one’s own company. If so, we would identify few people who did not have similar scores (either high/high, medium/medium, or low/low and all points between) on all three attributes. Thus, these three attributes would not make a difference when assessing the factor structure, nor would they correlate differently with other constructs. Put differently, they would not make a conceptual or empirical difference, and could be ignored safely. And, although it would require paring some items in existing measures because they lacked validity (e.g., see Levine & McCroskey, 1990), we would expect these many measures to fit a second order unidimensional factor model. We could then affix one label to them and relegate the alternative terms to the construct dump. Such a hypothesis awaits empirical test.
Example 4. Loneliness and Social Support. The first two examples suggest that we might advance theory and research by expanding constructs; abandoning Machiavellianism for four constructs, and replacing opinion leadership with three constructs. The third example took the opposite tack, suggesting the consolidation of the many terms and measures used to refer to communication anxiety. The fourth example continues in this direction, although in a different manner. It suggests the possibility that we might have redundant constructs so construed because they lie at opposite ends of a continuum.
Both loneliness and social support refer to the quality of our attachments with others. The former refers to their absence, the latter their presence. Because loneliness and social support measures predict various facets of physical and emotional well-being, they play an important role for scholars focusing on interpersonal relationships and on health communication.
In a 1986 manuscript Newcomb and Bentler examined the psychometric properties of two social support measures, the Social Resources Scale (see Newcomb & Bentler, 1986, p. 524 for items) and the Socially Supportive Relationships Scale (see Newcomb & Bentler, 1986, p. 524 for items), and two loneliness measures, The Differential Loneliness Scale (Schmidt & Sermat, 1983) and the UCLA Loneliness Scale (Russell et al., 1980). A second order factor analysis indicated that these four measures formed a single unidimensional second order factor. Newcomb and Bentler write,
. . . a single higher-order factor accounts for the correlations among the loneliness and social support constructs. Thus, the current data do not support a conceptual model that gives priority to support as generating decreased loneliness. The current data imply strong equivalence between the constructs. (1986, p. 532)
Judging from subsequent research (e.g., Zhang & Dong, 2022) scholars have either overlooked Newcomb and Bentler’s (1986) findings or, perhaps, found that they failed to hold for other measures of loneliness, social support, or both. Nevertheless, the point remains. We need exhibit caution in order to avoid mistaking an extreme on a continuum for a construct.
More examples would only add redundancy. Nevertheless, I should add two points. First, one can find misconstrued constructs in other content areas in communication science as well as those mentioned in the examples (e.g., organizational communication, Manata & Grubb, 2022). And, second, I have found measures employed commonly in research focusing on personality and social attitude research (such as authoritarianism: Adorno et al., 1950; Altemeyer, 1988 and dogmatism: Rokeach, 1960), particularly likely to have radically multidimensional factor structures, and thus fail to meet the criteria of a construct.
The examples provided illustrate two errors that we can make. We can identify, as a construct, a concept that does not meet the criteria required for a dimension of individual difference, and we can fail to identify a concept that meets those criteria. And, as the examples also illustrate, when we make the first error, we usually make the second.
On a positive note though, the first, second, and fourth examples also illustrate that we can avoid these errors by conducting careful psychometric work. Doing so would involve conducting and publishing measurement studies. And, in studies that do not focus on measurement alone (perhaps those focusing on cause and effect), it would entail engaging in rigorous tests of our measures before examining the effect of our focal construct on other variables, or the effect of other variables on our focal construct.
In the main, we would use confirmatory factor analysis as the primary statistical procedure to test for the dimensionality of our measures, focusing on the internal consistency and parallelism (see Hunter & Gerbing, 1982) of our indicators as a means of identifying those indicators with low validity. And, particularly, we would profit from using the internal consistency and parallelism theorems to generate predicted correlations among all indicators, subtracting these predicted correlations from those obtained, and then examining the magnitude of the resulting residuals, so that we could discard those indicators with large residuals (i.e., those likely to have low validity). Such a procedure, while time consuming, provides a much more fine-grained analysis than only examining commonly employed fit indices.
Popular confirmatory factor analysis algorithms assume linear item characteristic curves (ICCs), relationships between indicators and true score, and linear ICCs dominate communication research. Nevertheless, other types of ICCs do occur, in particular the ogive appearing occasionally. In such cases formulae generating predicted correlations for internal consistency and parallelism theorems differ, but do exist (Keating & Boster, 2019; Tracey, 2000).
The examples discussed involved self-report measures. Notably, the same points hold with respect to measures obtained from observers, such as coding group interaction, measures of artifacts, physiological measures, and the like. And, they also hold with respect to experimental inductions. Our inductions, like our measures, may lack validity, i.e., they may fail to induce what they intend to and nothing else. Or, put another way, they may induce confounds. Perhaps they deserve their own induction dump. And, as with our measures, careful pretesting and conducting pilot studies would contribute substantially to diminishing this problem.
Having the ability to operationalize, or operationize, would make our task easier. The problems with establishing that our measures and inductions have high validity or not would vanish. We would have no misidentified constructs. But, we do, and we also do know how to attack the problem. We must demand of ourselves and other scholars that we demonstrate that our purported constructs meet the criteria of a dimension of individual differences, and assign those that do not to the construct (or induction) dump.
|1.||Adorno, T. W., Frenkeel-Brunswick, E., Levinson, D. J., & Sanford, R. N. (1950). The authoritarian personality. Harper & Row.|
|2.||Altemeyer, B. (1988). Enemies of freedom: Understanding right-wing authoritarianism. Jossey-Bass.|
|3.||Boster, F. J., Kotowski, M. R., Andrews, K. R., & Serota, K. (2011). Identifying influence: Development and validation of the connectivity, persuasiveness, and mavens scales. Journal of Communication, 61(1), 178–196.
|4.||Bridgman, P. W. (1927). The logic of modern physics. MacMillan.|
|5.||Burgoon, J. K. (1976). The unwillingness-to-communicate scale: Development and validation. Communication Monographs, 43(1), 60–69.
|6.||Christie, F., & Geis, F. L. (1970). Studies in Machiavellianism. Academic Press.|
|7.||Clevenger, T., Jr. (1959). A synthesis of experimental research in stage fright. Quarterly Journal of Speech, 45(2), 134–145.
|8.||Eysenck, H. J. (1973) Eysenck on extraversion. John Wiley & Sons.|
|9.||Flynn, L. R., Goldsmith, R. E., & Eastman, J. K. (1996). Opinion leaders and opinion seekers: Two new measurement scales. Journal of the Academy of Marketing Science, 24(2), 137–147.
|10.||Hunter, J. E., & Gerbing, D. W. (1982). Unidimensional measurement, second order factor analysis, and causal models. In B. M. Staw & L. L. Cummings (Eds.), Research in organizational behavior (Vol. 4, pp. 267–320). JAI Press.|
|11.||Hunter, J. E., Gerbing, D. W., & Boster, F. J. (1982). Machiavellian beliefs and personality: Construct invalidity of the Machiavellianism dimension. Journal of Personality and Social Psychology, 43(6), 1293–1305.
|12.||Katz, E., & Fialkoff, Y. (2017). Six concepts in search of retirement. Annals of the International Communication Association, 41(1), 86–91.
|13.||Katz, E., & Lazarsfeld, P. F. (1955). Personal influence, the part played by people in the flow of mass communications. The Free Press.|
|14.||Keating, D. M., & Boster, F. J. (2019). Nonlinear unidimensionality in communication science: Tests, examples, and implications. Communication Research Reports, 36(1), 67–77.
|15.||Leary, M. R. (1983). Social anxiousness: The construct and its measurement. Journal of Personality Assessment, 47(1), 66–75.
|16.||Levine, T. R., & McCroskey, J. C. (1990). Measuring trait communication apprehension: A test of rival measurement models of the PRCA-24. Communication Monographs, 57(1), 62–72.
|17.||Manata, B. M., & Grubb, S. (2022). Conceptualizing leader-member exchange as a second order construct. Frontiers in Psychology, 13, 1–13.
|18.||McClelland, D. C. (1985). Human motivation. Scott, Foresman and Company.|
|19.||McCroskey, J. C. (2005). An introduction to rhetorical communication (9th ed.). Prentice Hall.|
|20.||Newcomb, M. D., & Bentler, P. M. (1986). Loneliness and social support: A confirmatory hierarchical analysis. Personality and Social Psychology Bulletin, 12(4), 520–535.
|21.||Rogers, E. (2003). Diffusion of innovation (5th ed.). Simon & Schuster.|
|22.||Rokeach, M. (1960). The open and closed mind. Basic Books.|
|23.||Russell, D., Peplau, L. A., & Cutrona, C. E. (1980). The revised UCLA Loneliness Scale: Concurrent and discriminant validity evidence. Journal of Personality and Social Psychology, 39(3), 472–480.
|24.||Schmidt, N., & Sermat, V. (1983). Measuring loneliness in different relationships. Journal of Personality and Social Psychology, 44(5), 1038–1047.
|25.||Stevens, S. S. (1935a). The operational definition of psychological concepts. Psychological Review, 42(6), 517–527.
|26.||Stevens, S. S. (1935b). The operational basis of psychology. The American Journal of Psychology, 47(2), 323–330.
|27.||Torgerson, W. S. (1958). Theory and method of scaling. John Wiley & Sons.|
|28.||Tracey, T. J. G. (2000). Analysis of circumplex models. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 641–664). Academic Press.
|29.||Zhang, X., & Dong, S. (2022). The relationships between social supports and loneliness: A meta-analysis. Acta Psychologica, 227, 1–11.
|30.||Zimbardo, P. G. (1977). Shyness. Jove.|