Chapter 5- Measurement- Linking Theory to Research

1. A researcher interested in measuring religiosity first defines the meaning and identifies possible dimensions of this concept. Which step in the measurement process does this represent? A) conceptualization B) validity assessment C) operationalization D) index construction 2. A researcher interested in measuring religiosity decides to ask respondents how often they pray and how often they listen to religious programs on the radio. These questions represent the researcher's attempt to A) provide a conceptual definition of the concept. B) refine the meaning of the concept. C) specify empirical indicators. D) assess the quality of the researcher's operational definition. 3. Some sociologists define “formalization” as the extent of regulation in an organization. In one study this was indicated by the number of the following types of documents that organizations had created: written employment contracts, information booklets, an organization chart, written job descriptions, and a manual of procedures. The total count of documents is an example of A) conceptualization. B) operationalization. C) reliability assessment. D) validity assessment. 4. Which of the following is an empirical indicator of alienation? A) a feeling of estrangement from a situation, group, or culture B) the inability to control one's immediate work processes C) lack of a sense of purpose that connects one's job to the overall organization of production D) whether an individual agrees with the statement “I am often left out of things going on around me” 5. In social measurement, indicator is to index as __________ is to __________. A) measured; manipulated B) one questionnaire item; two or more items C) observation; verbal reports D) reliable; unreliable 6. (Box 5.1) Creating an index by adding together multiple indicators of a concept Page 1

A) is likely to compound the error associated with each of the individual indicators. B) will improve the quality of the measure if the error of individual indicators tends to be random. C) tends to obscure the meaning of the operational definition. D) works better with simple than with complex concepts. 7. Manipulation operations are used most commonly in A) experiments. B) surveys. C) qualitative field research. D) the analysis of existing data. 8. In Pager's study of the impact of criminal record on employment opportunity, what type of operational definition did she use to measure “employment opportunity”? A) manipulation operations B) verbal report C) observation D) archival records 9. A variable's level of measurement indicates the A) number of questions used to measure the variable. B) number of variable categories. C) kinds of comparisons that can be made between cases in different variable categories. D) degree of correspondence between conceptual and operational definitions. 10. Which of the following is an example of nominal-level measurement? A) family size (number of children in a family) B) political participation (number of times voted in the last 10 civic elections) C) educational attainment (highest year or grade in school completed) D) political party affiliation (Republican/Democrat/Independent/Other/None) E) religiosity (whether one attends religious services frequently, occasionally, or never) 11. A survey question measuring frequency of exercise asks respondents, “On how many of the past 7 days did you exercise or participate in sports activities?” The answer categories are 0, 1 – 3, 3 – 5, 5 or more. This set of categories is A) exhaustive but not mutually exclusive. B) mutually exclusive but not exhaustive. C) both exhaustive and mutually exclusive. Page 2

E) use quantitative measures whenever possible. 17. Which of the following questions most clearly concerns validity and not reliability? A) Do self-reports of alcohol consumption underestimate the incidence of binge drinking? B) Do respondents' answers to the question change with repeated questioning? C) What is the average inter-item correlation among the scale items? D) Do different coders agree in their ratings of violence on Saturday morning cartoons? 18. Which of the following questions concerns reliability ? A) What does the Scholastic Aptitude Test (SAT) really measure? B) Do attitude scales tap personal feelings or just what people think investigators want to hear? C) Is poverty best represented by measures of low absolute income (a minimum subsistence) or low relative income (e.g., 50 percent of national median income)? D) How closely does self-reported church attendance correspond to actual church attendance? E) Assuming that students did not learn the answers in the interim, would they score the same on this exam if they took it again tomorrow? 19. (Box 5.3) Scales differ from indexes in terms of A) what they are used to measure. B) how items are combined to create the measure. C) the type of indicators or survey items included in the measure. D) how their relationships with other variables can be analyzed. 20. (Box 5.3) A researcher creates a composite measure of attitudes toward research methods consisting of 20 statements with which respondents indicate their level of agreement or disagreement (“strongly agree” to “strongly disagree”). The researcher's measure is an example of A) an index. B) a Guttman scale. C) a summated ratings scale. D) a cumulative scale. 21. Survey estimates of church attendance based on self-reports are very stable, as respondents tend to give consistent answers. Observations of church attendance indicate, however, that it is about one-half the level reported in surveys. This suggests that, as a measure of church attendance, self-reported attendance is Page 4

A) reliable but not valid. B) valid but not reliable. C) neither reliable nor valid. D) both reliable and valid. 22. Suppose a researcher asks the exact same question at two different points in a lengthy interview, near the beginning and near the end. If respondents give the same answer each time the question is asked, this supports the question's A) test-retest reliability. B) internal consistency. C) convergent validity. D) construct validity. 23. For various reasons, which form of reliability assessment is least likely to be used? A) inter-rater reliability B) internal consistency C) test-retest reliability 24. Two observers rate individual leadership by recording the number of times that each member of a group initiates discussion. The level of agreement between the observers' independent records provides evidence of A) reliability. B) validity. C) neither reliability nor validity. D) both reliability and validity. 25. Which of the following statements correctly applies to the reliability of this exam? A) If nearly everyone misses a particular question, this will lower the exam's reliability. B) The best way to assess the exam's reliability is to carefully judge how representative the questions are of the course content. C) Ambiguously worded questions that evoke guessing have little or no effect on reliability. D) Increasing the number of questions on the exam is likely to increase its reliability. 26. Reliability assessment is a matter of checking for __________, whereas validity assessment is a matter of determining the degree of __________. A) accuracy; consistency B) consistency; accuracy Page 5

32. The feedback loop in the measurement process A) is consistent with the cyclical relation between theory and data. B) moves from theoretical concepts to data. C) reflects the deductive model of inquiry. D) consists of conducting tests for reliability and validity. 33. The measurement process moves from the concrete to the abstract. A) True B) False 34. The first steps in the measurement process—conceptualization and operationalization—follow the deductive model of inquiry. A) True B) False 35. Operational definitions are intended to capture the meaning of conceptual definitions. A) True B) False 36. Operational definitions may be manipulated or measured. A) True B) False 37. The most common form of social measurement is direct observation. A) True B) False 38. One empirical indicator is as good as another. A) True B) False 39. Measurement level is determined by the number of empirical indicators used to operationalize a concept. A) True B) False Page 7

40. In nominal measurement, the numbers assigned to categories may be added together but not multiplied. A) True B) False 41. Interval-level measures have an arbitrary zero point. A) True B) False 42. Class rank in a graduation class is an example of ratio measurement. A) True B) False 43. There is almost always an imperfect correspondence between conceptual and operational definitions. A) True B) False 44. It is possible for a measure to be reliable but invalid. A) True B) False 45. It is possible for a measure to be valid but unreliable. A) True B) False 46. Indexes and scales are types of composite measures. (Box 5.3) A) True B) False 47. Indexes are generally superior to scales. (Box 5.3) A) True B) False 48. Guttman scaling is a method of constructing a unidimensional scale. (Box 5.3) A) True B) False Page 8

57. Levels of measurement indicate the inferences that can be drawn when comparing one category or value of a variable with another: Do they differ? Is one ranked higher than the other? What is the numerical difference? What is the ratio of the values? Identify the four levels of measurement. Then, for each level describe the inferences that can be made when comparing units (e.g., people) who are in different variable categories. 58. (Box 5.3) Sociologists Richard Schwartz and James Miller proposed that changes in societies' legal systems parallel changes in societal complexity (or, in Durkheim's terms, the societal “division of labor”). Fully developed legal systems are seen to have the following three characteristics: a. Mediation : regular use of third parties to settle disputes b. Police : use of specialized, armed force to enforce laws c. Counsel : regular use of specialized, nonkin advocates (e.g., lawyers) Assuming that these three characteristics, in order, represent a progression of complexity, present a table that shows the response pattern of a perfect Guttman scale. Then give an example of a “nonscale” response pattern. 59. Describe the relationship between reliability and validity. (NOTE: Your description should answer the following two questions: Is it possible to have a reliable but invalid measure? Is it possible to have an unreliable but valid measure?) 60. Explain how construct validation (a) depends on the theory underlying the concept that is being measured, and (b) is based on an accumulation of evidence. 61. Figure 5.2 in the text presents a flowchart of the measurement process. Using this figure as a reference, discuss how this process reflects the cyclical relationship between theory and data in science. What part of the figure reflects the deductive logic of inquiry? What part reflects the inductive logic of inquiry? Page 10

Answer Key 1. A 2. C 3. B 4. D 5. B 6. B 7. A 8. C 9. C 10. D 11. A 12. B 13. D 14. C 15. D 16. E 17. A 18. E 19. B 20. C 21. A 22. A 23. C 24. A 25. D 26. B 27. D 28. D 29. B 30. D 31. B 32. A 33. B 34. A 35. A 36. A 37. B 38. B 39. B 40. B 41. A 42. B 43. A 44. A Page 11