PSY 2307 - Chapter 5 Lecture

pdf

School

University of Houston, Downtown *

*We aren’t endorsed by this school

Course

2307

Subject

Psychology

Date

Oct 30, 2023

Type

pdf

Pages

Uploaded by GrandSnow12337

Chapter 5: Identifying Good Measurement PSY 2307 CHAPTER 5 IS DUE FRIDAY, OCTOBER 27, 2023, AT 11:59 PM, CST. 1

Outline • Three Common Types of Measures − Self-report Measures − Observational Measures − Physiological Measures • Scales of Measurement − Categorical vs. Quantitative Variables • Reliability of Measurement • Validity of Measurement • The Relationship Between Reliability and Validity 2

Introduction • Systematic observations require good measurement tools • Most psychological phenomena are difficult to measure directly • Learning Objective : Evaluate the reliability and validity of measured variables in psychological science No matter what a psychologist is interested in studying, every scientist faces the challenge of measurement. When researchers test theories or pursue empirical questions, they have to systematically observe the phenomena by collecting data. Such systematic observations require measurements, and these measurements must be good ones — or else they are useless. Measurement in psychological research can be particularly challenging. Many of the phenomena psychologists are interested in — motivation, emotion, thinking, reasoning — are difficult to measure directly. Happiness, the topic of much research, is a good example of a construct that could be hard to assess. Is it really possible to quantify how happy people are? Are the measurements accurate? This chapter explains how to ask questions about the quality of a study’s measures. You will learn, first, about different ways researchers operationalize measured variables. Then you’ll learn how you can evaluate the reliability and validity of those measurements. 3

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Three Common Types of Measures • There are three common types of measures: 1. Self-report 2. Observational 3. Physiological In Chapter 3, you learned about operationalization, the process of turning a construct of interest into a measured or manipulated variable. Much psychological research requires two definitions of each variable. The conceptual definition, or construct, is the researcher’s definition of the variable in question at a theoretical level (e.g., intelligence). The operational definition represents a researcher’s specific decision about how to measure or manipulate the conceptual variable (e.g., IQ test, GPA). The process of measuring variables involves some key decisions. As researchers decide how they should operationalize each variable in a study, they choose among three common types of measures: self-report , observational , and physiological . They also decide on the most appropriate scale of measurement for each variable they plan to investigate. 4

Self-report Measures • Self-report : Recording people’s answers to questions about themselves in a questionnaire or interview − E.g., Likert scales • Can be completed by oneself or another person A self-report measure operationalizes a variable by recording people’s answers to questions about themselves in a questionnaire or interview. To assess happiness or satisfaction with life, for example, a researcher my develop a rating scale with ten items that ask you to respond to statements like “I am satisfied with my life,” “In most ways, my life is close to my ideal,” and “The conditions of my life are excellent.” or How much do you agree or disagree with a particular statement (e.g., Do you think you are smarter than average). Self-reports do not always need to be completed by the person or population of interest. For example, in research on children, self-reports may be replaced with parent reports or teacher reports. These measures ask parents or teachers to respond to a series of questions, such as describing the child’s eating behaviors, the words the child knows, or the child’s typical classroom behaviors. 5

Observational Measures • Observational : Recording observable behaviors or physical traces of behavior − E.g., Counting the frequency of an observable behavior An observational measure, sometimes called a behavioral measure, operationalizes a variable by recording observable behaviors or physical traces of behaviors. For example, a researcher could operationalize happiness by observing how many times a person smiles. Intelligence tests can be considered observational measures, because the people who administer such tests in person are observing people’s intelligent behaviors (such as being able to correctly solve a puzzle or quickly detect a pattern). Observational measures may record physical traces of behavior. Stress behaviors could be measured by counting how many times someone fidgets in a certain time frame. 6

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Physiological Measures • Physiological : Recording biological data − E.g., Brain activity, hormone levels, heart rate • Requires the use of equipment to amplify, record, and analyze data A physiological measure operationalizes a variable by recording biological data, such as brain activity, hormone levels, or heart rate. Physiological measures usually require the use of equipment to amplify, record, and analyze biological data. For example, moment-to-moment happiness has been measured using facial electromyography (EMG) — a way of electronically recording tiny movements in the muscles in the face. Facial EMG can be said to detect a happy facial expression because people who are smiling show particular patterns of muscle movement around the eyes and cheeks. Other constructs might be measured using a brain scanning technique called functional magnetic resonance imaging, or fMRI. In a typical fMRI study, people engage in a carefully structured series of psychological tasks (such as looking at three types of photos or playing a series of rock-paper-scissors games) while lying in an MRI machine. The MRI equipment records and codes the relative changes in blood flow in particular regions of the brain. When more blood flows to a brain region while people perform a certain task, researchers conclude that brain area is activated because of 7

the patterns on the scanned images. A physiological way to operationalize stress might be to measure the amount of the hormone cortisol released in saliva because people under stress show higher levels of cortisol. Skin conductance, an electronic recording of the activity in the sweat glands of the hands or feet, is another way to measure stress physiologically. 7

Scales of Measurement • Categorical variable : When the levels of a variable are categories − E.g., Sex, race, marital status, level of education • Quantitative variable: When the levels of a variable have meaningful numbers − E.g., Height, weight, IQ score, GPA All variables must have at least two levels (see Chapter 3). The levels of operational variables, however, can be coded using different scales of measurement. Examples are sex, whose levels are male and female; race, martial status, level of education. In contrast, the levels of quantitative variables are coded with meaningful numbers. Height and weight are quantitative because they are measured in numbers. Same with someone's IQ score, or GPA. 8

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Reliability of Measurement • Reliability : How consistent the results of a measure are 1. Test-retest 2. Interrater reliability 3. Internal reliability Now that we’ve established different types of operationalizations, we can ask the important questions about reliability and validity. Before deciding on the measures to use in a study, researchers collect their own data or review data collected by others. They use data because establishing the reliability of a measure is an empirical question. A measure’s reliability is just what the word suggests: whether or not researchers can rely on a particular score. If an operationalization is reliable, it will yield a consistent pattern of scores every time. Reliability refers to how consistent the results of a measure are. Reliability can be assessed in three ways, depending on how a variable was operationalized, and all three involve consistency in measurement. With test-retest reliability , the researcher gets consistent scores every time he or she uses the measure (getting similar results on an IQ test). With interrater reliability , consistent scores are obtained no matter who measures the variable. In other words, two or more independent observers will come up with consistent (or very similar) findings. Interrater reliability is most relevant for 9

observational measures. Suppose you are assigned to observe the number of times each child smiles in 1 hour at a daycare playground. Your lab partner is assigned to sit on the other side of the playground and make his own count of the same children’s smiles With internal reliability , a study participant gives a consistent pattern of answers, no matter how the researcher has phrased the question. The third kind of reliability, internal reliability, applies only to self-report scales with multiple items. 9

Validity of Measurement • Validity : Determines whether the operationalization is measuring what it is supposed to be measuring 1. Face validity 2. Content validity Before using particular operationalizations in a study, researchers not only check to be sure the measures are reliable; they also want to be sure they get at the conceptual variables they were intended for. A measure has face validity if it is subjectively considered to be a plausible operationalization of the conceptual variable in question. If it looks like a good measure, it has face validity. For example, speed of problem solving, vocabulary size, and creativity have higher face validity as operationalizations of intelligence. Researchers generally check face validity by consulting experts. Content validity also involves subjective judgment. To ensure content validity, a measure must capture all parts of a defined construct. For example, consider this conceptual definition of intelligence, which contains distinct elements, including the ability to “reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly, and learn from experience” (Gottfredson, 1997, p. 13). To have adequate content validity, any operationalization of intelligence should include questions or items to assess each of these seven components. 10

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

So, how do you identify a good measurement? • Reliability and validity are NOT the same • Reliability is necessary (but not sufficient) for validity • Reliability and validity information can be found in the Method section of an empirical article One essential point is worth reiterating: The validity of a measure is not the same as its reliability. A journalist might boast that some operationalization of behavior is “a very reliable test,” but to say that a measure is “reliable” is only half the story. It is also important to note that reliability is necessary (but not sufficient) for validity. For example, suppose you used your pedometer to count how many steps there are in your daily walk from your parking spot to your building. If the pedometer reading is very different day to day, then the measure is unreliable — and of course, it also cannot be valid because the true distance of your walk has not changed. Therefore, a measure cannot be more valid than it is reliable. When you read a research study, you should be asking: Did the researchers collect evidence that the measures they are using have construct validity? If they didn’t do it themselves, did they review construct validity evidence provided by others? In empirical journal articles, you’ll usually find reliability and validity information in the Method section, where the authors describe their measures. 11

Chapter 5 Key Concepts • Three Common Types of Measures − Self-report, observational, physiological • Scales of Measurement − Categorical vs. Quantitative Variables • Reliability of Measurement − Test-retest, interrater, internal • Validity of Measurement − Face, content • The Relationship Between Reliability and Validity 12