zyBooks- Chapter 1

1.1 Example: Paired designs Learning goals Identify a study design as having paired or independent groups. Identify a study design as paired using repeated measures or paired using matching. Does listening to music with lyrics interfere with studying? Many students like to study while listening to music. What does research say about how well students can focus while listening to music with lyrics? Researchers plan to design an experiment with 27 students with two treatments: listening to music with lyrics and listening to music without lyrics. The researchers will then compare students' performance on a memorization game: Students will study a list of 25 common ±ve-letter words for 90 seconds and then write down as many of the words as they can remember. PARTICIPATION ACTIVITY 1.1.1: Possible experimental design. Randomly assigned to groups 27 students With Lyrics Without Lyrics ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 1/78

PARTICIPATION ACTIVITY 1.1.2: Music study design. 1) The experimental units are _____. the students the lyrics the music 2) The explanatory variable is _____, which is a _____ variable. the number of words memorized; quantitative whether the student listened to music with lyrics; categorical Animation content: Static ±gure: A possible study design to measure the effect of listening to music with lyrics on memorization. Step 1: Suppose 14 students are randomly assigned to listen to music with lyrics and 13 students are randomly assigned to listen to music with no lyrics. Step 2: Each of the 27 individuals provides exactly one response value, which is the number of words memorized. Data Student Music Type # Words 1 With lyrics 5 ... ... ... 14 With lyrics 10 15 No lyrics 5 ... ... ... 27 No lyrics 10 Animation captions: 1. Suppose 14 students are randomly assigned to listen to music with lyrics and 13 students are randomly assigned to listen to music with no lyrics. 2. Each of the 27 individuals provides exactly one response value, which is the number of words memorized. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 2/78

Randomly assigned first treatment With Lyrics Without Lyrics Animation content: Static ±gure: A paired study design is explored to minimize the chance of unequal groups on the observed difference in effect. Step 1: In an independent groups design for the music study, more of the people with better memorization skills could end up in the without lyrics group by random chance. Step 2: To account for the variability in scores due to prior differences in memorization skills between people, a paired study design instead asks each student in the study to play the memorization game twice, once with lyrics and once without. Step 3: Randomization is used to decide which treatment the students participate in ±rst. Step 4: The resulting dataset has two measurements of the number of words remembered for each person. Paired study design data Student First Music Type # Words Second Music Type # Words 1 With lyrics 5 No lyrics 6 ... ... ... ... ... 14 With lyrics 10 No lyrics 7 15 No lyrics 5 With lyrics 8 ... ... ... ... ... 27 No lyrics 10 With lyrics 11 Animation captions: 1. In an independent groups design for the music study, more of the people with better memorization skills could end up in the without lyrics group by random chance. 2. To account for the variability in scores due to prior differences in memorization skills between people, a paired study design instead asks each student in the study to play the memorization game twice, once with lyrics and once without. 3. Randomization is used to decide which treatment the students participate in ±rst. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 4/78

PARTICIPATION ACTIVITY 1.1.4: Paired design in the music study. 1) In a paired study design, randomness is present in _____. deciding which treatment is given to each experimental unit deciding which experimental units are paired deciding the orders of the treatments 2) Using a paired study design, any difference in words memorized by a particular student will be mainly due to _____. randomness the explanatory variable confounding variables 3) In using the paired study design, the Lyrics and Without Lyrics groups are ___ to be balanced on general memorization skill. likely guaranteed not likely Other paired designs A paired design using repeated measurements is a study in which each experimental unit is measured twice under different conditions where the order of the conditions is randomly assigned. Randomizing the order of the treatments is important to guard against any changes over time. Ex: Both familiarity with the memorization game and fatigue can change over time and could affect the number of words memorized. But changes over time are accounted for in the music study by randomizing the order of the treatments. Having each subject experience both treatments is not always an option, and other paired designs are possible. A paired design using matching creates pairs by matching up two very similar experimental units and then randomly assigning one to each treatment. 4. The resulting dataset has two measurements of the number of words remembered for each person. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 5/78

1) In the above paired design using matching, randomness is present in _____. deciding which treatment each experimental unit receives deciding which experimental units are paired deciding the orders of the treatments 2) In the above paired design using matching, the Lyrics and Without Lyrics groups are _____ to be balanced on general memorization skill. likely not likely Can observational studies use pairing? The main goal of a paired design is to create treatment groups that are even more likely to be similar to each other, at least on "important" potential confounding variables. Ex: General memorization ability is clearly related to the response variable for the music study, and the study would be much less informative if random assignment happened to put more of the better memorizers into one of the treatment groups than another. A paired design using matching guards against such unfortunate events and helps the researchers better compare the explanatory variable conditions of interest. Less important variables are hopefully still balanced by the random assignment process. Ex: Age is not as relevant as general matching skills to the number of words memorized, but the random assignment process will still help account for differences between older and younger participants in number of words. Paired designs are not limited to experiments, as pairing can also be part of the design of an observational study. Randomization is no longer used within each pair, but pairing will still "control" for an important source of variation. PARTICIPATION ACTIVITY 1.1.7: Advantages of paired designs. Research question Independent groups design Paired design Advantage of paired design In heterosexual couples, do married men tend to be older than married women? Record the ages of a random sample of married men and an independent random sample of married women. Record the ages for both the wife and the husband of a random sample of married couples. The difference in ages the same couple will te to be smaller than diffe in ages among married and married women. Does Store A tend to have cheaper Record the prices of a random sample of Record the prices at both store A and store B for a Pairing accounts for the variation in prices of gr ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 7/78

PARTICIPATION ACTIVITY 1.1.8: Identifying designs as paired or independent. For the proposed studies below, identify the study plan as paired or independent groups prices than Store B? products from store A and an independent sample of products from store B. random sample of products common to both stores. store products. Animation content: Static ±gure: An independent groups design is compared to a paired design for two possible research questions. Step 1: For the research question, "In heterosexual couples, do married men tend to be older than married women?", how would an independent groups design differ from a paired design using matching? Independent groups design: Record the ages of a random sample of married men and an independent random sample of married women. Paired design: Record the ages for both the wife and husband of a random sample of married couples. Step 2: Why is the pairing likely to be helpful? The differences will have less variability than the ages. The difference in ages in the same couple will tend to be smaller than differences in ages among married men and married women. Step 3: For the research question, "Does Store A tend to have cheaper prices than Store B?", how would an independent groups design differ from a paired design using matching? Independent groups design: Record the prices of a random sample of products from store A and an independent sample of products from store B. Paired design: Record the prices at both store A and store B for a random sample of products common to both stores. Step 4: What source of variation does the pairing account for? The variation in the differences in prices of the same product should be smaller than the variation in grocery store products. Pairing accounts for the wide variation in prices of grocery store products. Animation captions: 1. For the research question, "In heterosexual couples, do married men tend to be older than married women?", how would an independent groups design differ from a paired design using matching? 2. Why is the pairing likely to be helpful? The differences will have less variability than the ages. 3. For the research question, "Does Store A tend to have cheaper prices than Store B?", how would an independent groups design differ from a paired design using matching? 4. What source of variation does the pairing account for? The variation in the differences in prices of the same product should be smaller than the variation in grocery store products. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 8/78

1.2 Example: Simulation-based approach for analyzing paired data Learning goals Understand the difference between independent samples and paired samples in terms of the study design, how variability can be lower in a paired design, and how this can in±uence the strength of evidence. Complete a simulation-based test of signi²cance of a paired design by writing out the hypothesis, determining the observed statistic, computing the p-value, and writing out an appropriate conclusion. Does the path taken to "round" a base make a difference? In baseball, after the batter hits the ball, the batter attempts to run to ±rst base (single), second base (double), or third base (triple) as fast as possible. Suppose a baseball player is trying to stretch a single to a double. Does the strategy used to "round" ±rst base make a difference? A graduate student ( Woodward, 1970 ) decided to compare times for 22 runners to run from home plate to second base using a narrow angle and a wide angle. Step 1: Do the running times using the narrow-angle strategy tend to differ from the running times using the wide-angle strategy? PARTICIPATION ACTIVITY 1.2.1: Rounding bases study design and data collection. Runner Narrow angle Wide angle 1 2 3 5.50 5.70 5.60 5.55 5.75 5.50 4 5 6 5.50 5.85 5.55 5.40 5.70 5.60 7 8 5.40 5.50 5.35 5.35 9 5.15 5.00 Narrow angle ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 10/78

PARTICIPATION ACTIVITY 1.2.2: Rounding bases study design. 10 ... 5.80 ... 5.70 ... Wide angle Animation content: Static ±gure: The study design and data of the rounding bases study. Step 1: Woodward wanted to determine whether rounding ±rst base at a narrow-angle strategy or a wide-angle strategy makes a difference in time to travel from home to second base. Narrow angle stays the same distance from the baseball diamond throughout, while a wide angle is far until ±rst base and then close between ±rst base and second base. Step 2: Each runner ran twice, once by taking a wide-angle strategy and once by taking a narrow- angle strategy, with a rest period in between. Both times are recorded in seconds. Step 3: The order of the strategy was randomized for each runner. Some runners ran the narrow angle before the wide angle and others ran the wide angle ±rst. Step 4: For each of the 22 runners, times are recorded for both the narrow-angle and wide-angle strategies. Rounding bases study data Runner Narrow angle (seconds) Wide angle (seconds) 1 5.50 5.55 2 5.70 5.75 3 5.60 5.50 ... ... ... Animation captions: 1. Woodward wanted to determine whether rounding ±rst base at a narrow-angle strategy or a wide-angle strategy makes a difference in time to travel from home to second base. 2. Each runner ran twice, once by taking a wide-angle strategy and once by taking a narrow-angle strategy, with a rest period in between. Both times are recorded in seconds. 3. The order of the strategy was randomized for each runner. 4. For each of the 22 runners, times are recorded for both the narrow-angle and wide-angle strategies. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 11/78

6) Randomizing which angle runners used ±rst and the rest period control for _____. runner-to-runner variability sample-to-sample variability fatigue as a confounding variable Base-running times as matched pairs A paired design allows a more direct comparison of the effects of the treatment conditions: the two base- running strategies. To analyze the data, rather than using the run times from each strategy, the analysis can use the differences (\(d\)) in the two running times for each runner: \[d = (\text{narrow angle time}) - (\text{wide angle time}).\] The time difference \(d\) is a single quantitative response variable, which means that the usual one-sample methods can be applied. In the context of a single sample of differences, the sample statistics can be written using the subscript "d": \(\overline{x}_d\) : mean of the differences in run time between the narrow-angle and wide-angle strategies \(s_d\) : standard deviation of the differences in run time between the narrow-angle and wide-angle strategies. PARTICIPATION ACTIVITY 1.2.3: Exploring paired data: Calculating the mean of the differences. Runner Narrow Wide Difference 1 2 22 ... ... ... ... 5 5.35 5.15 4.95 5.45 5 0.05 -0.10 0.15 Time (sec) Differences (sec) ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 13/78

Animation content: Static ±gure: Calculating the men of the differences in running times. Step 1: The data for the narrow-angle and wide-angle times are paired, so such data should not be treated as coming from two independent samples. Instead, the differences in times are examined. Rounding bases study data Runner Narrow angle (seconds) Wide angle (seconds) Difference (seconds) 1 5.00 4.95 0.05 2 5.35 5.45 -0.10 ... ... ... ... 22 5.15 5.00 0.15 Both the narrow and wide running times cluster around 5.5 and range from approximately 4.9 to 6.4. Step 2: Runner number 1 took longer on the narrow angle than the wide angle, so the difference in time is 0.05 seconds. Step 3: Runner number 2 took longer on the wide angle than on the narrow angle, so the difference in time is -0.10 seconds. Step 4: The distribution of the differences in running times has a mean of \(\overline{x}_d\) = 0.075 sec. and a standard deviation of \(s_d\) = 0.0883 sec. The distribution of differences range from -0.1 to 0.2. The majority of the differences are greater than 0. Summary statistics Sample size, \(n\) Sample mean Sample SD Narrow 22 \(\bar{x}_{n} = 5.534\) \(s_{n} = 0.260\) Wide 22 \(\bar{x}_{w} = 5.459\) \(s_{w} = 0.273\) Difference 22 \(\bar{x}_{d} = 0.075\) \(s_{d} = 0.088\) Animation captions: 1. The data for the narrow-angle and wide-angle times are paired, so such data should not be treated as coming from two independent samples. Instead, the differences in times are examined. 2. Runner number 1 took longer on the narrow angle than the wide angle, so the difference in time is 0.05 seconds. 3. Runner number 2 took longer on the wide angle than on the narrow angle, so the difference in time is -0.10 seconds. 4. The distribution of the differences in running times has a mean of \(\overline{x}_d\) = 0.075 sec. and a standard deviation of \(s_d\) = 0.0883 sec. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 14/78

5) The variability of the differences in running time is _____ the variability in running times between runners. less than about the same as greater than Step 4: A chance model for paired-base running data As hoped with the paired design, the variation in the time differences (\(s_{d}\) = 0.088 second) is smaller than the variation in running times for each of the base-running strategies (\(s_{narrow}\) = 0.260, \(s_{wide}\) = 0.273 seconds). To decide whether the observed value of the mean difference (\(\bar{x}_d\) = 0.075 seconds) in run times is signi±cantly above 0, the paired design also needs to be re²ected in the chance model. The null hypothesis assumes no association between strategy and run time (a mean of the differences in run times equal to 0), so within each pair of individual run times, the narrow and wide run times are interchangeable. So to generate a set of "could-have-been" outcomes, the simulation will randomly decide whether or not to swap the two observations for each runner. PARTICIPATION ACTIVITY 1.2.5: A chance model for the mean of the differences for paired data. Narrow Wide Swap? Diff ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 5.50 5.55 0.05 5.70 5.75 - 0.05 5.60 5.50 0.10 5.50 5.40 0.10 5.85 5.70 0.15 5.55 5.60 - 0.05 5.40 5.35 0.05 5.50 5.35 0.15 5.15 5.00 0.15 5.80 5.70 0.10 5.20 5.10 0.10 5.55 5.45 0.10 5.35 5.45 - 0.10 5.00 4.95 0.05 5.50 5.40 0.10 Y N N Y ? ? ? ? ? ? ? ? ? ? ? - Differences Outcomes -0.007 { 1000 repetitions 0.039 0.052 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 16/78

Average difference 16 17 18 19 20 21 22 5.55 5.50 0.05 5.55 5.35 0.20 5.50 5.55 - 0.05 5.45 5.25 0.20 5.60 5.40 0.20 5.65 5.55 0.10 5.30 5.25 0.05 ? ? ? ? ? ? ? { 1000 Animation content: Static ±gure: Creating a null distribution of simulated average differences. Step 1: If the wide and narrow strategies take the same amount of time, the times in each pair can be swapped at random (Ex: ²ip a coin) to generate a repetition of the study, assuming the type of angle doesn't matter. Step 2: If a coin lands heads, the wide and narrow times are swapped for that observation. When the times are swapped, the sign of the difference changes. Ex: a runner's narrow time is 5.50 and their wide time is 5.55 which corresponds to a difference of 5.50 - 5.55 = -0.05. The coin lands on heads, so the times are switched. The new difference is narrow - wide = 5.55 - 5.50 = 0.05. Step 3: A coin is ²ipped for each pair. In the dot plot of the differences, the difference for each runner either stays the same or changes sign. Step 4: The value of the mean of the simulated differences in run time is calculated for the ±rst repetition. The mean of the simulated differences is -0.007. Step 5: A second set of shu³es produces a second simulated value of the mean difference in run time assuming the wide and narrow angles take the same amount of time in the long run. The data is shu³ed by the same process. The second simulation produces a mean difference of 0.039. Step 6: A simulated null distribution for the sample mean difference in run times, assuming the two running angles take the same amount of time, is made by performing 998 additional repetitions. 1000 total randomizations creates a null distribution of average differences. The distribution is bell shaped and centered at 0. The values range from approximately -0.7 to 0.7. Animation captions: 1. If the wide and narrow strategies take the same amount of time, the times in each pair can be swapped at random (Ex: ²ip a coin) to generate a repetition of the study, assuming the type of angle doesn't matter. 2. If a coin lands heads, the wide and narrow times are swapped for that observation. When the times are swapped, the sign of the difference changes. 3. A coin is ²ipped for each pair. In the dot plot of the differences, the difference for each runner either stays the same or changes sign. 4. The value of the mean of the simulated differences in run time is calculated for the ±rst repetition. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 17/78

2) A repetition of the simulation generating a negative \(\overline{x}_d\) means _____. something went wrong as negative times are impossible all runners ran faster using the narrow angle individual runners' times using the narrow path were faster on average 3) The center of the null distribution should be 0 because _____. the chance model assumes that \(H_0\) is true the chance model assumes that all groups are paired the statistic is close to zero 4) The shape of the null distribution is _____. skewed to the left skewed to the right bell-shaped 5) A p-value could be calculated by ±nding the proportion of repetitions in the null distribution which have a statistic \ (\bar{x}_d\) of _____. 0 or greater 0.052 or greater 0.052 or greater and -0.052 or smaller 0.075 or greater 0.075 or greater and -0.075 or less Step 4, cont.: Measuring the strength of evidence The chance model can be used to estimate the p-value and standardized statistic as usual. ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 19/78

PARTICIPATION ACTIVITY 1.2.7: Measuring strength of evidence using the simulated null distribution of the average differences. Average difference Measuring strength of evidence p-value \( = \frac{0+0}{1000}\) Standardized statistic : \( \ \ \ \ \ \ \ \ \ \bar{x}_d - \) Mean = 0.000 SD = 0.024 \(0.024\) \(0\) Animation content: Static ±gure: Measuring the strength of evidence using the simulated null distribution. The observed average difference is \(\bar{x}_d = 0.075\). Step 1: The simulated null distribution of the average differences in run time for the two strategies found a mean of 0 and a standard deviation of 0.024. The simulated null distribution is created from 1000 repetitions. The distribution is approximately bell shaped. Step 2: The two-sided p-value can be calculated by counting the number of repetitions that have a simulated mean difference at least as extreme as 0.075 seconds. The estimated p-value is less than 0.001. None of the simulated average differences are less than or equal to -0.075 and none of the simulated average differences are greater or equal to 0.075. Therefore, the simulation-based p-value \(= \frac{0+0}{1000}\), Step 3: A standardized statistic can be calculated by subtracting the hypothesized value of the parameter and dividing by the standard error of the average difference, which can be estimated from the null distribution. Standardized statistic \(= \dfrac{\bar{x}_d - 0}{0.024}\). Animation captions: 1. The simulated null distribution of the average differences in run time for the two strategies found a mean of 0 and a standard deviation of 0.024. 2. The two-sided p-value can be calculated by counting the number of repetitions that have a simulated mean difference at least as extreme as 0.075 seconds. The estimated p-value is less ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 ©zyBooks 06/05/24 21:24 2090300 Melinie Weaver MTH_218_58079156 6/5/24, 9:24 PM zyBooks https://learn.zybooks.com/zybook/MTH_218_58079156/chapter/1/print 20/78