chap 10 Expect_The_Unexpected_A_First_Course_In_Biostatist..._----_(Statistics) (3)

Chapter 10 Comparison of Two Independent Samples Biologists are often interested in the comparison of groups. Consider the following examples. Do two different species of swallow produce similar eggs on average? Does a type of fertilizer produce larger plants on average, com- pared to another type of fertilizer? In this chapter, we introduce methods to compare two independent groups. We discuss how interval estimation and hypothesis testing can be used to infer whether there are differences be- tween the two populations. We first discuss techniques to compare means, and end the chapter with techniques to compare proportions. 10.1 Study/Experimental Design When analyzing data, it is important to consider the design of the study or experiment. This is especially true when comparing groups. The design of the study often dictates the probability model that will be used to describe the data collection process from the populations of interest. It is only when the probability model is appropriate, that we can generalize our results from the samples to the populations. Scientists often want to compare groups that are outcomes from a con- trolled experiment which is run under different experimental conditions. For example, a simple experiment might be designed to test a claim that a particular type of fertilizer produces taller plants compared to another type of fertilizer. The response variable in this instance is the height of the plants. The primary factor for this experiment is the fertilizer. The levels of the factor are called treatments . So the treatments in this case are the types of fertilizer. In a controlled experiment we assign the treatments to the experimental units, which could be plots with one seedling in this case. This assignment determines the treatment groups. 163 Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

164 Expect the Unexpected: A First Course in Biostatistics It is possible that there are uncontrolled factors that might affect the response variable. These are called nuisance factors . For example the genetic predisposition of a seedling to produce a tall plant might be a nuisance factor. Randomization is used to average the effects of the nuisance factors over the different groups. We should randomly assign the types of fertilizer to the seedlings. The purpose of a controlled experiment is to determine if there is a cause-and-effect relationship. In our case, this means that the use of the new fertilizer produces taller plants on average. If the controlled experi- ment is randomized and the treatment groups are statistically significantly different, then we can be confident that there is indeed a cause-and-effect relationship. One of the simplest experimental designs is called a completely random- ized design . For completely randomized designs, the levels of the primary factor are randomly assigned to the experimental units. Our fertilizer ex- periment has such a design. The tools introduced in this chapter apply to experiments with a completely randomized design. In some circumstances, the distribution of the response variable can be highly spread-out. This variability might be due to nuisance factors. For example, females and males might react differently to a particular drug. This noise can be prohibitive, in the sense that we would need very large samples in order to identify significant treatment effects. To reduce this noise we can construct homogeneous subgroups, called blocks . The variance within each block should be smaller than the variance of the entire sample. So the estimates within the blocks should be more precise. As we combine the estimates across blocks, we should obtain an estimate of the treatment effect that is more precise than without blocking . If we randomly assign all of the treatments to the experimental units within each block, then we say that the experiment has a randomized com- plete block design . As an example, if we want to compare a drug to a placebo and we believe that the gender has also an effect on the response, we divide the subjects into blocks according to their gender. If we have ten subjects of each gender, we randomly assign the drug to five subjects of each gender. The remainder of the subjects are given the placebo. We do not discuss the analysis of block designs in this chapter. The techniques presented in this chapter do not apply only to completely randomized experiments. They are also applicable in a non-experimental setting. Consider the study [64], where the authors compare the breeding biology of the Welcome Swallow in Australia and New Zealand. The factor Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

166 Expect the Unexpected: A First Course in Biostatistics (see Theorem 7.1). Similar to the estimation of the mean in the one sample case, the larger the sample sizes, the more precise is the estimate. Further- more, as we standardize X 1 - X 2 , we obtain that X 1 - X 2 - ( μ 1 - μ 2 ) p σ 2 1 /n 1 + σ 2 2 /n 2 has approximately an N (0 , 1) distribution. When both sample sizes are large (i.e. n 1 ≥ 40 and n 2 ≥ 40), we can use the sample variances instead of the population variances. More precisely, X 1 - X 2 - ( μ 1 - μ 2 ) p S 2 1 /n 1 + S 2 2 /n 2 has approximately an N (0 , 1) distribution. (10.1) This approximation can be used even if the populations are not normally distributed. To justify it, recall that by Theorem 8.1, X 1 has approximately an N ( μ 1 , S 2 1 /n 1 ) distribution, and X 2 has approximately an N ( μ 2 , S 2 2 /n 2 ) distribution. Moreover, X 1 and X 2 are independent random variables, since the two populations are independent. By Theorem 7.2, it follows that X 1 - X 2 has also approximately a normal distribution, with mean μ 1 - μ 2 and variance S 2 1 /n 1 + S 2 2 /n 2 . Relation (10.1) follows by standardization. The theory that we present in this section is based upon the standard- ization (10.1). We emphasize that this standardization should be used only when both sample samples are greater than or equal to 40. We consider the inference concerning the difference μ 1 - μ 2 . The null hypothesis is of the form H 0 : μ 1 - μ 2 = δ 0 , where δ 0 is a given numeric value. Note that when δ 0 = 0, the null hypothesis becomes H 0 : μ 1 - μ 2 = 0, or equivalently H 0 : μ 1 = μ 2 . We use the following test statistic: Z 0 = X 1 - X 2 - δ 0 p S 2 1 /n 1 + S 2 2 /n 2 . (10.2) If H 0 holds, then the sampling distribution of Z 0 is approximately standard normal. Hence we can use Table 18.3, to compute the corresponding p - value. Recall that the p -value is the probability of observing a value as extreme as the current observed value, under the assumption that the null hypothesis holds. Since our definition of an extreme value depends on the alternative hypothesis, the computation of the p -value depends on the alternative hypothesis. Table 10.1 gives the p -value for testing the null hypothesis H 0 : μ 1 - μ 2 = δ 0 against one of the alternative hypotheses H 1 . In this table, Z has a standard normal distribution and z 0 = x 1 - x 2 - δ 0 p s 2 1 /n 1 + s 2 2 /n 2 Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

Comparison of Two Independent Samples 167 is the observed value of the test statistic Z 0 given by (10.2). This will produce a large sample test for comparing the means. Table 10.1 The p -value for comparison of two means: large samples Alternative Hypothesis p -value H 1 : μ 1 - μ 2 > δ 0 P ( Z > z 0 ) H 1 : μ 1 - μ 2 < δ 0 P ( Z < z 0 ) H 1 : μ 1 - μ 2 6 = δ 0 2 P ( Z > | z 0 | ) The p -value is a measure of how much evidence we have against the null hypothesis. The smaller the p -value, the greater the inconsistency between the data and the null hypothesis. Actually, the p -value is the smallest level of significance at which the null hypothesis can be rejected with the given data. We will use the same rule as in Section 9.1: if p -value < α, then we reject H 0 if p -value ≥ α, then we fail to reject H 0 . This rule ensures that the probability of type I error is approximately equal to α . When the null hypothesis H 0 : μ 1 - μ 2 = 0 is rejected, it is often said that the difference between μ 1 and μ 2 is statistically significant. The p -value is a valuable statistic that measures the risk associated with rejecting the null hypothesis. However, it does not give us the whole picture. Think of the hypothesis test as a diagnostic tool. We must assess its specificity and its sensitivity (often called power in the context of hypothesis testing). We can control its specificity (our chances of failing to reject H 0 when H 0 is true) with the use of a significance level. We can use a confidence interval to assess the sensitivity (our chances of rejecting H 0 when H 1 is true). A confidence interval is also useful as a stand-alone tool if the goal is simply to estimate the difference in means. An (approximate) confidence interval for μ 1 - μ 2 at a level of confidence of (1 - α ) 100% is x 1 - x 2 ± z s s 2 1 n 1 + s 2 2 n 2 where z is a value such that P ( - z < Z < z ) = 1 - α and Z follows a standard normal distribution. This means that P ( Z > z ) = α/ 2, i.e. z = z α/ 2 . Regardless of whether the difference is found to be statistically signifi- cant or not, it is important to assess the sensitivity of the hypothesis test. Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

Comparison of Two Independent Samples 169 the hypothesis that the lake whitefish have equal mean lipid content in both lakes. Using their good judgment and experience the researchers had deter- mined before hand that the absolute difference | μ 1 - μ 2 | would have to be at least 1 to be of biological importance. The biological significance cannot be determined from the p -value. We must analyze the error of the estimate of μ 1 - μ 2 . A point estimate for μ 1 - μ 2 is x 1 - x 2 = - 0 . 13 and its estimated stan- dard error is p s 2 1 /n 1 + s 2 2 /n 2 = 0 . 0625 . A 95% (approximate) confidence interval for μ 1 - μ 2 is x 1 - x 2 ± 1 . 96 s s 2 1 n 1 + s 2 2 n 2 = - 0 . 13 ± 0 . 1225 = [ - 0 . 25; - 0 . 01] . We are 95% confident that | μ 1 - μ 2 | < 1. The statistically significant difference between the means has no biological importance. 10.3 Confidence Intervals and Tests for Means: Small Samples In this section, we consider the same problem of comparison of the means μ 1 and μ 2 of two independent populations, in the case of small samples. We use the same notation as in Section 10.2. In addition, we suppose that both X 1 and X 2 are normally distributed. Under this assumption, by Theorem 7.3, we know that X 1 has an N ( μ 1 , σ 2 1 /n 1 ) distribution, and X 2 has an N ( μ 2 , σ 2 2 /n 2 ) distribution. Therefore, X 1 - X 2 has an N ( μ 1 - μ 2 , σ 2 1 n 1 + σ 2 2 n 2 ) distribution . We consider two cases: (1) the population variances are equal; (2) the population variances are not equal. Case (1). Normal Populations with Equal Variances In this case, the underlying assumptions of our model are independent normal populations with equal variances: σ 2 1 = σ 2 2 . In addition, the sample sizes could be small. We denote the common variance by σ 2 . With the added assumption of homogeneity of the variance, the standardization of the estimator X 1 - X 2 becomes X 1 - X 2 - ( μ 1 - μ 2 ) σ p 1 /n 1 + 1 /n 2 has an N (0 , 1) distribution. Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

170 Expect the Unexpected: A First Course in Biostatistics Since σ 2 is unknown, we cannot base our inference on this statistic. Denoting by S 2 i the sample variance from population i , for i = 1 , 2, and using the fact that E ( S 2 i ) = σ 2 i = σ 2 , this means that both S 2 1 and S 2 2 are unbiased estimators of the common variance σ 2 . We combine them to obtain a better estimator of σ 2 . One possible combination is to take a weighted average of the variances with weights based on their respective degrees of freedom. This gives us an unbiased estimator of σ 2 , known as the pooled sample variance : S 2 p = ν 1 ν 1 + ν 2 S 2 1 + ν 2 ν 1 + ν 2 S 2 2 = ( n 1 - 1) S 2 1 + ( n 2 - 1) S 2 2 n 1 + n 2 - 2 , where ν i = n i - 1, for i = 1 , 2. The pooled sample standard deviation is S p = q S 2 p . As we replace σ by S p in the standardization of X 1 - X 2 , we get the following studentization: X 1 - X 2 - ( μ 1 - μ 2 ) S p p 1 /n 1 + 1 /n 2 has a T ( n 1 + n 2 - 2) distribution. (10.3) For testing H 0 : μ 1 - μ 2 = δ 0 , we use the test statistic: T 0 = X 1 - X 2 - δ 0 S p p 1 /n 1 + 1 /n 2 . If H 0 is true, T 0 has a T ( n 1 + n 2 - 2) distribution. A hypothesis test based on this test statistic is called Student’s two-sample t -test . The p -value is given in Table 10.2, where t 0 = x 1 - x 2 s p p 1 /n 1 + 1 /n 2 is the observed value of the test statistic T 0 , and T has a T ( n 1 + n 2 - 2) distribution. Table 10.2 The p -value for comparison of two means: σ 2 1 = σ 2 2 Alternative Hypothesis p -value H 1 : μ 1 - μ 2 > δ 0 P ( T > t 0 ) H 1 : μ 1 - μ 2 < δ 0 P ( T < t 0 ) H 1 : μ 1 - μ 2 6 = δ 0 2 P ( T > | t 0 | ) A (1 - α ) 100% confidence interval for μ 1 - μ 2 is x 1 - x 2 ± t s p r 1 n 1 + 1 n 2 , Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

172 Expect the Unexpected: A First Course in Biostatistics Fig. 10.1 Normal probability plots and comparative box plots for the plant heights The p -value is P ( T < t 0 ) = P ( T < - 1 . 18) = P ( T > 1 . 18) , where T has a T ( n 1 + n 2 - 2) = T (14) distribution. Referring to row ν = 14 in Table 18.4, 1.18 falls between 0 . 692 and 1 . 345, which have areas to the right of 0 . 25 and 0 . 10. Thus, 0 . 10 < p -value < 0 . 25. Using a statistical package, we see that p -value = 0.129. At a significance level of α = 0 . 05, we cannot reject H 0 . The data do not appear to support the hypothesis that the use of the new fertilizer produces taller plants. A 95% confidence interval for μ 1 - μ 2 is x 1 - x 2 ± t s p r 1 n 1 + 1 n 2 = - 4 . 65 ± 8 . 4554 = [ - 13 . 11 , 3 . 81] , where t = t 0 . 025 , 14 = 2 . 145. We are 95% confident that the difference in means is from - 13 . 11 cm to 3 . 81 cm. We are highly confident that the absolute difference in means is not larger than 14 cm. However we cannot say the same about 5 cm, since - 5 lies in the confidence interval. In the next example, we see that we can sometimes use a log- transformation to satisfy the underlying conditions to use Student’s two- sample t -test. Example 10.3. Dichloromethane is a volatile liquid that is widely used as a solvent. A chemical engineer wants to compare the dichloromethane concentration at two treatment water plants near industrial facilities. She Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

Comparison of Two Independent Samples 173 suspects that the distributions of the dichloromethane concentration are skewed to the right due to occasional higher discharges from the industrial facilities. She verifies her hunch with histograms (see Figure 10.2). She decides to apply a log transformation, that is, the new measure- ments are read in ln( μg/L ). The normal probability plots for the data in the original scale and the log scale are given in Figure 10.3. It is evident from the normal probability plots that the data in the original scale are Fig. 10.2 Histograms for the dichloromethane concentrations from plants 1 and 2 Fig. 10.3 Normal probability plots for the concentrations and log-concentrations Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

Comparison of Two Independent Samples 175 (for the population or the sample), we introduce the following definition. Definition 10.1. Let X 1 , X 2 , . . . , X n be a random sample from a popula- tion represented by the random variable X . The geometric mean of the population is G = e μ , where μ = E (ln X ). An estimate for G is the (sam- ple) geometric mean defined by g = e (1 /n ) ∑ n i =1 ln x i = ( Q n i =1 x i ) 1 /n , where x 1 , . . . , x n are the observed values of the random sample X 1 , . . . , X n . Example 10.3 (continued). We construct a 95% confidence interval for the difference in means of the log-concentrations for the data from Ex- ample 10.3. Since it is reasonable to assume that the two populations of log-concentrations are independent and normally distributed with equal variances, then a 95% confidence interval for μ 1 - μ 2 is x 1 - x 2 ± t s p r 1 n 1 + 1 n 2 = 0 . 270 ± 0 . 61930 = [ - 0 . 349 , 0 . 889] , where t = 2 . 011 satisfies 95% = P ( - t < T < t ) and T follows a T (48) distribution. (The value of t = 2 . 011 was obtained using a statistical pack- age.) We are 95% confident that μ 1 - μ 2 is between - 0 . 349 and 0 . 889 (in ln( μg/L )). Since 0 lies within the confidence interval, the means of the log-concentrations do not appear to be different. We denote by G i the geometric mean of the population i , consisting of the dichloromethane concentrations (in μg/L ) from plant i , for i = 1 , 2. Note that G i = e μ i , where μ i is the mean of the log-concentration from plant i , for i = 1 , 2. Exponentiating the difference in means gives us the ratio of the geometric means, that is e μ 1 - μ 2 = e μ 1 /e μ 2 = G 1 /G 2 . Since we are 95% confident that - 0 . 349 < μ 1 - μ 2 < 0 . 889, then we are also 95% confident that 0 . 71 = e - 0 . 349 < G 1 /G 2 < e 0 . 889 = 2 . 43. Since 1 lies within the interval, there appears to be no difference between the geometric means of the concentrations. Case (2). Normal Populations with Unequal Variances The assumption of equality of the two variances is sometimes not rea- sonable. So we should try to adapt our techniques to the case of unequal variances: σ 2 1 6 = σ 2 2 . This is known as the Behrens-Fisher problem . There are exact solutions to Behrens-Fisher problem (see [21]). These solutions are beyond the scope of this book. We present an approximate solution. In 1938, Welch [71] proposed an approximate solution to the Behrens- Fisher problem. Welch argued that the inference concerning μ 1 - μ 2 for Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

176 Expect the Unexpected: A First Course in Biostatistics two independent normal population can be based on X 1 - X 2 - ( μ 1 - μ 2 ) p S 2 1 /n 1 + S 2 2 /n 2 , (10.4) which follows approximately a T distribution with ν degrees of freedom, where ν = ( s 2 1 /n 1 + s 2 2 /n 2 ) 2 ( s 2 1 /n 1 ) 2 / ( n 1 - 1) + ( s 2 2 /n 2 ) 2 / ( n 2 - 1) . (10.5) ν is called Welch’s number of degrees of freedom . It follows that we can construct the following approximate (1 - α ) 100% confidence interval for μ 1 - μ 2 : x 1 - x 2 ± t s s 2 1 n 1 + s 2 2 n 2 , where P ( - t ≤ T ≤ t ) = 1 - α and T has a T ( ν ) distribution. Note that t = t α/ 2 ,ν since P ( T > t ) = α/ 2. Since the number of degrees of freedom must be an integer, we round down ν to the nearest integer. This rounding procedure is for conservative reasons. For instance, if ν = 7 . 8, we need to decide between ν = 7 and ν = 8. Since the value t 0 . 025 (7) = 2 . 365 is greater than t 0 . 025 (8) = 2 . 306, the 95% confidence interval based on the T distribution with ν = 7 degrees of freedom will be larger than the 95% confidence interval based on the T distribution with ν = 8 degrees of freedom. Hence, the smaller interval (based on ν = 8) may not actually contain the value μ 1 - μ 2 . By work- ing with a larger interval, we minimize the risk that the interval does not contain the value μ 1 - μ 2 . To test H 0 : μ 1 - μ 2 = δ 0 , we use the test statistic T 0 = X 1 - X 2 - δ 0 p S 2 1 /n 1 + S 2 2 /n 2 . A test based on this test statistic is often called Welch’s approximate two- sample t -test . This test is sometimes also called the Welch–Satterthwaite t -test or the Satterthwaite t -test. The p -value of this test is given in Table 10.3, where t 0 is the observed value of T 0 , T has a T ( ν ) distribution, and ν is given in (10.5). Welch’s method is not exact, but is generally a good approximation. However, if the population variances are equal, or if the sample sizes are Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

178 Expect the Unexpected: A First Course in Biostatistics T ( ν ) distribution with the following number of degrees of freedom: ν = ( s 2 1 /n 1 + s 2 2 /n 2 ) 2 ( s 2 1 /n 1 ) 2 / ( n 1 - 1) + ( s 2 2 /n 2 ) 2 / ( n 2 - 1) = 12 . 89 . We round down the number of degrees of freedom to the nearest integer, that is ν = 12. Referring to row ν = 12 in Table 18.4, 2.11 falls be- tween 1 . 782 and 2 . 179, which have areas to the right of 0 . 05 and 0 . 025, respectively. Thus, 0 . 05 < p -value < 0 . 10. The p -value computed with a statistical package is 0.056. At a level of significance of α = 0 . 10, we can accept the alternative hypothesis that the egg mass of the two species are different on average. Fig. 10.4 Normal probability plots and comparative box plots for the egg masses Technology Component using R : Assume that the data for the two populations are saved in the numerical vectors x1 and x2 , respectively. • To produce the overlayed QQ-plots for x1 (in blue) and x2 (in red), together with the fitted lines, we use: lmts=range(x1,x2) qqnorm(x1,ylim=lmts,col="blue") abline(mean(x1),sd(x1),col="blue") par(new=T) qqnorm(x2,ylim=lmts,col="red") abline(mean(x2),sd(x2),col="red") Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

Comparison of Two Independent Samples 179 par(new=F) Remark: The above procedure gives the plot of the pairs ( z i , y i ) with the fitted line of equation y = ˆ μ + ˆ σz with ˆ μ = ¯ x and ˆ σ = s , for each of the two variables. This procedure is used for verifying the assumption that the two populations are normally distributed with equal variances. We say that the two populations are normally distributed if both plots seem to be linear. We say that the two populations have equal variances if the two lines seem to be parallel. • To produce side-by-side boxplots, we use: boxplot(x1,x2) Remark: If you assigned the data to a dataframe (for example, using the function read.table() ), refer to the last item of the Technology component at the end of Section 7.3 to see how to produce side-by-side boxplots and overlayed normal probability plots in the same graphics window. • To test the hypothesis H 0 : μ 1 = μ 2 against μ 1 6 = μ 2 and calculate a 95% confidence interval for μ 1 - μ 2 when the two populations are normally distributed with equal variances, we use: t.test(x1,x2,var.equal=TRUE) Remark: In the case of normal populations with unequal vari- ances, we use the same command as above, but without including var.equal=TRUE . To change the confidence level to 98% (or any other value), we use: t.test(x1,x2,conf.lev=0.98,var.equal=TRUE) • To test the hypothesis H 0 : μ 1 = μ 2 against μ 1 > μ 2 when the two populations are normally distributed with equal variances, we use: t.test(x1,x2,alternative="greater",var.equal=TRUE) Remark: This procedure produces also a one-sided confidence interval which is not discussed in this book. In the case of normal popula- tions with unequal variances, we use the same command as above, but without including var.equal=TRUE . • To test the hypothesis H 0 : μ 1 = μ 2 against μ 1 < μ 2 when the two populations are normally distributed with equal variances, we use: Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

Comparison of Two Independent Samples 181 when n 1 and n 2 are large. What are large sample sizes? This is not an easy question to answer. A common rule of thumb is to not use the latter normal approximation when the observed number of successes or the observed number of failures in either one of the groups is less than 5. Using (10.7), we construct the (approximate) confidence interval for p 1 - p 2 at a level of confidence of (1 - α ) 100%. This interval is: b p 1 - b p 2 ± z s b p 1 (1 - b p 1 ) n 1 + b p 2 (1 - b p 2 ) n 2 , where z is value such that P ( - z < Z < z ) = 1 - α, and Z follows a standard normal distribution. In practice, we usually want to compare our data against a model with equal proportions. In other words, we would like to test the null hypothesis H 0 : p 1 - p 2 = 0 (or equivalently H 0 : p 1 = p 2 ) against an appropriate alternative hypothesis. Assuming that H 0 holds, then the probability of success is the same for all trials in both experiments. This common prob- ability is p = p 1 = p 2 . If this is the case, we can consider the n = n 1 + n 2 observations as a sample from a binomial distribution with n trials and probability p of success. The corresponding sample proportion (called the pooled sample proportion ) is b p = Y 1 + Y 2 n = n 1 n b p 1 + n 2 n b p 2 . Note that the pooled sample proportion is a weighted average of the respec- tive sample proportions, where the weights are the relative sample sizes. Assuming that H 0 is true (i.e. p 1 = p 2 ) the standardization in (10.6) becomes b p 1 - b p 2 - 0 p p (1 - p ) p 1 /n 1 + 1 /n 2 . Using b p instead of p , we get the following test statistic: Z 0 = b p 1 - b p 2 p b p (1 - b p ) p 1 /n 1 + 1 /n 2 . Since the p -value is the probability of observing a value as extreme as z 0 (which is the observed value of the test statistic) in the direction of the al- ternative hypothesis, this hypothesis must be taken in consideration when computing the p -value. We usually want to test the null hypothesis of equal- ity against one of the following three alternative forms. Table 10.4 gives the corresponding p -value in the three cases. Here Z has approximately a standard normal distribution. The test is a large sample test. Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.

182 Expect the Unexpected: A First Course in Biostatistics Table 10.4 The p -value for the comparison of two proportions Alternative Hypothesis p -value H 1 : p 1 - p 2 > 0 P ( Z > z 0 ) H 1 : p 1 - p 2 < 0 P ( Z < z 0 ) H 1 : p 1 - p 2 6 = 0 2 P ( Z > | z 0 | ) Example 10.5. Refer to Example 3.7. We denote by p 1 and p 2 the proportions of recaptured moths in the light-colored population, respec- tively the dark-colored population. Among the n 1 = 137 light-colored moths, y 1 = 18 were recaptured, whereas among the n 2 = 493 dark-colored moths, y 2 = 131 were recaptured. The proportions of recaptured moths are: b p 1 = 0 . 131 for the light-colored moths, and b p 2 = 0 . 266 for the dark- colored moths. Is there a statistical difference between the proportions of recaptured moths, at a level of significance of α = 0 . 05? If so, we wish to investigate the biological (practical) significance. We assume that the samples are independent. Since both sample sizes are large and the observed number of successes and of failures are not too small, we can safely perform a large sample test. To test H 0 : p 1 - p 2 = 0 against H 1 : p 1 - p 2 6 = 0, we compute the test statistic: z 0 = b p 1 - b p 2 p b p (1 - b p ) p 1 /n 1 + 1 /n 2 = 0 . 131 - 0 . 266 p (0 . 2365)(1 - 0 . 2365) p 1 / 137 + 1 / 493 = - 3 . 29 , where the pooled sample proportion is b p = y 1 + y 2 n 1 + n 2 = 18 + 131 137 + 493 = 0 . 2365 . The p -value is the probability of observing a difference in proportions as extreme as b p 1 - b p 2 = - 0 . 135, under the assumption that both proportions are equal. This is approximately equal to 2 P ( Z > | z 0 | ) = 2 P ( Z > 3 . 29), where Z follows a standard normal approximately. Using Table 18.3, we can argue that the p -value is 0 . 001. There is a statistical significant difference between the proportions. To investigate the biological significance, we construct a 95% confidence interval for p 1 - p 2 : b p 1 - b p 2 ± 1 . 96 q b p 1 (1 - b p 1 ) n 1 + b p 2 (1 - b p 2 ) n 2 = - 0 . 135 ± 0 . 0687 = [ - 0 . 204 , - 0 . 066] . We are 95% confident that the difference in proportion p 2 - p 1 is between 6.6% to 20.4%. Recall that biological significance cannot be determined by a test. Only by using their Balan, R., & Lamothe, G. (2017). Expect the unexpected : A first course in biostatistics (second edition). World Scientific Publishing Company. Created from ottawa on 2023-09-29 20:15:50. Copyright © 2017. World Scientific Publishing Company. All rights reserved.