300_Lec18_MultipleComparisons_2post

Multiple Comparisons STAT 300: Intermediate Statistics for Applications Lecture 18 University of British Columbia (UBC) Multiple Comparisons 1 / 17

Announcements/Reminders WeBWork homework 5 - Due:Monday, March 1 I Access through Canvas Written Assignment 1 I Due: Tuesday, March 2 Labs 5 - Due: Thursday, March 4 I Access the Lab assignment and instructions in the Canvas page I upload answers to Crowdmark Help in Breakout Rooms I Introduce TAs: Harper, Jenkin, Qiong and Yang (UBC) Multiple Comparisons 2 / 17

Motivation ANOVA can only tell us whether one or more groups are different from the others I Cannot tell you which groups are different Today: we will look at multiple comparisons I Post-hoc method F Performs a set of pairwise comparisons to assess which groups differ I Involves performing several tests F We must adjust for the increased risk of making a Type I error (UBC) Multiple Comparisons 4 / 17

Case study Assessing the strength of the rubber from 4 different machines M1 M2 M3 M4 15 16 17 18 19 20 Machine Tensile strength (UBC) Multiple Comparisons 5 / 17

Hints When X and Y are independent random variables: I Var( X + Y ) = Var( X ) + Var( Y ) When c is a constant: I Var( cX ) = c 2 Var( X ) The variance of the sample mean: I Var( ¯ X ) = Var( X ) n (UBC) Multiple Comparisons 7 / 17

Comparing 2 groups We are interested in comparing machines M1 and M2 I What is an estimate of the difference in the mean rubber strength from the two machines? F ¯ y 1 . - ¯ y 2 . I Could we use this as a test statistic to compare the groups? (UBC) Multiple Comparisons 8 / 17

Common variance σ 2 What’s the variance of the estimated difference in rubber strength from machines M1 and M2? I Assuming all machines produce rubber with tensile strength having variance σ 2 Recall: the assumptions for the ANOVA are: I observations within each group are independent I observations in different groups are independent I observations within groups are from Normal distributions F with the same variance (UBC) Multiple Comparisons 10 / 17

Var(¯ y 1 . - ¯ y 2 . ) Var(¯ y 1 . - ¯ y 2 . ) = Var(¯ y 1 . ) + Var(¯ y 2 . ) Var(¯ y 1 . - ¯ y 2 . ) = Var( y 1 . ) n 1 + Var( y 2 . ) n 2 Var(¯ y 1 . - ¯ y 2 . ) = σ 2 n 1 + σ 2 n 2 Var(¯ y 1 . - ¯ y 2 . ) = σ 2 5 + σ 2 5 Var(¯ y 1 . - ¯ y 2 . ) = 2 σ 2 5 (UBC) Multiple Comparisons 11 / 17

Comparing probabilities E 1 , E 2 , . . . , E 6 are any six events in a sample space How would you verify the following? P ( E 1 or E 2 or · · · or E 6 ) ≤ ∑ 6 i =1 P ( E i ) (UBC) Multiple Comparisons 13 / 17

Several pairwise comparisons Test at the 5% significance level whether a difference between 2 machines appeared significant I i.e. Assess whether the statistic computed is bigger in magnitude than the 97.5% percentile of the appropriate t distribution We need to perform 6 tests to make all pairwise comparisons What is the problem with this approach? (UBC) Multiple Comparisons 14 / 17

Summary: Multiple Comparison tests We often want to compare pairs of treatment group means to see if they differ To do so, we used a t-test with the within group mean square as the pooled sample variance If we want to test several such pairs I We must adjust for performing several tests to keep the overall risk of Type I error from growing too large We adjust the significance level of each pairwise test to: I α G I Ensure that the chance of making at least one type I error would be no more than α (UBC) Multiple Comparisons 16 / 17

Before the next class ... Visit the course website at canvas.ubc.ca Read the activity solution Topic of next class: Interactions I Watch Introducing two-way ANOVA pencast I Read Section 5.4 (up to and including 5.4.2) of the Design and ANOVA notes (UBC) Multiple Comparisons 17 / 17