HW 6-AB Testing

[7]: def ab_shuffle (tab): shuffle_group = tab . sample(with_replacement = False ) . column( 0 ) shuffled_tab = tab . with_column( "Shuffled Grouping" ,shuffle_group) . , → select( 2 , 1 ) return shuffled_tab Notes - The input for our ab-shuffle function is a table. - The function shuffles the first column. - The function then returns a table with two columns. - The returned table’s first column is the grouping variable. - The returned table’s second column should be the numeric variable. To illustrate, let’s create a table called ‘narc’ using the ‘personality’ dataset. Execute the code block below with no changes. Now, we can apply our ab_shuffle function to our new table narc . [8]: ab_shuffle(narc) . show( 5 ) <IPython.core.display.HTML object> Note that the Grouping variable column is shuffled. Try executing the code block several times, and notice how the shuffling of the first column labels is different each time. The data in the second column is unchanged. We can use both of our new functions at once to find the difference in means in our shuffled table. Execute the code block below to see that in action. [9]: ab_diff(ab_shuffle(narc)) [9]: -0.03341523341523356 Run the code block several times and note how the output changes. 1.4 ab_hist Let’s create our last function, one that will display a visual representation of our results from the A/B test. Execute the code block below. [10]: def ab_hist (myArray, observed_value): tab = Table() . with_column( 'A/B Differencs' ,myArray) tab . hist( 0 ) _ = plots . plot([observed_value, observed_value], [ 0 , 0.1 ], color = 'red' , ␣ , → lw =2 ) Notes - The function input requires two things, an array and an observed value. - We run a “for loop” to generate the array. - The observed value is the ab_diff from the original data in narc . 1.5 The for loop First, let’s create a variable observed_diff which will be the difference in groups A and B in the original narc dataset. 3

1.5.3 Question 4 Where do you think the red line would fall in the histogram if the p-value were p = 0 . 25 ? What if p = 0 . 8 or p = 0 . 9 ? Explain your reasoning in the block below. If the p-value were p = 0 . 25 , I believe it would go up with the difference. 1.5.4 Question 5 Given the p-value, what do you think of your guess in Question 1? Is there a difference in the levels of narcissism between males and females? Why do you think so? Are males or females (or neither) more narcissistic? After seeing the results of A/B testing, I believe that there is a differnece and males are in fact more narcissistic. 1.6 Practice Problems Pick at least one of the following practice problems. In the code blocks below, run the indicated independent samples t-test. Calculate an exact p-value, and say what you think the output means in the real world. 1. Using the exact same procedure, test for a difference between male and female perfectionsim scores. The following code will help extract the table you need to work with. perf = personality.select('Sex','Perf') perf.show(5) 2. Use an A/B test to determine if older students drink more caffeine than younger students. The G21 variable has Yes/No responses to the question, “Are you 21 years old or older?” The following code will help get the needed table. caff = personality.select('Sex','Caffeine') caff.show(5) 3. Use an A/B test to determine if older students are less naive about relationships. The variable TxRel is based on the “Toxic Relationship Beliefs” scale. The G21 variable has Yes/No responses to the question, “Are you 21 years old or older?” Higher TxRel scores indicate the person is more naive aobut relationships. Determine if younger students are more naive about relationships. The following code will help get the needed table. rel = personality.select('Sex','TxRel') rel.show(5) 4. Use an A/B test to determine if the thrill-seeking behaviors of males and females is different. The numeric variable Thrill indicates a person’s interesting thrill-seeking behaviors with higher scores indicating more interest. The following code will help get the needed table. thrill = personality.select('Sex','Thrill') thrill.show(5) [27]: perf = personality . select( 'Sex' , 'Perf' ) perf . show( 5 ) 6