Lab08.ipynb - Colaboratory

Welcome to Lab 8! In this lab we will go over the topic of sampling and distributions . The data used in this lab will contain salary data and other statistics for basketball players from the 2014-2015 NBA season. This data was collected from the following sports analytic sites: Basketball Reference and Spotrac . First, set up the tests and imports by running the cell below. Lab 8: Sampling and Distributions - Sampling Basketball Data # Run this cell, but please don't change it. # These lines import the Numpy and Datascience modules. import numpy as np from datascience import * # These lines do some fancy plotting magic import matplotlib %matplotlib inline import matplotlib.pyplot as plt plt.style.use('fivethirtyeight') Run the cell below to load player and salary data that we will use for our sampling. Name Age Team Games Rebounds Assists Steals Blocks Turnovers Points James Harden 25 HOU 81 459 565 154 60 321 2217 Chris Paul 29 LAC 82 376 838 156 15 190 1564 Stephen Curry 26 GSW 80 341 619 163 16 249 1900 ... (489 rows omitted) PlayerName Salary Kobe Bryant 23500000 Amar'e Stoudemire 23410988 Joe Johnson 23180790 ... (489 rows omitted) PlayerName Salary Age Team Games Rebounds Assists Steals Blocks Turnovers Points A.J. Price 62552 28 TOT 26 32 46 7 0 14 133 Aaron Brooks 1145685 30 CHI 82 166 261 54 15 157 954 Aaron Gordon 3992040 19 ORL 47 169 33 21 22 38 243 ... (489 rows omitted) player_data = Table().read_table("player_data.csv") salary_data = Table().read_table("salary_data.csv") full_data = salary_data.join("PlayerName", player_data, "Name") # The show method immediately displays the contents of a table. # This way, we can display the top of two tables using a single cell. player_data.show(3) salary_data.show(3) full_data.show(3)

Rather than getting data on every player (as in the tables loaded above), imagine that we had gotten data on only a smaller subset of the players. For 492 players, it's not so unreasonable to expect to see all the data, but usually we aren't so lucky. If we want to make estimates about a certain numerical property of the population (known as a statistic, e.g. the mean or median), we may have to come up with these estimates based only on a smaller sample. Whether these estimates are useful or not often depends on how the sample was gathered. We have prepared some example sample datasets to see how they compare to the full NBA dataset. Later we'll ask you to create your own samples to see how they behave. To save typing and increase the clarity of your code, we will package the analysis code into a few functions. This will be useful in the rest of the lab as we will repeatedly need to create histograms and collect summary statistics from that data. We've de±ned the histograms function below, which takes a table with columns Age and Salary and draws a histogram for each one. It uses bin widths of 1 year for Age and $1,000,000 for Salary . def histograms(t): ages = t.column('Age') salaries = t.column('Salary')/1000000 t1 = t.drop('Salary').with_column('Salary', salaries) age_bins = np.arange(min(ages), max(ages) + 2, 1) salary_bins = np.arange(min(salaries), max(salaries) + 1, 1) t1.hist('Age', bins=age_bins, unit='year') plt.title('Age distribution') t1.hist('Salary', bins=salary_bins, unit='million dollars') plt.title('Salary distribution') histograms(full_data) print('Two histograms should be displayed below')

[26.536585365853657, 4269775.7662601592] # TEST stats = compute_statistics(full_data) plt.close() plt.close() round(float(stats[0]), 2) == 26.54 True # TEST stats = compute_statistics(full_data) plt.close() plt.close() round(float(stats[1]), 2) == 4269775.77 True One sampling methodology, which is generally a bad idea , is to choose players who are somehow convenient to sample. For example, you might choose players from one team who are near your house, since it's easier to survey them. This is called, somewhat pejoratively, convenience sampling . Suppose you survey only relatively new players with ages less than 22. (The more experienced players didn't bother to answer your surveys about their salaries.) Question 2. Assign convenience_sample to a subset of full_data that contains only the rows for players under the age of 22. Convenience sampling

PlayerName Salary Age Team Games Rebounds Assists Steals Blocks Turnovers Points Aaron Gordon 3992040 19 ORL 47 169 33 21 22 38 243 Alex Len 3649920 21 PHO 69 454 32 34 105 74 432 Andre Drummond 2568360 21 DET 82 1104 55 73 153 120 1130 Andrew Wiggins 5510640 19 MIN 82 374 170 86 50 177 1387 Anthony Bennett 5563920 21 MIN 57 216 48 27 16 36 298 Anthony Davis 5607240 21 NOP 68 696 149 100 200 95 1656 Archie Goodwin 1112280 20 PHO 41 74 44 18 9 48 231 Ben McLemore 3026280 21 SAC 82 241 140 77 19 138 996 Bradley Beal 4505280 21 WAS 63 241 194 76 18 123 962 Bruno Caboclo 1458360 19 TOR 8 2 0 0 1 4 10 ... (34 rows omitted) convenience_sample = full_data.where(full_data.column('Age') < 22) convenience_sample # TEST convenience_sample.num_columns == 11 True # TEST convenience_sample.num_rows == 44 True Question 3. Assign convenience_stats to an array of the average age and average salary of your convenience sample, using the compute_statistics function. Since they're computed on a sample, these are called sample averages . age=convenience_sample.column('Age') salary=convenience_sample.column('Salary') convenience_stats = ([sum(age)/len(age),sum(salary)/len(salary)]) convenience_stats [20.363636363636363, 2383533.8181818184] # TEST len(convenience_stats) == 2 True # TEST round(float(convenience_stats[0]), 2) == 20.36 True # TEST round(float(convenience_stats[1]), 2) == 2383533.82 True

Sometimes, it’s useful to take random samples even when we have the data for the whole population. It helps us understand sampling accuracy. sample The table method sample produces a random sample from the table. By default, it draws at random with replacement from the rows of a table. It takes in the sample size as its argument and returns a table with only the rows that were selected. Run the cell below to see an example call to sample() with a sample size of 5, with replacement. Producing simple random samples PlayerName Salary Bruno Caboclo 1458360 Alec Burks 3034356 Tony Wroten 1210080 Andre Roberson 1160880 Kendrick Perkins 9154342 # Just run this cell salary_data.sample(5) The optional argument with_replacement=False can be passed through sample() to specify that the sample should be drawn without replacement. Run the cell below to see an example call to sample() with a sample size of 5, without replacement. PlayerName Salary Noah Vonleh 2524200 Aron Baynes 2077000 Mike Dunleavy 3326235 Ben McLemore 3026280 Eric Bledsoe 13000000 # Just run this cell salary_data.sample(5, with_replacement=False) Question 5. Produce a simple random sample of size 44 from full_data . Run your analysis on it again. Run the cell a few times to see how the histograms and statistics change across different samples. my_small_srswor_data = full_data.sample(44) print(my_small_srswor_data) def histograms(s): ages = s.column('Age') salaries = s.column('Salary')/1000000 s1 = s.drop('Salary').with_column('Salary', salaries) age_bins = np.arange(min(ages), max(ages) + 2, 1)

salary_bins = np.arange(min(salaries), max(salaries) + 1, 1) s1.hist('Age', bins=age_bins, unit='year') plt.title('Age distribution') s1.hist('Salary', bins=salary_bins, unit='million dollars') plt.title('Salary distribution') plt.show() histograms(my_small_srswor_data) my_small_stats = ([sum(my_small_srswor_data.column('Age'))/len(my_small_srswor_data.column('Age')),sum(my_small_srswor_data.column('Salary'))/len(my_small_srswor_data.column('Salary'))*1000000]) my_small_stats

[25.780000000000001, 4206290650000.0005] Answer the following questions in English: Do the histogram shapes seem to change more or less across samples of 100 than across samples of size 44? Are the sample averages and histograms closer to their true values/shape for age or for salary? What did you expect to see? 1. Yes,It appears that histogram shapes vary more with samples of 100 compared to 44. The inclusion of more bins caused these variations in the histogram's shape. 2. The age and pay to sample averages and histograms were closer to their true shapes; they weren't exact, but they were close. The graph can be deemed perfect if it is more transparent. Congratulations, you're done with Lab 8! Be sure to... run all the tests, print the notebook as a PDF, and submit both the notebook and the PDF to Canvas.