STAT311W24Week4

Rmd

School

University of Washington *

*We aren’t endorsed by this school

Course

200

Subject

Statistics

Date

Feb 20, 2024

Type

Rmd

Pages

Uploaded by CommodoreSeahorseMaster945

--- title: "Lab 2: Introduction to Data" author: "STAT 311" date: "Winter 2024" output: openintro::lab_report --- Adapted from: https://openintrostat.github.io/oilabs-tidy/02_intro_to_data/intro_to_data.html ```{r load-packages, message=FALSE, warning=FALSE} library(tidyverse) library(openintro) ``` In this lab we explore flights, specifically a random sample of domestic flights that departed from the three major New York City airports in 2013. We will generate simple graphical and numerical summaries of data on these flights and explore delay times. Since this is a large data set, along the way you’ll also learn the indispensable skills of data processing and subsetting. First, we’ll view the nycflights data frame.Let's load it into our environment. ```{r} data(nycflights) ``` The data set nycflights is a data matrix, with each row representing an observation and each column representing a variable. R calls this data format a data frame, which is a term that will be used throughout the labs. For this data set, each observation is a single flight. To view the names of the variables, type the command ```{r} names(nycflights) ``` The codebook (description of the variables) can be accessed by pulling up the help file: ```{r} ?nycflights ``` Remember that you can use glimpse to take a quick peek at your data to understand its contents better. ```{r} glimpse(nycflights) ``` The nycflights data frame is a massive trove of information. Let’s think about some questions we might want to answer with these data: - How delayed were flights that were headed to Los Angeles? - How do departure delays vary by month?

- Which of the three major NYC airports has the best on time percentage for departing flights? Let’s start by examining the distribution of departure delays of all flights with a histogram. ```{r} ggplot(data = nycflights, aes(x = dep_delay)) + geom_histogram() ``` This function says to plot the dep_delay variable from the nycflights data frame on the x-axis. It also defines a geom (short for geometric object), which describes the type of plot you will produce. Histograms are generally a very good way to see the shape of a single distribution of numerical data, but that shape can change depending on how the data is split between the different bins. You can easily define the binwidth you want to use: ```{r} ggplot(data = nycflights, aes(x = dep_delay)) + geom_histogram(binwidth = 15) ``` ```{r} ggplot(data = nycflights, aes(x = dep_delay)) + geom_histogram(binwidth = 150) ``` ### Exercise 1 **Question:** Look carefully at these three histograms. How do they compare? Are features revealed in one that are obscured in another? #### Solution  If you want to visualize only delays of flights headed to Los Angeles, you need to first filter the data for flights with that destination (dest == "LAX") and then make a histogram of the departure delays of only those flights. ```{r} lax_flights <- nycflights %>% filter(dest == "LAX") lax_flights ``` (Note, it’s common to add a break to a new line after %>% to help readability) Let’s decipher this command: Take the nycflights data frame, filter for flights headed to LAX, and save the result as a new data frame called lax_flights. - == means “if it’s equal to”.

- LAX is in quotation marks since it is a character string. ```{r} #Histogram of departure delays for flights flying to LAX ggplot(data = lax_flights, aes(x = dep_delay)) + geom_histogram() ``` Logical operators: Filtering for certain observations (e.g. flights from a particular airport) is often of interest in data frames where we might want to examine observations with certain characteristics separately from the rest of the data. To do so, you can use the filter function and a series of logical operators. The most commonly used logical operators for data analysis are as follows: - == means “equal to” - != means “not equal to” - > or < means “greater than” or “less than” - >= or <= means “greater than or equal to” or “less than or equal to” You can also obtain numerical summaries for these flights: ```{r} lax_flights %>% summarise(mean_dd = mean(dep_delay), median_dd = median(dep_delay), n = n()) ``` Note that in the summarise function you created a list of three different numerical summaries that you were interested in. The names of these elements are user defined, like mean_dd, median_dd, n, and you can customize these names as you like (just don’t use spaces in your names). Calculating these summary statistics also requires that you know the function calls. Note that n() reports the sample size. Summary statistics: Some useful function calls for summary statistics for a single numerical variable are as follows: - mean - median - sd - var - IQR - min - max Note that each of these functions takes a single vector as an argument and returns a single value. ### Exercise 2 **Question:** Create a new data frame that includes flights headed to SFO in February, and save this data frame as `sfo_feb_flights.` How many flights meet these criteria? #### Solution ```{r code-chunk-label} # creating a data set with flights flying to SFO in Feb

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

#outputting the number of flights to SFO in February ``` ### Exercise 3 **Question:** Describe the distribution of the **arrival** delays of these flights using a histogram and appropriate summary statistics. #### Solution Exploring a histogram with the data. ```{r, message = FALSE, warning = FALSE} ``` Looking at the summary statistics: ```{r} # display summary statistics ``` The histogram shows a possible right skew in the data. This is further reinforced by the fact that the mean is larger than the median. A right skew would indicate that most flights are early, but there are a few heavily delayed flights that are skewing the mean towards higher values.  Another useful technique is quickly calculating summary statistics for various groups in your data frame. For example, we can modify the above command using the group_by function to get the same summary stats for each origin airport: ```{r} # sfo_feb_flights %>% # group_by(origin) %>% # summarise(median_dd = median(dep_delay), iqr_dd = IQR(dep_delay), n_flights = n()) ``` ### Exercise 4 **Question:** Calculate the median and interquartile range for `arr_delays` of flights in in the `sfo_feb_flights` data frame, grouped by carrier. Which carrier has the most variable arrival delays? #### Solution ```{r}

``` We can see that there are only 5 carriers that operated flights (in our data set) to SFO in February of 2013. Based on the IQR values on the previous slide, it looks like Delta Airlines and United Airlines have the most variable arrival delays because they have the highest IQR.  ### Exercise 4.1 **Question:** Which month has the highest average delay departing from an NYC airport? ```{r} ``` ### Exercise 5 **Question:** Suppose you really dislike departure delays and you want to schedule your travel in a month that minimizes your potential departure delay leaving NYC. One option is to choose the month with the lowest mean departure delay. Another option is to choose the month with the lowest median departure delay. What are the pros and cons of these two choices? #### Solution **Con of using the median departure delay:** **Con of using the mean departure delay.**  ### Exercise 6 Suppose you will be flying out of NYC and want to know which of the three major NYC airports has the best on time departure rate of departing flights. Also supposed that for you, a flight that is delayed for less than 5 minutes is basically “on time.”" You consider any flight delayed for 5 minutes or more to be “delayed”. **Question:** If you were selecting an airport simply based on on time departure percentage, which NYC airport would you choose to fly out of? In order to determine which airport has the best on time departure rate, you can - first classify each flight as “on time” or “delayed”, - then group flights by origin airport, - then calculate on time departure rates for each origin airport, - and finally filter the summary output to include only the aiport with the highest on time departure rate. Let’s start with classifying each flight as “on time” or “delayed” by creating a new variable with the mutate function.

```{r} nycflights <- nycflights %>% mutate(dep_type = ifelse(dep_delay < 5, "on time", "delayed")) nycflights ``` The first argument in the mutate function is the name of the new variable we want to create, in this case dep_type. Then if dep_delay < 5, we classify the flight as "on time" and "delayed" if not, i.e. if the flight is delayed for 5 or more minutes. Note that we are also overwriting the nycflights data frame with the new version of this data frame that includes the new dep_type variable. We can handle all of the remaining steps in one code chunk: ```{r} nycflights %>% group_by(origin) %>% #group nyc flights by origin airport summarise(ot_dep_rate = sum(dep_type == "on time") / n()) %>% #on-time dep rate filter(ot_dep_rate == max(ot_dep_rate)) #airport with best on-time dep rate ``` We can see that `LGA` has the best on time departure percentage. You can also visualize the distribution of on on time departure rate across the three airports using a segmented bar plot. ```{r} ggplot(data = nycflights, aes(x = origin, fill = dep_type)) + geom_bar() ``` ### Exercise 7 **Question:** Mutate the data frame so that it includes a new variable that contains the average speed, `avg_speed` traveled by the plane for each flight (in mph). **Hint:** Average speed can be calculated as distance divided by number of hours of travel, and note that `air_time` is given in minutes. #### Solution This exercise is asking us to create the new `avg_speed` variable. Recall that the equation for average speed in miles per hour is given by $$ \text{ avg speed} = \dfrac{\text{distance in miles}}{\text{time in hours}} $$ Also recall that the `time` variable is given in minutes. This means $$ \mathtt{avg\underline{}speed} = \dfrac{\mathtt{distance \ in \ miles}}{\ mathtt{time \ in \ minutes}} \; \mathtt{\ast \ \dfrac{\mathtt{60 \ minutes}}{\ mathtt{1 \ hr}}} $$ Coding this in R: ```{r} ```

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

### Exercise 8 **Question:** Make a scatterplot of `avg_speed` vs. `distance.` Describe the relationship between average speed and distance. Hint: Use `geom_point()`. #### Solution ```{r, message=FALSE} ``` ##### Exercise 8 Follow Up **Why are the `distance` values on the x-axis represented as a "stack" of points?** To determine the answer to this question, it is best to start by looking at the first 20 observations for the `distance` variable. ```{r} head(nycflights$distance, 20) ``` Notice that these values are all integers. The fact that `distance` is made up of only integer values (whole numbers) means there is a finite (limited) number of choices for the value of `distance`, which makes "stacked" points almost inevitable for large data sets. Let's look at how many unique distance's were traveled. ```{r} ``` Note that our data set has 32735 observations in total, but only 204 different distances were traveled. ### Exercise 9 **Question:** Replicate the plot at the bottom of this page: https://openintrostat.github.io/oilabs-tidy/02_intro_to_data/intro_to_data.html **Hint:** The data frame plotted only contains flights from American Airlines, Delta Airlines, and United Airlines, and the points are `color`ed by `carrier`. Once you replicate the plot, determine (roughly) what the cutoff point is for departure delays where you can still expect to get to your destination on time.   #### Solution Recall that the desired plot only contains observations for flights flown by Delta Airline, American Airlines, and United Airlines. This necessitates that we `filter` out the data based on this condition. Let's start by filtering the flights flown by Delta (`DL`), American (`AA`), and United (`UA`).

```{r} # filtering flights from Delta, American, and United ``` ```{r} # plotting the filtered data, colored by carrier ``` In addition to replicating the above plot, we were also asked to determine what the "cutoff" point was for a flight that departed late to still arrive on time. ```{r} #determine what departure/arrival flights are considered "on time" vs. "delayed" based on the 5 min criteria from exercise 6 ``` ```{r} #find the max flight departure delay that does not results in a delayed arrival ``` ##### Fun Fact about the `color = ` Option The `color = ` option gets applied to any items you add to the plot after the `aes` function. To see this, observe what happens when we use the `geom_smooth` function to add trend lines for each carrier. ```{r} # ggplot(nycflights_filtered) + # aes(x = dep_delay, y = arr_delay, color = carrier) + # geom_point() + # geom_smooth(method = "lm", se = FALSE) + # labs(title = "Depature Delay Time vs. Arrival Delay Time", # subtitle = "Plotted by Carrier for Delta Airlines, American Airlines, and United Airlines") + # xlab("Length of Departure Delay (in minutes)") + # ylab("Length of Arrival Delay (in minutes)") ``` Notice that adding lines to the plot also added lines to the legend.