Figure A.1. A scientific method that incorporates the hypothetico-deductive approach and falsificationist procedure. If the hypothesis was supported, something positive is now known about that phenomenon and other aspects can be examined by constructing and testing other hypotheses. If the hypothesis was rejected, something else is known about that phenomenon, albeit something negative. At the same time other hypotheses should also be constructed and tested. As you can see, via the hypothetico- deductive approach, it is possible to go on learning about things forever. Consequently, there is always the possibility that a new hypothesis and test will show a previous piece of "knowledge" to be false. This self-correcting mechanism is an important aspect of the scientific method. The falsificationist procedure : The falsificationist procedure is a simple way of increasing the power of conclusions deduced using the hypothetico-deductive approach. It merely involves taking the prediction (hypothesis) of an effect (H1 above) and creating a null hypothesis . For the purpose of this course, we will state that a null hypothesis (H0) predicts no effect or no difference between two or more tested samples. The reason for doing this is that hypotheses can be disproved much more easily than they can be proved. When we are formulating statistically testable hypotheses, they need to meet certain criteria. A good hypothesis is one that is both specific and testable. Specific:

One of the statistics that measures the central tendency of a variable is the mean. The mean is more commonly known as the “arithmetic average.” The mean of a sample (X ̄ ) is calculated as the sum of all measurements in the sample divided by the sample size ( n ). However, the mean is only a good estimate of the central tendency of a set of data if the data’s distribution is bell-shaped (symmetric single-humped with thin tails). Mean = X ̄ = (X +X +X +...+X ) / n = ∑X / n When is it OK to use the mean? Rule of Thumb When is it OK to use the mean to describe a data set? We can set out a rule of thumb. Let’s say that a distribution is “bell-shaped” if it is: symmetric single humped thin tailed Then our rule of thumb is that when a distribution is bell-shaped, the use of the mean value to describe the data is OK. Median The median is a measure of central tendency that works even if the data doesn’t fit these requirements, so it is often a better measure than the mean. The median is the measurement located at the middle of the ordered set of data. In other words, there are just as many observations larger than the median as there are smaller. If the sample size is odd, the median is the middle measurement of the ordered series. If the sample size is even, the median is the average between the two middle measurements. For example, Series A: 1.5, 3.7, 3.9, 4.5, 6.3, 7.1, 8.0, 8.8, 9.4 Series B: 1.5, 3.7, 3.9, 4.5, 6.3, 7.1, 8.0, 8.8, 9.4, 10.5 The median for Series A is 6.3 and the median for Series B is (6.3 + 7.1) ÷ 2 = 6.7 1 2 3 i i

The MIT has several cognitive measures. Prior to taking any form of the test, subjects are presented with a pre-test that allows them to adjust to the format of the test. A series of ten items flashes on the screen. The item type (either a word or image) varies depending on the test stimulus type chosen. Participants identify the item using the arrow keys. Once the participant has completed the pre-test they will begin the memory recall test. The picture memory tests [pictures, faces, designs, and kanji] flash images onto the screen, while the word memory test flashes written words. In the auditory test, the subject wears headphones and listens to lists of words with no visual cues. Each version of the MIT consists of four memory tests and a reaction time test: Tests 1, 2, and 3 are identical. Each presents a target list of twenty items and then a recognition list of fifty items. The recognition list consists of the twenty target items randomly interspersed among thirty additional items (referred to as distractors). The subject identifies which items they recognize from the previously presented target list. Test 4 presents an additional recognition list of sixty items, consisting of ten items from each of the target lists of Tests 1, 2, and 3, together with thirty distractors. The subject is asked to identify which of the items in the recognition list appeared in the three previously presented target lists. Test 5 is a test of reaction times only, independent of any memory effects. It presents a group of fifty items, consisting of twenty squares and thirty circles. The subject is required to identify which items are squares and which are circles, and the computer records his or her reaction time on each identification. Regardless of which type of test is taken the subject is exposed to the same three lists of items in the same order. In addition to recording right and wrong answers, the program measures reaction time. The computerized test takes approximately fifteen minutes to complete, and an additional five minutes to fill out the demographic survey. Please keep in mind that the MIT is not a measure of intelligence or education. It simply tests the subject's memory at a particular point in time. Results can vary widely, depending on many factors including sleep, stress, time of day, etc. This variability is one of the most interesting aspects of the test, and it is what allows students to formulate and test research hypotheses. Once you have had the option of taking the MIT, you can begin to think about a factor you would like to test. For example, do people recall pictures and words differently? Take a look at the many independent variables available in the database to get an idea of what you might like to test. Think about a factor in which you are genuinely curious - you will be expected to write an individual lab report on this topic, and it is much easier to write about something that interests you. MIT Manual This manual will guide you through the web interface for comparing two groups using the aggregated database. To begin, we'll walk you through the process of defining the data, specifically focusing on the dependent variables that can be extracted from the aggregated database . You will get a chance to practice this process during lab in week 1.

experiment (Stimulus Type: Pictures), while green bars represent independent variable group B of each experiment (Stimulus Type: Word). The respective effect sizes (the absolute difference between the medians of the two compared groups) are displayed in each panel's top-right corner. To calculate the p-value, you'll need to run a simulation using the resampling method. To run the simulation, simply click on the gray button labeled in red as ' Calculate p-value ' ( Figure A.5b ). Figure A.5c. Once the button is pushed the simulation will start ( Figure A.5c ) and might take a few seconds to finish, depending on the sample size. Figure A.5d. In Figure A.5d , both experiments have been assessed. The two new graphs at the center in the lower panel illustrate crucial metrics: A histogram of the simulated effect sizes , the observed effect size and the p-value . Considering these results, would you deem them statistically significant? Would you choose to either reject or fail to reject your null hypothesis?

Figure A.6b. 2. Choosing Independent Variables: Read through the header questions in each panel to pinpoint your independent variable of interest (e.g. “What time of day the MIT was completed [start]”). Selecting Two Groups: Proceed to select your two groups for comparison (eg. ‘morning’ & ‘afternoon’). You are only able to compare groups listed under each independent variable. The order in which you choose "Group A" and "Group B" has no bearing on your results. (as illustrated in Figure A.6c ). Figure A.6c. Variable Selection Criteria: Note that due to the ongoing nature of this project, some independent variables may not display all possible options.

effect size (>=13ms), which is p <0.01%. This indicates that results as extreme as 13ms happen under the Null Hypothesis less than 0.01% of the time. The Aggregated Database In Table 1 , you'll find a list with the questions for various sub-windows as they appear on the web interface. It's important to note that if the sample size (n) falls below 50, the data may not be accessible for testing. It's worth keeping in mind that this database is dynamic, and the available data will undergo changes throughout the quarter as data gets added. Table 1: Questions of Demographic Sub-Windows 0-7 Category 0 1 2 3 4 5 Theme Situational Demographic Language Background History Substanc Use Questions Day of Week the MIT was completed. Age Fluent in how many languages Country of Birth Have you ever received special education services? Coffee Frequency/ How long a How fast the MIT was completed. Gender Primary Language Use (Spoken) City of Birth Ever had loss of consciousness? Alcohol Frequency/ How long a What time of day the MIT was started. Orientation Primary Language Use (Reading) If you live in the US which part if yes duration (indicate worst) Tea Frequency/ How long a Is this your first time performing the URI- UCLA Memory Interference Test? Race/Ethnicity Primary Language Use (Writing) How many years have you lived here Loss of consciousness incident Caffeinated Frequency/ How long a

Last time performed this task: Education COMPLETED Primary Language Spoken by Father What best describes your area Do you have family history of left handedness? Tobacco Frequency/ How long a Please indicate what trial this is: Ethnic Group Primary Language Spoken by Mother Handedness Fluent in how many languages? 8. Appendix: An Example of Comparing Two Groups Using Resampling (Used in Scientific Methodology and the MIT) Based on Section 6.3 of “Understanding Data: an Experimental Approach to Statistics” by Alan Garfinkel and Yina Guo In a hypothetical case study, we are evaluating a potential treatment for an immune disease by testing its effect on the number of T-cells in culture (more T-cells means the treatment is better). This test is unpaired (also called independent because the data come from two different groups of subjects. We could have made this a paired test by applying both treatments to pairs of genetically identical T-cells. We have 49 measurements in the Control group and 45 measurements in the Treatment group, depicted in bee swarm and box plots in Figure AA.1 below. Since the data is not “bell-shaped” with unimodal, symmetric, and thin-tailed distribution, we should use the descriptive statistic of the median. Our effect size is the difference in medians, which is 69.5 - 59.8 = 9.7 T-cells per unit area (calculated as Treatment - Control since Treatment is larger). Return to top  

Figure AA.1. Experimental hypothesis: Treatment plates have a different median number of T-cells per unit area than Control plates. Null hypothesis: There is no difference in the number of T-cells between the Control and Treatment plates (and any observed difference is due to random sampling). When we combine the two groups of collected measurements (in a “Big Box”) and resample new datasets 10,000 times, we found results equally or more extreme than our difference of medians never occurred. Thus, the p-value is less than 1 in 10,000 (0.0001). Therefore, we can conclude that the drug treatment produced an absolute increase in T-cell numbers that was statistically significant and reject our null hypothesis.

Figure AA.2. To learn more about statistical analysis, we recommend taking LS 40: Statistics of Biological Systems and reading its textbook from which this example was taken.