Article Evaluation Worksheet 07042023.docx657880ac9215620608 (2)

docx

School

Moi University *

*We aren’t endorsed by this school

Course

1016

Subject

Sociology

Date

Nov 24, 2024

Type

docx

Pages

Uploaded by BrigadierBarracuda1555

INFO 4080/5080 Research Methods Final Exam Article Evaluation STEP-BY-STEP GUIDE FOR EVALUATION WORKSHEET Complete and submit this document as your final exam. Address each item that is applicable to your article. Do not answer simply as “yes” or “no”, but include your thoughts on whether quality of the item and how it impacts on the overall usability of the article. Your Name: Naga Valli Kiranmayee Kaja Provide Article Citation: Evaluate The Journal of Academic Librarianship article by Gutierrez, C., & Wang, J. (2001), titled “ A comparison of an electronic vs. Print Workbook for Information Literacy Instruction” 1. Framework (Everything prior to the detailed description of the researcher's methods: Problem Statement, Literature Review, Hypotheses, and Definitions) Problem Statement (State the problem of the study either by quoting it or, if no problem statement was given, in your own words. Is the problem stated in such a way as not to imply bias toward the subject studied? Is the problem actually solvable - what factors will have to be identified?) Definitions (Are definitions of important constructs provided? Are they sufficient for the understanding of this study? .Are definitions conceptual (based on theory) or operational (made up for this study?- the article does not include definitions) Literature Review (What is the researcher's orientation? Do they seem to have the expertise to perform the research? Are recent works included? Are both sides of issues presented? Is a historical basis for the study provided? Are findings explained or just "box-scored"? Is "probable cause" for the expected results provided? This is typically done by presenting one or more theories that provide a foundation for why the study should have useful results) Literature Review:

Researcher's Orientation: The researchers seem to have a neutral orientation towards the topic. They present a balanced review of previous research on both print and electronic workbooks, acknowledging the advantages and disadvantages of each format. They do not appear to have a personal preference or bias towards one format over the other. Expertise: The researchers are librarians with experience in information literacy instruction. They demonstrate their expertise by providing a comprehensive and accurate review of the literature, including relevant references and citations. They also demonstrate their understanding of the research design process. Recency of Works: The literature review includes several recent works, including studies published within the past few years. This suggests that the researchers are aware of the latest research on the topic. Both Sides of Issues: The researchers present both sides of the issue by discussing the advantages and disadvantages of both print and electronic workbooks. They acknowledge that there is no single "best" format and that the most effective format may vary depending on the specific context and needs of the learners. Historical Basis: The literature review provides a brief overview of the historical development of print and electronic workbooks. This helps to contextualize the current research and provides a foundation for understanding the evolution of information literacy instruction. Findings Explained: The researchers do not just "box-score" the findings of previous research. They explain the results of each study and discuss their implications for the current research. They also identify gaps in the existing literature and highlight areas for future research. Probable Cause: The researchers provide several theoretical frameworks to explain why electronic workbooks might be more effective than print workbooks. These include the potential for

interactivity, multimedia content, and immediate feedback. They also acknowledge that print workbooks may have certain advantages, such as lower cost and familiarity. Overall, the literature review is well-written and informative. It demonstrates the researchers' expertise and understanding of the topic, and it provides a strong foundation for the current research. Variables (What are the independent and dependent variables? Are the independent and dependent variables stated or clearly implied? Are the variables adequately related to the research question(s)? Are other important variables held constant or otherwise accounted for?) Variables: Independent Variable:  Workbook format: This is the independent variable, as it is the factor that the researchers are manipulating (print vs. electronic). Dependent Variable:  Student performance on post-test: This is the dependent variable, as it is the outcome that the researchers are measuring to see if it is affected by the independent variable. Independent and Dependent Variables Stated or Implied: While not explicitly stated as "independent" and "dependent," the roles of workbook format and student performance on the post-test are clearly implied throughout the article. Relationship to Research Question: The independent and dependent variables are directly related to the research question, which is: "Are electronic workbooks more effective than print workbooks for teaching information literacy skills to freshman college students?" By manipulating the workbook format (independent variable) and measuring student performance (dependent variable), the researchers can answer this question.

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Control of Other Variables: The researchers attempt to control for other important variables by:  Using a quasi-experimental design with two groups (print and electronic)  Randomly assigning students to groups  Administering the same pre- and post-tests to both groups  Providing the same lecture on information resources to both groups  Collecting demographic data on the students to identify any potential confounding variables Overall, the researchers have adequately defined and accounted for the independent and dependent variables, ensuring that their study is valid and reliable. Hypothesis (What is the hypothesis (hypotheses)? Provide the hypothesis (hypotheses) as stated in the article and/or ones that you inferred. If stated, is it a null or directional format? If hypotheses are not stated in the null provide the hypotheses in a null format in addition to those above. Are the stated hypotheses, if any, related to the statement of the problem?) Hypotheses: The article explicitly states two hypotheses: 1. Students will prefer using the electronic workbook to the print workbook. (This is a directional hypothesis) 2. Students assigned the electronic workbook will perform better on post-tests than those who receive a print workbook. (This is a directional hypothesis) In addition, the following null hypotheses can be inferred:  H01: There will be no significant difference in student preference for the electronic vs. print workbook.  H02: There will be no significant difference in post-test performance between students using the electronic and print workbooks. Relationship to the Problem Statement:

The stated hypotheses are directly related to the problem statement, which investigates the effectiveness of electronic workbooks compared to print workbooks for teaching information literacy skills. The hypotheses address the student's preference and performance, which are key aspects of evaluating the effectiveness of a teaching tool. The null hypotheses are also logically related to the problem statement, as they provide the baseline assumption that there is no difference between the electronic and print workbooks in terms of student preference and performance. Rejecting the null hypotheses would provide evidence in favor of the alternative hypotheses, which suggest that the electronic workbook is superior in these aspects. Overall, the hypotheses and null hypotheses are well-defined and closely aligned with the research problem, providing a clear framework for the study. 3. Methodology (Population , Describe the population. Is the population stated or did it have to be inferred? If the latter, how difficult was it to infer the population? Does the derivation of the population make sense based on the literature? Is the population mostly opportunistic?): Sample (Describe the sample. Was the sample drawn at random? If not, does it represent a clearly biased portion of the population? Is the sample size appropriate?) Population: Description: The population is not explicitly stated in the article. However, it can be inferred that the population is all freshman college students who are enrolled in a library instruction program. Inference Difficulty: Inferring the population was a relatively straightforward process, as the context of the study and the research question clearly point towards freshmen enrolled in library instruction. Derivation and Literature: The derivation of the population is consistent with the literature on information literacy instruction, which often focuses on undergraduate students as they develop their research skills. The focus on freshmen is also logical, as they are new to the academic

environment and likely require more support in developing their information literacy skills. Opportunistic: The population is not entirely opportunistic, as the researchers are not simply studying a readily available group of students. They are specifically targeting a population relevant to their research question: freshmen who need support in developing their information literacy skills. . Sample (Describe the sample. Was the sample drawn at random? If not, does it represent a clearly biased portion of the population? Is the sample size appropriate?) Sample: Description: The sample consisted of 134 students enrolled in six freshman seminar classes. The researchers assigned students within these classes to either the print or electronic workbook group. Random Sampling: The article does not explicitly state whether the students were randomly assigned to groups. However, the use of a quasi-experimental design with two groups suggests some form of controlled assignment, potentially involving random allocation. Bias and Representation: Without further information about the selection process, it is difficult to assess whether the sample represents a biased portion of the population. However, the fact that the students were drawn from six different freshman seminar classes suggests some diversity in the sample. Sample Size:

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

The sample size of 134 students is considered moderate for a study of this nature. While a larger sample size would have been preferable, the current size is sufficient to provide statistically significant results, particularly given the quasi-experimental design. Overall, while the information provided about the population and sample could be more detailed, the available information suggests that the researchers have chosen a relevant and representative sample for their study Data Gathering (Briefly describe the process used to gather data including its clarity of the data gathering description. What are strengths and weaknesses of the process? Impact on validity? Could you replicate the data gathering process with the information provided?) Instrumentation (Was the instrument developed for this study or was an existing instrument used? Is the instrument used appropriate to the research questions? Was a portion or all of the instrument provided with the report or available online? If not, how significant an impact is there on your ability to judge the report? .If accessible, comment on the quality of the survey/questionnaire.Was the instrument pilot tested to work out problems? Was reliability of the instrument addressed? Was data provided? (e.g., split-half, test-retest, parallel form, inter coder reliability) Was validity of the instrument addressed , i.e., is the instrument really measuring what it is supposed to? (validity-face, concurrent, construct (factor analysis)) Method (Describe the method used in the study, e.g., Survey, Quasi Experimental, Experimental, Historical, or Qualitative. Are important details of the method used provided, including important descriptions? e.g. characteristics of assistants, time of day, room conditions, and other characteristics. If experimental design, chart and describe the design) Provide a brief summary of what occurred in the implementation of the study including response rates and any problems the researchers experienced. Are the results clearly provided in both tabular and narrative form? Are all results provided? If not, does the missing data jeopardize the study. Are all observations attributed to data collected? .Is unimportant, confusing detail avoided? Do the statistical analyses used make sense given the level of data of the variables and other factors? Does the results section flow clearly by interweaving statistics (when appropriate) and narration? Ideally, the narration should make sense when read without the numbers.

Discussions/Conclusion ( Does the Discussion/Conclusion section begin with a brief summary of the study's purpose? Is the hypothesis (either one stated or one that you inferred) addressed? Is the hypothesis addressed in terms of rejection? If the hypothesis was rejected, did the researcher discuss possible Type I errors (That the hypothesis was rejected in error and should not have been)? If the hypothesis was not rejected, did the researcher discuss possible Type II errors (That the hypothesis should have been rejected and was not)? Briefly summarize the conclusions. Are conclusions warranted by the data? Are findings discussed in terms of the literature search? If the results are contrary to the literature search, does the author address this?) As an example: “This study found that research chemists left their lab and walked across campus to the library to conduct an online search. This was not expected in light of Roberts's (1985) finding. However, in this study, the research chemists had been used to conducting searches on the office computer, but the modem was malfunctioning. In Roberts study the researchers had no record of online searching.” .Are alternate explanations for the study’s findings considered and set aside? Discussion/Conclusion: Purpose Summary: The discussion section does not explicitly begin with a brief summary of the study's purpose. However, it does quickly establish the context and research question. Hypothesis Addressing: The article addresses the two originally stated hypotheses: 1. Students will prefer using the electronic workbook to the print workbook. (This hypothesis was supported by the findings) 2. Students assigned the electronic workbook will perform better on post-tests than those who receive a print workbook. (This hypothesis was not supported by the findings) Type I and Type II Errors:

The article does not explicitly discuss the possibility of Type I or Type II errors. This omission weakens the discussion section and leaves room for misinterpretation of the results. Conclusion Summary: The main conclusion is that students prefer using electronic workbooks over print workbooks, but there is no significant difference in their performance on post-tests between the two formats. The study also suggests that the frequency of library usage is the most significant factor in information literacy development, regardless of the workbook format. Warranting Conclusions: While the conclusions generally align with the data, the discussion section lacks a deeper analysis and explanation of the findings. This makes it difficult to fully assess whether the conclusions are completely warranted. Literature Comparison: The article mentions relevant literature to support its findings, but it could benefit from a more comprehensive discussion of how the results relate to existing research. Addressing any discrepancies or inconsistencies would strengthen the overall argument. Alternate Explanations: The discussion section does not adequately consider alternative explanations for the findings. This limits the depth and comprehensiveness of the analysis. Exploring potential confounding variables and alternative interpretations would enhance the overall validity of the study. Overall, the discussion/conclusion section provides a basic overview of the results, but it lacks depth and critical analysis. Addressing the limitations mentioned above would significantly improve the quality and usefulness of this section. 5. Recommendations Briefly summarize the recommendations, if any. .Are recommendations justified by the findings?

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

.Check all internal and external validity factors. Recommendations: The article recommends the following:  Librarians should consider using electronic workbooks for information literacy instruction due to their user-friendliness and potential cost savings.  Librarians need to find ways to convince students of the value of information literacy instruction and encourage them to use the library more frequently.  Faculty should incorporate assignments into their courses that require students to use library resources, which can help students develop and practice their information literacy skills. Justification: The recommendations are partially justified by the findings. The study found that students prefer using electronic workbooks and that the frequency of library usage is the most important factor in information literacy development. However, the lack of significant performance difference between the two workbook formats weakens the argument for recommending electronic workbooks over print ones. Validity Factors: Internal Validity:  Quasi-experimental design: The non-random assignment of participants raises concerns about internal validity and potential bias.  Limited control over extraneous variables: Other factors, such as prior knowledge or learning styles, could influence the results.  Unclear information about data analysis: Makes it difficult to assess the validity of the findings. External Validity:  Limited sample size: Limits the generalizability of the findings to other populations.  Specific context of freshman seminar classes: May not be applicable to other settings.

 Focus on one information literacy program: Findings might not be applicable to other programs with different approaches. Overall, the recommendations are somewhat justified by the findings, but the limitations of the study weaken their overall strength. Addressing the internal and external validity issues and providing more concrete evidence would enhance the recommendations. 6. In about 200-300 words, summarize your thoughts about the quality of the article based on your input above. Include any observations not covered in 1-5 above. Conclude with a statement regarding this article that you would make to a researcher pursuing this topic at the time of the article’s publication. Overall Assessment of the Article: The article provides a framework for investigating the effectiveness of electronic workbooks for information literacy instruction. However, the research suffers from several limitations that weaken its overall quality. Strengths:  Identifies a relevant and timely topic in information literacy education.  Employs a quasi-experimental design to attempt to isolate the impact of workbook format.  Utilizes multiple data collection methods, including pre- and post-tests and a satisfaction survey.  Presents findings in both tabular and narrative formats.  Aligns conclusions with the data to some extent. Weaknesses:  Lacks information about the population and sampling process.  Provides insufficient details about the instruments used.  Does not address potential sources of bias.  Uses basic statistical analyses without clear interpretation.

 Presents a limited discussion and fails to consider alternative explanations.  Makes recommendations based on partial evidence and limited internal and external validity. Observations:  The article focuses on student preference and post-test performance, neglecting other potentially relevant outcomes of information literacy instruction.  The study design could be strengthened by incorporating additional control variables and implementing a true random assignment of participants.  The discussion and conclusion would benefit from a more thorough analysis of the findings, including acknowledging potential limitations and exploring alternative interpretations. Advice for Researchers: While the topic of electronic workbooks for information literacy instruction holds promise, further research is needed to address the limitations identified in this article. Researchers should consider:  Utilizing a stronger research design with robust internal and external validity.  Providing detailed descriptions of the instruments and data collection procedures.  Addressing potential sources of bias and conducting pilot testing to ensure instrument quality.  Employing appropriate statistical analyses and providing clear interpretations of the results.  Engaging in a comprehensive discussion that considers the literature, alternative explanations, and limitations.  Focusing on a broader range of outcomes beyond student preference and post- test performance. By addressing these issues, future research can contribute significantly to understanding the effectiveness of electronic workbooks in information literacy instruction and provide valuable insights for librarians and educators.

Your preview ends here

Eager to read complete document? Join bartleby learn and gain access to the full version

Access to all documents
Unlimited textbook solutions
24/7 expert homework help

Article Evaluation Worksheet 07042023.docx657880ac9215620608 (2)

Related Documents