| 1. |
- Kalmendal, André, 1989-, et al.
(författare)
-
Visible learning, best practice or boondoggle? : Challenges in assessing a meta-meta-analysis
- 2023
-
Ingår i: Presented at Unconference on Open Scholarship Practices in Education Research, Centre for Open Science, Charlottesville, United States of America.
-
Konferensbidrag (refereegranskat)abstract
- In 2009, John Hattie released the meta-meta-review Visible Learning which summarized 800 meta-analyses into 138 possible influences on student achievement. The influences were all re-coded to a standard metric (Cohen’s d) and ranked based on their effect sizes, ranging from negative (e.g. retention), little effect (e.g., student personality), to strong influences on student achievement (e.g., Response to intervention). To this day, the general criticism has focused on discovering examples of flaws in Hattie’s approach which has been referred to as cherry-picking by proponents of Visible Learning. The purpose of this project is to conduct a rigorous systematic assessment of the presented material. This talk will go through the syntheses made in Visible Learning and also how the quality assessment of the material is done. For example, previous research indicates that several influences have combined meta-analyses despite not having similar population, intervention, comparison groups, outcomes, and study types (PICOS). The talk will also contain a practical demonstration of the codesheet and coding of the influences. The approach taken includes resources when conducting or assessing any type of meta-review.
|
|
| 2. |
- Nilsonne, Gustav, et al.
(författare)
-
”Sluta betala för att få publicera forskning”
- 2023
-
Ingår i: Svenska Dagbladet. - 1101-2412. ; :2023-03-28
-
Tidskriftsartikel (populärvet., debatt m.m.)abstract
- Vetenskapliga tidskrifter som gömmer sina forskningsresultat bakom betalväggar har spelat ut sin roll. Nu har vi chansen att få 500 miljoner mer till forskning – bara genom att säga nej till tidskrifterna, skriver debattörer.
|
|
| 3. |
- Nordström, Thomas, Filosofie doktor i psykologi, 1977-, et al.
(författare)
-
Evaluating the Simple View of Reading Model : Longitudinal Testing and Applicability to the Swedish Language
- 2023
-
Ingår i: Presented at the Quantitative research methods in education conference, QRM, Gothenburg, Sweden, June 12-13.
-
Konferensbidrag (refereegranskat)abstract
- This study investigates the challenges associated with statistically evaluating the Simple View of Reading model (SVR) and its applicability to the Swedish language. The SVR model, a widely-accepted and popular framework, posits that reading comprehension (RC) is a product of two independent factors: decoding (D) and language comprehension (LC), expressed as RC = D x LC. While various statistical approaches have been employed to validate the model in English, a deep and non-transparent orthography, a consensus on a formal testing method has not been reached. Additionally, the model's functionality in other languages, such as Swedish, which has a semi-transparent orthography, remains unclear. This study has two primary objectives: 1) to longitudinally test the SVR model's validity from year 1 to 3, examining the relative contributions of decoding and language comprehension factors to reading comprehension over time using a latent variable approach, and 2) to assess the model's applicability to the Swedish language. To achieve these goals, we utilize an extensive dataset from the LegiLexi foundation, comprising data from 43,127 students across 2,666 schools and 18,006 classes. The presentation will discuss methodological and statistical considerations necessary for evaluating the SVR model, as well as the contributions of decoding and language comprehension factors to reading comprehension during primary school's learning-to-read process.
|
|
| 4. |
- Nordström, Thomas, Filosofie doktor i psykologi, 1977-, et al.
(författare)
-
Risk of bias and open science practices in systematic reviews of educational effectiveness : A meta-review
- 2023
-
Ingår i: Review of Education. - : John Wiley & Sons. - 2049-6613. ; 11:3
-
Tidskriftsartikel (refereegranskat)abstract
- In order to produce the most reliable syntheses of the effectiveness of educational interventions, systematic reviews need to adhere to rigorous methodological standards. This meta-review investigated risk of bias occurring while conducting a systematic review and the presence of open science practices like data sharing and reproducibility of the review procedure, in recently published reviews in education. We included all systematic reviews of educational interventions, instructions and methods for all K-12 student populations in any school form with experimental or quasi-experimental designs (an active manipulation of the intervention) with comparisons and where the outcome variables were academic performance of any kind. We searched the database Education Resources Information Center (ERIC) through the years 2019–2021. In parallel we hand-searched four major educational review journals for systematic reviews: Educational Research Review (Elsevier), Educational Review (Taylor & Francis), Review of Education (Wiley), and Review of Educational Research (AERA). Systematic reviews were assessed with the risk of bias tool ROBIS and whether the studies had pre-registered protocols, shared primary research data, and whether a third party could reproduce search strings and details of where exactly primary research data were extracted. A total of 88 studies that matched our PICOS were included in this review; of these, 10 educational systematic reviews were judged as low risk of bias (approximately 11%) . The rest were classified as high risk of bias during a shortened ROBIS assessment or assessed as high risk or unclear risk of bias following a full ROBIS assessment. Of the 10 low risk of bias reviews, 6 had detailed their search sufficiently enough for a third party to reproduce, 3 reviews shared the data from primary studies, however none had specified how and from where exactly data from primary studies were extracted. The study shows that at least a small part of systematic reviews in education has a low risk of bias, but most systematic reviews in our set of studies have high risk of bias in their methodological procedure. There are still improvements in this field to be expected as even the low risk of bias reviews are not consistent regarding pre-registered protocols, data sharing, reproducibility of primary research data and reproducible search strings.
|
|
| 5. |
- van den Akker, Olmo R., et al.
(författare)
-
Increasing the transparency of systematic reviews : presenting a generalized registration form
- 2023
-
Ingår i: Systematic Reviews. - : BioMed Central (BMC). - 2046-4053. ; 12
-
Tidskriftsartikel (refereegranskat)abstract
- This paper presents a generalized registration form for systematic reviews that can be used when currently available forms are not adequate. The form is designed to be applicable across disciplines (i.e., psychology, economics, law, physics, or any other field) and across review types (i.e., scoping review, review of qualitative studies, meta-analysis, or any other type of review). That means that the reviewed records may include research reports as well as archive documents, case law, books, poems, etc. Items were selected and formulated to optimize broad applicability instead of specificity, forgoing some benefits afforded by a tighter focus. This PRISMA 2020 compliant form is a fallback for more specialized forms and can be used if no specialized form or registration platform is available. When accessing this form on the Open Science Framework website, users will therefore first be guided to specialized forms when they exist. In addition to this use case, the form can also serve as a starting point for creating registration forms that cater to specific fields or review types.
|
|
