| 1. |
|
|
| 2. |
- Carlsson, Rickard, 1984-, et al.
(författare)
-
Community Augmented Meta-analysis
- 2021
-
Ingår i: Swedish Reproducibility Network (SweRN) and 2nd annual conference of Open Science Community Sweden, Stockholm, November 25, 2021.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- The presentation covers an overview of community-augmented meta-analysis (CAMA) and why CAMAs are to be preferred over traditional meta-analyses, for example that CAMAS can easily be updated when new research is published but also that syntheses are accessible and interactive to any user. The presentation also covers newly started and planned CAMAs; Evidence in learning and didactics (ELD) and in Disability research.
|
|
| 3. |
- Kalmendal, André, 1989-, et al.
(författare)
-
Protocol: Strategy instruction for improving short‐ and long‐term writing performance on secondary and upper‐secondary students : A systematic review
- 2024
-
Ingår i: Campbell Systematic Reviews. - : John Wiley & Sons. - 1891-1803. ; 20:2
-
Tidskriftsartikel (refereegranskat)abstract
- This is the protocol for a Campbell systematic review. The objectives are as follows. This review aims to investigate the effectiveness of all types of teacher-delivered classroom-based strategy instruction aimed at students in the general population (all students) including struggling students (with or at-risk of academic difficulties) in ages 12–19 for increasing writing performance. The majority of previous reviews scoped all outcomes presented in the primary studies. This review will solely focus on covering three most common outcomes: story quality, story elements and word count/length.
|
|
| 4. |
- Kalmendal, André, 1989-, et al.
(författare)
-
Visible learning, best practice or boondoggle? : Challenges in assessing a meta-meta-analysis
- 2023
-
Ingår i: Presented at Unconference on Open Scholarship Practices in Education Research, Centre for Open Science, Charlottesville, United States of America.
-
Konferensbidrag (refereegranskat)abstract
- In 2009, John Hattie released the meta-meta-review Visible Learning which summarized 800 meta-analyses into 138 possible influences on student achievement. The influences were all re-coded to a standard metric (Cohen’s d) and ranked based on their effect sizes, ranging from negative (e.g. retention), little effect (e.g., student personality), to strong influences on student achievement (e.g., Response to intervention). To this day, the general criticism has focused on discovering examples of flaws in Hattie’s approach which has been referred to as cherry-picking by proponents of Visible Learning. The purpose of this project is to conduct a rigorous systematic assessment of the presented material. This talk will go through the syntheses made in Visible Learning and also how the quality assessment of the material is done. For example, previous research indicates that several influences have combined meta-analyses despite not having similar population, intervention, comparison groups, outcomes, and study types (PICOS). The talk will also contain a practical demonstration of the codesheet and coding of the influences. The approach taken includes resources when conducting or assessing any type of meta-review.
|
|
| 5. |
- Nordström, Thomas, Filosofie doktor i psykologi, 1977-, et al.
(författare)
-
Meta-Review of Systematic Reviews in Educational Research : Risk of Bias and Open Practices
- 2022
-
Ingår i: 2022 Unconference on Open Scholarship Practices in Education Research, Centre for Open Science, Charlottesville, United States of America.
-
Konferensbidrag (övrigt vetenskapligt/konstnärligt)abstract
- As part of a larger meta-review project that aims to assess the quality of systematic reviews of educational intervention, this study provides a snapshot of reviews published between 2019 and 2021. Main goal was to assess the current state of literature and have the best studies published as CAMAs. We looked for reviews that investigated educational interventions’ effectiveness for the k-12 population using experimental designs (RCT, QED, SCD). We searched for systematic reviews in the ERIC database and four journals which publish educational reviews. Studies that were included in the full-text screening were assessed using the ROBIS (risk of bias in systematic reviews) tool, first by assessing if the PICOS fit ours, then moved on to first stage ROBIS screening, which was conducted for all articles included in the full-text reading phase. Preliminary results of the first stage ROBIS screening indicate the lack of preregistration and data sharing practice, no standardized approach in conducting searches and reporting results, and often absent quality check of studies included in the reviews.
|
|
| 6. |
- Nordström, Thomas, Filosofie doktor i psykologi, 1977-, et al.
(författare)
-
Risk of bias and open science practices in systematic reviews of educational effectiveness : A meta-review
- 2023
-
Ingår i: Review of Education. - : John Wiley & Sons. - 2049-6613. ; 11:3
-
Tidskriftsartikel (refereegranskat)abstract
- In order to produce the most reliable syntheses of the effectiveness of educational interventions, systematic reviews need to adhere to rigorous methodological standards. This meta-review investigated risk of bias occurring while conducting a systematic review and the presence of open science practices like data sharing and reproducibility of the review procedure, in recently published reviews in education. We included all systematic reviews of educational interventions, instructions and methods for all K-12 student populations in any school form with experimental or quasi-experimental designs (an active manipulation of the intervention) with comparisons and where the outcome variables were academic performance of any kind. We searched the database Education Resources Information Center (ERIC) through the years 2019–2021. In parallel we hand-searched four major educational review journals for systematic reviews: Educational Research Review (Elsevier), Educational Review (Taylor & Francis), Review of Education (Wiley), and Review of Educational Research (AERA). Systematic reviews were assessed with the risk of bias tool ROBIS and whether the studies had pre-registered protocols, shared primary research data, and whether a third party could reproduce search strings and details of where exactly primary research data were extracted. A total of 88 studies that matched our PICOS were included in this review; of these, 10 educational systematic reviews were judged as low risk of bias (approximately 11%) . The rest were classified as high risk of bias during a shortened ROBIS assessment or assessed as high risk or unclear risk of bias following a full ROBIS assessment. Of the 10 low risk of bias reviews, 6 had detailed their search sufficiently enough for a third party to reproduce, 3 reviews shared the data from primary studies, however none had specified how and from where exactly data from primary studies were extracted. The study shows that at least a small part of systematic reviews in education has a low risk of bias, but most systematic reviews in our set of studies have high risk of bias in their methodological procedure. There are still improvements in this field to be expected as even the low risk of bias reviews are not consistent regarding pre-registered protocols, data sharing, reproducibility of primary research data and reproducible search strings.
|
|
