| 1. |
- Botvinik-Nezer, Rotem, et al.
(författare)
-
Variability in the analysis of a single neuroimaging dataset by many teams
- 2020
-
Ingår i: Nature. - : Springer Science and Business Media LLC. - 0028-0836 .- 1476-4687. ; 582, s. 84-88
-
Tidskriftsartikel (refereegranskat)abstract
- Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.
|
|
| 2. |
|
|
| 3. |
- Aczel, Balazs, et al.
(författare)
-
Consensus-based guidance for conducting and reporting multi-analyst studies
- 2021
-
Ingår i: eLIFE. - : eLife Sciences Publications. - 2050-084X. ; 10
-
Tidskriftsartikel (refereegranskat)abstract
- Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.
|
|
| 4. |
- Anderson, Christopher J., et al.
(författare)
-
Response to Comment on "Estimating the reproducibility of psychological science"
- 2016
-
Ingår i: Science. - : American Association for the Advancement of Science (AAAS). - 0036-8075 .- 1095-9203. ; 351:6277
-
Tidskriftsartikel (övrigt vetenskapligt/konstnärligt)abstract
- Gilbert et al. conclude that evidence from the Open Science Collaboration's Reproducibility Project: Psychology indicates high reproducibility, given the study methodology. Their very optimistic assessment is limited by statistical misconceptions and by causal inferences from selectively interpreted, correlational data. Using the Reproducibility Project: Psychology data, both optimistic and pessimistic conclusions about reproducibility are possible, and neither are yet warranted.
|
|
| 5. |
|
|
| 6. |
- Bourget, M. H., et al.
(författare)
-
Microscopy-BIDS: An Extension to the Brain Imaging Data Structure for Microscopy Data
- 2022
-
Ingår i: Frontiers in Neuroscience. - : Frontiers Media SA. - 1662-4548 .- 1662-453X. ; 16
-
Tidskriftsartikel (refereegranskat)abstract
- The Brain Imaging Data Structure (BIDS) is a specification for organizing, sharing, and archiving neuroimaging data and metadata in a reusable way. First developed for magnetic resonance imaging (MRI) datasets, the community-led specification evolved rapidly to include other modalities such as magnetoencephalography, positron emission tomography, and quantitative MRI (qMRI). In this work, we present an extension to BIDS for microscopy imaging data, along with example datasets. Microscopy-BIDS supports common imaging methods, including 2D/3D, ex/in vivo, micro-CT, and optical and electron microscopy. Microscopy-BIDS also includes comprehensible metadata definitions for hardware, image acquisition, and sample properties. This extension will facilitate future harmonization efforts in the context of multi-modal, multi-scale imaging such as the characterization of tissue microstructure with qMRI.
|
|
| 7. |
- Buchanan, E. M., et al.
(författare)
-
The Psychological Science Accelerator's COVID-19 rapid-response dataset
- 2023
-
Ingår i: Scientific Data. - : Springer Science and Business Media LLC. - 2052-4463. ; 10:1
-
Tidskriftsartikel (refereegranskat)abstract
- In response to the COVID-19 pandemic, the Psychological Science Accelerator coordinated three large-scale psychological studies to examine the effects of loss-gain framing, cognitive reappraisals, and autonomy framing manipulations on behavioral intentions and affective measures. The data collected (April to October 2020) included specific measures for each experimental study, a general questionnaire examining health prevention behaviors and COVID-19 experience, geographical and cultural context characterization, and demographic information for each participant. Each participant started the study with the same general questions and then was randomized to complete either one longer experiment or two shorter experiments. Data were provided by 73,223 participants with varying completion rates. Participants completed the survey from 111 geopolitical regions in 44 unique languages/dialects. The anonymized dataset described here is provided in both raw and processed formats to facilitate re-use and further analyses. The dataset offers secondary analytic opportunities to explore coping, framing, and self-determination across a diverse, global sample obtained at the onset of the COVID-19 pandemic, which can be merged with other time-sampled or geographic data.
|
|
| 8. |
- Czeszumski, Artur, et al.
(författare)
-
#EEGManyLabs: Investigating the Replicability of Influential EEG Experiments
- 2024
-
Annan publikation (övrigt vetenskapligt/konstnärligt)abstract
- There is growing awareness across the neuroscience community that the replicability of findings on the relationship between brain activity and cognitive phenomena can be improved by conducting studies with high statistical power that adhere to well-defined and standardized analysis pipelines. Inspired by efforts from the psychological sciences, and with the desire to examine some of the foundational findings using electroencephalography (EEG), we have launched #EEGManyLabs, a large-scale international collaborative replication effort. Since its discovery in the early 20th century, EEG has had a profound influence on our understanding of human cognition, but there is limited evidence on the replicability of some of the most highly cited discoveries. After a systematic search and selection process, we have identified 27 of the most influential and continually cited studies in the field. We plan to directly test the replicability of key findings from 20 of these studies in teams of at least three independent laboratories. The design and protocol of each replication effort will be submitted as a Registered Report and peer-reviewed prior to data collection. Prediction markets, open to all EEG researchers, will be used as a forecasting tool to examine which findings the community expects to replicate. This project will update our confidence in some of the most influential EEG findings and generate a large open access database that can be used to inform future research practices. Finally, through this international effort, we hope to create a cultural shift towards inclusive, high-powered multi-laboratory collaborations.
|
|
| 9. |
- Darai-Ramqvist, Eva, et al.
(författare)
-
Microenvironment-dependent phenotypic changes in a SCID mouse model for malignant mesothelioma
- 2013
-
Ingår i: Frontiers in Oncology. - : Frontiers Media SA. - 2234-943X. ; 3
-
Tidskriftsartikel (refereegranskat)abstract
- Background and Aims: Malignant mesothelioma is an aggressive, therapy-resistant tumor. Mesothelioma cells may assume an epithelioid or a sarcomatoid phenotype, and presence of sarcomatoid cells predicts poor prognosis. In this study, we investigated differentiation of mesothelioma cells in a xenograft model, where mesothelioma cells of both phenotypes were induced to form tumors in severe combined immunodeficiency mice.Methods: Xenografts were established and thoroughly characterized using a comprehensive immunohistochemical panel, array comparative genomic hybridization (aCGH) of chromosome 3, fluorescent in situ hybridization, and electron microscopy.Results: Epithelioid and sarcomatoid cells gave rise to xenografts of similar epithelioid morphology. While sarcomatoid-derived xenografts had higher growth rates, the morphology and expression of differentiation-related markers was similar between xenografts derived from both phenotypes. aCGH showed a convergent genotype for both xenografts, resembling the original aggressive sarcomatoid cell sub-line.Conclusion: Human mesothelioma xenografts from sarcomatoid and epithelioid phenotypes converged to a similar differentiation state, and genetic analyses suggested that clonal selection in the mouse microenvironment was a major contributing factor. This thoroughly characterized animal model can be used for further studies of molecular events underlying tumor cell differentiation.
|
|
| 10. |
|
|
| 11. |
- Dreber, Anna, et al.
(författare)
-
Olika slutsatser från samma data : Den analytiska flexibiliteten i medicinsk forskning är ofta stor
- 2023
-
Ingår i: Läkartidningen. - : Läkartidningen Förlag AB. - 1652-7518 .- 0023-7205. ; 120
-
Tidskriftsartikel (refereegranskat)abstract
- Analysis of research data entails many choices. As a result, a space of different analytical strategies is open to researchers. Different justifiable analyses may not give similar results. The method of multiple analysts is a way to study the analytical flexibility and behaviour of researchers under naturalistic conditions, as part of the field known as metascience. Analytical flexibility and risks of bias can be counteracted by open data sharing, pre-registration of analysis plans, and registration of clinical trials in trial registers. These measures are particularly important for retrospective studies where analytical flexibility can be greatest, although pre-registration is less useful in this context. Synthetic datasets can be an alternative to pre-registration when used to decide what analyses should be conducted on real datasets by independent parties. All these strategies help build trustworthiness in scientific reports, and improve the reliability of research findings.
|
|
| 12. |
- Drude, Natascha Ingrid, et al.
(författare)
-
Planning preclinical confirmatory multicenter trials to strengthen translation from basic to clinical research : a multi-stakeholder workshop report
- 2022
-
Ingår i: Translational Medicine Communications. - : Springer Nature. - 2396-832X. ; 7:1
-
Tidskriftsartikel (refereegranskat)abstract
- Clinical translation from bench to bedside often remains challenging even despite promising preclinical evidence. Among many drivers like biological complexity or poorly understood disease pathology, preclinical evidence often lacks desired robustness. Reasons include low sample sizes, selective reporting, publication bias, and consequently inflated effect sizes. In this context, there is growing consensus that confirmatory multicenter studies -by weeding out false positives- represent an important step in strengthening and generating preclinical evidence before moving on to clinical research. However, there is little guidance on what such a preclinical confirmatory study entails and when it should be conducted in the research trajectory. To close this gap, we organized a workshop to bring together statisticians, clinicians, preclinical scientists, and meta-researcher to discuss and develop recommendations that are solution-oriented and feasible for practitioners. Herein, we summarize and review current approaches and outline strategies that provide decision-critical guidance on when to start and subsequently how to plan a confirmatory study. We define a set of minimum criteria and strategies to strengthen validity before engaging in a confirmatory preclinical trial, including sample size considerations that take the inherent uncertainty of initial (exploratory) studies into account. Beyond this specific guidance, we highlight knowledge gaps that require further research and discuss the role of confirmatory studies in translational biomedical research. In conclusion, this workshop report highlights the need for close interaction and open and honest debate between statisticians, preclinical scientists, meta-researchers (that conduct research on research), and clinicians already at an early stage of a given preclinical research trajectory.
|
|
| 13. |
- Gau, Rémi, et al.
(författare)
-
Brainhack : Developing a culture of open, inclusive, community-driven neuroscience
- 2021
-
Ingår i: Neuron. - : Elsevier. - 0896-6273 .- 1097-4199. ; 109:11, s. 1769-1775
-
Tidskriftsartikel (refereegranskat)abstract
- Brainhack is an innovative meeting format that promotes scientific collaboration and education in an open, inclusive environment. This NeuroView describes the myriad benefits for participants and the research community and how Brainhacks complement conventional formats to augment scientific progress.
|
|
| 14. |
- Hallinan, D., et al.
(författare)
-
International transfers of personal data for health research following Schrems II: a problem in need of a solution
- 2021
-
Ingår i: European Journal of Human Genetics. - : Springer Science and Business Media LLC. - 1018-4813 .- 1476-5438. ; 29, s. 1502-1509
-
Tidskriftsartikel (refereegranskat)abstract
- On 16 July 2020, the Court of Justice of the European Union issued their decision in the Schrems II case concerning Facebook’s transfers of personal data from the EU to the US. The decision may have significant effects on the legitimate transfer of personal data for health research purposes from the EU. This article aims: (i) to outline the consequences of the Schrems II decision for the sharing of personal data for health research between the EU and third countries, particularly in the context of the COVID-19 pandemic; and, (ii) to consider certain options available to address the consequences of the decision and to facilitate international data exchange for health research moving forward. © 2021, The Author(s).
|
|
| 15. |
- Hardwicke, Tom E., et al.
(författare)
-
Data availability, reusability, and analytic reproducibility : evaluating the impact of a mandatory open data policy at the journal Cognition
- 2018
-
Ingår i: Royal Society Open Science. - : The Royal Society. - 2054-5703. ; 5:8
-
Tidskriftsartikel (refereegranskat)abstract
- Access to data is a critical feature of an efficient, progressive and ultimately self-correcting scientific ecosystem. But the extent to which in-principle benefits of data sharing are realized in practice is unclear. Crucially, it is largely unknown whether published findings can be reproduced by repeating reported analyses upon shared data ('analytic reproducibility'). To investigate this, we conducted an observational evaluation of a mandatory open data policy introduced at the journal Cognition. Interrupted time-series analyses indicated a substantial post-policy increase in data available statements (104/417, 25% pre-policy to 136/ 174, 78% post-policy), although not all data appeared reusable (23/ 104, 22% pre-policy to 85/136, 62%, post-policy). For 35 of the articles determined to have reusable data, we attempted to reproduce 1324 target values. Ultimately, 64 values could not be reproduced within a 10% margin of error. For 22 articles all target values were reproduced, but 11 of these required author assistance. For 13 articles at least one value could not be reproduced despite author assistance. Importantly, there were no clear indications that original conclusions were seriously impacted. Mandatory open data policies can increase the frequency and quality of data sharing. However, suboptimal data curation, unclear analysis specification and reporting errors can impede analytic reproducibility, undermining the utility of data sharing and the credibility of scientific findings.
|
|
| 16. |
- Helgesson, Gert, et al.
(författare)
-
Editors publishing in their own journals : A systematic review of prevalence and a discussion of normative aspects
- 2022
-
Ingår i: Learned Publishing. - : Wiley. - 0953-1513 .- 1741-4857. ; 35:2, s. 229-240
-
Forskningsöversikt (refereegranskat)abstract
- Journal editors are the main gatekeepers in scientific publishing. Yet there is a concern that they may receive preferential treatment when submitting manuscripts to their own journals. The prevalence of such self-publishing is not known, nor the consequences for reliability and trustworthiness of published research. This study aimed to systematically review the literature on the prevalence of editors publishing in their own journals and to conduct a normative ethical analysis of this practice. A systematic review was performed using the following databases: Medline, PsycInfo, Scopus and Web of Science. Articles that provided primary data about editors publishing in own journals were included. We identified 15 studies meeting inclusion criteria. There was large variability of self-publishing across fields, journals and editors, ranging from those who never published in their own journal to those publishing extensively in their own journal. Many studies suffered from serious methodological limitations. Nevertheless, our results show that there are settings where levels of self-publication are very high. We recommend that editors-in-chief and associate editors who have considerable power in journals refrain from publishing research articles in their own journals. Journals should have clear processes in place about the treatment of articles submitted by editorial board members.
|
|
| 17. |
|
|
| 18. |
- Huntington-Klein, Nick, et al.
(författare)
-
Subjective evidence evaluation survey for many-analysts studies
- 2024
-
Ingår i: Royal Society Open Science. - : The Royal Society. - 2054-5703 .- 2054-5703. ; 11:7
-
Tidskriftsartikel (refereegranskat)abstract
- Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.
|
|
| 19. |
- Ingre, Michael, et al.
(författare)
-
Estimating statistical power, posterior probability and publication bias of psychological research using the observed replication rate
- 2018
-
Ingår i: Royal Society Open Science. - : The Royal Society. - 2054-5703. ; 5:9
-
Tidskriftsartikel (refereegranskat)abstract
- In this paper, we show how Bayes' theorem can be used to better understand the implications of the 36% reproducibility rate of published psychological findings reported by the Open Science Collaboration. We demonstrate a method to assess publication bias and show that the observed reproducibility rate was not consistent with an unbiased literature. We estimate a plausible range for the prior probability of this body of research, suggesting expected statistical power in the original studies of 48-75%, producing (positive) findings that were expected to be true 41-62% of the time. Publication bias was large, assuming a literature with 90% positive findings, indicating that negative evidence was expected to have been observed 55-98 times before one negative result was published. These findings imply that even when studied associations are truly NULL, we expect the literature to be dominated by statistically significant findings.
|
|
| 20. |
- Ingre, Michael, 1965-
(författare)
-
P-hacking in academic research : a critical review of the job strain model and of the association between night work and breast cancer in women
- 2017
-
Doktorsavhandling (övrigt vetenskapligt/konstnärligt)abstract
- P-hacking can be described as a more or less deliberate, explorative approach to data analysis with a flexible/opportunistic search space and the reporting of primarily statistically significant findings. This leads to inflated type-1 error rates and to bias in reported estimates in the scientific literature.This thesis aims to describe how p-hacking can be manifested in academic research and to illustrate how bias due to p-hacking is expected to affect the veracity of published findings using two specific examples from the literature. This thesis also argues that when evaluating published findings in the current academic environment, we should assume a priori that biases due to p-hacking and publication bias are present.The thesis used Monte Carlo simulations and systematic reviews of the literature in two specific fields: the proposed associations between exposure to night work and breast cancer in women, and between job strain and coronary heart disease.A general model and mathematical framework to predict expected bias from p-hacking was developed, and can be used for a priori defined protected inferences of any published finding, under explicit assumptions of various levels of p-hacking. The model indicated a close to 100% chance of demonstrating a false positive association in larger studies, but also showed that even minimal p-hacking results in substantial bias in estimates.The literature review identified large flexibility in the analytical process, allowing for the final model to be picked from a large pool of available models, with an implied search space of thousands of estimates. Some of the specific observations made here could be used to argue evidence for high risk of p-hacking and publication bias in the reviewed literature:None of the 17 reviewed studies on job strain and coronary heart disease reported the proper estimate of the job strain interaction (chapter 6) and our analysis showed that the proper estimate would not have been statistically significant in any of the studies (chapter 7).One study described a data driven approach with an implied search space of at least 502 models, where adjusting for confounding did not reduce the strength of the association, as would be expected, but instead increased its strength so it fell above the threshold for statistical significance (chapter 5).One study was based on a speculative and marginally significant estimate after arbitrarily restricting the analysis to a subgroup, when estimates on the full group were available and indicated a non-significant association (chapter 5).Statistical power analyses on research into night work and breast cancer indicated that statistically significant findings were over-represented in the literature (p≈.001) suggesting the presence of bias from p-hacking or selective publishing of significant findings (chapter 5).The findings also suggest that previously reported estimates in meta-analyses was likely to represent prevailing bias in the two fields reviewed here. A bias-adjusted meta-analysis on the job strain model and coronary heart disease with a total of 462,220 subjects and 6,836 CHD events indicated no support for the job strain interaction (RR=1.00; 95% CI: 0.88--1.14). In addition, it did not show an increased risk due to high job demand (RR=1.03; 95% CI: 0.97--1.11) but it did confirm previously reported risks due to low job control (RR=1.11; 95% CI: 1.03--1.20).
|
|
| 21. |
- Klein, Olivier, et al.
(författare)
-
A Practical Guide for Transparency in Psychological Science
- 2018
-
Ingår i: Collabra: Psychology. - : University of California Press. - 2474-7394. ; 4:1
-
Forskningsöversikt (refereegranskat)abstract
- The credibility of scientific claims depends upon the transparency of the research products upon which they are based (e.g., study protocols, data, materials, and analysis scripts). As psychology navigates a period of unprecedented introspection, user-friendly tools and services that support open science have flourished. However, the plethora of decisions and choices involved can be bewildering. Here we provide a practical guide to help researchers navigate the process of preparing and sharing the products of their research (e.g., choosing a repository, preparing their research products for sharing, structuring folders, etc.). Being an open scientist means adopting a few straightforward research management practices, which lead to less error prone, reproducible research workflows. Further, this adoption can be piecemeal – each incremental step towards complete transparency adds positive value. Transparent research practices not only improve the efficiency of individual researchers, they enhance the credibility of the knowledge generated by the scientific community.
|
|
| 22. |
- Koba, Cemal, et al.
(författare)
-
Spontaneous eye movements during eyes-open rest reduce resting-state-network modularity by increasing visual-sensorimotor connectivity
- 2021
-
Ingår i: Network Neuroscience. - : MIT Press - Journals. - 2472-1751. ; 5:2, s. 451-476
-
Tidskriftsartikel (refereegranskat)abstract
- During wakeful rest, individuals make small eye movements during fixation. We examined how these endogenously driven oculomotor patterns impact topography and topology of functional brain networks. We used a dataset consisting of eyes-open resting-state (RS) fMRI data with simultaneous eye tracking. The eye-tracking data indicated minor movements during rest, which correlated modestly with RS BOLD data. However, eye-tracking data correlated well with echo-planar imaging time series sampled from the area of the eye-orbit (EO-EPI), which is a signal previously used to identify eye movements during exogenous saccades and movie viewing. Further analyses showed that EO-EPI data were correlated with activity in an extensive motor and sensorimotor network, including components of the dorsal attention network and the frontal eye fields. Partialling out variance related to EO-EPI from RS data reduced connectivity, primarily between sensorimotor and visual areas. It also produced networks with higher modularity, lower mean connectivity strength, and lower mean clustering coefficient. Our results highlight new aspects of endogenous eye movement control during wakeful rest. They show that oculomotor-related contributions form an important component of RS network topology, and that those should be considered in interpreting differences in network structure between populations or as a function of different experimental conditions.
|
|
| 23. |
- Koenig, Julian, et al.
(författare)
-
Cortical thickness and resting-state cardiac function across the lifespan : A cross-sectional pooled mega-analysis
- 2021
-
Ingår i: Psychophysiology. - : Wiley. - 0048-5772 .- 1469-8986 .- 1540-5958. ; 58:7
-
Tidskriftsartikel (refereegranskat)abstract
- Understanding the association between autonomic nervous system [ANS] function and brain morphology across the lifespan provides important insights into neurovisceral mechanisms underlying health and disease. Resting-state ANS activity, indexed by measures of heart rate [HR] and its variability [HRV] has been associated with brain morphology, particularly cortical thickness [CT]. While findings have been mixed regarding the anatomical distribution and direction of the associations, these inconsistencies may be due to sex and age differences in HR/HRV and CT. Previous studies have been limited by small sample sizes, which impede the assessment of sex differences and aging effects on the association between ANS function and CT. To overcome these limitations, 20 groups worldwide contributed data collected under similar protocols of CT assessment and HR/HRV recording to be pooled in a mega-analysis (N = 1,218 (50.5% female), mean age 36.7 years (range: 12–87)). Findings suggest a decline in HRV as well as CT with increasing age. CT, particularly in the orbitofrontal cortex, explained additional variance in HRV, beyond the effects of aging. This pattern of results may suggest that the decline in HRV with increasing age is related to a decline in orbitofrontal CT. These effects were independent of sex and specific to HRV; with no significant association between CT and HR. Greater CT across the adult lifespan may be vital for the maintenance of healthy cardiac regulation via the ANS—or greater cardiac vagal activity as indirectly reflected in HRV may slow brain atrophy. Findings reveal an important association between CT and cardiac parasympathetic activity with implications for healthy aging and longevity that should be studied further in longitudinal research.
|
|
| 24. |
- Lakens, Daniel, et al.
(författare)
-
Justify your alpha
- 2018
-
Ingår i: Nature Human Behaviour. - : Nature Publishing Group. - 2397-3374. ; 2:3, s. 168-171
-
Tidskriftsartikel (refereegranskat)abstract
- In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently report and justify all choices they make when designing a study, including the alpha level.
|
|
| 25. |
|
|
