| 1. |
- Albano, Anthony D., et al.
(författare)
-
Linking With External Covariates : Examining Accuracy by Anchor Type, Test Length, Ability Difference, and Sample Size
- 2019
-
Ingår i: Applied psychological measurement. - : Sage Publications. - 0146-6216 .- 1552-3497. ; 43:8, s. 597-610
-
Tidskriftsartikel (refereegranskat)abstract
- Research has recently demonstrated the use of multiple anchor tests and external covariates to supplement or substitute for common anchor items when linking and equating with nonequivalent groups. This study examines the conditions under which external covariates improve linking and equating accuracy, with internal and external anchor tests of varying lengths and groups of differing abilities. Pseudo forms of a state science test were equated within a resampling study where sample size ranged from 1,000 to 10,000 examinees and anchor tests ranged in length from eight to 20 items, with reading and math scores included as covariates. Frequency estimation linking with an anchor test and external covariate was found to produce the most accurate results under the majority of conditions studied. Practical applications of linking with anchor tests and covariates are discussed.
|
|
| 2. |
- Edvardsson, Bo, 1944-, et al.
(författare)
-
O - factor analysis of mood ratings
- 1981
-
Ingår i: Applied psychological measurement. - : SAGE Publications. - 0146-6216 .- 1552-3497. ; 5:4, s. 459-465
-
Tidskriftsartikel (refereegranskat)abstract
- In O-factor analysis an analysis is made on data from many variables with only one individual who is measured on several occasions. The occasions are then correlated across the variables. This paper demonstrates the usefulness of O-factor analysis in studying how feelings are structured and how situational factors influence feelings. Fours subjects made self-ratings on several occasions using variables they had chosen themselves. Data were O-factor analyzed, and factors were interpreted in terms of feelings common for situations which loaded on the factors. Problems and applicability of the method in psychotherapy research are discussed.
|
|
| 3. |
- González, Jorge, et al.
(författare)
-
A note on the Poisson's binomial distribution in Item Response Theory
- 2016
-
Ingår i: Applied psychological measurement. - : SAGE Publications. - 0146-6216 .- 1552-3497. ; 40:2, s. 302-310
-
Tidskriftsartikel (refereegranskat)abstract
- The Poisson's binomial (PB) is the probability distribution of the number of successes in independent but not necessarily identically distributed binary trials. The independent non-identically distributed case emerges naturally in the field of item response theory, where answers to a set of binary items are conditionally independent given the level of ability, but with different probabilities of success. In many applications, the number of successes represents the score obtained by individuals, and the compound binomial (CB) distribution has been used to obtain score probabilities. It is shown here that the PB and the CB distributions lead to equivalent probabilities. Furthermore, one of the proposed algorithms to calculate the PB probabilities coincides exactly with the well-known Lord and Wingersky (LW) algorithm for CBs. Surprisingly, we could not find any reference in the psychometric literature pointing to this equivalence. In a simulation study, different methods to calculate the PB distribution are compared with the LW algorithm. Providing an exact alternative to the traditional LWapproximation for obtaining score distributions is a contribution to the field.
|
|
| 4. |
- Leôncio, Waldir, et al.
(författare)
-
Evaluating equating transformations in IRT observed-score and Kernel Equating methods
- 2023
-
Ingår i: Applied psychological measurement. - : Sage Publications. - 0146-6216 .- 1552-3497. ; 47:2, s. 123-140
-
Tidskriftsartikel (refereegranskat)abstract
- Test equating is a statistical procedure to ensure that scores from different test forms can be used interchangeably. There are several methodologies available to perform equating, some of which are based on the Classical Test Theory (CTT) framework and others are based on the Item Response Theory (IRT) framework. This article compares equating transformations originated from three different frameworks, namely IRT Observed-Score Equating (IRTOSE), Kernel Equating (KE), and IRT Kernel Equating (IRTKE). The comparisons were made under different data-generating scenarios, which include the development of a novel data-generation procedure that allows the simulation of test data without relying on IRT parameters while still providing control over some test score properties such as distribution skewness and item difficulty. Our results suggest that IRT methods tend to provide better results than KE even when the data are not generated from IRT processes. KE might be able to provide satisfactory results if a proper pre-smoothing solution can be found, while also being much faster than IRT methods. For daily applications, we recommend observing the sensibility of the results to the equating method, minding the importance of good model fit and meeting the assumptions of the framework.
|
|
| 5. |
- Miller, R, et al.
(författare)
-
On the Relation Between the (Censored) Shifted Wald and the Wiener Distribution as Measurement Models for Choice Response Times
- 2018
-
Ingår i: Applied psychological measurement. - : SAGE Publications. - 1552-3497 .- 0146-6216. ; 42:2, s. 116-135
-
Tidskriftsartikel (refereegranskat)abstract
- Inferring processes or constructs from performance data is a major hallmark of cognitive psychometrics. Particularly, diffusion modeling of response times (RTs) from correct and erroneous responses using the Wiener distribution has become a popular measurement tool because it provides a set of psychologically interpretable parameters. However, an important precondition to identify all of these parameters is a sufficient number of RTs from erroneous responses. In the present article, we show by simulation that the parameters of the Wiener distribution can be recovered from tasks yielding very high or even perfect response accuracies using the shifted Wald distribution. Specifically, we argue that error RTs can be modeled as correct RTs that have undergone censoring by using techniques from parametric survival analysis. We illustrate our reasoning by fitting the Wiener and (censored) shifted Wald distribution to RTs from six participants who completed a Go/No-go task. In accordance with our simulations, diffusion modeling using the Wiener and the shifted Wald distribution yielded identical parameter estimates when the number of erroneous responses was predicted to be low. Moreover, the modeling of error RTs as censored correct RTs substantially improved the recovery of these diffusion parameters when premature trial timeout was introduced to increase the number of omission errors. Thus, the censored shifted Wald distribution provides a suitable means for diffusion modeling in situations when the Wiener distribution cannot be fitted without parametric constraints.
|
|
| 6. |
- van der Linden, Wim J., et al.
(författare)
-
Local observed-score equating with anchor-test designs
- 2010
-
Ingår i: Applied psychological measurement. - : SAGE. - 0146-6216 .- 1552-3497. ; 34:8, s. 620-640
-
Tidskriftsartikel (refereegranskat)abstract
- For traditional methods of observed-score equating with anchor-test designs, such as chain and poststratification equating, it is difficult to satisfy the criteria of equity and population invariance. Their equatings are therefore likely to be biased. The bias in these methods was evaluated against a simple local equating method in which the anchor-test score was used as a proxy of the proficiency measured by the test and the equating was conditional on this score. The results showed substantial bias for the two traditional methods under a variety of conditions but much smaller bias for the local method. In addition, unlike the traditional methods, the local method appeared to be quite robust with respect to changes in the difficulty and accuracy of the two tests that were equated. But like these methods, it appeared to be sensitive to a decrease in the accuracy of the anchor test as a proxy of the ability measured by the tests.
|
|
| 7. |
- Wallin, Gabriel, et al.
(författare)
-
How Important is the Choice of Bandwidth in Kernel Equating?
- 2021
-
Ingår i: Applied psychological measurement. - : Sage Publications. - 0146-6216 .- 1552-3497. ; 45:7-8, s. 518-535
-
Tidskriftsartikel (refereegranskat)abstract
- Kernel equating uses kernel smoothing techniques to continuize the discrete score distributions when equating test scores from an assessment test. The degree of smoothness of the continuous approximations is determined by the bandwidth. Four bandwidth selection methods are currently available for kernel equating, but no thorough comparison has been made between these methods. The overall aim is to compare these four methods together with two additional methods based on cross-validation in a simulation study. Both equivalent and non-equivalent group designs are used and the number of test takers, test length, and score distributions are all varied. The results show that sample size and test length are important factors for equating accuracy and precision. However, all bandwidth selection methods perform similarly with regards to the mean squared error and the differences in terms of equated scores are small, suggesting that the choice of bandwidth is not critical. The different bandwidth selection methods are also illustrated using real testing data from a college admissions test. Practical implications of the results from the simulation study and the empirical study are discussed.
|
|
| 8. |
- Wallmark, Joakim, et al.
(författare)
-
Efficiency analysis of item response theory kernel equating for mixed-format tests
- 2023
-
Ingår i: Applied psychological measurement. - : Sage Publications. - 0146-6216 .- 1552-3497. ; 47:7-8, s. 496-512
-
Tidskriftsartikel (refereegranskat)abstract
- This study aims to evaluate the performance of Item Response Theory (IRT) kernel equating in the context of mixed-format tests by comparing it to IRT observed score equating and kernel equating with log-linear presmoothing. Comparisons were made through both simulations and real data applications, under both equivalent groups (EG) and non-equivalent groups with anchor test (NEAT) sampling designs. To prevent bias towards IRT methods, data were simulated with and without the use of IRT models. The results suggest that the difference between IRT kernel equating and IRT observed score equating is minimal, both in terms of the equated scores and their standard errors. The application of IRT models for presmoothing yielded smaller standard error of equating than the log-linear presmoothing approach. When test data were generated using IRT models, IRT-based methods proved less biased than log-linear kernel equating. However, when data were simulated without IRT models, log-linear kernel equating showed less bias. Overall, IRT kernel equating shows great promise when equating mixed-format tests.
|
|
| 9. |
- Wiberg, Marie, 1976-
(författare)
-
Alternative linear item response theory observed-score equating methods
- 2016
-
Ingår i: Applied psychological measurement. - : SAGE Publications. - 0146-6216 .- 1552-3497. ; 40:3, s. 180-199
-
Tidskriftsartikel (refereegranskat)abstract
- Item response theory observed-score equating (IRTOSE) is widely used in many testing programs. The aim of this study was to empirically examine three alternative linear IRTOSE methods compared with the traditional IRTOSE method and to discuss these methods in light of previously suggested alternatives. This contribution is both conceptual, by exploring three alternative methods that fit into the current observed-score equating framework, and empirical by comparing the methods through simulations and with real data. The results show that the local linear (kernel) IRTOSE methods yield low bias and low values on loss measures. However, using only a linear IRTOSE method results in excessive bias and cannot be recommended because of the ease with which IRTOSE with full distributions can be performed. An example using real data showed considerable differences in the equated scores with the alternative methods as well as in comparison with the traditional IRTOSE method. Practical considerations are given in the concluding remarks.
|
|
| 10. |
- Wiberg, Marie, et al.
(författare)
-
Kernel Equating Under the Non-Equivalent Groups With Covariates Design
- 2015
-
Ingår i: Applied psychological measurement. - : SAGE Publications. - 0146-6216 .- 1552-3497. ; 39:5, s. 349-361
-
Tidskriftsartikel (refereegranskat)abstract
- When equating two tests, the traditional approach is to use common test takers and/or common items. Here, the idea is to use variables correlated with the test scores (e.g., school grades and other test scores) as a substitute for common items in a non-equivalent groups with covariates (NEC) design. This is performed in the framework of kernel equating and with an extension of the method developed for post-stratification equating in the non-equivalent groups with anchor test design. Real data from a college admissions test were used to illustrate the use of the design. The equated scores from the NEC design were compared with equated scores from the equivalent group (EG) design, that is, equating with no covariates as well as with equated scores when a constructed anchor test was used. The results indicate that the NEC design can produce lower standard errors compared with an EG design. When covariates were used together with an anchor test, the smallest standard errors were obtained over a large range of test scores. The results obtained, that an EG design equating can be improved by adjusting for differences in test score distributions caused by differences in the distribution of covariates, are useful in practice because not all standardized tests have anchor tests.
|
|
