| 1. |
|
|
| 2. |
- Dalberg, Tobias, 1985-, et al.
(författare)
-
Samstämmighet i läsbedömning : Statistisk analys vid bedömning av ett nationellt läsförståelseprov
- 2020
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Rater agreement in reading assessment. Statistical analysis of rating reading comprehension in Swedish national tests. In order to support consistent and commensurate grading, which has important consequences for individual pupils, tests need to demonstrate high levels of inter-reliability. Rater agreement is an important component of the overall test inter-reliability needed when assessing complex tasks such as written or oral assignments. In this report, we analyse rater agreement of constructed response items in reading comprehension tests assigned to students within the framework of the national test in Swedish 1 at upper secondary school. The purpose of the analysis is to examine agreement levels between different teachers’ ratings, and predict expected levels of reliability, given factors such as number of items and raters involved. We observe rater agreement using a range of measures based on different assumptions about whether raters should only be consistent, or whether they should reach consensus. The statistical measures include exact agreement, Cohen’s and Fleiss’ κ statistics, intraclass correlation coefficient, many-facets Rasch measurement and generalizability coefficients. To aid the interpretability of our generalizations, we also provide probability estimates based on Bayesian inference. We find the reliability of ratings of constructed response items to be well within acceptable range, and on par with levels accepted by large-scale international tests such as PISA. Finally, the results indicate that the overall reliability of the test would benefit more from increasing the number of items rather than increasing the number of raters.
|
|
| 3. |
- Dalberg, Tobias, 1985-
(författare)
-
Samstämmighet i skrivbedömning : Statistisk analys vid bedömning av två nationella skrivprov
- 2019
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Rater agreement in the assessment of writing. Statistical analysis of the rating of two national tests. This report presents a statistical analysis of assessment in national tests. It analyses rater agreement in the assessment of student texts written within the framework of two different course tests in Swedish at upper secondary school; the national test in Swedish 1 and the national test in Swedish 3. The purpose of the analysis is to discover the level of agreement between different trained raters, given the current design of the test. The paper aims to predict expected levels of reliability, given alternative assessment circumstances such as number of raters involved. Rater agreement is observed using a set of measures based on different assumptions about what might be expected regarding the ability of the raters to either be consistent or reach consensus. The statistical measures include exact agreement, Cohen’s and Fleiss’ κ statistics, intraclass correlation coefficient, many-facets Rasch measurement and generalizability coefficients. The statistical analyses based on the different approaches provide an overall picture of the reliability of ratings of student texts written within the framework of national tests in Swedish 1 and Swedish 3. In conclusion, the results suggest that the ratings meet conventionally acceptable levels of reliability reported in the literature. In addition, the results show that increasing the number of raters, either through calibrating the ratings of multiple individual raters or through social moderation in groups of raters, increases the reliability of the ratings. Although this second conclusion may seem self-evident, studies with a design that is sufficiently statistically sound to merit such conclusions are rare. The present report contributes to such a foundation.
|
|
| 4. |
|
|
| 5. |
- Hagberg-Persson, Barbro, et al.
(författare)
-
Elever visar vad de kan : Två studier kring ämnesprovet i svenska som andraspråk för årskurs 3
- 2013
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- In this study, more detailed knowledge is gathered in two key areas related to Sweden’s national test for Grade 3 pupils in the subjects of Swedish and Swedish as a second language. The first investigation (see chapter 2) takes as its starting point the fact that pupils who follow the national curriculum for Swedish perform better than pupils who follow the national curriculum for Swedish as a second language. Consequently, an analysis is conducted of the Grade 3 test results for a number of multilingual pupils for the years 2009–2011.The second investigation (see chapter 3) proceeds from the assumption that the tests should contribute to as equivalent an assessment as possible. This study is limited to a calculation of points for the reading texts included in the Grade 3 test for the period 2009–2012.The aim of the first investigation is thus to study in more detail a number of multilingual pupils who followed the national curriculum for Swedish as a second language from 2009 to 2011 and analyse what their Swedish language ability is like, as seen through the Grade 3 test on the subject.The results of this study indicate that the majority of the multilingual pupils meet the standards required for the different parts of the test but that more pupils meet the standards required for the oral parts than for the reading and writing parts. In a more in-depth analysis, it is found that pupils had the greatest difficulty meeting the standards required for the parts that test reading of a factual text, followed by reading of a narrative text.The second investigation is aimed at mapping in what way the reading material included in the Grade 3 test for the period 2009–2012 can be considered to meet the requirement of equivalent assessment over time. A comparison is made between the pupils’ results for the different reading texts and an assignment of points determined in advance for the lowest passing level required for the different texts.The results of this study indicate that the texts investigated meet the requirement for equivalent assessment over time. The adjustment of points assigned to the texts, moving the level a point or two up or down relative to the degree of the text’s difficulty, worked well. Furthermore, the majority of pupils achieve high points on the two reading tests, but the narrative texts are easier for pupils to process than the factual texts.
|
|
| 6. |
- Hagberg-Persson, Barbro, et al.
(författare)
-
Ämnesprov i svenska och svenska som andraspråk för årskurs 3 : en utprövningsomgång
- 2010
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Subject test of Swedish and of Swedish as a second language for Grade 3 – an initial trial run. In autumn 2007, the Swedish National Agency for Education commissioned the Department of Scandinavian Languages at Uppsala University to develop and construct the subject test of Swedish and of Swedish as a second language for Grade 3. An initial trial run was to be ready for implementation in spring 2009. This report first documents the work on these trials and what it meant for the future of the test. A large part of the report is then given over to the field study carried out in spring 2009. Following this is an account of teachers’ views and pupil results collected along with some reflections on the prospects for the test over the coming years. The first trial run for the Grade 3 subject test was held March–May, 2009. During this period, five teachers from the university observed the proceedings in the form of a field study. In all, 11 schools with a total of 222 pupils were involved. The results from the field study indicate that the different components of the exam were carried out in a way that worked well in general. Both the field observations and teachers’ views confirm this since experiences were on the whole predominantly positive. However, with respect to exam components A and B, the teachers were concerned about whether they would manage to observe every student’s oral proficiency at the same time, even though the two oral components were carried out separately with each half of the class. The pupils’ reading proficiency was tested in two exam components (C and D). Both the field observations and teachers’ views indicate that the two components worked well in general but that the time spent introducing them and the amount of detail in doing so varied among teachers. Components E/F test the pupils’ ability to produce a written narrative. The results here indicate that the written narrative task worked well in general but that some parts should be given more attention. This primarily concerns the instructions and graphic material for the writing task. Component G tests the pupils’ ability to write a descriptive text. Also apparent here is the importance of the clarity of the teacher’s instructions so that pupils can be provided with the best conditions for successfully completing this writing task.
|
|
| 7. |
- Nordberg, Olle
(författare)
-
Att finnas till som läsare – skönlitterär läsning i ett elevperspektiv : Didaktiska tillämpningar av en empirisk studie baserad på elevers egna texter om sin läsning
- 2013
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Throughout their schooling, children and young adults hear from their teachers how important it is to read fiction. Many times, parents and other adults close to these youths share this opinion. School policy documents and the editorial pages of major newspapers also highlight this. But what do young people themselves really say? And where can a student perspective on reading lead the discussion on didactics?This report is based on a student perspective of reading fiction. In this empirical investigation, the results of three tasks carried out in national tests in the 2000s by upper secondary students in their final year of studies are analysed and compared. The students, who wrote essays, reflected in their texts on their reading based on different perspectives. In the analysis, the students’ arguments for and against the reading of fiction were systematically categorised and divided into groups. Their arguments were then related to reception theory and analysed in the report based on four main concepts:identity, interpretive communities, literary repertoire and literary competence. These concepts are integral to the general questions in the report on receptive processes and personal development in connection with reading. The concepts constitute a framework in a concluding section on didactics, with the results of the empirical study transformed into arguments about didactics and practical pedagogy.The investigation shows that the different tasks in the test have served as arenas for different types of readers and reading but that all groups describe transformative, meaning-creating processes. As for the question of identity and personal development, the investigation indicates that belonging to an upper secondary study programme and over time belonging to a class guide a person’s view of reading material and the act of reading in a way that shapes identity, where the question of fiction or non-fiction can be seen as a clear example. In the concluding section on didactics, a number of possible ways of working in the classroom are proposed based on these results, with the emphasis on students having a different approach to reading than the academic approach. The most important conclusion of this investigation is that reading fiction really plays a role and can be made meaningful for all groups of students.
|
|
| 8. |
- Nordberg, Olle, Doktor, 1972-
(författare)
-
Berättelser som förändrar : Utvärdering och didaktisk diskussion kring ett nationellt läsprojekt för ungdomar
- 2019
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- Olle Nordberg: Berättelser som förändrar. Utvärdering och didaktisk diskussion kring ett nationellt läsprojekt för ungdomar. Svenska i utveckling nr 35. Uppsala universitet, Uppsala.Narratives that change. Evaluation and pedagogical discussion on a national literary reading project for young people. During 2017, a comprehensive national project concentrating on literary reading was performed in secondary and upper secondary schools all over Sweden. Around 30,000 teenagers were involved, reading novels as whole classes with the literary discussion in the centre. This report is an evaluation of that project, made with the participant experiences in focus, and from a literary teaching point of view.The project was initiated and designed by the foundation Läsrörelsen. The aim of the project was to highlight the discussion around the content of books, bookclub-style, not fixating on technical details, grades and tests, biographical details or similar. Accordingly, ten novels with rather “strong” or even poignant content were part of the project. Each school or teacher then made their own choice among the titles available.The rich evaluation material submitted by the participating pupils, teachers, librarians and principals clearly shows that the project really had numerous beneficial effects. This was seen most of all among young readers who expressed the meaning by discussing, disagreeing, engaging, empathising, reflecting and making references to the world around them, all based on the current novel.From a literary pedagogical perspective, the outcome of the project points out several opportunities for teaching literature in schools of today.
|
|
| 9. |
|
|
| 10. |
- Palmér, Anne, 1961-, et al.
(författare)
-
En utvecklande tolkningsgemenskap? : Matrisanvändning, kontext och interaktion i bedömningssamtal om ett nationellt prov i muntlig framställning
- 2017
-
Rapport (övrigt vetenskapligt/konstnärligt)abstract
- An enriching interpretative community? presents two studies on the assessment of students’ oral performance in Sweden’s national test in Swedish and Swedish as a second language. One conserns the use of rubrics in assessing while the other consider interaction and context in dialogues between three raters comparing their assessments. Both studies are based on sociocultural understanding of assessment dialogues as a means of strengthening the interpretative community to which the raters belong. The material in both studies consists of audio recorded assessment dialogues between three raters – two test developers and one Swedish teacher whose class has just completed the national oral test. In study 1, the raters’ explanatory statements are identified, categorized and quantified in order to investigate how the rubrics are used as an assessment resource. The explanatory statements range along a continuum, from those close to the descriptors to those demonstrating a more independent and flexible use of the rubrics. This latter is interpreted as so-called professional use of rubrics whereby the raters’ observations are compared to the descriptors in the rubrics in view of their teaching experience and rhetorical knowledge.Study 2 analyses context and interaction in the assessment dialogues based on Linell’s (2011) theory of communicative activity types. The analysis shows that these assessment dialogues have a great deal in common, in some but not in all respects, with assessment dialogues in general. The test developers are primarily interested in validating the rubrics and thus, in what way the teachers use them. The teachers, in turn, aim to reach agreement in their assessment. One effect of this is that both the agenda and the participant’s roles as experts and laypeople shift during the dialogues. Often, the test developers refrain from changing the teacher’s agenda, and all three of them start to use different strategies to reach consensus. Thus, agreement is a more common feature in the dialogues than disagreement, which is also reflected in the more detailed analysis of the interaction.
|
|
