| 1. |
|
|
| 2. |
- Dalberg, Tobias, 1985-, et al.
(author)
-
Samstämmighet i läsbedömning : Statistisk analys vid bedömning av ett nationellt läsförståelseprov
- 2020
-
Reports (other academic/artistic)abstract
- Rater agreement in reading assessment. Statistical analysis of rating reading comprehension in Swedish national tests. In order to support consistent and commensurate grading, which has important consequences for individual pupils, tests need to demonstrate high levels of inter-reliability. Rater agreement is an important component of the overall test inter-reliability needed when assessing complex tasks such as written or oral assignments. In this report, we analyse rater agreement of constructed response items in reading comprehension tests assigned to students within the framework of the national test in Swedish 1 at upper secondary school. The purpose of the analysis is to examine agreement levels between different teachers’ ratings, and predict expected levels of reliability, given factors such as number of items and raters involved. We observe rater agreement using a range of measures based on different assumptions about whether raters should only be consistent, or whether they should reach consensus. The statistical measures include exact agreement, Cohen’s and Fleiss’ κ statistics, intraclass correlation coefficient, many-facets Rasch measurement and generalizability coefficients. To aid the interpretability of our generalizations, we also provide probability estimates based on Bayesian inference. We find the reliability of ratings of constructed response items to be well within acceptable range, and on par with levels accepted by large-scale international tests such as PISA. Finally, the results indicate that the overall reliability of the test would benefit more from increasing the number of items rather than increasing the number of raters.
|
|
| 3. |
- Dalberg, Tobias, 1985-
(author)
-
Samstämmighet i skrivbedömning : Statistisk analys vid bedömning av två nationella skrivprov
- 2019
-
Reports (other academic/artistic)abstract
- Rater agreement in the assessment of writing. Statistical analysis of the rating of two national tests. This report presents a statistical analysis of assessment in national tests. It analyses rater agreement in the assessment of student texts written within the framework of two different course tests in Swedish at upper secondary school; the national test in Swedish 1 and the national test in Swedish 3. The purpose of the analysis is to discover the level of agreement between different trained raters, given the current design of the test. The paper aims to predict expected levels of reliability, given alternative assessment circumstances such as number of raters involved. Rater agreement is observed using a set of measures based on different assumptions about what might be expected regarding the ability of the raters to either be consistent or reach consensus. The statistical measures include exact agreement, Cohen’s and Fleiss’ κ statistics, intraclass correlation coefficient, many-facets Rasch measurement and generalizability coefficients. The statistical analyses based on the different approaches provide an overall picture of the reliability of ratings of student texts written within the framework of national tests in Swedish 1 and Swedish 3. In conclusion, the results suggest that the ratings meet conventionally acceptable levels of reliability reported in the literature. In addition, the results show that increasing the number of raters, either through calibrating the ratings of multiple individual raters or through social moderation in groups of raters, increases the reliability of the ratings. Although this second conclusion may seem self-evident, studies with a design that is sufficiently statistically sound to merit such conclusions are rare. The present report contributes to such a foundation.
|
|
| 4. |
|
|
| 5. |
- Hagberg-Persson, Barbro, et al.
(author)
-
Elever visar vad de kan : Två studier kring ämnesprovet i svenska som andraspråk för årskurs 3
- 2013
-
Reports (other academic/artistic)abstract
- In this study, more detailed knowledge is gathered in two key areas related to Sweden’s national test for Grade 3 pupils in the subjects of Swedish and Swedish as a second language. The first investigation (see chapter 2) takes as its starting point the fact that pupils who follow the national curriculum for Swedish perform better than pupils who follow the national curriculum for Swedish as a second language. Consequently, an analysis is conducted of the Grade 3 test results for a number of multilingual pupils for the years 2009–2011.The second investigation (see chapter 3) proceeds from the assumption that the tests should contribute to as equivalent an assessment as possible. This study is limited to a calculation of points for the reading texts included in the Grade 3 test for the period 2009–2012.The aim of the first investigation is thus to study in more detail a number of multilingual pupils who followed the national curriculum for Swedish as a second language from 2009 to 2011 and analyse what their Swedish language ability is like, as seen through the Grade 3 test on the subject.The results of this study indicate that the majority of the multilingual pupils meet the standards required for the different parts of the test but that more pupils meet the standards required for the oral parts than for the reading and writing parts. In a more in-depth analysis, it is found that pupils had the greatest difficulty meeting the standards required for the parts that test reading of a factual text, followed by reading of a narrative text.The second investigation is aimed at mapping in what way the reading material included in the Grade 3 test for the period 2009–2012 can be considered to meet the requirement of equivalent assessment over time. A comparison is made between the pupils’ results for the different reading texts and an assignment of points determined in advance for the lowest passing level required for the different texts.The results of this study indicate that the texts investigated meet the requirement for equivalent assessment over time. The adjustment of points assigned to the texts, moving the level a point or two up or down relative to the degree of the text’s difficulty, worked well. Furthermore, the majority of pupils achieve high points on the two reading tests, but the narrative texts are easier for pupils to process than the factual texts.
|
|
| 6. |
- Hagberg-Persson, Barbro, et al.
(author)
-
Ämnesprov i svenska och svenska som andraspråk för årskurs 3 : en utprövningsomgång
- 2010
-
Reports (other academic/artistic)abstract
- Subject test of Swedish and of Swedish as a second language for Grade 3 – an initial trial run. In autumn 2007, the Swedish National Agency for Education commissioned the Department of Scandinavian Languages at Uppsala University to develop and construct the subject test of Swedish and of Swedish as a second language for Grade 3. An initial trial run was to be ready for implementation in spring 2009. This report first documents the work on these trials and what it meant for the future of the test. A large part of the report is then given over to the field study carried out in spring 2009. Following this is an account of teachers’ views and pupil results collected along with some reflections on the prospects for the test over the coming years. The first trial run for the Grade 3 subject test was held March–May, 2009. During this period, five teachers from the university observed the proceedings in the form of a field study. In all, 11 schools with a total of 222 pupils were involved. The results from the field study indicate that the different components of the exam were carried out in a way that worked well in general. Both the field observations and teachers’ views confirm this since experiences were on the whole predominantly positive. However, with respect to exam components A and B, the teachers were concerned about whether they would manage to observe every student’s oral proficiency at the same time, even though the two oral components were carried out separately with each half of the class. The pupils’ reading proficiency was tested in two exam components (C and D). Both the field observations and teachers’ views indicate that the two components worked well in general but that the time spent introducing them and the amount of detail in doing so varied among teachers. Components E/F test the pupils’ ability to produce a written narrative. The results here indicate that the written narrative task worked well in general but that some parts should be given more attention. This primarily concerns the instructions and graphic material for the writing task. Component G tests the pupils’ ability to write a descriptive text. Also apparent here is the importance of the clarity of the teacher’s instructions so that pupils can be provided with the best conditions for successfully completing this writing task.
|
|
| 7. |
- Nordberg, Olle
(author)
-
Att finnas till som läsare – skönlitterär läsning i ett elevperspektiv : Didaktiska tillämpningar av en empirisk studie baserad på elevers egna texter om sin läsning
- 2013
-
Reports (other academic/artistic)abstract
- Throughout their schooling, children and young adults hear from their teachers how important it is to read fiction. Many times, parents and other adults close to these youths share this opinion. School policy documents and the editorial pages of major newspapers also highlight this. But what do young people themselves really say? And where can a student perspective on reading lead the discussion on didactics?This report is based on a student perspective of reading fiction. In this empirical investigation, the results of three tasks carried out in national tests in the 2000s by upper secondary students in their final year of studies are analysed and compared. The students, who wrote essays, reflected in their texts on their reading based on different perspectives. In the analysis, the students’ arguments for and against the reading of fiction were systematically categorised and divided into groups. Their arguments were then related to reception theory and analysed in the report based on four main concepts:identity, interpretive communities, literary repertoire and literary competence. These concepts are integral to the general questions in the report on receptive processes and personal development in connection with reading. The concepts constitute a framework in a concluding section on didactics, with the results of the empirical study transformed into arguments about didactics and practical pedagogy.The investigation shows that the different tasks in the test have served as arenas for different types of readers and reading but that all groups describe transformative, meaning-creating processes. As for the question of identity and personal development, the investigation indicates that belonging to an upper secondary study programme and over time belonging to a class guide a person’s view of reading material and the act of reading in a way that shapes identity, where the question of fiction or non-fiction can be seen as a clear example. In the concluding section on didactics, a number of possible ways of working in the classroom are proposed based on these results, with the emphasis on students having a different approach to reading than the academic approach. The most important conclusion of this investigation is that reading fiction really plays a role and can be made meaningful for all groups of students.
|
|
| 8. |
- Nordberg, Olle, Doktor, 1972-
(author)
-
Berättelser som förändrar : Utvärdering och didaktisk diskussion kring ett nationellt läsprojekt för ungdomar
- 2019
-
Reports (other academic/artistic)abstract
- Olle Nordberg: Berättelser som förändrar. Utvärdering och didaktisk diskussion kring ett nationellt läsprojekt för ungdomar. Svenska i utveckling nr 35. Uppsala universitet, Uppsala.Narratives that change. Evaluation and pedagogical discussion on a national literary reading project for young people. During 2017, a comprehensive national project concentrating on literary reading was performed in secondary and upper secondary schools all over Sweden. Around 30,000 teenagers were involved, reading novels as whole classes with the literary discussion in the centre. This report is an evaluation of that project, made with the participant experiences in focus, and from a literary teaching point of view.The project was initiated and designed by the foundation Läsrörelsen. The aim of the project was to highlight the discussion around the content of books, bookclub-style, not fixating on technical details, grades and tests, biographical details or similar. Accordingly, ten novels with rather “strong” or even poignant content were part of the project. Each school or teacher then made their own choice among the titles available.The rich evaluation material submitted by the participating pupils, teachers, librarians and principals clearly shows that the project really had numerous beneficial effects. This was seen most of all among young readers who expressed the meaning by discussing, disagreeing, engaging, empathising, reflecting and making references to the world around them, all based on the current novel.From a literary pedagogical perspective, the outcome of the project points out several opportunities for teaching literature in schools of today.
|
|
| 9. |
|
|
| 10. |
- Palmér, Anne, 1961-, et al.
(author)
-
En utvecklande tolkningsgemenskap? : Matrisanvändning, kontext och interaktion i bedömningssamtal om ett nationellt prov i muntlig framställning
- 2017
-
Reports (other academic/artistic)abstract
- An enriching interpretative community? presents two studies on the assessment of students’ oral performance in Sweden’s national test in Swedish and Swedish as a second language. One conserns the use of rubrics in assessing while the other consider interaction and context in dialogues between three raters comparing their assessments. Both studies are based on sociocultural understanding of assessment dialogues as a means of strengthening the interpretative community to which the raters belong. The material in both studies consists of audio recorded assessment dialogues between three raters – two test developers and one Swedish teacher whose class has just completed the national oral test. In study 1, the raters’ explanatory statements are identified, categorized and quantified in order to investigate how the rubrics are used as an assessment resource. The explanatory statements range along a continuum, from those close to the descriptors to those demonstrating a more independent and flexible use of the rubrics. This latter is interpreted as so-called professional use of rubrics whereby the raters’ observations are compared to the descriptors in the rubrics in view of their teaching experience and rhetorical knowledge.Study 2 analyses context and interaction in the assessment dialogues based on Linell’s (2011) theory of communicative activity types. The analysis shows that these assessment dialogues have a great deal in common, in some but not in all respects, with assessment dialogues in general. The test developers are primarily interested in validating the rubrics and thus, in what way the teachers use them. The teachers, in turn, aim to reach agreement in their assessment. One effect of this is that both the agenda and the participant’s roles as experts and laypeople shift during the dialogues. Often, the test developers refrain from changing the teacher’s agenda, and all three of them start to use different strategies to reach consensus. Thus, agreement is a more common feature in the dialogues than disagreement, which is also reflected in the more detailed analysis of the interaction.
|
|
| 11. |
- Palmér, Anne, 1961-
(author)
-
Nationella skrivprov baserade på två olika läroplaner : Genre, kommunikationssituation och skrivdidaktiska diskurser
- 2013
-
Reports (other academic/artistic)abstract
- National tests in writing based on two national curricula: Genre, the communicative situation and discourses of writing is a study aimed at formulating a theoretical context for national tests in writing. The tests under discussion are given in Swedish upper secondary school in the subjects of Swedish as a mother tongue and Swedish as a second language. A discourse analysis is conducted of the 1994 and 2011 national curricula and of the tests in writing associated with them. Inspired by Roz Ivanič (2004), the analysis examines three discourses of writing and learning to write – a genre discourse, a process discourse and a social practice discourse. It also considers two different notions of genre – a rhetorical notion of genre (Miller 1984) and a more linguistic notion from the Sydney school in Australia (Martin & Rose 2008).The results indicate that neither the curricula nor the tests can be readily associated with just one discourse of writing and learning to write. One conclusion is that the curricula and the tests are all part of hybrid discourses since there are references to all three discourses in the data investigated. Minor theoretical changes between the curricula and tests from 1994 to 2011 are noted; there is an opening to a genre discourse in the 2011 curricula and tests. Another conclusion is that theoretical assumptions about writing, like the discourses considered, influence both curricula and national tests and that the tests follow the assumptions of the curricula. The study also contains a critical discussion of Ivanič’s conceptual framework.
|
|
| 12. |
|
|
| 13. |
- Palmér, Anne, 1961-, et al.
(author)
-
Skrivutveckling och textkvalitet vid övergången från mellan- till högstadiet. : En undersökning av elevtexter från nationella prov
- 2023
-
Reports (other academic/artistic)abstract
- Argumentative writing is of considerable importance in writing instruction at Swedish schools and for the development of writing in young people. This investigation of writing development focusses on writing argumentative texts compared to writing narrative texts. The data consists of a corpus of 240 student texts. The text analysis starts with a descriptive, statistical analysis of language features and word class distribution (N=240), followed by a word frequency analysis of the argumentative corpus (N=120) and a qualitative, lexicogrammatical analysis of a sample of argumentative texts (N=9). Additionally, a tentative analysis is conducted on the predictability of the nominal ratio of student texts from year 6 according to a model of the verbal–nominal dimension (Hellspong & Ledin 1997).The statistical results indicate that argumentative genre writing may pose a great challenge to students of this age, although the corpus also contains some signs of adaptation to this genre. According to word class distribution, the style of the argumentative texts are described as verbal and close to the spoken language, except for some texts that seem to be more of a more nominal style. Further, the word classes of adverbs and participles seem to be important indicators of text quality of argumentative writing by students of this age, while lots of adjectives are associated with lower text quality, presumably because the adjectives frequently used in the corpus are common vocabulary words in Swedish. Moreover, the lexicogrammatical analysis shows how texts with different levels of the nominal ratio vary according to word class distribution, the use of nominal phrases, and syntax. Finally, the analysis of the predictability of the nominal ratio of these texts indicates that the nominal ratio can predict word class distribution and length of the nominal phrases, but not the syntax used.We conclude that most, but not all, of the student texts may be seen as the expression of a basic, and critical, phase of language development in which they master the written language using a verbal communication style. In this phase adverbs are used to express the writer’s attitude as well as to create text cohesion, and subordination as well as coordination are used for developing the syntax. Some of the student texts may be seen as an expression of a second phase of writing development characterised by the use of a more nominal, information-dense style, where signs of text quality include a rhetoric use of participles and simple sentences with longer phrases.
|
|
| 14. |
- Schüssler, Eija L.
(author)
-
Visuell kommunikation - modell för bedömning av gymnasieelevers bruk av bildspel vid muntliga anföranden
- 2014
-
Reports (other academic/artistic)abstract
- Visual communication – a model for assessing upper secondary student use of slideshows in speeches. In Sweden’s upper secondary school curriculum (Gy11) for Swedish as a school subject, the use of presentation techniques in oral presentations is a requirement. This study uses the visual grammar formulated by Kress & van Leeuwen (2006) to examine the slideshows used during an oral presentation in class. The aim of the study is to determine whether and how students use visual communication and multimodal competence in their work. The curriculum’s grading criteria for presentation techniques are tested on eight presentations. The results indicate that the criteria are not sufficient to identify multimodal competence.
|
|
| 15. |
|
|
| 16. |
- Östlund-Stjärnegårdh, Eva
(author)
-
Att förmedla egna och andras tankar : Om gymnasisters källhantering i det nationella provets skrivuppgift
- 2006
-
Reports (other academic/artistic)abstract
- The material for this report consists of 55 student texts written in the national test of Swedish during the last compulsory course in the upper secondary school (18–19 year-olds). The task was to argue for more literature and reading for children in order to develop the children’s language, reasoning skills and fantasy. The student texts are to be seen as contributions to the debate in a local newspaper. Consideration is also given to the test material, that is texts both fiction and non-fiction, and the written instruction. The aim of the study is to investigate how the students handle the demand to use ideas and thoughts from the texts they have read before the actual writing test. Teachers sometimes find this part of the task hard to assess and ask for strict guidelines in the assessment material. The texts show that the students generally know that they are supposed to use facts and thoughts from the texts, and most of them also know that they are supposed to give references to the texts they quote from or refer to. Chapter 3 lists different ways of referring, from the elegant or neat to very circumstantial references or no reference at all. Using some part of a text without a reference is called “theft” for pedagogical reasons. A frequent pattern is to give a reference the first time a text is used but not the second or third time. Chapter 4 goes deeper into what the students choose to refer to or quote, that is how the content of the texts is made use of. The majority of the students get facts from two of seveal possible sources and build their argumentation on those. Only one shows a critical approach towards the texts in question. The student texts are also compared with published contributions to the debate in a local newspaper. First the question of voice in the two text materials is studied. The students speak more as an “I” and the published debaters as a “we” Then references in the newspaper texts are compared to those in the school texts. The recommendation to teachers is to keep on working with references but not at an academic level. The ideal for the students is to reach a control of different references for different text types and genres. Most important it is a question of honesty, not to steal but to announce loans from other sources.
|
|
| 17. |
|
|
| 18. |
|
|
