| کلیدواژههای انگلیسی مقاله |
Language tests, Narration, Autism spectrum disorder, Psychometrics, What&,rsquo s Known Assessment of narrative discourse in children with language impairments is essential. Several instruments are available to assess different types of narratives, including retelling and self-generated stories. However, none of these tools are suitable for the consecutive assessment of narrative skills. What&,rsquo s New For the first time, a new test is developed for consecutive assessment and monitoring of narrative skills in school-age children. The validity and reliability of the test were confirmed. Discriminant validity of the test was confirmed following testing with typically developing and autistic children. IntroductionMost parents and teachers inspire preschool children to talk about events that occurred during the day or in a distant past. 1, Narration is an important aspect of language and defined as stories about real or imaginary events with a causal relationship or which are in temporal order. 2, Narrative discourse plays an important role in children&,rsquo s socio-cultural and language development and is considered a predictor of literacy achievement, particularly reading comprehension. 3,, 4, It is therefore essential that children understand and effectively produce appropriate narrative forms. 5, There are three types of narrative assessment methods, namely story comprehension, story retelling, and personal narrative. Story comprehension involves reading a story to a child and asking questions about the story. In story retelling, an examiner reads a model story to a child, and the child should retell the story. Finally, personal narrative requires children to formulate stories from pictures or telling their personal story that is thematically related to a model story. Retelling and personal narrative provide different insights about children&,rsquo s storytelling skills. 6, Story grammar model is a common way for macro-structural analysis of narrative. According to this model, each story has a setting and an episode system. 7, The setting provides information about the time or the place where the story had occurred. The episode system comprises three elements. The first element is a problem or an initiating event and/or internal response, which is an occurrence that motivates actions by the main character(s). The second, goal-directed actions for solving the problem known as attempts and the third element is the consequence (or outcome) associated with both the initiating event and the actions. 8, Another area for the development of narratives is literate language and cohesive devices. Literate language is an essential area of narrative microstructure. 9, Narrative competency is using elaborated noun phrases, metalinguistic verbs such as say or tell, and metacognitive verbs namely think or believe. They are main literate language components. 10, Narrative skills are diminished in children with specific language impairments and those with language/learning disabilities. 11, In order to discriminate between children with typical development (TD) and children with language disorders, macrostructural analysis has been used. 12, There are several instruments for assessing narrative performance. These instruments are divided into two groups, namely norm-referenced and criterion-referenced. Norm-referenced tests are designed to compare one child with another, based on which a therapist can rapidly and precisely compare children&,rsquo s narratives to the peers. Criterion-referenced tests are used to measure the knowledge and skills of a child against a set of predetermined criteria. 13, Two standardized norm-referenced instruments including the Renfrew bus story and the test of narrative language are available for assessment of narratives. 2, The narrative &,ldquo Frog, where are you?&,rdquo 14, and the Persian narrative norms instrument 15, are the commonly known criterion-referenced methods in Iran for analyzing language content, form, and use in children. In these methods, a child is asked to tell a story based on a wordless picture book from which a language sample is analyzed. 16, Checklists, rating scales, or rubrics are mainly applied as tools to analyze the results of criterion-referenced tests, such as monitoring indicators of scholarly language (MISL) 17, and narrative scoring scheme (NSS). 5, The present study aimed to develop a valid and reliable criterion-referenced test. Compared to other narrative assessment tests, this newly developed test is specifically designed for consecutive assessment of spontaneous narratives in young school-age children in a short period of time. It is known that when clinicians or researchers use the same test repeatedly for progress monitoring, the results may not be reliable due to familiarization. The uniqueness of our test is that it consists of several independent tests for progress-monitoring, all of which assess the same item and the corresponding scores are comparable with one another. Furthermore, criterion-referenced tests facilitate intervention planning, since they indicate the weaknesses and strengths of a specific area in detail.Materials and MethodsParticipantsTwo groups of children were recruited in the study, including typically developing children (TD) as the control group (n=50) and children with autism spectrum disorder (ASD) as the group with language impairments (n=7). The TD children were recruited from public schools in Kerman, Iran, 2018. Based on a general guideline, a sample size of at least 50 participants was considered adequate for the assessment of the psychometric parameters. 18, The inclusion criteria for the TD children were average achievement scores in core subjects at school and no history of hearing, visual, or gross neurological impairment, as well as psychiatric diagnosis, or learning disability. Moreover, preschool screening tests also confirmed their normal intelligence level. The parents of these children were approached to inquire about their native language and the history of language development. Subsequently, the inclusion criteria were extended to include native Persian-speaking children and normal language development. The speech-language pathology (SLP) test was performed to confirm normal oral structure with no signs of stuttering, speech errors, language impairment, or attention deficit disorder in each child in the TD group. As part of the selection process, the teachers at those public schools were informed about the study protocol and requested to identify qualified children.The ASD children were recruited from private clinics in Kerman, Iran, 2018. Their disorders were confirmed by a qualified child psychiatrist using the DSM-V criteria 19, and the Persian Gilliam autism rating scale, second edition (GARS-2). 20, In the selection process, only boys were considered, since the prevalence of ASD is much higher in boys than in girls, 21, and we could only approach boys. The children in this group were monolingual (Persian) speakers with a nonverbal IQ of &,ge 85 (89.39&,plusmn 3.15) on the Wechsler preschool and primary scale of intelligence (WPPSI). 22, Due to the high prevalence of language impairments in children with ASD, we used the free-play method to collect language samples from each child. The inclusion criteria were the ability to produce 100 utterances and a mean length of utterances (MLU) of four. The choice for the 100 utterances was to ensure that a low score on the test was not due to their pragmatic disorder. Note that MLU is a useful criterion for indicating syntactic complexity. 23, As a result of the above, the age range of the children in the ASD group was wider that was the TD group. Besides verbal consent from each child, written informed consent was obtained from the children&,rsquo s parents. The study protocol was approved by the Ethics Committee of Tehran University of Medical Sciences, Tehran, Iran (REC.FNM.TUMS.IR.137.1397). Pictures and StoriesTo elicit story retelling by the children, 14 single-scene pictures (figure 1,) as the main test and a single image (figure 2,) as an exercise test were developed. The pictures depicted human characters based on which 15 stories were scripted in accordance with the Stein and Glenn model. 7, Each of the 15 stories constituted a discrete test. Since familiarization is crucial in populations such as children, an exercise session was included in the study. All stories were of a single-episode with two or three characters. Each story contained a clear initiating event (e.g., a child getting injured, tearing off clothes, painting on a wall) to set a plot in motion as well as the associated goal-directed actions such as wound dressing and mended torn clothes.Figure 1. A sample of the 14 single-scene pictures that was illustrated.Figure 2. The figure shows one of the pictures used in the exercise session.A panel of narrative experts was asked to comment on the developed stories in terms of structure and relevance to children. Based on their feedback, the stories were modified and subsequently, a professional cartoonist drew the final version of the pictures in accordance with the scripts. The pictures depicted characters, setting (park, house, or doctor&,rsquo s office), initial event, and a problem-solving clue. The face of the human characters was designed to depict an external event in emotions. Except for the main components of the stories, pictures did not show any confusion or extra stimulants. All pictures were colorful and of the same size (A5). The narrative experts reviewed the final version of the pictures and confirmed their appropriateness and representativeness. Data CollectionData were collected by three student research assistants in speech-language pathology. The first author fully informed the children as well as their parents about the goal of the study, test method, and the data collection procedure. The children were assessed individually in a quiet room in their own school. The tests were performed in two sessions on two consecutive days, each covering seven pictures (i.e., tests) and lasting approximately 30 minutes. The pictures were randomly presented to the children to reduce the negative impact of fatigue. At first, an exercise session was carried out using a dedicated picture and its associated story. The picture depicted two main characters (a little boy and his mother) in a room with a window through which a snowy landscape could be seen. The boy seemed to have a cold and coughed, and his mother looked sad and offered him cough medicine. The research assistant presented the picture to each participating child and only gave an overall description (the presence of two individuals and an event). Each child was requested to carefully look at the picture while listening to the pre-defined story (based on the Stein and Glen model). Children were asked to retell the story afterward without seeing the picture to ensure that they would narrate the story as literally as possible and in an understandable fashion. Every effort was made to avoid any association between the exercise session and the main sessions (the 14 single-scene pictures). The main sessions were similarly conducted as the exercise session. To examine test-retest reliability, the 2-day sessions were repeated after one month. With prior permission from the children&,rsquo s parents, recordings of the stories retold by the children were made using the Recordium IOS application software for iPhone. To ensure the correct conduct of the tests, the first author randomly reviewed 20% of all audio recordings using a pre-defined verification checklist. The recorded data were transcribed verbatim by the research assistants. The transcribed utterances were divided into communication units. Each communication unit included one main clause with all dependent phrase or clauses attached to it. 17, All irrelevant and unintelligible utterances were noted but excluded from the analysis. To ensure correct transcription of the recorded data, a PhD student in speech-language pathology (blinded to the goal of the study) transcribed 10% of the recorded data. The agreement between transcriptions was 98% for C-unit segmentation and 97% for word-by-word agreement (agreement divided by agreement plus disagreement). The MISL, with its macrostructure and microstructure subscales, was used to analyze the transcribed stories. The macrostructure subscale includes seven story elements (character, setting, initiating event, internal response, plan, action, and consequence). The microstructure subscale contains six items (coordinating conjunction, subordinating conjunction, adverbs, metacognitive verbs, metalinguistic verbs, and elaborated noun phrases). The sum of macrostructure and microstructure scores (total MISL score) represents an index of overall narrative complexity (NC). 17, For each macrostructure subscale, a story element was rated zero when the story did not contain that element. A score of 1 demonstrated that the story included an element that was not directly related to the story. A score of 2 indicated that an element was correctly related to the story, and a score of 3 showed that there were two or more of those elements. 17, The scoring system for the microstructure sub-scale was according to the rating of the items. A score of 0 was given if there were not any items in the story, a score of 1 was given when there was one example of the item in the story, a score of 2 was given when there were two various examples, and a score of 3 was given if three or more various examples were present. To facilitate the scoring, a table was prepared by the first author with the support of a Farsi linguist for the definition of story elements and examples of each element. 17, Inter-rater ReliabilityA PhD student in speech-language pathology (the coder) blind to the goal of the study was trained in the application of the MISL. During a preliminary training, five randomly selected stories were scored by the coder and the results were cross-checked with those of the first author. The scoring method was discussed and potential ambiguities were resolved. Then, the coder and the first author independently scored five additional stories (20% of the total transcripts) and the extent of agreement between the two sets of results was calculated. Since a simple inter-rater agreement does not include the effect of chance, Cohen&,rsquo s kappa coefficient was used to calculate agreement on the NC of each story. The coefficient indicates the agreement between individual scoring categories and the frequency of different categories among raters other than those expected by chance. 24, Statistical AnalysisStatistical analysis was performed using SPSS software version 18.0. Spearman&,rsquo s rank-order correlation coefficient was used to measure construct validity. Validity was also determined by displaying differences between distinct groups that were predicted empirically. 25, Previous studies have shown that children with ASD have problems with comprehending and producing narratives. 12,- 26, Therefore, we specifically recruited these children in the present study. Due to the small sample size in the ASD group, the nonparametric Mann-Whitney U test was used to compare the mean NC between the TD and ASD groups. The internal consistency and test-retest reliability were assessed using Cronbach&,rsquo s alpha and intraclass correlation coefficient (ICC), respectively. Repeated measures ANOVA was used to compare the tests.ResultsThe TD group included 50 children (25 boys, 25 girls) with a mean age of 7.7&,plusmn 0.66 years (range 6.6-8.4 years). The ASD group included seven boys with a mean age of 8.8&,plusmn 0.53 years (range, 8.2-9.5 years). The bivariate correlation between the total MISL score of each test and each MISL item confirmed the construct validity of the test. The corresponding Spearman&,rsquo s correlation coefficients are presented in table 1,. The results showed a high correlation between the total MISL score of each test and each MISL item (r&,gt 0.5). 27, Moreover, the correlation coefficient between the macrostructure and microstructure subscales was between r=69 and r=86 for all stories. In terms of discriminant validity, we anticipated a difference in the total MISL score between the ASD and TD groups. Based on the Mann-Whitney U test, there was a significant difference in the mean NC score of all stories between the two groups (3.95&,lt Z&,lt 4, P&,lt 0.001). The inter-rater reliability for each story was examined using the kappa coefficient (table 2,). The coefficients were significant at P&,lt 0.001, indicating an almost perfect inter-rater agreement. 24, We examined reliability via internal consistency reliability. Cronbach&,rsquo s alpha coefficients of the NC scores and the two MISL subscales are presented in table 3,. In accordance with a previous study, a reliability coefficient of &,ge 0.70 was considered acceptable. 27, The internal consistency reliability of the macrostructure subscale was greater than 0.70 in 86% of the pictures, but it was slightly lower for the microstructure subscale of most of the pictures (a&,gt 0.60). The ICC (P&,lt 0.05) was used to measure test-retest reliability. 28, The ICC values for each story are presented in table 4,. According to a subjective guideline provided by Landis and Koch, 29, an ICC value between 0.61 and 0.80 indicates substantial agreement and values between 0.81 and 1.00 indicates almost perfect agreement. Finally, we examined whether all pictures in our instrument assessed the same item, and if their corresponding scores were comparable. The results of the repeated measures ANOVA (within-subject) showed no significant difference between the 14 tests (F=1.87, P=0.15), and the observed power was 0.741. T1T2T3T4T5T6T7T8T9T10T11T12T13T14Macrostructures |