The Development of a Screening Schedule for Use by Teachers to Describe the Communication Abilities of Children with Severe Disabilities

The study developed a screening schedule for teachers to describe the communication abilities of children with severe disabilities Teachers were trained in the use of such an assessment procedure. Both the schedule as well as the training were evaluated. Results indicated that IKe schedule can be regarded as adequate for future use as teachers found it easy to use and applicable to their situation. Scoring the schedule requires more practise and training as this is a more advanced skill. Specific issues had an effect on the results, e.g., the teachers'familiarity with the type of disability observed, their qualifications and experience.


INTRODUCTION
The increasing need for service delivery to all children with severe disabilities (CSD) necessitates the use of teachers to facilitate the children's general interaction.Teachers are ideally suited to assess their childrens' capabilities as they are familiar with their individual needs and skills.In this context, the transdisciplinary method of intervention whereby the teacher is trained to fulfil certain intervention functions in the classroom can be a most effective way of dealing with the'severely disabled population (Hogg & Raynes, 1978).
As assessment can be seen as the beginning of the intervention process, teachers should also be trained to perform this task.Researchers such as Skuy, Westaway, Makula and Perold (1988) have found teachers valuable, parsimonious and accurate in rating students' performances in psychometric testing.The use of teachers in the communication assessment of CSD becomes especially relevant as they are able to functionally assess students during daily class routines.A functional assessment can thus be described as an assessment method aimed at describing the child's performance in real life contexts.This approach to assessment is of the utmost importance when working with the majority of children with severe disabilities as they experience difficulty in learning new tasks due to the severity of their cognitive and/or physical dis-abilities.It is therefore important that the tasks used in the assessment process should be relevant and applicable to daily routines to facilitate the children's increasing independence.
Which evaluative procedure should be used if teachers are involved in the assessment process?A very limited choice of non-verbal tests are available, e.g., the Nonspeech Test (Huer, 1983), and are for exclusive use by speech and language therapists.Many speech and language therapists who work with CSD make use of tests which were developed to test language in depth.The problem is that many of the children with severe disabilities are unable to speak which makes it very difficult to obtain reliable results.The need for an AAC assessment schedule to describe the skill areas of functioning relevant to AAC which could be used by teachers of CSD is evident.However, various issues must be taken into account in the development of such an assessment schedule.
Firstly, the level of impairment in CSD often necessitates the use of an augmentative system for more effective communication.The variation in symptoms of the child with severe disabilities clearly indicates that more than one skill area can be involved.It is therefore important to assess all skill areas involved in augmentative communication, such as communication, cognitive, motor, sensory and social/emotional skills.Successful intervention should not take only abilities into account, but should also focus Die Suid-Afrikaanse Tydskrif vir Kommunikasieafwykings, Vol. 43, 1996 Reproduced by Sabinet Gateway under licence granted by the Publisher (dated 2012) on needs (Beukelman & Mirenda, 1992) Secondly the heterogeneity of this population means that every^ child has unique problems, abilities and needs.A schedule has to address the development of an individual programme.Thirdly, the characteristics of the child with severe disabilities such as his inconsistent behaviour, passivity, short attention span, anti-social behaviour (Durant, 1990) and motor problems (Bergen, 1990), make it necessary to observe such behaviour during ordinary functional routines as was recently proposed by ASHA (1992).Researchers such as Light (1989) as well as Mirenda and Iocono (1990), regard functional communication as an integral part of all areas of development particularly in the areas of social, emotional and cognitive development which must therefore also be accommodated in the assessment process of CSD.Fourthly, for a schedule to be effective for use in the classroom, it needs to be easy to administer and time effective.
The development of such a screening schedule for teachers of CSD has various advantages in that its use would enable teachers to describe their students' problems, it would help them to know when to refer to specialised services if necessary, and lastly, they would be able to draw up an individualised program plan for each child which enables them to monitor progress.It is against this background that the present study was conducted to investigate the use of an AAC screening schedule for use by teachers.

Anna-Marie Wium and Erna Alant
Apart from their subjective evaluation on the ease of use of the schedule, the results of their assessment were compared to those done by a group of three experienced AAC specialists to assess their ability to identify items and to score them.The results obtained by this group was, considered as a norm or a reference to guide the training of observation skills.

RESEARCH DESIGN
A small group, experimental design was used, which involved elements of the single subject, multiple-baseline design (using replication across 12 subjects).This is similar to the study described by Light, Dattilo, English, Gutierrez and Harts (1992).In this referred study of Light et. al (1992), data was presented for each individual subject.Data for the evaluation of changes that had occurred in the teachers as a result of the training in observation skills were collected five times in total for each of the 12 teachers and then compared to a norm.This was done for reliability as it is recommended by Barlow and Hersen (1984) that at least four baselines be collected for convincing results in this type of design.The general flow of events in the design of this study encompassed the following: -Firstly the development of the schedule which, in turn, was evaluated for ease of use in a pilot study with a sample of eight teachers.-After the schedule had been refined, the main study trained twelve teachers to use it.-The results were processed and analyzed.
To describe the application of a schedule to identify the communication abilities of CSD as used by teachers of children with severe disabilities (CSD).More specifically the following aspects of implementation will be highlighted: -The evaluation of the ease of use of the schedule -The identification of questions in the schedule which were poorly phrased and thus poorly understood by the teachers -The identification of problematic questions which were difficult to score Teachers used the schedule to functionally assess the abilities of three children with severe disabilities (Cases A, Β and C) and were then asked to evaluate the schedule.

MATERIALS USED
The schedule which was used to document observation skills of the teachers was designed to cover 47 items of five skill areas associated to AAC.They are the communication, cognitive, motor, sensory and social/emotional skill areas.The questions were formulated in user-friendly terms and examples of these can be seen in Tables 1 and  3.The schedule has to be completed by observing four functional tasks.Each task needs to be rated on a six-point rating scale (0-5).In order to guide the teacher, the rating scale is categorized as poor, sub-average and average which has to be selected prior to attributing a specific score.Table 1 has a summarized description of the schedule 1 .

DATA COLLECTION PROCEDURE
Training was conducted as a workshop according to nciples recommended for adult training (Wium, 1994).TVaining took place at the Centre for Augmentative and Alternative Communication (CAAC), Dept.Communicafrn pathology, University of Pretoria, over a two-day period The content and procedure of training are described Table 2. Data collection procedures are highlighted.The focus of this study was to.develop an AAC assessment schedule which involved two aspects: Firstly, the schedule had to be evaluated and secondly, the ability of the teachers to identify (recognize) the various items and to score them in the schedule had to be described.

Evaluation of the schedule by the teachers:
The teachers evaluated the ease of use of the schedule by corn-Day 1:-Descriptive information of teachers: Teachers from schools for children with severe disabilities in the Pretoria area registered for the workshop.Each received a folder consisting of handouts, three unused schedules and each of the three above-mentioned questionnaires.These teachers had to have a minimum experience of six months with children with severe disabilities.The seats were arranged in a semi-circle facing the researcher/trainer in order to encourage interaction amongst the trainees.Each seat was numbered according to their anti-clockwise position from 1-12 (from right to left) for material to be distributed and collected.
-Pre-training knowledge of skill areas: Prior to training, teachers completed an open-ended questionnaire which recorded their untrained knowledge of the five skill areas to be assessed in AAC as well as their expectations of the course.These questionnaires were collected after completion.
-Introductory and background information on a functional assessment was presented by means of audio-visual material (overhead projector and transparencies).The schedule (see Table 1) was introduced and the questions as well as the instructions for scoring were read through with them and demonstrated.Teachers were required to score every item in every observation.The assessment schedules were not collected until the teachers had scored four children and had calculated the results.
-Pre-training observation skills (Al): Firstly they were shown a video (A) of an autistic child (noted as observation Al).After three consecutive viewin'gs, they were requested to complete the schedule.The use of the schedule specifies that four functional tasks should be observed and documented.This first documentation was regarded as Task 1 on the schedule.After completion of the first observation, questions were answered and the scoring of some of the items was compared within the group as part of the discussion regarding their problems.
-Pre-training observation skills (Bl): They were then shown a second video (B) of a cei-ebral palsied child (noted as observation Bl) three times consecutively and they were required to complete the schedule as for Task 2 on the schedule (see Appendix A).The same procedure as in Observation A1 was followed after training.Again teachers required that scores of some items be compared.These were then discussed in the class.
-Training in the completion of the schedule was conducted by means of video material of children with severe disabilities.Atraining video of a young, non-ambulatory, cognitively high-functioning child was shown (as many times as needed) and scored by all teachers.The child was severely communicatively disabled but demonstrated all precursors to communication while playing with a speech-language therapist during therapeutic prone standing.Open discussion of all items in the schedule was continuously encouraged.Teachers scored this child as the third of the four tasks on their schedules.Scoring of these items were compared and discussed by the group.Practical demonstrations and role play were used to demonstrate the items in the schedule.
Day 2: -At the beginning of the second day handouts for the specific day were distributed.The previous day's work was reviewed, questions answered and handouts were discussed.
-Another training video of a non-verbal child was then shown as many times as necessary.The child identified pictures and symbols and the teachers had to score this video as well.The training video was discussed and scores compared.Teachers have now completed the four tasks on the schedule and could then learn to calculate the results.Each teacher had to calculate her four observations (Al, Bl, task 3 and task 4), and present the results as a graph.The completed schedules were then collected.Documentation of tasks 3 and 4 were not used for the research but merely to train teachers in the use of the schedule, the various skill areas and to calculate the results.
-Post-training observation skills (A2): After tea, the first video (A) was again shown (as often as necessary) and scored on an unused schedule as Observation A2.Questions were again answered and several scores compared.Evaluation of the items in the schedule: The approach followed emphasised the different interpretations given to the same set of data, and focused on the ability to observe the items listed in the schedule from the observation of video cases as well as an evaluation of their ability to score the items.The ability to identify items described in the schedule was evaluated by using the data obtained from the schedules completed during training.All correct responses were given a value of 1 and all errors a value of 0 which implied that they could or could not identify the skill.Responses were considered as correct when the teachers' responses were the same as the norm (which was established by three AAC experts prior to the main study).
A distribution of all 0's and l's was calculated before (Al and Bl) and after (A2, B2, C) training for all teachers and presented in two graphs.Four types of results were observed.They are summarised in Table 3.
The results of both Observations A and B, before (Al, Bl) and after (A2,B2) training, were calculated as percentages and visually presented in Figures 3 and 4. Questions in the schedule which caused confusion (1-0), or where no gains were made (0 -0), should be altered to be better understood or should be specifically focused upon in training.On the other hand, questions where existing knowledge (1 -1), or gained knowledge (0 -1) were observed, could be regarded as adequate for future use.

EVALUATION OF SCORING
This information was obtained by assessing the margin of error exhibited for each question.Therefore all scores from the completed schedules were compared to a norm (the value of which was allocated by a specialist team prior to training), and then statistically analysed..This analysis was done by obtaining the Root Mean Square (RMS) (Kirkpatrick, 1974) of the deviations from the norm for all 5 observations (Al, Bl, A2, B2, C), and then comparing them.This information in turn determined not only the magnitude of errors per question, but also the degree of complexity of each case.The deviations from the norm were calculated by subtracting the teacher's score from the norm.The RMS's of these deviations were then used as basis of assessing the questions: the larger the deviation from the norm, the less accurate the observation.An overall impression of suitable questions can be obtained by

Anna-Marie Wium and Erna Alant
considering the distribution of the deviations.This is presented in Fig. 7, where the number of cases (expressed as a percentage of the total number), was grouped in intervals of 25% of the deviation from the norm.
The accuracy of teachers' observations as they deviated from the norm was calculated for the individual questions for all three cases (A,B,C).The scores documented before training were compared with those after training and indicated questions to be changed.In calculating an average error per question for all cases (A,B,C) observed, an indication of problematic questions could be obtained.Questions with a >1.5 variation were considered as those questions which elicited poor scoring accuracy and should therefore be changed or receive more attention in future training.Questions with a <1.5 accuracy were considered adequate for future use.

RESULTS AND DISCUSSION
The schedule was evaluated from two perspectives: -The teachers evaluated it for ease of use -Content and structure of the schedule were evaluated by identifying questions (items), that proved to be difficult to assess.

EVALUATION OF THE SCHEDULE BY TEACHERS
The data from this evaluation was obtained from a closed-ended questionnaire and the results are presented as a distribution graph.The results describing the ease of use of the schedule are presented in Figure 1 below.
From Figure 1 it is clear that in general the teachers regarded the schedule as good or very good, and easy to use.None of the teachers evaluated any aspect on this questionnaire as poor or unacceptable although they had these options to choose from if they wanted to.The instructions for use as well as the questions within the schedule were regarded as clear, and its appearance was well received.The length of the schedule for classroom use was considered acceptable (the alternative options to choose from were "too long" or "too short") which actually reflects the optimum choice.In general, the schedule was evaluated very favourably by the teachers and they thought that it would be of great future use to them.j

EVALUATION OF CONTENT VERSUS STRUCTURE OF THE SCHED ULE
The problematic questions in the schedule were identified by using a two-level approach: ] TABLE 3: Four types of results for identification of items (0-1): Gains made by training in ability to identify a function.
The number of teachers who could not identify the function before training but could do so afterwards.A decrease of errors after training made by the group indicates that they had gained in observation skills regarding their ability to identify functions.Reproduced by Sabinet Gateway under licence granted by the Publisher (dated 2012) The Development of a Screening Schedule for Use by

Abilities of Children with Severe Disabilities
Firstly, it was established whether teachers could recognise the particular skill (described by the item in the schedule) as exhibited by the student -thus whether the skill was displayed.
Secondly the accuracy with which teachers could score (rank) their observation on a six-point rating scale was investigated.
It was necessary to differentiate between these two aspects in assessing a particular item to prevent confusion.It is much less difficult to identify an item (skill) as present or absent by attributing a positive (1) or negative (0) value than to scale (score) it from 0 -5.Scoring could be considered a more advanced skill which requires extensive training.The data (collected before and after training), was used to measure the success with which teachers could recognise items in the schedule as well as to identify questions where items were unreliably scored on a rating scale.

RECOGNITION OF ITEMS FOR ALL ITEMS AND FOR INDIVIDUAL CASES
The questions of concern are those where the item could not be recognised correctly after training.A summary of correct and incorrect answers (after training) was com-

Teachers to Describe the Communication 21
piled, and is presented in Figure 2. From this figure it is clear that after training approximately 31% of the items were identified incorrectly, while 69% were correctly recognised as being present or absent.It is significant to note that the fact that they could identify 69% of the questions before training, could imply that the questions in the schedule were formulated in a user friendly manner and that most people could understand it without any training.
An analysis of the schedule for Cases A and Β together may be misleading, therefore Cases A and Β were studied separately.These results are presented in Figures 3 and  4, where the results obtained prior to training were also included.Four classes of answers could therefore be distinguished and are presented in Table 3.

GENERAL ANALYSIS OF INDIVIDUAL CASES
-The sum of the first two categories in the correct group (1:1, 0:1), corresponds with the "correct" class in Figure 2, while the latter two can be regarded as being equivalent to the "incorrect" class in that figure .From Figure 3 it is apparent that for Case A, a total of 65.7% (45.7% + 20.0%) is correct after training, compared with 69.3% for both cases.

Schedule
Ease of feraphing Ease of Arithmetic Ease of Scoring Ease of Assessment  Kommunikasieafwykings, Vol. 43, 1996 Reproduced by Sabinet Gateway under licence granted by the Publisher (dated 2012)

Comparison of Observations
-In Figure 4, the total of correct items in Case Β amounts to 72.9% (57.3% + 15.6%), or a difference of 7,2% between the two cases. .It is also important to evaluate the impact of the training: For Case A (Figure 3), the correctness of answers improved for 20% of the answers, and decreased for 11.7% of the answers (a net gain of 8.3%).For Case B, (Figure 4), a net gain of 3.9% was recorded which is much less than for Case A.

(a) Questions litems in the schedule that the teachers found difficult to recognise
The results were finally used to identify individual questions where teachers found it difficult to recognise the presence or absence of a skill.These results are presented in Figures 5 and 6 and should be looked at as a pair in order to distinguish different patterns in shading.Particular attention should be paid to the darker and solid areas as these represent correctly identified items, while the lighter areas represent incorrect observations.
It is obvious when observing Figures 5  and 6, that there is a marked difference in the patterns between Cases A and B. Questions which were poorly identified in Case A were much better identified in Case B, and vice versa.A good example of this is where question no.7 was poorly identified by 9 (75%) of the teachers in Case A but only by 2 (16%) in Case Β after training.This shows that the teachers understood the questions but that they might have had other difficulties, e.g., unfamiliarity with the case presented (which is discussed later).
By studying Figures 5 and 6 specific questions were identified as difficult.All the questions which could not be identified by more than 25% of the teachers after training, were regarded as difficult and are listed in Table 4 below.It was only after all these questions were listed (see Table 4), that similarities could be detected which in turn identified problematic questions.Questions which were repeatedly regarded as difficult in both Cases A and B, consist of 12% (6 out of 47) of the total number of questions.These questions (all AB or BA responses), were labelled as problematic and are numbers 21, 26, 31, 33, 38 and 39.These questions need more attention during training by more demonstrations and explanations.
The motor skill area seems to be relatively difficult as 2/3 of the questions in this skill area are listed as difficult, which also includes half of the problematic questions on this list.This indicates that teachers found the terminology and assessment of Reproduced by Sabinet Gateway under licence granted by the Publisher (dated 2012) The Development of a Screening Schedule for Use by Abilities of Children with Severe Disabilities motor skill area difficult.The majority of teachers in this study had not yet worked with physically disabled hildren and therefore had no knowledge of this skill area.This is therefore not a reflection on the schedule, but more on the teachers' experience.

(b) Teachers' accuracy in scoring (rating 0-5) the items in the the schedule
The data obtained from the completed schedules also provided information on the questions which elicited poor scoring.Teachers scored their observations which were compared with the norm, on a scale from 0 to 5. The size of the deviation from the norm indicated their ability/inability to score a specific item, e.g., when the teachers' scores deviated more than two numerals on a scale from 0-5 then their scoring ability was considered as poor.When their scores deviated one numeral (on a scale 0-5), then

Teachers to Describe the Communication 23
their scoring ability was considered as inconsistent.
An overall impression of suitable questions can be obtained by considering the distribution of the deviations.This, is presented in Figure 7, where the number of cases (expressed as a percentage of the total number), was grouped in intervals of 25% of the deviation from the norm.
From Figure 7 it is clear (when adding the number of questions in the two intervals of deviations between 26% and 75% from the norm), that there is a large percentage (45%) of questions in this section.Also, none of the questions deviates more than 200% (2 numerals on the scale 0-5), from the norm.It must be made clear that the term % might be misleading and should be regarded as a unit (e.g., 100% deviates in fact 1 score from the norm).
A more detailed account of the specific problematic questions was obtained when the average error per question was calculated and it became clear that 12,7% of the questions deviated more than 150%.The specific questions are  1988), who stated that the assessor should be skilled.When the scoring of the individual skill areas was studied, it became dear tha the teachers were too broadly trained in the skill areas to be tested with such a relatively fine measuring instrument.However, the repeated use of the rating scale is useful for monitoring progress in the classroom.In general, the items in the schedule were not significantly better scored after training.Scoring is considered an advanced skill which requires more training.

PROBLEMATIC QUESTIONS IN THE SCHEDULE
When both the ability to identify the items as well as the scoring of the schedule are taken into account, six questions were found to be difficult, of which four were found in the motor skill area of the schedule.Of these only one question needs to be rephrased.Difficult questions require specific attention in future training.

CONCLUSIONS
The results indicated that the schedule is adequate for future use as teachers found it easy to use and the questions were adequately phrased.The training of teachers in the use of this schedule, however, needs to focus more on unfamiliar skill areas depending on the experience of the teacher (e.g., teachers who work with children with only cognitive disabilities would need more familiarisation with the motor skill area as their children are not physically disabled).The scoring routine of the schedule should be considered as an advanced skill which may need more practise and further training.The results obtained in this study highlight various issues which could have had an effect.This study showed that some children are more difficult to assess for specific teachers than others.Two important factors which have to be kept in mind are: -The teachers' familiarity with the disability: The majority of teachers in this workshop were unfamiliar with physically disabled children as they were used to working with primarily cognitively disabled children.The terminology applicable to the motor skill area was therefore especially at risk.This was pointed out as two thirds of the questions regarded as difficult occurred in the motor skill area.The effect of familiarity with the Anna-Marie Wium and Erna Alant disability observed could be seen in their ability to identify items as well as in scoring.Furthermore, the fact that the items which were poorly identified, and the items poorly scored did not match each other, supports the notion that the questions in the schedule are not so much poorly phrased, but that teachers were less familiar with certain skill areas due to their experience with specific disabilities.These results underline the complexity of assessing the severely disabled child (Culp & Carlyle, 1988), and also the fact that it is not easy to learn to observe objectively and in depth.It is therefore essential that future initial training workshops train a specific group of teachers with a familiar disability, as more reliable results could be obtained.Additional disabilities can be introduced with more advanced follow-up courses in order to make teachers better observers.Secondly, the severity of the disability: In addition to the above, teachers were also not familiar with the severity of the disabilities observed.The majority of teachers who attended this workshop had not been exposed to severely physically disabled children as their own .students were mainly severely or moderately cognitively disabled.
Familiarity in the use of the schedule and the ability to observe depends on repetition (Stevens, 1978).This also became clear during training where it was observed that the more experienced and skilled teachers became in the use of the schedule , the less time was required to complete it and the more accurate their scoring became (as was continually noted in observation of Case C).Perhaps the most significant contribution of the schedule and the training was the effect it had on the selfconfidence of the teachers.The results in Figure 1 indicated they felt confident of their ability to apply the schedule in future.This study has provided teachers with a screening tool as well as training in the use of the tool.The ability to effectively assess students by using the schedule made teachers feel efficient and in control.Practise and continual use of the schedule will_ make them feel even more confident and efficient in assessment.This implies that teachers who are able to use the schedule effectively, become empowered to perform their role as teachers more successfully.
Most importantly in this study, the issue of a transdisciplinary approach was addressed.One of the problems in AAC service delivery was that the responsibility of assessment had largely been considered as that of the professional speech therapists (Hogg & Raynes, 1987).Such measures resulted in it being an exclusive service in this country, focusing on the community with the !pning instrument which can be used by a much crroup of people, which in turn makes the service ΐ£ΙΙ ^βΓ Γν of AAC more available and accessible to those work with children with severe disabilities.The f this schedule enables teachers to identify prob-Γ δβ ° to refer to a specialised multi-disciplinary team when necessary, to plan communication objectives and to monitor progress.This in turn results in more effective service delivery where problems such as limited manpower and funding are addressed.

-
Post-training observation skills (B2): The second video (B) was then shown (as often as necessary) and then scored as Observation B2.Questions were again answered and several scores compared.-Post-training observation skills (C): A third video (of an unfamiliar child) was then shown and also scored.This observation was documented as Observation C. Results were compared and questions anwered.The second schedule (with the results of documentation after training), were then collected.-Post-training evaluation of schedule: Teachers had to complete a closed-ended questionnaire to evaluate the schedule.-Post-training knowledge of skill areas and evaluation of the training: They also had to complete the open-ended questionnaire after training to assess the change that had occurred in their knowledge of the skill areas.Comments and recommendations regarding the training were also given.-At the end of Day 2 teachers received certificates of attendance.pleting a closed-ended questionnaire.Responses were calculated as percentages and displayed as a distribution graph.Comments and recommendations were tabled.
the teacher could not identify the function before training and neither could she do it after training.ίΒ^ΒβΛΗ^ϋίΒΛ Existing knowledge.These results imply that the teacher could identify the function before as well as after training.e^Bp^Blie^pllll^llillii Incorrect The teacher identified the correct function before training but could not do so after training.The South African Journal of CommunicationDisorders, Vol.43, 1996

FIGURE 7 .
FIGURE 5. Difficult items to identify in observation A

FIGURE 2. Total of all questions Comparison of Observations Pre-& Post Training: Case Β
FIGURE 3. Difficult questions in observation A FIGURE 4. Difficult questions in observation Β Die Suid-Afrikaanse Tydskrif vir

TABLE 5 : Questions poorly scored
41How well does he attend auditory?Sensory 45 How often does self-stimulation occur?Social/EmotionalThe South African Journal of CommunicationDisorders, Vol.43, 1996Reproduced by Sabinet Gateway under licence granted by the Publisher (dated 2012) looment of a

Screening Schedule for Use by The De ^of Children with Severe Disabilities
I' ancial resources to afford it.This study provides a A