While randomised controlled trials (RCTs) are considered the gold standard of research, prior study is needed to determine the feasibility of a future large-scale RCT study.
This pilot study, therefore, aimed to determine feasibility of an RCT by exploring: (1) procedural issues and (2) treatment effect of the Classroom Communication Resource (CCR), an intervention for changing peer attitudes towards children who stutter.
A pilot cluster stratified RCT design was employed whereby the recruitment took place first at school-level and then at individual level. The dropout rate was reported at baseline, 1 and 6 months post-intervention. For treatment effect, schools were the unit of randomisation and were randomised to receive either the CCR intervention administered by teachers or usual practice, using a 1:1 allocation ratio. The stuttering resource outcomes measure (SROM) measured treatment effect at baseline, 1 and 6 months post-intervention overall and within the constructs (positive social distance, social pressure and verbal interaction).
For school recruitment, 11 schools were invited to participate and 82% (
A high recruitment rate of schools and participants was observed with a high dropout rate of participants. Significant differences were only noted at 6 months post-intervention within one of the constructs of the SROM. These findings suggest that a future RCT study is warranted and feasible.
Among the various levels of evidence that are valuable in clinical practice, the randomised controlled trial (RCT) is regarded as the gold standard because of its design strength (Evans,
It is further reported that the procedural rigour of the study is as important as the treatment effect benefit when evaluating the feasibility of an RCT (Oakley et al.,
The process evaluations of a study are specifically recommended in longitudinal studies, such as this one, where repeated measures occur (Oakley et al.,
At present, there are no documented feasibility studies in South Africa within the domain of classroom-based intervention, which is essential for planning of future large-scale studies. Comment on feasibility is also important, as research within the school context is challenging. Common challenges relate to procedural aspects and treatment effect such as consent, participation and ethical concerns resulting from the vulnerable nature of conducting research with children. While it could be argued that this is the case for all studies, the complexity of school research adds to the level of difficulty that is often experienced when conducting school-based research.
In terms of treatment effect, it is essential to also determine the potential treatment effect in a pilot study design prior to conducting an RCT. Lancaster et al. (
Stuttering, a communication disorder, presents with personal and social implications (negative self-perceptions, teasing and bullying), often occurring at primary school (Dijkstra, Lindenberg, & Veenstra,
Internationally, persistent reports of teasing and bullying of children who stutter (CWS) led to the development and study of the Teasing and Bullying (TAB) resource, a classroom-based intervention in Canada (Langevin,
The TAB resulted in the development of the Classroom Communication Resource (CCR) intervention for South Africa, the intervention being subjected to testing in this study. It was required specifically in South Africa because of the prevalence of teasing and bullying and requests from teachers for support (Abrahams, Harty, St Louis, Thabane & Kathard,
The CCR intervention was studied and developed through small-scale studies by the University of Cape Town between 2009 and 2014 (Badroodien et al.,
The aim of this study was to determine the feasibility of an RCT through conducting a pilot study.
The study had two objectives:
Primary objective: to determine the recruitment rates of schools and participants and the dropout rate of participants
Secondary objective: to determine the treatment effect of attitudes towards stuttering among Grade 7 students based on the SROM and its subscales – the PSD, VI and SP.
A pilot, cluster, stratified RCT design was used, where schools were the unit of randomisation. The cluster stratified RCT design was emulated using a pilot study design to accurately comment on the feasibility of a future RCT. The schools were stratified into two quintile groups (lower vs. higher) and randomised to receiving the CCR intervention or usual practice, using a 1:1 allocation ratio.
The eligibility criteria included Grade 7 participants, aged 11 years and older, in mixed-gender schools where the language of learning and teaching was English. The participants attended public primary schools in the lower (two and three) and higher quintiles (four and five). Quintiles were included to ensure a representative sample was included. The schools were situated in the Western Cape metropolitan urban area in South Africa. Participants were not financially compensated in any way. All participating schools were provided with their own copy of the CCR intervention. Schools could have CWS in the classroom; however, once CWS were identified they were approached to obtain consent to determine if the study could be conducted in their school. Exclusion criteria included participants aged younger than 11 years from same-sex schools and schools within Quintile 1.
A total sample size of
The CCR intervention included three key components, namely a social story, role play and teacher-led discussion. The teacher read the social story to his or her class. Once the story was complete, the teacher selected participants to act out the role play. The purpose of the role play was to emphasise the story but also to provide participants with a first-hand account of how the characters of the story may have felt. Finally, the teacher facilitated the discussion by using the guidelines in the CCR intervention. The discussion aimed to promote acceptance of diversity and difference related to stuttering, communication and generally, as well as discussions around teasing and bullying and how this related to what was happening at each school.
The CCR intervention is considered self-sufficient for the most part. However, the teacher was provided with basic training on how to administer the CCR intervention. The focus on training was placed on the discussion aspect of the intervention, as many teachers had queries and concerns about how to best administer this section. In doing so, the CCR could be considered a supported classroom-based intervention that was used as a single-dose intervention. The researchers observed, without interference, the administration of the CCR intervention. The CCR intervention was only administered in the intervention groups, while control group teachers continued with their teaching without drawing attention to stuttering in any way. Any questions that were asked after the intervention were to be answered and recorded by the teachers.
The recruitment rate described the number of schools that were invited compared to those who agreed to participate during the recruitment phase of this study. This was described at a school-level, as this is how participants were initially recruited. Thereafter individual recruitment was described in terms of those recruited from the eligible sample (based on school recruitment). The dropout rate described the loss of participants at baseline, 1 and 6 months post-intervention because of the longitudinal nature of this study.
This study is concerned with the observation of a positive shift in the treatment effect (magnitude and direction) at 1 and 6 months post-intervention from baseline. The treatment effect was commented on using the global and sub-scale scores on the SROM. The SROM consisted of 20 questions making use of a Likert scale. The SROM sub-scales, including PSD, VI and SP, are psychometrically approved constructs (Walters,
The SROM was developed on the Peer Attitudes towards Children Who Stutter (PATCS). The PATCS was developed by Langevin (
Once-off randomised sampling took place to track participants from baseline, to 1 and 6 months post-intervention. Continuous sampling was therefore not practical.
Upon obtaining the relevant consent and assent, all participants viewed a video of a CWS. The participants were all asked to complete the SROM at baseline. Thereafter the teachers in the intervention groups received training, over a 60–90 min session, to administer the CCR intervention. The CCR intervention was then administered by the teacher to the participants in the intervention groups. No intervention took place in the control group. At 1 and 6 months post-intervention, all participants completed the SROM. Thereafter the control group teachers were provided with a copy of the CCR as well as teacher training.
The procedural aspects are calculated as follows:
The school recruitment rate was determined by examining how many schools were invited and agreed to participate. Individual recruitment was similarly reported.
The dropout rate was calculated into a percentage value at each time interval (from baseline, 1 month and 6 months post-intervention). It was reported as it is a common occurrence within the school setting and accounts for the participant numbers noted in this study. The inclusion of this information is essential for future planning of an RCT.
The treatment effect is calculated as follows.
Each participant’s SROM scores were captured in Microsoft Excel, and R Studio version 1.0.143 (
Ethical approval was obtained from the University of Cape Town Health Sciences Human Research Ethics committee (510/2013). Thereafter, permission was provided by the Western Cape Education Department. Consent and assent were obtained from schools, principals, parents and participants. The ethical principles of autonomy, confidentiality, beneficence, non-maleficence and distributive justice were upheld at all times, as stipulated by the Declaration of Helsinki (Williams,
A total of 11 schools were invited, 10 schools responded to the invitation to participate, nine schools accepted the invitation and only eight participated in this study, as one school withdrew from the study. The recruitment rate was therefore 82%, as 9 out of the 11 schools invited agreed to participate in this study. Based on the school recruitment,
The dropout rate in the intervention group at baseline was 23% (
Dropout rate at baseline, 1 and 6 months.
Time point | Dropout | |||
---|---|---|---|---|
Intervention ( |
Control ( |
|||
% | % | |||
Baseline | 34 | 23 | 15 | 6 |
1 month | 10 | 7 | 15 | 6 |
6 months | 10 | 7 | 44 | 17 |
A total of
Baseline characteristics of study participants by study group.
Variable | Intervention ( |
Control ( |
|||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
% | Mean | SD | Min | Max | % | Mean | SD | Min | Max | ||||
Number of clusters | 4 | - | - | - | - | - | 4 | - | - | - | - | - | |
Cluster size | - | - | 37 | - | 25 | 57 | - | - | 63 | - | 25 | 141 | |
Gender: male | 63 | 42 | - | - | - | - | 105 | 42 | - | - | - | - | |
SROM | - | - | 73.17 | 12.05 | - | - | - | - | 71.48 | 12.80 | - | - | |
PSD | - | - | 38.65 | 7.63 | - | - | - | - | 38.21 | 7.55 | - | - | |
SP | - | - | 19.89 | 3.68 | - | - | - | - | 19.15 | 4.14 | - | - | |
VI | - | - | 14.62 | 3.28 | - | - | - | - | 14.12 | 3.37 | - | - |
SROM, Stuttering resource outcomes measure; PSD, positive social distance; SP, social pressure; VI, verbal interaction.
As shown in
Forest plot of treatment effect at 1 and 6 months on the stuttering resource outcomes measure and its subscales positive social distance, social pressure and verbal interaction (
A sensitivity analysis was also conducted ignoring clustering, which showed similar results as shown in
Sensitivity analysis of treatment effect at 1 and 6 months on the stuttering resource outcomes measure sub-scales (
Variable | Outcome | Method | Difference (I-C) | 95% CI | ICC | |
---|---|---|---|---|---|---|
At 1 month | SROM | Random effects | 2.0100 | −1.07, 5.11 | 0.2020 | < 0.001 |
Linear regression | 2.0140 | −1.09, 5.12 | 0.2030 | - | ||
PSD | ||||||
Random effects | 1.0800 | −1.09, 3.07 | 0.3210 | 0 | ||
Linear regression | 0.9900 | −1.10, 3.08 | 0.3530 | - | ||
SP | ||||||
Random effects | 0.5100 | −0.31, 1.29 | 0.2210 | 0 | ||
Linear regression | 0.5000 | −0.30, 1.29 | 0.2210 | - | ||
VI | ||||||
Random effects | 0.5300 | −0.29, 1.35 | 0.2070 | < 0.001 | ||
Linear regression | 0.5300 | −0.30, 1.36 | 0.2110 | - | ||
At 6 months | SROM | Random effects | 3.0100 | −0.69, 6.69 | 0.0920 | 0.015 |
Linear regression | 2.4600 | −1.05, 5.98 | 0.1690 | - | ||
PSD | ||||||
Random effects | 2.5700 | 0.67, 4.46 | 0.0076 | 0 | ||
Linear regression | 2.5700 | 0.66, 4.47 | 0.0085 | - | ||
SP | ||||||
Random effects | 1.0400 | 0.18, 1.89 | 0.0180 | 0 | ||
Linear regression | 1.0362 | 0.17, 1.90 | 0.0186 | - | ||
VI | ||||||
Random effects | 1.3500 | 0.58, 2.13 | 0.0010 | 0.043 | ||
Linear regression | 1.2100 | 0.46, 1.96 | 0.0017 | - |
ICC, Intra-school correlation coefficient; SROM, stuttering resource outcomes measure; PSD, positive social distance; SP, social pressure; VI, verbal interaction.
It should be noted that the findings of this study reflect schools in the Western Cape, South Africa, from Quintiles 2, 3, 4 and 5. Findings should, therefore, be interpreted with caution when considering other provinces within South Africa.
Several challenges were encountered during this study, despite the anticipation of some general challenges that often arise during school-based research. It could be argued that all researchers experience varying degrees of difficulty with conducting research, while the complexity of school research added to the level of difficulty that was experienced in this study. The common challenges were experienced, such as consent and participation, which affected the recruitment of participants.
The results indicate that the recruitment rate was high because schools were approached early in the year and could thus foresee making time available for the researchers, showing that school recruitment may be a feasible method of initial recruitment. While there is disparity in the numbers of control versus intervention groups, it should be noted that this was as a result of consent not being provided and absenteeism, all factors out of control of the researcher. Furthermore, this was taken into account when interpreting the findings of this study. Once the challenge of school recruitment was overcome, the researcher faced difficulty with recruiting individual participants. Based on the eligible participants from school recruitment, far fewer participants were recruited, as a result of poor consent. It was challenging because the researcher relied on schools, principals, teachers, parents and participants to provide permission, consent and assent required for recruitment, while the researcher only had access to principals and some of the teachers. Because of the strict design of RCTs, the study will only be successful should schools agree to participate and facilitate return of consent forms from parents. Once participants were recruited, the next challenge was to retain participants and prevent a large dropout of participants to ensure power analysis of this study. It was reported by schools that clearer communication is required. It is, therefore, vital that, in future, schools and teachers are made explicitly aware of the time commitments required of them so that they may make an informed decision as to whether they are able to participate in a future study and not experience the burden of participating in a study.
Upon discovering that data were to be collected at three separate intervals (baseline, 1 month and 6 months) in addition to another visit to the school where the teacher administered the CCR, schools expressed anecdotally great concern around the time commitments required of them. As a result, a dropout of participants was noted over time as well as difficulty arranging for data collection dates. This is commonly reported in longitudinal studies (Galea & Tracy,
In terms of organisation, early planning and scheduling, logistically it was challenging to find suitable times for data collection, given the pre-existing busy academic calendar. Schools found the research time-consuming and reported that they would not have committed to it had they realised the extent of the time needed to dedicate to this study. Consequently, there were serious implications in terms of motivation to participate and the relationship between the researcher and the school. This was found to be especially true where telephonic contact was made. While it appeared to be the most convenient method, it was viewed as impersonal. Face-to-face contact and direct contact is reported to improve building a relationship with schools, principals, teachers, participants and the researcher (Galea & Tracy,
In addition to the administrative challenges discussed, related to planning and scheduling, other challenges included relationships and consistency of researchers. These factors collectively affected the recruitment and dropout rates. Schools reported that the use of research assistants was inconsistent and reported that they were unable to build a relationship with and get to know the researcher and research assistant at their school. Building a relationship early on with the school is recommended by Galea and Tracy (
Though no significant result was observed at 1 month post-intervention, it is possible that it was too early for participants to have internalised their learning. This is supported by Kathard et al. (
Overall, both procedural aspects and treatment effect trends provide important information about the feasibility of an RCT. It is illustrated that collectively these factors suggest that an RCT is feasible. The recruitment and dropout rates specifically showed that several factors should be considered to improve the feasibility of a future RCT in terms of the procedural aspects of this study. Additionally, the treatment effect shows that 6 months post-intervention proved to be an optimal and feasible time to determine the treatment effect, whereas in this study a significant result was noted only in one of the constructs at 6 months. It would be important to retain a sample to test the effectiveness of the CCR intervention in a more robust way. Furthermore, it would be impractical to measure post-intervention attitudes at three intervals in future because of time constraints (reported by schools) and because of the repeated use of the same outcomes measure.
The main strength of this study was its ability to achieve the objectives of determining the feasibility of an RCT by drawing on the findings of this pilot study. The limitation of this study is the way in which schools experienced the study. Clinical implications include that an RCT is feasible and that there is a need for further research to enrich South African literature on classroom-based stuttering intervention.
An RCT is recommended, with further development of the process. In order to conduct a methodologically sound RCT, there are several factors that need to be considered and put into place, as described in the discussion. There are two main recommendations for this study: (1) to reduce the dropout rate of participants through stringent methods and (2) to determine treatment effect at baseline and 6 months post-intervention only. No significant results were noted at 1 month, suggesting that perhaps only 6 months post-intervention data may be necessary, as this is where the shift in treatment effect begins. By reducing the number of data collection intervals and being transparent about the number of visits that are required, the researcher may also alleviate time pressure and any burden schools may experience.
R.B.M. and Prof. H. Kathard wish to acknowledge the Programme for Enhancement of Research Capacity grant, University of Cape Town that partially contributed to this study. Prof. L. Thabane wishes to acknowledge and was supported in part by funds from the Carnegie African Diaspora Fellowship Program. This study was funded in part by the Programme for Enhancement of Research Capacity grant, University of Cape Town, and the Carnegie African Diaspora Fellowship Program.
The authors declare that they do not have any competing interests. There are no personal and/or financial relationships that may have influenced the writing of this article.
R.B.M., the corresponding author completed the research as part of her Master’s study and was responsible for writing and compiling drafts of this paper. Prof. H. Kathard was the primary supervisor and Prof. L. Thabane was a consultant in this research study. A.S.M.B. conducted the statistical analysis of this study. H.K., L.T. and A.S.M.B. contributed to the writing and reviewing of this paper through a number of drafts.