Traditionally, the emphasis in research in the fields of speech-language therapy and Audiology has been on quantitative research, in large part as a result of the medical model that dominated the field for many years. The qualitative research paradigm started to gain popularity in the late 1980s shifting to a more biopsychosocial approach as propagated by the World Health Organization (
Johnson, Onwuegbuzie and Turner (
research questions that call for real-life contextual understanding, multilevel perspectives and cultural influences
employs rigorous quantitative research assessing the magnitude and frequency of constructs and rigorous qualitative research exploring the meaning and understanding of constructs
utilises multiple methods (e.g. intervention trials and in-depth interviews)
intentionally integrates or combines these methods to draw on the strengths of each and framing the investigation within philosophical and theoretical positions.
Yin (
Mixed-methods research is a relatively new approach in the fields of speech-language therapy and audiology and has been under-represented in these disciplines as compared, to mono-method approaches. Suleman and Hopper (
There appears to be some uncertainty as to what can be considered as MMR and what such studies entail. Many studies that incorporate both quantitative and qualitative methods for data collection do not include the term MMR in the title of the research (refer to
Mixed-methods research articles published in the South African Journal of Communication Disorders (2008–2018).
Number | Publication | Report: single or both methods |
---|---|---|
1 | Wium, A. M., Louw, B., & Eloff, I. (2010). Speech-language therapists supporting foundation-phase teachers with literacy and numeracy in a rural and township context. |
Quantitative and qualitative (QUAN+QUAL) |
2 | Wium, A. M., Louw, B., & Eloff, I. (2011). Evaluation of a programme to support foundation-phase educators to facilitate literacy. |
Qualitative |
3 | Wium, A. M., & Louw, B. (2011). Teacher support – An exploration of how foundation-phase teachers facilitate language skills. |
Qualitative |
4 | Wium, A. M., & Louw, B. (2012). Continued professional development of teachers to facilitate language used in numeracy and mathematics. |
Quantitative and qualitative (QUAN+QUAL) |
5 | Teixeira, L., & Joubert, K. (2014). Availability of audiological equipment and protocols for paediatric assessment and hearing aid fitting in Gauteng, South Africa. |
Quantitative and qualitative (QUAN+qual) |
6 | Navsaria, I., Pascoe, M., & Kathard, H. (2011). ‘It’s not just the learner, it’s the system!’ Teachers’ perspectives on written language difficulties: Implications for speech-language therapy. |
Qualitative |
7 | Mdlalo, T., Flack, P. S., & Joubert, R. (2016). Are South African speech-language therapists adequately equipped to assess English Additional Language (EAL) speakers who are from an indigenous linguistic and cultural background? A profile and exploration of the current situation. |
Quantitative and qualitative (QUAN→QUAL+quan) |
8 | Lundie, M., Erasmus, Z., Zsilavecz, U., & Van der Linde, J. (2014). Compilation of a preliminary checklist for the differential diagnosis of neurogenic stuttering. |
Quantitative and qualitative (QUAN→qual) |
9 | Wium, A. M., & Gerber, B. (2016). Ototoxicity management: An investigation into doctors’ knowledge and practices, and the roles of audiologists in a tertiary hospital. |
Quantitative and qualitative (QUAN+qual) |
10 | Schütte, U. (2016). Culturally sensitive adaptation of the concept of relational communication therapy as a support to language development: An exploratory study in collaboration with a Tanzanian orphanage. |
Quantitative |
11 | Andrews, M., & Pillay, M. (2017). Poor consistency in evaluating South African adults with neurogenic dysphagia. |
Quantitative and qualitative (QUAN+qual) |
12 | Abdoola, F., Flack, P. S., & Karrim, S. B. (2017). Facilitating pragmatic skills through role-play in learners with language-learning disability. |
Quantitative and qualitative (QUAN+qual) |
Researchers also tend to report results obtained from MMR as a single method (refer to
Considering these aforementioned reasons for confusion, some clarification is required for researchers and clinicians in terms of MMR concepts and methods. The aim of this tutorial is to provide researchers, speech-language therapists and audiologists with an overview of MMR to serve as alternative to the traditional quantitative and qualitative research approaches in their pursuit of evidence-based practice (EBP). Based on a comprehensive and in-depth literature review, the tutorial firstly provides a background that explains MMR in terms of the philosophy and its relationship to EBP. Next, the definition and clarification of concepts, as well as the characteristics of MMR are explained. The article discusses the various issues to be considered in the critical appraisal of MMR, followed by a description of the process of conducting MMR. A critical review of MMR describes the advantages, as well as the controversies and challenges in MMR. Lastly, a conclusion highlights the importance of MMR for the disciplines of speech-language therapy and audiology.
As a methodology, MMR involves philosophical assumptions that guide the direction of the collection and analysis of data and the mixture of qualitative and quantitative data in a single study or series of studies.
New methodologies evolved that combine and integrate quantitative and qualitative research approaches, which open up more possibilities in answering specific research questions (Glogowska,
When considering the nature of the research, there are definite differences between quantitative, qualitative and MMR approaches (Creswell & Plano Clark,
In terms of the epistemology, the nature of knowing in quantitative research is objective, as opposed to qualitative research, where the nature of knowing is subjective. Once again MMR takes the middle ground as the nature of knowing is intersubjective.
The role of values in the interpretation of results also differs, as quantitative researchers strive to be unbiased and take precautions to avoid bias, which is different from qualitative research where the researchers are integrally part of the research and, therefore, are fundamentally biased. In MMR, the interpretation of findings is both biased and unbiased as both quantitative and qualitative methods are used. However, in MMR, care is taken to limit bias by putting specific measures in place. In the interpretation of results, quantitative results can be generalised because of the sample size and selection methods being used, which is not the case in qualitative research where smaller samples are typically used which makes the findings more context specific. In MMR, the findings are transferrable to similar contexts and population groups because thick descriptions are used in the qualitative strand and also because a sufficient sample size is being used in the quantitative strand. The three research approaches also differ in terms of causality: in quantitative research the cause results in effect, whereas in qualitative research the cause cannot be isolated from the effect. In MMR, causality cannot be determined. It can, therefore, be concluded that because both the quantitative and qualitative approaches are included in MMR, it has a unique underlying philosophy that is based in the common ground between these two methodologies.
In the case of MMR, the claim of knowledge (philosophical assumption) often is pragmatic because the rigid interpretations of methodologies have begun to fade (Onwuegbuzie & Dickinson,
Research in speech-language therapy and audiology aims to not only accumulate knowledge concerning communication and its disorders but also consider the therapeutic issues that could improve the quality of life of individuals with communication disorders. Such knowledge should inform practice. Evidence-based practice emerged as an important principle in the delivery of speech-language therapy and audiology services in the past decade (American Speech-Language-Hearing Association,
Evidence-based practice involves the integration of best current research evidence, clinical expertise and the needs, abilities, values, preferences and interests of clients and their families in making clinical decisions to provide high-quality services and, therefore, EBP is central to the disciplines of speech-language therapy and audiology (American Speech-Language-Hearing-Association,
Evidence-based practice.
However, a gap often exists in the application of knowledge to practice, which opens up the field of implementation science (Olswang & Prelock,
Systematic reviews were developed to support EBP and can be performed on MMR by analysing both quantitative and qualitative research evidence. In this case, it is necessary for data analyses to be carried out separately in order to ensure the integrity of the research. Mixed-method systematic reviews are suitable to understand the barriers and facilitators to the implementation of certain interventions, to assess cost-effectiveness and to address the core competencies of practitioners. These systematic reviews add important value to traditional systematic reviews as the inclusion of qualitative studies provides a greater understanding of the appropriateness of an intervention beyond documentation of its effectiveness (Orlikoff, Schiavetti, & Metz,
Mixed-methods research provides the opportunity to seek answers to complex clinical questions and to explore evidence from both quantitative and qualitative perspectives in the pursuit of improving outcomes. It is important for speech-language therapists and audiologists to be cognisant of the MMR approach and how it can be applied in EBP.
Mixed-methods research is often criticised for a lack of a common definition, confusing terminology and limited guidelines for when it is suitable to select MMR methods (Creswell,
The most probable answers to this question are when either the quantitative or the qualitative methods by itself appear to be inadequate to create a complete understanding of the research problem, or to develop multiple perspectives of a phenomenon, for example, when quantitative outcome measures are enhanced or explained by using qualitative data. On the other hand, a qualitative exploration, for example, a focus group discussion prior to the development of an instrument or measurement tool, may inform the contents and approach thereof.
Creswell and Plano Clark (
Several terms are used to describe the nature and extent of mixing methods (e.g. mixing, combining and integrating), and these are often used interchangeably (Creswell, Plano Clark, Gutmann, & Hanson,
Mixed-methods research is where the mixing occurs in the methods stage of the study and it can be on a continuum from simple to complex (Onwuegbuzie & Johnson,
Mixed-model research is where the mixing occurs in all stages of the study (the research question, methods, data collection and analysis and inference processes).
There are definite differences that distinguish MMR from mono-method designs (Garuth,
Morse (
QUAN or quan refers to quantitative data
QUAL or qual refers to qualitative data
MM refers to mixed-methods
→ data collected sequentially
+ data collected simultaneously
= converged data collection
( ) one method embedded in the other.
Knowledge of the notation system used in MMR makes it easier to read and to critically appraise such studies.
Speech-language therapists and audiologists are required to critically appraise research before selecting the findings for making EBP decisions. Many research studies include both quantitative and qualitative data collection and analysis methods. The quality of research depends on how well the accepted form for both qualitative and quantitative research designs is followed (Garuth,
It is recommended that the process be initiated by performing a general screening procedure that involves scanning the abstract and methodology sections of an article to determine where in the research process the mixing occurred (e.g. at the research questions stage, the sampling stage, the data collection or analyses). It is important to search for key words that refer to terms, for example, mixed- or multimethods, or qualitative, quantitative, triangulation, integrating methods, and combining methods. Once the initial scanning process has been completed it is necessary to conduct a rigorous content analysis to look for rich descriptions of the content, particularly on how the mixing and/or combining was performed. At this stage of the appraisal, it is necessary to identify the design of the study, the type of sampling and the data collection and analysis. Next, one has to consider issues of validity and whether there are any signs of using meta-inferences that could indicate inference quality (Tashakkori & Teddlie,
The second step in the critical appraisal of MMR is to identify the research design and to ensure that a pattern is used throughout the research. The specific mixed-methods design is determined by two main factors (Tashakkori & Teddlie,
Concurrent Triangulation Designs:
Concurrent designs may also comprise data being collected by using both open-ended and close-ended questions with a survey instrument (Creswell & Plano Clark,
Embedded designs have two research questions, which should be answered separately. The supplemental data set should answer the second research question. Danzak (
Sequential Explanatory Design: an alternative is the
Sequential Exploratory Design: the qualitative strand helps to develop or inform the quantitative strand and to connect the data between the two phases (e.g. in instrument design, theory building or testing). The emphasis usually is on QUAL, but can be QUAN, or it can be equal. The mixing occurs in the interpretation of the results – (QUAL → quan). The QUAL part of the study will provide information to uncover variables of interest. The second phase would be where the quan phase is used to develop the tool from the results obtained in the first phase. A focus group in a school could explore how teachers determine perceived barriers to receive speech-language services at their schools (qual), and then these ideas will be used to conduct a large-scale survey in a district by asking the participants to rate the impact of predetermined barriers (QUAN) (Suleman & Hopper,
The third step in the critical appraisal of MMR is to review the sampling process. Collins, Onwuegbuzie and Sutton (
Identical sampling is where the qualitative and quantitative samples include the same participants.
Parallel sampling design is where different qualitative and quantitative samples are drawn from the same population. There is parallel use for probability and purposive strategies, either concurrently or sequentially. Examples are where one set may be a subset of the other, or where both studies use the same total sample.
Nested sampling design is where the participants from one component of the investigation represent a subset of those who were included in another phase of the study.
Multilevel sampling makes use of sample sets from different populations at different levels of the study. In this case, probability and purposive sampling techniques are used at different levels of analysis (e.g. therapists and clients).
The fourth step in the critical appraisal of MMR is to evaluate the data collection. Data are collected simultaneously (concurrently) or sequentially (Creswell & Plano Clark,
In a sequential study, Overby et al. (
In a nested MMR design, Bryman et al. (
It is essential that one evaluates or assesses which data analyses were employed when critically appraising MMR. Various data analyses methods can be used in MMR. Quantitative data analysis is either descriptive and/or inferential, whereas qualitative data analysis is carried out descriptively and through thematic analyses of text or image. Overby et al. (
In the Langevin et al. (
Once the initial quantitative and qualitative data analyses have been completed, the MMR analysis should be carried out. Creswell (
From the integration process, several outcomes are possible in MMR (Brannen,
The process of legitimising the research (which is the mixed-methods nomenclature for validity, reliability and trustworthiness) determines quality (Onwuegbuzie & Johnson,
Aspects related to legitimising the research
Three processes determine the value of MMR, namely methodological rigour, interpretive rigour and inference transferability (refer to
The methodological rigour (also known as design quality) is concerned with the application of method and provides the standards for the assessment of MMR. Dellinger and Leech (2006) propose a validation framework consisting of concepts outside the more common terms. These terms include, for example,
The criteria for determining
As a result of issues in meeting these stringent criteria, additional measures have to be introduced to ensure quality in qualitative research (Bryman,
Bryman (
Inference quality, which refers to validity within a mixed-methods context, that allows the researcher to draw meaningful and accurate conclusions.
Consequential validity, which is steered by the pragmatist approach as it guides the triangulation design which results in an overarching validity when researchers draw evidence from different sets of data that provide better results than any of the two data sets can do alone.
In MMR, it is important that each method is complete and meets the criteria for rigour as if it would when standing alone (Woolley,
In MMR, researchers firstly have to determine whether a mixed-methods study is practical and what the justification for the mixing of methods is. This is followed by formulating the research questions and collecting and analysing the data. Lastly, the report has to be written (Creswell & Plano Clark,
Researchers need to be cognisant of both the strengths or advantages and the challenges or controversies related to MMR that need to be considered.
Mixed-methods research maximises the strengths and reduces the limitations of single methods (Polit & Beck,
When quantitative and qualitative methods are used together, they both contribute to a common understanding of the research phenomenon (O’Cathain, Murphy, & Nicholl,
Despite these advantages of using MMR, there are several controversies and unresolved issues that researchers need to be cognisant of before embarking on such a venture. It is necessary to identify the potential threats or challenges that could have occurred during data collection and analysis, as it could impact on the validity of the findings. In order to enhance the quality of the MMR, the researchers should also indicate how such threats were countered by them (Venkatesh et al.,
Controversies and challenges may occur in the conceptualisation stage of research (in providing a rationale for the mixing of methods), the operationalisation stage (how the methods will be used) or the synthesis stage where the findings of various methods used are interpreted. Teddlie and Tashakkori (
The mixed-methods research process.
Phase | Steps |
---|---|
Phase 1: Formulation Phase | Step 1: Addresses the long-term aim of the study, |
Step 2: Objectives (e.g. to measure change; to understand complex phenomena; to test or generate new ideas; to inform constituencies; and to examine the past three goals). | |
Phase 2: Planning and Design | Step 3: Determines the research or mixing rationale that explains why the study is needed and why quantitative and qualitative approaches should be mixed. Collins et al. described four main rationales for MMR: (1) participant enrichment (e.g. when recruiting participants or to obtain participant feedback), (2) instrument fidelity, which assesses the suitability and use of research instrument and to validate individual scores on outcome measures, (3) treatment integrity (i.e. refining intervention implementation and the variables related with its context), and (4) significance enhancement (i.e. expanding the interpretation of the results and enhancing the interpretation of significant findings). |
Step 4: Consists of stating the mixing purpose, which explains what will be undertaken in the study and the purpose of mixing these two approaches. Collins et al. ( Complementarity, which allows for mutual viewpoints about similar experiences or associations. This is to enhance and clarify the findings from one method with the results from another. Completeness, which is to confirm that there is total representation of experiences or that associations are reached. Developmental, which is to develop questions from one method that emerge from the inferences of a prior method or one method presents assumptions that can be tested in a subsequent method. One method informs the development of another method (e.g. interviews inform the development of a survey). Expansion, which is to explain and elaborate on the knowledge gained from a prior method. This adds breadth and scope to a project through the use of various methods for different components, or where one method could be nested within another method to provide insight into different levels of analyses. Corroboration or confirmation, which is to evaluate the trustworthiness of inferences gained from one method. Triangulation is convergence and corroboration of findings from different methods that study the same phenomenon (Morse, Compensation, which is to counter the flaws in one method by using the other. Diversity, which is to find contradictory or opposing viewpoints of the same experiences or associations (Venkatesh et al., |
|
Step 5: Research questions guide the research as they determine the research design in terms of the stages and sequence of collecting the data (Onwuegbuzie & Leech, |
|
Phase 3: Early development and pilot testing: | Step 6: Sampling design: Leech and Onwuegbuzie ( identical, where the same participants’ sample members participate in both the quantitative and qualitative components parallel, as the quantitative and qualitative samples are different but drawn from a common population nested (i.e. sample members selected for one phase of the study represent a subset of participants chosen for the other facet of the investigation) multilevel (i.e. using two or more sets of samples that are extracted from different levels of the study). |
Step 7: Mixed-Methods Design: The data from the quantitative and qualitative components are to be collected concurrently or sequentially. Data from the two components can also be collected partially or fully and can have equal or unequal status (Leech & Onwuegbuzie, |
|
Step 8: Early development and pilot study. | |
Phase 4: Advanced development | Step 9: Data collection: Data are collected, either simultaneously, concurrently or sequentially (Creswell & Plano Clark, |
Step 10: Data analysis: Data analyses can be carried out through either transformation, exploration of articles, and instrument development or examination of multiple levels. | |
Step 11: Data validation: In the case of a sequential design (e.g. in the case where one cycle informs the design of data collection procedure in the second cycle), more data have to be collected, analysed and validated. After the validity has been established in both quantitative and qualitative strands of the research, the researcher has to consider the validity of the mixing process in the entire MMR study. | |
Step 12: Interpretation: interpretation of the findings takes place only once all the data have been collected, analysed and validated. The goal of the interpretation phase is to make meta-inferences from combining quantitative and qualitative inferences (Teddlie & Tashakkori, |
|
Report writing | Step 13 consists of report writing, in which the researchers have to decide how to present the quantitative and qualitative components of the research. It is important to emphasise the contribution of the mixed-methods approach in the report (Creswell et al., |
These aforementioned controversies were later encapsulated in four challenges described by Onwuegbuzie and Dickinson (
Another threat is appropriateness of the sample (Onwuegbuzie, Jiao, & Bostick,
Fourthly, there is a challenge concerning
Many MMR projects end up being published separately as mono-method research. Reasons for this may be that the authors expect readers to have an interest in one specific aspect of the study or because of the strict page and word limitations in some journals. Mixed-methods research is longer than mono-methods research, and researchers may find such limitations challenging to get their work published. In sequential studies, the timing of the various components of the research may also be the cause for not publishing mixed-methods studies as researchers may feel pressured to publish results obtained from earlier phases first.
There may also be logistical challenges, which are related to availability of resources (Onwuegbuzie & Teddlie,
Researchers should, however, not be discouraged by these challenges but should familiarise themselves with the MMR literature and explore the exciting possibilities that mixed-methods designs can offer. The challenges can, however, be turned into advantages as it requires researchers to be versatile and to work in teams and across disciplines. Interprofessional collaborative practice in both health care and education settings is supported across the globe (Garuth,
The MMR approach offers an exciting avenue for exploring multidimensional and complex questions in the disciplines of speech-language therapy and audiology. Speech-language therapists and audiologists serve clients with complex conditions such as speech, language, hearing, balance and swallowing problems. These clients are affected by the physical, social and attitudinal environments in which they live (World Health Organization,
The therapeutic process itself can be viewed as a MMR process, as assessment procedures consist of quantitative data that are complemented by qualitative data obtained from interviews with the clients and their families. A dearth of knowledge continues to exist regarding the most efficacious intervention approaches for clients with a variety of communication disorders.
Interprofessional research is becoming increasingly popular and lends itself to MMR. Researchers working in teams can draw on the strengths of different disciples and their research approaches. Such collaborative enterprises will contribute to capacity building. Researchers, speech-language therapists and audiologists are encouraged to make use of MMR to address the complex research issues in the multicultural, multifaceted South African context. Mixed-methods research makes an important contribution to the understanding of individuals with communication disorders, and in turn, researchers in the two disciplinary fields of speech-language therapy and audiology can contribute to the development of this research approach. Mixed-methods research is well suited to the complexity of South African contexts and its populations, as it can provide multiple perspectives on a topic.
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
A-M.W. was the project leader and prepared the conceptual draft and edited the article. B.L. gave conceptual input and edited the article.