Quality in Qualitative Research

Keywords:

Rigor, trustworthiness, internal, external, validity, reliability, credibility, transferability, coherence, replicability, dependability, authenticity, plausibility, confirmability, triangulation, consistency, neutrality, objectivity, applicability, fittingness, auditability, consistency, replicability.

)|(

To establish the “trustworthiness” of findings from studies using qualitative methods, Lincoln and Guba (1985) propose that we assess the research in terms of its:

Credibility. The “truth” of the findings, as viewed through the eyes of those being observed or interviewed and within the context in which the research is carried out.
Transferability. The extent to which findings can be transferred to other settings. In order for findings to be transferable, the contexts must be similar. Therefore, it is the role of the researcher to identify key aspects of the context from which the findings emerge and the extent to which they may be applicable to other contexts.
Dependability. The extent to which the research would produce similar or consistent findings if carried out as described, including taking into account any factors that may have affected the research results.
Confirmability. Researchers need to provide evidence that corroborates the findings. Such evidence should come directly from subjects and research context, rather than the researcher’s biases, motivations, or perspectives.

(Devers, 1999)

)|(

(Creswell & Miller, 2000)

)|(

Reliability and Validity in Quantitative Research

Reliability means consistency or the degree to which a research instrument measures a given variable consistently every time it is used under the same condition with the same subjects. It is important to note that reliability applies to data not to measurement instruments. From different perspectives or approaches, researchers can evaluate the extent to which their instruments provide reliable data.

Validity refers to the accuracy of research data. A researcher’s data can be said to be valid if the results of the study measurement process are accurate. That is, a measurement instrument is valid to the degree that it measures what it is supposed to measure.

To judge the quality or (a) credibility and (b) dependability of a qualitative study, the following questions compiled from various studies can be asked (Miles & Huberman, 1994, pp. 278–279):

Credibility (instead of validity) questions:

How context-rich and detailed are the basic descriptions?
Does the account ‘ring true’, make sense, seem convincing or plausible, enable a ‘vicarious presence’ for the reader?
Is the account rendered comprehensive, respecting the conﬁguration and temporal arrangement of elements in the local context?
Did triangulation among complementary methods and data sources generally lead to converging conclusions? If not, is there a coherent explanation for this?
Are the presented data linked to the categories of prior or emergent theory if used?
Are the ﬁndings internally coherent and concepts systematically related?
Were guiding principles used for conﬁrmation of propositions made explicit?
Are areas of uncertainty identiﬁed?
Was negative case or evidence sought for? Found? What happened then?
Have rival explanations been actively considered? What happened to them?
Were the conclusions considered to be accurate by the participants involved in the study?
If not, is there a coherent explanation for this?

Dependability (instead of reliability) questions:

Are research questions clearly deﬁned and the features of the study design congruent with them?
Are basic paradigms and analytic constructs clearly speciﬁed?
Are the researcher’s role and status within the site explicitly described?
If multiple ﬁeld-researchers are involved, do they have comparable data collection protocols?
Do multiple observers’ accounts converge, in instances, settings, or times when they might be expected to?
Were data connected across the full range of appropriate settings, times, respondents suggested by the research questions?
Were coding checks made and did they show adequate agreements?
Were data quality checks for bias, deceit, informant knowledgeability etc. made?
Do ﬁndings show meaningful parallelism across data sources (informants, contexts, and times)?
Were any forms of peer or colleague review employed?

To assess the rigour of qualitative research, Lincoln and Guba (1985) resort to the concepts of credibility, transferability, dependability, and conﬁrmability to express the quantitative concepts of internal validity, external validity (generalisability), reliability, and objectivity respectively. Credibility means that the participants involved in the study ﬁnd the results of the study true or credible.

Transferability is achieved if the ﬁndings of a qualitative study are transferable to other similar settings. Thick description of the setting, context, people, actions, and events studied is needed to ensure transferability or external validity in quantitative terms. The study has dependability (reliability) if the process of selecting, justifying and applying research strategies, procedures and methods is clearly explained and its effectiveness evaluated by the researcher and conﬁrmed by an auditor, which is called ‘audit trail’. The study enjoys conﬁrmability when its ﬁndings are based on the analysis of the collected data and examined via an auditing process, i.e. the auditor conﬁrms that the study ﬁndings are grounded in the data and inferences based on the data are logical and have clarity, high utility or explanatory power.

In short, terms such as credibility, trustworthiness, authenticity, neutrality or conﬁrmability, dependability, applicability or transferability and the like are those that qualitative researchers use most in their discussion of the concepts of reliability and validity

(Yilmaz, 2013)

)|(

While some argue for the same criteria as quantitative research (Morse et al., 2002), others argue for different criteria (Sandelowski, 1986; Koch and Harrington, 1998), while still others reject any pre-determined criteria (Hope and Waterman, 2003; Johnson and Waterﬁeld, 2004; Rolfe, 2006).

While it has been argued that each approach requires different criteria (Koch and Harrington, 1998), the commonly accepted criteria that are applied across approaches are summarized Table 4. Transferability is contingent on credibility, which in turn is contingent on dependability and conﬁrmability. Trustworthiness refers to the conﬁdence or trust one can have of a study and its ﬁndings (Robson, 2011) and is determined by those assessing a study (c.f. quantitative research that refers to validity which is judged by researcher).

(Petty, Thomson, & Stew, 2012)

References

Creswell, J. W., & Miller, D. L. (2000). Determining Validity in Qualitative Inquiry. Theory into Practice, 39(3), 124–130. http://doi.org/10.1207/s15430421tip3903_2

Devers, K. J. (1999). How Will We Know “Good” Qualitative Research When We See It? Beginning the Dialogue in Health Services Research. HSR: Health Services Research, 34(5), 1153–1188. http://doi.org/http://www.hsr.org/

Petty, N. J., Thomson, O. P., & Stew, G. (2012). Ready for a paradigm shift? Part 2: Introducing qualitative research methodologies and methods. Manual Therapy, 17(5), 378–384. http://doi.org/10.1016/j.math.2012.03.004

Yilmaz, K. (2013). Comparison of Quantitative and Qualitative Research Traditions : epistemological , theoretical. European Journal of Education, 48(2), 311–325. http://doi.org/doi:10.1111/ejed.12014