validated and/or adequately described?
appropriate to the research question?
clearly described?
accounts for bias?
Does it permit an examination of the research question?
Are the issues clearly described?
Are issues of special importance fully discussed?
the decisions researchers have made, but our criticism should be sympathetic to what is reasonably (or even theoretically) possible.
We have covered a considerable number of these methodological choices in earlier chapters, and many commentators have devised brief checklists for reviewers to help them address methodological issues. Some of these checklists may be found on the websites or in the further reading at the end of this chapter. However, we have also provided a synopsis of the issues in Table 22.1.
Additionally, there are issues specific to or mainly associated with qualitative and quantitative research, and these are presented in Tables 22.2 and 22.3.
Table 22.2 Methodological issues particularly associated with qualitative research.
| Design – | Is a theoretical framework described and justified? |
| Sample – | Where saturation is used to determine eventual sample, are criteria for determining saturation described? |
| Measures – | Is an interview guide given? |
| Procedure – | Is there examination of appropriateness of interviewer behaviour? Is there evidence of reflexivity? |
| Data analysis – | Were independent raters used? Is there an adequate audit trail including details of all stages of decision making in the examination of transcripts and generation of categories and themes? What provision was made for validation of the data? |
| Ethics – | Have precautions been taken to protect the identity of participants where sample sizes are small and quote material is used? |
Table 22.3 Methodological issues particularly associated with quantitative research.
| Design – | Has bias been adequately controlled for? |
| Sample – | Has a power calculation been performed? Was a control group used where appropriate? Have randomisation procedures (where appropriate) been adequately applied? (Do all prospective participants have a fair and equal chance of inclusion?) |
| Measures – | Has a main outcome measure associated with the power calculation been identified? Is it indicative of important clinical change? Does the sample adequately reflect the target population? |
| Procedure – | In outcome comparisons, are raters and/or patients and/or clinicians unaware of the treatment allocation (blinded)? In clinical outcome studies, do the procedures adequately reflect clinical practice? In outcome comparison studies, is there adequate specification of the different treatments (including manualisation and staff training where appropriate)? |
| Data analysis – | Is the statistical approach optimal? |
| Ethics – | If treatment is withheld, is there an adequate rationale for this? |
User involvement in research
We discussed the matter of patient and public involvement in research in Chapters 2 and 5. However, it should be noted that this issue is also important in judging the quality of published research, even though it rarely finds its way into guidelines about how to assess the quality of research. Thus, we recommend that reviewers ask the following questions when reviewing a research paper: how far were patients and carers involved in all stages of the research? If patients and carers were not involved, what implications do these have for the research? If this paper is intended for a general audience, how accessible is the writing to non-specialist readers?
Four broad areas of patient and public involvement in research
Identifying and prioritising research topics
Being part of research advisory groups and steering groups
Undertaking research projects and collecting and analysing data
Reporting and communicating research findings
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
