Clinical trials use elaborate methods to make sure that everybody does the exact thing as they planned. Measuring treatment fidelity is checking the agreement between study plan and practice. Some health problems require complex changes. How to measure fidelity in trials of complex interventions? Here are some ideas for fidelity checking.
The National Institutes of Health established a workgroup for treatment fidelity, as part of their behaviour change consortium (1999). They surveyed each centre in the consortium to find out which fidelity measures they use in trials. The workgroup recommendations span five areas: study design, training providers, delivery of treatment, receipt of treatment and enactment of treatment skills. They are useful for investigators who want to measure and improve their treatment fidelity. The key areas for our study are design, training, delivery and receipt.
Fidelity in our PINTA study
Our feasibility study has several aims. The first is to estimate parameters for a fully powered clinical trial. Secondly, we also want to know whether our intervention works. As a complex intervention, it targets multiple levels – doctor and patient level. We hope to improve doctors’ practices and patients’ health behaviour. Intervention fidelity in a multi-level study means adhering to different guidelines and processes. Our trainers must deliver uniform training to all learners groups. The doctors must provide consistent interventions to all patients in the intervention group.
Availability of personal portable audio recorders, e.g. smartphones, provides new and exciting opportunities for fidelity checking, but it raises some ethical issues. Doctors and other interventionists can easily record their consultations with patients and email them to researchers for fidelity checking, but email is not safe.
To avoid the potential confidentiality breach, the researchers can ring the doctors, give them a one-sentence brief and ask them what would they respond should this patient appear in their next appointment. Recording such phone calls is not a technical or ethical problem; it is not without limitations, though. Telephonic consultation with researcher in the role of patient does not reflect real life consultations and, as such, cannot be an accurate skills check. Doctors may not want to be called and recorded for quality assurance purposes, even if it’s anonymous and does not affect their income or professional standing.
When designing measures to improve treatment fidelity in our study, we have to consider how they will be perceived by our participants and providers. These are the strategies for monitoring and improving treatment fidelity that we plan to use:
Measuring fidelity in trials of complex interventions is important. It is not technically demanding. Ultimately this becomes a question of personal development and credibility – willingness to have one’s skills analysed and improved is the basis of reflective practice.
What should doctors do differently when screening for alcohol use and delivering brief interventions for agonist patients in primary care? General principles remain the same like for other people, but:
the screening and treatment processes should be more systematic and proactive in all problem drug users, especially in those with concurrent chronic illnesses or psychiatric co-morbidity,
lower thresholds should be applied for both identification and intervention of problem alcohol use and referral to specialist services,
special skills and specialist supervision is required if managing persistent/dependent alcohol use in primary care.
UL researchers have been awarded €300,000 by Ireland’s Health Research Board to develop and evaluate psychosocial interventions for drug and alcohol users in primary care. Psychosocial interventions are described as aimed at reducing consumption behaviour or alcohol-related problems by using psychological approaches
“UL Researchers Undertake Study to Evaluate Impact of Psychology Based Treatments for Drugs Users“
Principal Investigator and Professor of General Practice at UL, Walter Cullen, explains: “The focus of this study is to evaluate the impact of psychology based treatments as opposed to the approach of medicating patients dealing with drug and alcohol addiction. There is a significant knowledge gap in this area internationally and we hope this study will help practitioners in Ireland assist their patients to deal with this issue.”
Led by Dr Jan Klimas Post-doctoral Researcher at the Centre for Interventions in Inflammation, Infection & Immunity (4i) (http://www.4i.ie) hosted by University of Limerick’s Graduate Entry Medical School, the study involves collaborators from a wide range of disciplines and agencies as well as international experts from the UK, USA and Australia.
The study, entitled ‘‘Are Psychosocial Interventions Effective for Problem Alcohol Use among Problem Drug Users’ (the PINTA study) will involve over 20 practices in the Midwest and Eastern regions.
The Centre for Interventions in Infection, Inflammation & Immunity (4i), Graduate Entry Medical School, UL brings together a multidisciplinary team of researchers focused on developing studies that impact health outcomes.
Director of the Centre, Professor Colum Dunne, complimented the team that successfully competed for this funding, adding “this study builds on previous work, also funded by the HRB, that qualitatively explored patients’ and practitioners’ experiences of problem drug and alcohol use. In a recent Cochrane review we identified gaps in the currently available scientific evidence relating to effectiveness treatments for problem alcohol use. This new study will add considerably to that field of practice.”
Read more here …
Ethical principles, ethics committees and research
This has been a busy week from an ethics perspective. All research should be ethical and adhere to the ethical guidelines. The gatekeepers for the ethical principles are usually ethical committees, institutional review boards or panels. Researchers submit their proposals for approval to the committees and they decide whether the proposed research is in line with the universally accepted principles of ethical research. For example: Research carried out on humans should be in compliance with the Helsinki Declaration (http://www.wma.net/en/30publications/10policies/b3/index.html).
In Ireland, we have 25 committees/ boards who can provide ethical approval and we submitted two applications for ethical approval to two different committees this week:
1. Are psychosocial interventions for problem alcohol use effective among problem drug users in primary care (PINTA): A Controlled before-and-after feasibility study
2. Prospective audit of characteristics of opioid overdoses in an emergency ambulance service
…so many trees going to the ethics.
As usual, we submitted the ethics application on the last day of the deadline.
I wasn’t even sure if the cyclist courier would take it on a bike to the addressee, because the package had surely over 2 kilos. Fingers crossed.