Fidelity questions

Clinical trials use elaborate methods to make sure that everybody does the exact thing as they planned. Measuring treatment fidelity is checking the agreement between study plan and practice. Some health problems require complex changes. How to measure fidelity in trials of complex interventions? Here are some ideas for fidelity checking.

The National Institutes of Health established a workgroup for treatment fidelity, as part of their behaviour change consortium (1999). They surveyed each centre in the consortium to find out which fidelity measures they use in trials. The workgroup recommendations span five areas: study design, training providers, delivery of treatment, receipt of treatment and enactment of treatment skills. They are useful for investigators who want to measure and improve their treatment fidelity. The key areas for our study are design, training, delivery and receipt.

Fidelity in our PINTA study

Our feasibility study has several aims. The first is to estimate parameters for a fully powered clinical trial. Secondly, we also want to know whether our intervention works. As a complex intervention, it targets multiple levels – doctor and patient level. We hope to improve doctors’ practices and patients’ health behaviour. Intervention fidelity in a multi-level study means adhering to different guidelines and processes. Our trainers must deliver uniform training to all learners groups. The doctors must provide consistent interventions to all patients in the intervention group.

Availability of personal portable audio recorders, e.g. smartphones, provides new and exciting opportunities for fidelity checking, but it raises some ethical issues. Doctors and other interventionists can easily record their consultations with patients and email them to researchers for fidelity checking, but email is not safe.

To avoid the potential confidentiality breach, the researchers can ring the doctors, give them a one-sentence brief and ask them what would they respond should this patient appear in their next appointment. Recording such phone calls is not a technical or ethical problem; it is not without limitations, though. Telephonic consultation with researcher in the role of patient does not reflect real life consultations and, as such, cannot be an accurate skills check. Doctors may not want to be called and recorded for quality assurance purposes, even if it’s anonymous and does not affect their income or professional standing.

When designing measures to improve treatment fidelity in our study, we have to consider how they will be perceived by our participants and providers. These are the strategies for monitoring and improving treatment fidelity that we plan to use:

Design:

  • Guidelines for primary care providers to manage problem alcohol use among problem drug users
  • Scripted curriculum for the group training of providers

Training:

  • Booster session (practice visits) to prevent drift in provider skills
  • Access of providers to research staff for questions about the intervention
Delivery:

  • Instructional video of patient–doctor interaction to standardize the delivery
  • Cards with examples of standard drinks and scripted responses – to standardize the delivery
  • Question about patient scenario in follow-up questionnaires (telephone contact)
Receipt:

  • SBIRT checklist for providers (process measure)
  • Pre- and post training test (knowledge measure)
  • Patient follow-up questionnaire will check whether each component of the intervention was delivered

Measuring fidelity in trials of complex interventions is important. It is not technically demanding. Ultimately this becomes a question of personal development and credibility – willingness to have one’s skills analysed and improved is the basis of reflective practice.