OSCE - How to appraise the evidence

“To believe or not to believe….?”

As doctors, we are called to practise in an increasingly regimented world, buttressed by a seemingly insurmountable massif of clinical guidelines, practice codes, system handbooks, procedural frameworks, decision algorithms…. the list goes on. Reaching the summit of the medical ‘evidence mountain’ can often seem like too much hard yakka. It’s much easier to assume that the guidebook is informed by good quality evidence. But how can we know for sure?

art and science of Critical Appraisal

The art and science of critically appraising research has been discussed at length in the literature – just google ‘critical appraisal’ and you are faced with 2.05 MILLION hits. To avoid reinventing the wheel I’ll take you to base camp and then challenge you to make your own climb up the summit.

Essentially there are only four questions you need to ask about anything anyone says about any aspect of medicine:

1. Does the study have a clear focus? Or has it bitten off more than it can chew?
2. Are the methods used to answer the question valid? Did the study on patient’s opinions actually use patient opinions to inform their results?
3. Are the results important in general? Does the study on healthcare satisfaction based on whether patients are seated on the left-hand versus the right-hand side of the waiting room really matter?
4. Are the results relevant to you? Does it apply to your patient, your practice or your population? The study on the effects of extreme sub zero temperatures on migraines in Antarctica may be interesting, but is it really relevant to your practice in tropical Queensland?

Appraising medical and health research

Medical and health research can be broadly categorised into quantitative and qualitative research; some are mixed. There are (more!) frameworks and algorithms available online to help you sift through how to analyse each type of research – for example, what does p-value mean? What is the significance of the odds ratio? How do you determine sensitivity and specificity? What is the level of bias? Was there triangulation of qualitative commentary? How was thematic analysis conducted?

practice makes perfect

The CASP website provides a comprehensive checklist to help you decide whether to believe or not to believe. These critical appraisal tools are designed to be used when reading the research articles on a variety of study designs. To practice your critical appraisal skills; find a relevant article, identify the appropriate tool and work your way through the appraisal steps.

Happy reading!