Session Five: How are clinical guidelines produced?

in

This episode is a live recording of the fifth session of #UnblindingResearch held in DREEAM 27th June 2018.  The group work has been removed for the sake of brevity.

This session of #UnblindingResearch looks at the different levels of evidence and how our clinical guidelines are produced using them.  Here are the slides for this session (p cubed of course), you can move between slides by clicking on the side of the picture or using the arrow keys.

Here's our Take Visually for this session:

There are levels of research evidence usually represented in a pyramid.  The higher up the pyramid the greater the evidence base, through the use of controls, randomisation and greater statistical analysis.  The higher up the pyramid the harder and more expensive the study is to perform and so there are fewer examples of the study in question.  The session contained an exercise to sort fictitious examples of research into order of evidence from lowest to highest.  These are the examples in the right order:

  • We present an article written by the RCN President who argues that research nurses should be given free sweets as a sign of good will which will undoubtedly cause their recruitment to studies to go up.

(Opinion)

  • We report the case of a research nurse who after being given 3 wine gums a day as well as their normal lunch had increased their recruitment to studies by 24%.

(Case report). Case reports can help identify new trends or diseases such as the MMWR in June 1981 which first described what would become  known as AIDS.  They also serve an educational purpose.  However, they are not generalisable and may focus on the rare and not actually that useful.

  • We present our study into access to sweets and research nurse retainment.  10 research nurses began working at our research department in May 2017.  They each received a free bag of wine gums on arrival.  By May 2018 6 of the nurses remained in post.  

(Case series)  Case series look at participants with a known exposure.  They have no comparison.  

  • We report our study into access to sweets and research nurse retainment.  We compared research nurses who left their post within a year to those who have remained in post after a year.  We found that the nurses who remained in post ate on average 1 packet of wine gums more a week than nurses who left their post.

(Case-control)  These compare participants with a known outcome with those who do not have that outcome.  It looks back to see the relationship between a risk factor or exposure and that outcome. As we know the outcome has occurred these are quicker and are useful for initial studies and rare diseases.  As they are retrospective they are susceptible to recall bias and are not good for diagnostic tests because we know the outcomes already.

  • We report our study into research nurse intake of sweets.  We compared a group of research nurses to a group of educator nurses and a group of staff nurses.  We found that research nurses eat on average 3.4 sweets more each than educator nurses and 1.2 more than staff nurses.     

(Cohort study) A study where one or more samples (cohort) are compared prospectively to assess the effects of certain factors on a particular outcome.  Participants in a group can be matched with a subject in another group with similar demographics so as to limit variables.  Cohort studies are easier to carry out than randomised control trials.  However, there is no randomisation, they may take a lot of time and are susceptible to confounding factors.

  • We present our study into sweets and research nurse recruitment to studies.  We compared Maynard wine gum to an identical tasting placebo.  We randomly allocated 100 research nurses into 2 groups of 50, on group had Maynard’s every day whilst the other group had the placebo.  Neither the participants or study staff knew the allocation.  The Maynard group recruited 12% more patients to studies over a 6 month period. 

(Randomised control trials)  RCTs randomly assign participants to either the treatment or placebo group.  Blinding is usually involved.  These trials are expensive and can't prove causation.

  • We report our study to determine the optimum sweets for research nurse productivity.  We performed a literature review and critically appraised 200 randomised controlled studies of sweets and research nurse productivity.  We find that that Maynard’s wine gums are the superior sweet for research nurse productivity.  

(Systematic review) These involve an exhaustive review of the current literature. They take less time than a new study and results can be extrapolated into general population more broadly than other studies.  However. they are very time consuming and researchers may not be able to combine some studies.

There was then a discussion about systematic reviews and meta-analysis. Systematic reviews are a type of literature review that use systematic methods to collect secondary data and critically appraise research studies to create an exhaustive summary of current evidence.  Meta-analyses assumes a common truth between a variety of different studies.  They use statistical methods to find a pooled estimate close to the common truth.  This led to The Cochrane and how they perform systematic reviews.  

The session then looked at how guidelines are formed using the example of NICE and their own protocol for producing clinical guidelines.  They broadly follow these steps:

  1. Choose a topic
  2. Produce the scope
  3. Develop a guideline using a literature review and considering costings
  4. Consult and revise the guidelines
  5. Sign-off and publish
  6. Update

There was a brief discussion about the limits of guidelines using the FeverPAIN Score as an example before looking at how the cost effectiveness of a treatment is used through 'Quality Adjusted Life Years' or QALYs.  QALY is measured on a scale from 0 (death) to 1 (perfect health).  A year of perfect health is 1 QALY.  Health is calculated using the EQ-5D model which has five dimensions (mobility, self-caring, activities of daily living, absence of pain/discomfort and absence of anxiety/depression.  Time is then factored in to make a QALY.  This means 2 years of half perfect health would be 1 QALY as would 4 years of 0.25 perfect health and so on. 

We then ended with an activity attempting to create guidelines for the fictional disease 'Maynard's disease'. 

You are a committee formed by NUH NHS Trust to create guidelines to protect our research nurses against Maynard’s disease

Maynard’s disease is an acquired condition effecting research nurses who do not get enough wine gums.  However, wine gums are expensive and contain a lot of sugar.  You want to create UK guidelines for the right number of wine gums needed to prevent Maynard’s disease whilst not causing harm to your nurses.  

Look at the evidence below and create your guidelines.  Be ready to explain your choice

Would you want to do any studies yourself to fill any holes in the evidence?

(There isn’t a right answer here, it’s just the process (and maybe debate))

  • A systematic review of RCTs in Japan recommended 5 wine gums a day reducing Maynard’s disease by 65% with a 10% rate of diabetes
  • A research unit in Leicester gave their nurses 10 wine gums a day for a year.  None of them have developed Maynard’s disease or diabetes
  • A single blind randomised control trial in the UK found that 2 wine gums a day reduced Maynard’s by 50% with a 2% rate of diabetes
  • A double-blind randomised control trial in the UK found that 3 wine gums a day reduced Maynards by 60% with a 5% rate of diabetes
  • A research unit in London has looked back at all their nurses employed in the past 15 years.  Their  nurses who developed Maynard’s disease ate on average 2 wine gums a day or fewer.  No nurse who ate 4 or more wine gums a day developed Maynard’s.  Their nurses who developed diabetes and obesity ate on average 5 or more wine gums a day.  None of their nurses who ate 2 or fewer wine gums a day developed diabetes
  • A 60 year old research nurse in Scotland has never eaten a single wine gum and has never developed Maynard’s disease.  She does have diabetes though
  • A meta-analysis of RCTs in the USA recommended 4 wine gums a day to reduced Maynard’s disease by 60% with a 15% rate of diabetes
  • A study in France compared research nurses with Maynard’s with research nurses without Maynard’s and found that the nurses without Maynard’s ate on average 2 wine gums a day.  
  • A study in Germany compared research nurses with diabetes and research nurses without diabetes and found that the nurses with diabetes ate on average 3 wine gums a day
  • A multi-centre double blind RCT in Europe has found that 2 or more wine gums a day offer no benefit against Maynard’s disease for nurses over 35 but increases the risk of diabetes by 1%