Evidence synthesis TSD series

The intention behind the TSDs is not to be prescriptive, but rather to explain the requirements for evidence syntheses set out in the and to provide guidance on methods that meet these requirements.

TSDs 2, 3, 4 and 5 include an extensive set of worked examples of analyses using Bayesian Markov chain Monte Carlo methods, using WinBUGS. The WinBUGS code should not be copied and pasted from the text; users are instead advised to download the WinBUGS system (.odc) files.

When all seven documents have been completed, those already on the website will be updated in order to harmonise and complete the cross-referencing. Instructions for citing these documents can be found on page 3 of each TSD.

TSD 18

A standard methodology for indirect comparisons is now used routinely in submissions to NICE. This allows estimates on the relative effect of C vs B to be estimated from an AB trial and an AC trial.

Recently, publications have appeared on “Matching-adjusted Indirect Comparisons” (MAICs) and on “Simulated Treatment Comparisons” (STCs). The aim of these methods is to hold out the promise of improving on standard indirect comparisons by using individual patient data (IPD) in a subset of trials to control for imbalances in “baseline characteristics” in the trial evidence.

These methods are now being used increasingly in STA submissions to NICE, particularly when the manufacturer has access to individual patient data on its own AB trial, but only aggregate data on the competitor’s AC trial.

The purpose of this TSD is to comment on the reliability of these methods and to provide recommendations on their use in the context of NICE appraisals.

TSD 20

The TSD describes the use of multivariate meta-analytic methods for synthesis of data from multiple correlated outcomes as well as for evaluation of surrogate endpoints, in the context of HTA.

Health technology assessment (HTA) agencies such as the National Institute for Health and Care Excellence (NICE) require evidence synthesis of existing studies to inform their decisions. For example; gathering information about the best available treatments with respect to multiple efficacy and safety outcomes. However relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Studies that do not provide direct evidence about a particular outcome or treatment of interest are often excluded from a meta-analysis evaluating that outcome or treatment.

This exclusion is unwelcome, especially if participants are otherwise representative of the population, clinical settings and condition of interest. Research studies require considerable costs and time, and involve precious patient participation, and simply discarding them could be viewed as research waste. Statistical models for multivariate and network meta-analysis address this by simultaneously analysing multiple outcomes and multiple treatments, respectively. This allows more studies to contribute toward each outcome and treatment comparison.

This Technical Support Document (TSD) describes the key methods of multivariate meta-analysis and their extensions to network meta-analysis of multiple outcomes. We focus on the use of multivariate meta-analytic methods for combining data from multiple correlated outcomes with the aim of including all relevant evidence and borrowing of information across outcomes, in particular when not all of the relevant studies report an outcome of interest. We also devote considerable attention to the use of multivariate meta-analysis for the purpose of surrogate endpoint evaluation.

We describe and apply the methods in the Bayesian framework of estimation and provide WinBUGS code for a Bayesian analysis using Markov chain Monte Carlo (MCMC) simulation.

TSD 25

This TSD describes methods for meta-analysis of diagnostic test accuracy, distinguishing between situations where (1) only a single estimate of sensitivity and specificity is available from each study, or (2) some studies report accuracy at multiple explicit diagnostic threshold values. It additionally discusses use of meta-analysis results in decision models, including choice of output to incorporate, identification of the optimal threshold according to some criterion, and the critical role of prevalence in the latter.

91ֱ��

NICE Decision Support Unit

NICE Decision Support Unit