Failure can occur at any phase of the lifecycle. However, most candidates fail early in the development process. In the grand scheme of things, there is potentially a pressure to fail early rather than late as less time and resources will have been invested. 

The U.S. Food and Drug Administration and the National Institutes of Health recently developed the Biomarkers, EndpointS, and other Tools (BEST) resource to aid researchers in this particular area. This should further the understanding of the definitions of different biomarkers and their applications could help mitigate future failures during biomarker development. 

Failures during Discovery

A combination of poor methods, selective publication and selective or incomplete reporting contributes to considerable waste in the biomarker discovery process. Two common pitfalls to be aware of during discovery are applying a hypothesis-driven or supervised selection of biomarkers based upon existing knowledge of the disease and feature selection using exhaustive machine learning and statistical modelling searches. Both have their fallacies.

Hypothesis-driven methods when driven by confirmational bias lead to cherry-picking biomarkers. On the other hand, many machine learning techniques can result in overfitting data when the operator applies it blindly to the data. Both can lead to a biomarker that fails to generalise across independent datasets.

Failures during analytical validation

Premature promotion of biomarker potential in advance of comprehensive performance evaluation. One notable example was the discredited Lancet publication on the identification of proteomic patterns in ovarian cancer. The article reported results with the potential to impact the classification, screening practice, and management of ovarian cancer due to the almost perfect discriminating ability of the technique. 

However, the significance of the research was called into question because of its lack of analytical validity. Attention should be paid to the use of independent and representative samples in robust analytical performance testing prior to any claims being made regarding clinical use or impact. 

Failures of clinical validation

The clinical validation stage aims to demonstrate a link between the biomarker and the outcome of interest. Some biomarkers fail at this stage and are shown to have little additional predictive ability when used in a wider clinical setting.  

The regulatory landscape surrounding biomarker approval is not as robust as those for drugs and biologics. Currently, in the US, there is the CLIA regulatory strategy or the Food and Drug Administration (FDA) regulatory strategy to choose from. Both strategies have high technical demands, but proof of effectiveness from trials is not typically required.

The BEST resource by the US FDA and NIH should improve best practices regarding the clinical translation of biomarkers and act to prevent failures by clarifying any uncertainty.

At present, there are very few randomized trials of diagnostic/prognostic/predictive tests. In general, the number of trials is in the range of a few hundred trials. There are fewer guidelines for using biomarkers in clinical practice than those outlining how to conduct clinical trials. 

Therefore, the design, analysis and reporting of studies are less clear. The lack of clear guidelines makes evaluating biomarkers in clinical practice difficult and may result in biased data analyses. 

Defining the clinical need and critical evaluation of the risk-benefit profile. A biomarker that correlates with the action of the therapeutic seems like the ultimate prize. However, the example of hemoglobin A1C reduction with rosiglitazone treatment is a cautionary tale that should remind developers to consider the systemic risk-benefit. 

Rosiglitazone was a thiazolidinedione used in the treatment of Type 2 Diabetes to lower blood sugar levels. It was approved for use on the basis of evidence demonstrating its ability to lower hemoglobin A1C.  However, as a study evaluated multiple studies published in the New England Journal of Medicine in 2007 reported,  Rosiglitazone treatment resulted in ‘significant increase in the risk of myocardial infarction’. 

Diabetes was previously known to be a risk factor for cardiovascular disease, with this being one of the main causes of death in Diabetes patients. In this case, clinical studies used to gain approval were focused on evaluating the impact of treatment on the biomarker and demonstrating efficacy. Still, they were not powered to assess the impact on cardiovascular risk. 

Non-optimised clinical translation

Many gene expression profiles in relation to cancer outcomes have been reported in the literature, many with the potential to impact the management of highly prevalent cancers. However, few of these signatures have moved into clinical development because many studies have a small sample size. The selection for translation into clinical use is often not determined by factors like comparative evidence of performance, merit, and ease of use but by commercial viability.

Discrepancies in access to biomarkers also impact their implementation and depend largely on the geographical distribution of expertise available at different hospitals or medical centres. There are many factors to consider, and the nature of the stages of biomarker development means that researchers at the earlier discovery stage, for instance, may need to be made aware of these considerations.  

Clinical reversal

After implementation into clinical practice continued surveillance provides evidence that the biomarker is not as useful as once proposed, or may even be harmful. Prostate-Specific antigen (PSA) test was considered relatively accurate and standardisable because assays were also simple and easy to use across laboratories. However predictive discrimination was poor, overdiagnosis rates were high (17%-50%) and treatment came with the risk of serious harm. 

Although uncommon, examples of fraud, such as the series of papers retracted by Potti and, more recently, the Theranos scandal, provide us with firm reminders that ethics and the fundamental standards of reproducible research must always exist in biomedical research. Dr Matthew Alderdice explores the scientific reproducibility crisis and outlines how organisations can drive reproducible discoveries in our 'How to solve the Scientific Reproducibility Crisis' article.