Why Do Biomarkers Fail?

Why Do Biomarkers Fail?

Sharing Knowledge

Why Do Biomarkers Fail?

Subscribe to stay up to date with the latest Sonrai content.

Author: Dr Joy Kavanagh, Diagnostic Programme Manager | Reading time: 12 Minutes

In this article we explore:

The Potential of Biomarkers
The Biomarker Lifecycle
The Difficulties in Development
Reasons For Failure
Success Stories
Strategies For Success

The Potential of Biomarkers

In Drug Development

The drug development process takes time and money and is fraught with difficulties; many drugs do not make it to the approval stage. However, when biomarkers are used to select clinical trial populations, the probability of successful drug approval increases from 10% to 25%Biomarkers have the potential to decrease the time, costs, and failure rate of drug development.

In Disease Detection

Significant efforts are ongoing to generate reliable and sensitive early-detection biomarkers that allow diseases to be caught early. These efforts have the potential to benefit patients by enabling more effective treatment and screening for predictors of future disease. Effective biomarkers could provide us with greater knowledge of the mechanisms of disease. They could also pave the way for introducing preventative measures such as informed lifestyle changes.

 

In this article, we will discuss why biomarkers may fail at different stages of development. Let’s start with an overview of the biomarker life cycle.

The Biomarker Life Cycle

Fig.1. An overview of the IVD development process. From the point where the potential need for a biomarker is identified to real-world clinical adoption, all biomarkers transition through the key stages of maturity, discovery, development; validation; and adoption. (Adapted from Davis, K.D., Aghaeepour, N., Ahn, A.H. et al. Discovery and validation of biomarkers to aid the development of safe and effective pain therapeutics: challenges and opportunities. Nat Rev Neurol 16, 381–400 (2020). https://doi.org/10.1038/s41582-020-0362-2 (https://www.nature.com/articles/s41582-020-0362-2#Fig2))

All InVitro Diagnostics (IVDs) exist because of a diagnostic, predictive or prognostic clinical need. Therefore, the biomarker life cycle begins with identifying the clinical need, which forms the basis of the IVD’s intended purpose and defines the intended use population. The clinical need will drive the search for datasets to discover candidate biomarkers or putative signatures and confirm representation in the intended use population across multiple studies or cohorts. We discuss defining the intended purpose in our ‘Biomarker Validation’ article. 

 

Once a candidate has been established as representative and specific, a test method is developed. This often occurs in an iterative manner beginning with a prototype test that can be used to demonstrate proof of concept. Developing the test method for an IVD will require identifying and acquiring a sample set representative of the intended use population, selecting technology suited to the biomarker and intended use, and identifying appropriate positive and negative controls. 

 

Analytical validation involves rigorous performance testing of the biomarker and developed test method to ensure robustness. Studies in this phase will use naive samples and will typically evaluate performance in terms of accuracy, precision, sensitivity and specificity. The evidence required to demonstrate the analytical performance of the biomarker and test method depends on the intended use and associated risk-benefit profile. 

 

Clinical validation demonstrated the clinical sensitivity and specificity of the IVD, in other words, its ability to appropriately meet the clinical need it is intended to address. Clinical studies are designed and conducted in accordance with the relevant regulator requirements and appropriate to the intended purpose. Such studies may be retrospective or prospective in nature; for diagnostic and predictive biomarkers, retrospective studies are usually sufficient. 

 

Planning for clinical Adoption of the IVD begins in parallel with the preceding stages, often as soon as the clinical need and proof of concept have been established. Data generation through analytical and clinical validation; clinical utility studies; engagements with regulatory agencies, key opinion leaders and payers lay the groundwork for the eventual adoption of the IVD in the targetted market(s) following approval. 

Sonrai Success Stories

The Sonrai Difference

Every organization is unique, and generic solutions fall short in addressing their distinct challenges. Sonrai is purpose-built to tackle the complexities of precision medicine, offering tailor-made solutions that align with specific needs.

AI Biomarker Discovery Workflow

Explore how easy it is to streamline end-to-end AI biomarker discovery in a no-code environment in Dr Craig Davison's Use Case.

Code-Optional Discovery

Bridge the gap between technical and non-technical researchers, collaborate effectively and leverage strengths to align objectives and timelines.

The difficulties of biomarker development

Several studies highlight the benefits of including biomarkers in drug development. This has generated enthusiasm for biomarkers and massive investments of resources into biomarker development. However, a recent analysis showed that whilst the European Medicines Agency (EMA) published 883 European public assessment reports, only 37 predictive biomarkers for 41 drugs were mentioned. 

 

Generally speaking, there are a few overarching difficulties faced by researchers investing in biomarkers: 

 

  • Skills gaps and lack of expertise underly the difficulty in validating clinically that a marker has utility. Lack of expertise is a process failure that could be avoided by effective planning but does seem to affect the outcome of biomarker development efforts. 
  • Lack of regulators to oversee the masses of data arising from exceedingly parallel assays such as microarrays, mass spectroscopy, or DNA sequencing. Most publications on biomarkers have not been validated clinically by outside investigators, or they fail external validation attempts when they are. 
  • Investigator-specific biases persist because of the omission of the necessary "blinding". Some biases are especially common to initial reports (the "beginner's luck bias" or "declining effect phenomenon"). 

 

In the next section, we outline some of the reasons why biomarkers fail and, in particular, identify at what stage of development this failure tends to occur.

Need some advice?

Meet our expert Biomarker team

We Listen to your problems
No hard sales conversations
We give you confidence to move forward

Reasons for failure

Failure can occur at any phase of the lifecycle. However, most candidates fail early in the development process. In the grand scheme of things, there is potentially a pressure to fail early rather than late as less time and resources will have been invested. 

 

The U.S. Food and Drug Administration and the National Institutes of Health recently developed the Biomarkers, EndpointS, and other Tools (BEST) resource to aid researchers in this particular area. This should further the understanding of the definitions of different biomarkers and their applications could help mitigate future failures during biomarker development. 

 

Failures during Discovery

 

A combination of poor methods, selective publication and selective or incomplete reporting contributes to considerable waste in the biomarker discovery process. Two common pitfalls to be aware of during discovery are applying a hypothesis-driven or supervised selection of biomarkers based upon existing knowledge of the disease and feature selection using exhaustive machine learning and statistical modelling searches. Both have their fallacies.

 

Hypothesis-driven methods when driven by confirmational bias lead to cherry-picking biomarkers. On the other hand, many machine learning techniques can result in overfitting data when the operator applies it blindly to the data. Both can lead to a biomarker that fails to generalise across independent datasets.

 

Failures during analytical validation

 

Premature promotion of biomarker potential in advance of comprehensive performance evaluation. One notable example was the discredited Lancet publication on the identification of proteomic patterns in ovarian cancer. The article reported results with the potential to impact the classification, screening practice, and management of ovarian cancer due to the almost perfect discriminating ability of the technique. 

 

However, the significance of the research was called into question because of its lack of analytical validity. Attention should be paid to the use of independent and representative samples in robust analytical performance testing prior to any claims being made regarding clinical use or impact. 

 

Failures of clinical validation

 

The clinical validation stage aims to demonstrate a link between the biomarker and the outcome of interest. Some biomarkers fail at this stage and are shown to have little additional predictive ability when used in a wider clinical setting.  

 

The regulatory landscape surrounding biomarker approval is not as robust as those for drugs and biologics. Currently, in the US, there is the CLIA regulatory strategy or the Food and Drug Administration (FDA) regulatory strategy to choose from. Both strategies have high technical demands, but proof of effectiveness from trials is not typically required.

 

The BEST resource by the US FDA and NIH should improve best practices regarding the clinical translation of biomarkers and act to prevent failures by clarifying any uncertainty.

 

At present, there are very few randomized trials of diagnostic/prognostic/predictive tests. In general, the number of trials is in the range of a few hundred trials. There are fewer guidelines for using biomarkers in clinical practice than those outlining how to conduct clinical trials

 

Therefore, the design, analysis and reporting of studies are less clear. The lack of clear guidelines makes evaluating biomarkers in clinical practice difficult and may result in biased data analyses. 

 

Defining the clinical need and critical evaluation of the risk-benefit profile. A biomarker that correlates with the action of the therapeutic seems like the ultimate prize. However, the example of hemoglobin A1C reduction with rosiglitazone treatment is a cautionary tale that should remind developers to consider the systemic risk-benefit. 

 

Rosiglitazone was a thiazolidinedione used in the treatment of Type 2 Diabetes to lower blood sugar levels. It was approved for use on the basis of evidence demonstrating its ability to lower hemoglobin A1C.  However, as a study evaluated multiple studies published in the New England Journal of Medicine in 2007 reported,  Rosiglitazone treatment resulted in ‘significant increase in the risk of myocardial infarction’. 

 

Diabetes was previously known to be a risk factor for cardiovascular disease, with this being one of the main causes of death in Diabetes patients. In this case, clinical studies used to gain approval were focused on evaluating the impact of treatment on the biomarker and demonstrating efficacy. Still, they were not powered to assess the impact on cardiovascular risk. 

 

Non-optimised clinical translation

 

Many gene expression profiles in relation to cancer outcomes have been reported in the literature, many with the potential to impact the management of highly prevalent cancers. However, few of these signatures have moved into clinical development because many studies have a small sample size. The selection for translation into clinical use is often not determined by factors like comparative evidence of performance, merit, and ease of use but by commercial viability.

 

Discrepancies in access to biomarkers also impact their implementation and depend largely on the geographical distribution of expertise available at different hospitals or medical centres. There are many factors to consider, and the nature of the stages of biomarker development means that researchers at the earlier discovery stage, for instance, may need to be made aware of these considerations.  

 

Clinical reversal

 

After implementation into clinical practice continued surveillance provides evidence that the biomarker is not as useful as once proposed, or may even be harmful. Prostate-Specific antigen (PSA) test was considered relatively accurate and standardisable because assays were also simple and easy to use across laboratories. However predictive discrimination was poor, overdiagnosis rates were high (17%-50%) and treatment came with the risk of serious harm. 

 

Although uncommon, examples of fraud, such as the series of papers retracted by Potti and, more recently, the Theranos scandal, provide us with firm reminders that ethics and the fundamental standards of reproducible research must always exist in biomedical research. Dr Matthew Alderdice explores the scientific reproducibility crisis and outlines how organisations can drive reproducible discoveries in our 'How to solve the Scientific Reproducibility Crisis' article. 

Biomarker Tips

Top Tips for Successful Biomarker Discovery & Development

For those beginning their biomarker journey - to those who have already embarked and need further guidance on overcoming common challenges.

Success stories

Recently, some notable successes and prospective studies have shed some light on how this field could grow and move forward

 

A great example of success in the field of biomarker development is the uptick in the adoption of gene expression profiles in oncology for prognosis and treatment guidance. Commercially available diagnostic biomarker tests include the gene signatures of MammaPrint and Prosigna in breast cancer, the Oncotype Dx tests (breast, prostate and colorectal) and the Prolaris prostate cancer test. 

 

The key aspect is the extensive and intensive development programmes undertaken for gene expression assays. A thorough analytical validation of the signatures is conducted, which then moves on to a careful clinical validation process involving hundreds of patient samples from multiple collections that benefit treatment decisions for patients.

 

In the cardiovascular field, amongst many important circulating biomarkers, brain (derived) natriuretic protein (BNP) has been recently proven to be useful for diagnosing cardiac failure in acutely dyspnoeic patients. To provide some prognostic information on cardiac disease and to distinguish between cardiac and pulmonary disease. These are in addition to more established monitoring biomarkers, such as LDL cholesterol, which have important implications in clinical care of heart disease.

 

For markers such as BNP to become part of clinical practice has taken almost thirty years of effort (BNP was first identified in 1988). Clinical validation has been established by the study of thousands of patients in many controlled trials of various sizes. Whilst LDL cholesterol is perhaps one of the most well-established biomarkers in clinical practice, there are still controversial aspects of its use. This highlights the need to address what makes a good biomarker in the first place.

 

The therapeutic area of inflammatory diseases provides another example of recent potential success. A recent biomarker has shown it can interpret the results of a number of measures of rheumatoid arthritis (Vectra). Across a variety of different patient populations with rheumatoid arthritis, the test score correlates with traditional measures of disease activity. However, most importantly, changes in the score over as little as two weeks of therapy can predict future clinical responses (measured at six weeks)

 

If the results with Vectra are reproducible, the biomarker could provide two main benefits. Firstly, the avoidance of costly trials of therapy in patients who won't benefit from the new drug, and secondly, to simplify phase 2 exploratory clinical studies with new drugs.

Strategies for Success

Creating a biomarker strategy at the earliest point possible in drug development increases the opportunity for selecting optimal targets and models for the preclinical stage. An early biomarker strategy offers an increased chance to develop a companion diagnostic, which will be required for such translation.

 

An in-depth understanding of the biology associated with a candidate biomarker is imperative. Critical factors to this understanding include:

 

  • Access to well-annotated biospecimens
  • Early attention to the characterization of the biomarker and standardization of assay methods
  • Clinical trial designs that allow early evaluation of the effectiveness of the biomarker in predicting responsiveness to the drug are often referred to as co-development
  • Ability to incorporate advancing technology and emerging data suggesting new or additional biomarkers that better characterize the target populations

 

In conjunction with a sound strategy for discovery, a rigorous validation strategy is needed that takes into account a number of factors:

 

  • Unbiased data analytic strategies for reproducible biomarker discovery
  • Use stratification and dimension reduction to address disease heterogenicity 
  • Study design as a means of minimizing confounding factors
  • Avoid bias in data acquisition and analysis
  • Avoid overfitting during model building and optimization

 

Biomarkers are becoming increasingly important for discovering novel therapeutics. Having a biomarker strategy in place early in the drug development process affords researchers time to select optimal targets that are more likely to reach the clinical setting. A clear strategy for biomarker development that considers the various pitfalls of the discovery, analytical validation and clinical validation stages of biomarker development maximizes the chances for the biomarkers to translate to the clinic. 

Sonrai-ICON

End-to-end data solution for Biomarker Discovery

We have solutions to improve every stage of your pipeline lifecycle and an expert team excited to support you. Achieving excellence has never been easier.

References

Related Posts