Reproducibility of Preclinical Academic Studies
This op-Ed piece by Dr. Bing-Cheng Wang, co-Leader of the GU Malignancies Program of the Case Comprehensive Cancer Center and Professor of Medicine at CWRU and MetroHealth Medical Center, comments on a global issue in scientific publications- concern over reproducibility in lab findings and the complexity of validation studies. This has implications in how we conduct research. Lab leaders should especially consider its implications.
Recently the biomedical research community was shaken by the widely publicized findings that an alarming proportion of peer-reviewed preclinical studies are not reproducible. The problem surfaced after publication of two commentaries detailing the inability of scientists at pharmaceutical companies to reproduce results from the published literature in 67% to 90% of cases (Nat. Rev. Drug Discov.10, 712, 2011; Nature 483, 531-533, 2012). Many of the studies are in the field of oncology. Surprisingly, the reproducibility of published data did not significantly correlate with journal impact factors, the number of publications on the respective target, or the number of independent groups that authored the publications.
The reaction was strong and swift. Within months, the National Institute of Neurological Disorders and Stroke (NINDS) issued a call for transparent reporting on preclinical animal studies (Nature 490:187, 2012). The guidelines recommend that at a minimum studies should report on sample-size estimation, whether and how animals were randomized, whether investigators were blind to the treatment, and the handling of data. NIH, which funds most of the studies in question, is mulling new rules for validating key results. There are suggestions that key findings in preclinical studies need to be validated before a grant can be funded - perhaps for applications that are likely to lead to clinical trials. Companies such as Science Exchange are already starting to perform validation studies at an estimated cost of $25,000 for each major paper (Nature 500:14, 2013). In an editorial last week, Science announced that the journal will now adopt NINDS recommendations in reviewing future manuscripts (Science 343:229, 2014), following earlier enforcement by Science Translational Medicine. More journals are likely to follow suit.
A larger question is the underlying causes for the irreproducibility problem. Although an occasional fraudulent study is possible, the sheer number of irreproducible papers suggests other factors. The experimental design and data handling, as pointed out by NINDS, are likely contributing causes. Another factor can be the verification process itself. Cutting-edge research can be very hard to reproduce in the run-of-the-mill labs. The less obvious and less tractable factors include the heterogeneous experimental conditions. This latter possibility is partially born out in a new study (Nature 504:389, 2013) specifically designed to investigate inconsistencies between two high-profile papers examining 500 different cancer cell lines for genomic signatures and their association with sensitivities to cancer drugs (Nature483:570, 2012; Nature 483:603, 2012). While genomic data are found to be consistent between the two papers, significant discrepancies are seen in drug sensitivities and their association with genomic features, even when the similar assays are performed on the same cell types, indicating methodological variables may contribute to the discordant results. Therefore, we are only in the early phase of the evolving controversies surrounding irreproducible results. Finding the causes and resources to remedy them without restricting creative process of scientific research will remain a challenge in the foreseeable future.
If you are interested in writing an editorial for our newsletter, please contact Gillian Irwin.
CASE CCC IN THE NEWS
HOT TOPICS 2014: Dermatology (video)
MedPage Today - Jan 18, 2014
What do you anticipate will be the most important clinical development in dermatology in 2014? We asked three high-profile specialists in the field for their views..Kevin Cooper, MD, chair of the department of dermatology at University Hospitals Case Medical Center..Their answers, presented in this video, focus on a new tattoo removal laser, melanoma therapies, and new changes in the way dermatologists practice.
Dr. Smitha Krishnamurthi on the Phase III RAINBOW Trial (video)
OncLive - Jan 17, 2014
Smitha Krishnamurthi, MD, a medical oncologist and associate professor of medicine at University Hospitals Case Medical Center and Case Western Reserve University, discusses the results of the phase III RAINBOW trial that explored ramucirumab plus paclitaxel as a second-line treatment in gastric cancer.
Seven Studies at SABCS Make Dr. Jame Abraham's List of 'Practice-Changing' Talks
The ASCO Post- Jan 15, 2014
From December 10 to 14, the American Association for Cancer Research, the Cancer Therapy & Research Center at The University of Texas Health Sciences Center at San Antonio, and Baylor College of Medicine once again hosted the San Antonio Breast Cancer Symposium (SABCS), presenting state-of-the-art breast cancer research to an audience of more than 7,000 clinicians, investigators, and advocates. Jame Abraham, MD, Director of the Breast Oncology Program at the Taussig Cancer Institute of the Cleveland Clinic in Ohio, told The ASCO Post what he considers the biggest news to emerge from the meeting.
Combo Active in Neuroendocrine Cancers
MedPage Today - Jan 16, 2014
More than 90% of patients with neuroendocrine tumors had objective responses or stable disease when treated with an off-the-shelf chemotherapy doublet, results of a small study showed...The trial, though small and ongoing, has yielded interesting results thus far, said press briefing moderator Smitha Krishnamurthi, MD. "The CAPTEM regimen led to responses in neuroendocrine tumors that are typically resistant to chemotherapy, including five of 12 patients with carcinoid tumors," said Krishnamurthi, of University Hospitals Case Medical Center in Cleveland. "This regimen is being investigated versus temozolomide in a cooperative group trial of patients with pancreatic neuroendocrine tumors."
Agent Adds 2 Months to Gastric Ca Survival
MedPage Today - Jan 14, 2014
Patients with advanced gastric cancer lived 2 months longer when treated with an investigational angiogenesis inhibitor and chemotherapy, a large randomized trial showed...The trial is the only one reported to date to demonstrate as much as a 2-month survival benefit for any regimen in second-line treatment of gastric cancer, said program moderator Smitha Krishnamurthi, MD, of University Hospitals Case Medical Center in Cleveland.
Trial Design, Clinician Advocacy Key to Acceleration of Research, Review Processes
Helio/HemOnc Today - Jan 10, 2014
The time required for an investigational treatment to move from clinical trials through FDA review is understandably lengthy due to the planning, peer review and quality control necessary to ensure efficacy and safety..."ASCO has sought to address a number of factors involved in clinical trials and study design," Neal Meropol, MD, division chief of hematology and oncology, at University Hospitals Case Medical Center, director of clinical research at Case Comprehensive Cancer Center at Case Western Reserve University and one of the executive editors of the ASCO report, told HemOnc Today...Dr. Dale Shepard provides point for the FDA streamlining their approval process.
Friday Feedback: Does 'Cancer Report' Improve Practice?
MedPage Today - Jan 10, 2014
Friday Feedback is a feature that presents a sampling of opinions solicited by MedPage Today in response to a healthcare issue, clinical controversy, or new finding reported that week. This week, the topic is the latest results from the American Cancer Society's annual review of cancer statistics called the Cancer Report. Participants include Mikkael Sekeres, MD, director, Leukemia Program at Cleveland Clinic
IBIS-II results expected to change role of aromatase inhibitors in breast cancer prevention (video)
Helio/HemOnc Today - Jan 10, 2014
Jame Abraham, MD, FACP, director of the Breast Cancer Program at the Cleveland Clinic Taussig Cancer Institute, reviews data from the IBIS-II trial, which showed anastrozole reduced breast cancer occurrence in high-risk postmenopausal women by 53%.
2014 Forecast Series-Insights from Neal Meropol, MD
OBR Oncology - Jan 2014
We spoke with Neal Meropol, MD, the Division Chief, Hematology and Oncology, University Hospitals Case Medical Center; Associate Director for Clinical Research of the Case Comprehensive Cancer Center; and Associate Director, Clinical Programs, UH Seidman Cancer Center, to get his insights into the outlook for 2014.
CWRU Researchers Lead amfAR Teams to Research Ways to Eliminate HIV
Drs. Eric Arts and Jonathan Karn received research funding from amfAR, The Foundation for AIDS Research, to lead collaborative research teams as part of amfAR's Research Consortium on HIV Eradication (ARCHE) program, an initiative launched in 2010 focused on exploring potential strategies for eliminating HIV infection.
Dr. Arts, Professor of Medicine at CWRU and director of the Uganda Laboratory Core at CWRU's Center for AIDS Research, will collaborate with Dr. Ronald Veazey at the Tulane Primate Center in Covington, Louisiana and a team of researchers in the United Kingdom to test an experimental vaccine in monkeys with simian immunodeficiency virus (SIV), with the hope that their research will one day cure HIV in humans.
Dr. Karn, Professor and Chairman of Molecular Biology and Microbiology at CWRU, will work with Dr. Nicolas Chomont of the Vaccine and Gene Therapy Institute of Florida and Dr. Steven Deeks of the University of California San Francisco, to study how gender-based differences can affect how HIV is cured. [more]
Case Comprehensive Cancer Center Team Discovers Key Mechanisms to Inhibit Triple Negative Breast Cancers
A team of researchers led by Drs. Khalid Sossey-Alaoui, Department of Molecular Cardiology at the Cleveland Clinic Lerner Research Institute, and William Schiemann, Associate Professor in the Division of General Medical Sciences-Oncology at CWRU School of Medicine and co-leader of the Breast Cancer Program at the Case Comprehensive Cancer Center, have identified critical complex mechanisms involved in the metastasis of deadly "triple negative" breast cancers (TNBC). These tumors are extremely difficult to treat, frequently return after remission, and are the most aggressive form of breast cancer in women. The discovery of this critical interaction of mechanisms could be used to develop new life saving treatments to kill metastatic tumors in TNBC. [more]
Researchers Receive Grant to Build Complex Polymer Nanostructures on Plant Virus Scaffolds
Researchers at Case Western Reserve University have received a $540,000 federal grant to devise methods for building minute structures tailored to precisely deliver medicines to tumors or carry dyes that help imaging technologies detect disease, create more efficient nanowires and nanoelectronics, and more. Dr. Jon Pokorski, Assistant Professor of Macromolecular Science and Engineering, and Dr. Nicole Steinmetz, Assistant Professor of Biomedical Engineering and member of the Case Comprehensive Cancer Center, will use the three-year grant from the National Science Foundation's Macromolecular, Supramolecular and Nanochemistry Program to test three methods of synthesizing rod-shaped nanostructures. [more]
Amy Zhang Honored with Distinguished Researcher Award
Congratulations to Dr. Amy Zhang, Associate Professor in the School of Nursing, who recently received the Distinguished Researcher Award from the Pain and Symptom Management Research Section the of Midwest Nursing Research Society. Dr. Zhang's research focuses on quality of life and survivorship of cancer patients.
Leukemia & Lymphoma Society Translational Research Program
The Leukemia & Lymphoma Society is accepting Letters of Intent for the Translational Research Program, which funds new and innovative research that shows high promise for translating basic biomedical knowledge to clinical application. The goal of translational research is to reduce the time between laboratory findings and actual treatment.
LOI Deadline: February 15
Application Deadline: March 1
Bladder Cancer Advocacy Network
The Bladder Cancer Advocacy Network is accepting applications for the 2014 Young Investigator Awards. These awards support the development of outstanding research scientists and clinical cancer research investigators who have demonstrated a commitment to improving the understanding and treatment of bladder cancer. Investigators may be working in basic, translational, clinical, epidemiologic, bioengineering or any other field, but must be working in a research environment capable of supporting transformational bladder cancer research.
Deadline: March 5
Brian Werbel Memorial Fund
Applications are currently being accepted for summer cancer research fellowship fundingWerbel2014, provided by the Brian Werbel Memorial Fund. This fund was established in 2007 at University Hospitals Case Medical Center, and is dedicated to providing summer fellows an opportunity to focus on cancer research, with the ultimate goal to aid in curing cancer. Brian's hope was to find a cure for cancer – to make a difference in the world, just as he made a difference in the lives of everyone he touched. Applicants must be conducting cancer research with a Case Comprehensive Cancer Center member. Awardees are required to attend the Annual Golf Outing Event, typically held in July.
Deadline: April 25
HHMI Investigator Program National Competition
The Howard Hughes Medical Institute (HHMI) announced last week that it seeks to appoint up to 25 new biomedical researchers through a national open competition. The initiative represents an investment of approximately $150 million in basic biomedical research over the next five years. The HHMI investigator competition is open to basic researchers and physician scientists at more than 200 eligible institutions who study significant biological problems in all of the biomedical disciplines, including plant biology, as well as in adjacent fields such as evolutionary biology, biophysics, chemical biology, biomedical engineering, and computational biology. Those selected in this competition will receive a five-year appointment to HHMI, which is renewable pending favorable scientific review.
Deadline: June 3