Wednesday, January 07, 2009

Has Bioinformatics Hit A Hard Wall of Stagnation?

Is the field of bioinformatics in trouble?

I ask this due to two disturbing trends. The first is the misuse of bioinformatics for "dubious" purposes. The second is, has bioinformatics itself reached a wall of stagnation due in part to disciplinary insularity and resultant inadequate collaboration with its medical counterpart?

Terminology note: Bio-informatics is the application of IT to the field of molecular biology, as opposed to the broader Bio-medical Informatics. Biomedical informatics includes both bioinformatics and my field, Medical informatics, which deals with application of IT to patients, along with fields at different granularities such as public health informatics. See a diagram of these relationships in the PDF article here.

On the first issue, consider the following:

In "A Professor and an Anti-Aging Tonic", Roy Poses commented on an academic who may have violated the trust of patients by getting entangled with commercial interests, in this case with what seems a magic potions operation. He wrote:

Academics have long been trusted to seek and disseminate the truth in a spirit of free enquiry. Academics who use their reputations to sell products, and take advantage of their reputation thus to enrich themselves, violate this trust. When the academics who do so are medical academics (even if they are not physicians), they also violate the trust of patients.

I will make observations about this story from different angles than Poses', that of misuse of a scientific discipline itself, and that of stagnation and insularity within that discipline, namely, bioinformatics.

In the WSJ story, it is noted that the protagonist, Harvard Medical School professor Dr. David Sinclair has a PH.D. in biochemistry and molecular genetics but isn't a medical doctor.

Also noted in the article:

In obtaining the backing of Dr. Sinclair this summer, Shaklee scored a coup. Dr. Sinclair knows resveratrol; in 2006, he led a study showing the molecule could counteract the ill effects of overfeeding laboratory mice. One notable benefit: resveratrol let overweight mice live about 114 weeks on average, compared with 102 weeks without the chemical.


Ph.D. in biochemistry and molecular genetics, and tests in mice.

If it works in mice, it's only a sign that human testing is then merited, of course. Well known from drug trials is that what works in animals often does not work, or can even be harmful to human subjects. Yet here we went from mice to people directly. A misuse of a scientific discipline, by either the company or the scientist, or both. Abracadabra!


Abracadabra!


This is indeed an unusual example. However, it seems many bioinformatics findings touted in the press and in biotech prospectuses are of associations that probably merit far more skepticism than received. When I see reports in the news that certain genetic markers are "associated with disease X", for example, especially in animal models, my wishful thinking radar (and/or fraud radar) lights up. See, for example, "Why Most Published Research Findings Are False" by John P. A. Ioannidis, PLoS Medicine, 2005 August; 2(8): e124 (full text and PDF available free here). Ioannidis writes:

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.

The article is worth reading in its entirety.

On the second issue, that of stagnation and insularity of bioinformatics:

Molecular genetics and bioinformatics are fields I am familiar with, having been a co-PI of IT in a genetics and birth defects collaboration with a desert kingdom where consanguinity rates are high, some years ago.

As a result of this background and obervations I made in collaborating with molecular and clinical geneticists, in 2002 I wrote the following, published in the journal Bio-IT World:

Bio-IT World
Aug. 13, 2002
Medical Informatics MIA


I enjoyed reading the article "Informatics Moves to the Head of the Class" (June Bio·IT World). Thank you for spotlighting the National Library of Medicine (NLM) training programs in medical informatics and bioinformatics, of which I am a graduate (Yale, 1994).

Bioinformatics appears to receive more media attention and offer more status, career opportunities, and compensation than the less-prestigious medical informatics.

This disparity, however, may impede the development of next-generation medicines. Bioinformatics discoveries may be more likely to result in new medicines, for example via pharmacogenomics, when they are coupled with large-scale, concurrent, ongoing clinical data collection. At the same time, applied medical informatics, as a distinct specialty, is essential to the success of extensive clinical data collection efforts, especially at the point of care.

Hospital and provider MIS personnel are best equipped for implementing business-oriented IT, not clinical IT. Implementing clinical IT in patient-care settings constitutes one of the core competencies of applied medical informaticists.

Informatics specialists with a bioinformatics focus — even those coming from the new joint programs — usually are not proficient in hospital business and management issues that impede adoption of clinical IT in patient care settings. Such organizational and territorial issues are in no small way responsible for the low utilization of clinical IT in patient care settings.

It will be important for medical informaticists focused in the clinical domain and bioinformaticists specializing in the molecular domain to collaborate with other specialists in order to best integrate clinical and genomic data. ...

Scot Silverstein, MD
Director, Published Information Resources & The Merck Index
Merck Research Laboratories


The key phrase is "It will be important for medical informaticists focused in the clinical domain and bioinformaticists specializing in the molecular domain to collaborate with other specialists in order to best integrate clinical and genomic data." In fact, the "with other specialists" was added to my original to appease others within the company who reviewed this letter for publication.

The Sinclair anti-aging matter reminded me of this issue. Here, we have a nonclinical molecular scientist experimenting on cells from animals, and apparently either he and/or the company extended these observations to possibilities in people, without involvement of clinician and/or Medical Informatics expertise. This is the issue I raised in my Merck letter.

There was no such collaboration at Merck. There is none at many universities, including my current one (the Ivies may be an exception; I enjoyed such collaboration a number of years ago as a Medical Informatics postdoc and then faculty at Yale School of Medicine). Colleagues report there was no such collaboration at GSK, or Wyeth, or several other pharmas.

I've also noted such organizations paradoxically don't even recognize Medical Informatics as an essential specialty e.g., "Why Pharma Fails", "CRO's: We don't need medical informatics here", "We don't need medical informatics here Part 2" and "GSK, Avandia and Medical Informatics: More on Why Pharma Fails."

I fear in the field of Bioinformatics, the lack of collaboration with Medical Informatics occurs frequently, and in fact billions of investor dollars might be going down the drain in the pursuit of cybernetic in-silico miracles as a result.

An observer and commentator with relevant industry and academic experience, Felix Fulmer, observed in addition to the misuse of bioinformatics, this second and even more critical issue. He surmises that perhaps bioinformatics itself has reached a point of stagnation, even when used with the best of intentions but without medical input:

People in specialized disciplines ignore closely related fields, often out of arrogance. They may suffer of professional disciplinary "blinders", and bioinformatics is a prime example.

Doesn't it seem that Bioinformatics has hit a hard wall of stagnation? There have been tens if not hundreds of billions of dollars invested in this area, and yet lately there have been few truly revolutionary, tangible products on the medical shelf as a result. Pharmas are re-evaluating their approaches to bioinformatics, in fact, and numerous restructurings (example) have occurred in what seems an increasingly desperate attempt to leverage this discipline beyond the 'interesting journal article.'

The bioinformatics field may need intentional, massive, deliberate involvement of medical informatics personnel in order to continue achieving the benefits it promises, rather than casual and semi-serious involvement as we see so far.

We can observe that in computer science there is not much to be learned from medical informatics. In fact, it would be somewhat strange if computer scientists became interested in medical informatics for anything other than an application of their research.

On the other hand, that "management science" shows no interest in medical informatics is not surprising since management science has taken the arrogant stance that it alone is a fundamental discipline requiring no input from others; that everything is a widget and everyone is a resource to be dealt with in exactly the same way with minimal input from specific domains. Management has a "domain agnostic approach" to practice. So if computer and management science don't collaborate with medical informatics, it is understandible.

But bioinformatics should certainly be aware of the vital necessity for the strongest of collaborations with medical informatics professionals.

I issue a challenge to any bioinformatics professional who reads this blog to tell me why medical informatics is not an essential discipline to work alongside in discovery of new cures. As a sign of disciplinary insularity, however, I will unfortunately bet that few if any bioinformatics professionals actually read this clinically related blog or blogs on related topics such as medical informatics.

(A corollary of the same phenomena of misuse of experimental fields such as bioinformatics, disciplinary insularity and overconfidence in computing is the use of informatics as a substitute for biological testing altogether. I wrote recently about that issue in "A 21st Century Plague? The Syndrome of Inappropriate Over-Confidence in Computing." I observed via "New Drugs, Virtual Tests", Wall Street Journal, Dec. 17, 2008, that the U.S. Food and Drug Administration plans to use "new computer technology" to simulate how some drugs in development are supposed to work, helping researchers and regulators spot safety and effectiveness issues before late-stage tests on humans are completed.)

As a final aside, perhaps VC's (venture capitalists) ought to take far more care when investing in bioinformatics. They should become better informed (e.g., by reading my site on difficulties in health IT due to failures of collaboration between technologists and clinicians), and invest in cross-disciplinary - not monodisciplinary - endeavors, because monodisiplinary approaches to medicine have probably been plowed dry.

There is likely little to be reaped, nor anything of truly major significance done by lab scientists alone. The low hanging fruit is all gone.

Once an oil field is dried up, it does not matter how many more wells one drills nor how deep one goes. One must accept that a field is dried up, and go elsewhere. Bioinformatics may be in denial and not willing to admit this understandable but unfortunate reality.

Final note: an excellent powerpoint presentation entitled "Seven Deadly Sins of Bioinformatics" by Prof. Carole Goble at the Univ. of Manchester, UK can be seen at this link.

Addendum: I've noted some comments at Reddit.com on this posting. Examples:

  • Uh, yeah. Some professor (who isn't even a bioinformaticist) may or may not have endorsed a sketchy nutritional supplement with exaggeration of his own results, so clearly the entire field is "in trouble".
  • Has Bioinformatics Hit a Hard Wall of Stagnation? No.
And these at The Life Scientists:

  • That blog post is all over the place - a lot of incoherent ranting, I don't see the main message.
  • Main messages - bioinformatics is stagnant (nor argument for it) and isolated from medical informatics (again no argument).

It is unclear whether the posters were bioinformaticists. However, it seems they do not fully understand how to proffer logical argumentation, nor respond to the challenge I plainly posted above.

Jan 8 - an interesting and useful discussion is now seen at the above Reddit site.

-- SS

8 comments:

Pedro Beltrao said...

I think it is clear that medical informatics is an important field and I never heard otherwise. Regarding you comment about bioinformatics becoming stagnant I don't see any proof of that. Has Lincoln Stein recently put it (http://genomebiology.com/2008/9/12/114), bioinformatics is alive and kicking. Regarding also the lack of medical relevant research in bioinformatics just have a look at the any of the recent issues of Nature Genetics and the continuous flood association studies that are in part possible to analysis methods.

shevy said...

The way I see it, the problem is twofold, or manifold.

Years ago when I digged into bioinformatics I was really really disappointed to realize that almost _everything_ there involved maths and algorithms. I was never the person to feel tremendous joy in finding better algorithms or the faster way to accurate BLAST results, or similar tools.

Lateron with the advent of system biology I was a bit happier because I felt it was closer to "virtual cells", simulations and the like, and I thought this would include bioinformatics as a field too.
But a few problems existed as well - the lack of serious complexity in it, the problem to integrate the best details of so many different fields, and of course that system biology is still a rather young field compared to old stuff like BioPerl.

Even projects like BioBrick, while really really cool, won't help that much there because they do not yet bring the whole field to a higher "dimension", or more importance ... eventually though, I stopped caring too much and switched my attitude:
- All aspects of "BioInformatics" as a field should attempt to HELP researcher instead of GUIDING them.
And that thought process made me happier, for if I see computers only as a tool to be somehow "better/faster" in what I (or others) do, then I no longer have a big problem that bioinformatics is dominated by algorithms and similar less interesting stuff. Well... to me it is still less interesting. :\

I just cherry-pick what I like and need, and discard the rest. And I think that is ultimately the strength here - you combine the less interesting with something that is more interesting. If Bioinformatics is more seen as appliance and tool, then it needs not be a great "standalone field". It can continue to exist as a helper.

PS: There is however still one _HUGE_ thing i regret, or that I think is missing in bioinformatics.
In traditional classical genetics or synthetic genetics, or biochemistry, or whatever else you need to design experiments and answer complicated questions, there are _so_ many strategies, ideas etc... the whole field continuously feels alive, spawns new ideas and strategies, and you can be amazed at the ideas created in the last some years (and this continues all the time).
But in bioinformatics, where are the really exciting ideas, strategies or visions these days? I have even seen several BioInformatics blogs die over the years. It is really weird. :/

It is ok if BioInformatics is mostly just a tool, but if it is a "standalone" field then it really lacks exciting strategies and visions!

PSS: Collaboration is of course important, but I am afraid that it is not always possible because there exists so much competition on all levels. Competition can be good and bad, sometimes it stifles thoughts...

InformaticsMD said...

Pedro Beltrão wrote:

I think it is clear that medical informatics is an important field and I never heard otherwise

Yet most pharmas and biotechs lack medical informatics professionals. Curious. Perhaps it's a problem of management (or, should I say, mismanagement) at high levels.

Thanks for the Stein link.

-- SS

InformaticsMD said...

shevegen wrote:

All aspects of "BioInformatics" as a field should attempt to HELP researcher instead of GUIDING them.

Yes, consider one's self facilitators instead of enablers. Problem is, seems too many bioinformatics dept. heads (in industry) and professors (in academia) think they will cure the world's diseases in silico without involvement of clinical expertise and real live messy patients and clinical data - the domain of medical informatics experts. That's my point.

It is ok if BioInformatics is mostly just a tool, but if it is a "standalone" field then it really lacks exciting strategies and visions!

Agree!

Competition can be good and bad, sometimes it stifles thoughts...

And ability to publish provocative ideas and polemics (such as this post) that challenge the prevailing attitudes and help break up the stagnation and complacency. That's why I blog...

-- SS

Anonymous said...

Hi --

I am a doctor (MD) by training. I'm currently majoring also in Applied Mathematics.

I started out in Applied Mathematics out of my enthusiasm for bioinformatics. Then I realized, much to my dismay, that the field was completely inundated by computer-science types who understood very little of what they were doing. I've seen things in congress that simply don't make sense (e.g., a graduate student doing time-series of RNA sequences). Biologists, OTOH, seem well served by the many interfaces to data banks people designed and don't really need to delve into the mathematics.

I've grown tired of it and tired of the publication racing game people are playing. I came with much enthusiasm and leave it disenchanted. I've since moved to Medical Informatics proper, where I seek to apply pattern recognition and artificial intelligence to telemedicine. I feel this field still has its foot on *real* problems and the problems are more complex and specialized, so that computer science and physics types don't feel so comfortable in approaching it in the naive way they do with molecular biology and genetics,, and, that way, leave some elbow room for the real domain specialists as well as some breathing space so we can talk about what's *relevant* and *pressing* and not what they fancy for their next grant (bioinformatics of cheese-making, perhaps?)

Also, I also feel the programming is sloppy. It is bellow standard and the software engineering in bioinformatics would never make it into the medical device industry. There's very little (almost non-existant) research done with formal methods (NB: formal methods are not unit-testing and UML diagrams). I feel the software can't be trusted: design errors, no formal methods, very little thought about numerical accuracy, etc. It's very, very, amateurish and gung-ho. Unfortunately, the data they're processing will be used in real products. Errors might creep in and the result is that patients will get hurt. But talking about these things to computer science folk is like preaching on the desert. Look at the computer languages they use. I feel we would have to be using a different type of arsenal and programming languages. For instance, you can trust C only so much, and only if if you treat it like radioactive material. C++ you can't trust. Java is the pits for numerics. Etc. Yet, what is the place in word for safer programming languages (say, with sophisticated type systems) and practices? It's useless to talk about these thing to computer science types, let alone biologists...IMHO, bioinformatics software should be submitted to the same rigorous standards as avionics and medical instrumentation software. This field deals with serious stuff, it's not the same as writing some software for bloggers! If I, a doctor, am all stressed-out about these things, how come they're so relaxed in the way they produce software?

Sadly, I've realized the field is all about algorithms nowadays. There's very little research in trying to gain new relevant insights. There's little thought about how to *model* the biological problem - and that to me is the worse thing. The major problem is correctly modeling the problems we face, not creating faster algorithms. Some problems probably require new maths and can't be tackled by, say, graph theory. Unfortunately, this fundamental lack of understanding of the problem domain will hamper progress. Math and computer types are so arrogant - some conferences on Computational Biology might be packed with non-biologists! This is absurd! The field is not Theoretical Physics, you know. There's real data out there...but you have to interpret it! For instance, I have on my desk a book full of crazy "DNA algebra" that we don't even know if there's any experimental support for so much speculation. Yet, it's there, a major publishing house published it. There's really a wall blocking dialogue. It's kind of hopeless. Medical doctors and biologist will have to learn Maths. That is the only solution. We will have to be able to state clearly what we want. We can't rely so much on other people. The field is about biology. But, to compound the problem, for physicians, there's no financial incentive to go out into this field. This means less diseases will be dealt with (since human pathology is not the domain of the biologist).

I apologize for English, it's not my native tongue.

InformaticsMD said...

Dr. Who wrote:

I am a doctor (MD) by training. I'm currently majoring also in Applied Mathematics.

Thanks for the long post. Very interesting insights.

Sadly, I've realized the field is all about algorithms nowadays.

Indeed. Read this post as an example of just how bad the tunnel vision has become.

-- SS

Anonymous said...

Quite frankly this entire blog post sounds like a whine about how bioinformatics is popular and no one cares about medical informatics. One of your quotes mentions how they think it strange that bioinformatics hasn't produced some wonderdrugs and miracles for us now and I think that's entirely unfair to bioinformatics. Bioinformatics has solved a number of problems in biology. At this moment we are still in an extremely data poor environment that has prevented a lot of predictive in silico models from being produced.

I don't discount that medical informatics is important in fact I think it's vital but to me this seems like you're just attacking bioinformatics and wringing your hands over medical informatics as the unpopular little brother, what's wrong lose your funding?

InformaticsMD said...

Anonymous December 14, 2009 8:11:00 PM EST writes:

I don't discount that medical informatics is important in fact I think it's vital but to me this seems like you're just attacking bioinformatics and wringing your hands over medical informatics as the unpopular little brother

Let me see. So you understand medical informatics is vital but I'm merely wringing my hands that a vital field is ignored.

In view of "A Decade Later, Human Gene Map Yields Few New Cures", NY Times, June 13, 2010, I'd say you and your muddled thinking and logically fallacious ad hominem are part of the very problem I write about.

-- SS