We are very grateful to the people that supported us in preparing the training data and in the evaluation of the results. Here we want to mention especially Lorrie Tanabe and John Wilbur from NCBI for providing data for the tasks 1A and Evelyn Camon and her team at the EBI for providing us with data for tasks 2 and examining the results.

The progress in the automation of experimental approaches in the last years such as yeast-two-hybrid based methods, DNA expression arrays and mass spectrometry determination of components of macro-molecular complexes has created new opportunities and challenges for biologists in the generation and analysis of massive amounts of data. On the other hand the available literature is also growing constantly (MEDLINE alone, the archive of abstracts of biomedical publications, already contains more than 11 million entries) and it has become clear that there is an urgent need not only for methods for the massive analysis of experimental data but that the already existing knowledge locked in the scientific literature has to be extracted and collected for the understanding of this new kind of data.

Due to the impressive amount of available literature, this is a time-consuming task with many uncertainties and drawbacks, e.g. finding the right articles, finding the right conclusions, finding the way to disseminate the knowledge with co-workers, etc. Therefore, the development of systems for the automatic processing and analysis of scientific literature is absolutely needed, saving valuable time and resources and providing a way for identifying important data. Since the 90's this has been a growing field with a considerable numbers of publications, associations of active researches in the field (e.g the BioLINK group with a special workshop dedicated to information extraction and text mining at ISMB and workshops at the Association of Computational Linguistics) and specially dedicated web resources (BioNLP.org).

But the bad news is that the field has stayed very academic and the systems have been developed for different purposes, applied to different data sets and evaluated according to different measures, which makes them fundamentally incomparable. We believe that the technology is progressing, but until these methods are applied to real biological data sets and evaluated on biologically relevant tasks, it will be difficult to convince biologists that the technology can assist them in accessing the vast biological literature. Therefore there is an urgent need to introduce standards in the field to compare different methods and systems and to make them more useful and applicable for biologists. Furthermore training opportunities have to be created to introduce people to existing systems and a strong and active community has to be built to guide the development and application of this technology.

We therefore have decided to set up an assessment of text mining systems in biology, by defining a common task, common data sets and a clearly defined evaluation to be able to give clear answers about the performance of these techniques in a real world scenario relevant for the biological community.

We will build on the experience in bioinformatics where a number of competitions revolutionized entire fields by the agreement on common targets and evaluation rules. This happened for example for protein structure predictions (CASP: Critical Assessment of techniques for protein Structure Prediction) or gene predictions in entire genomes (at the "Genome Based Gene Structure Determination" symposium held on the Wellcome Trust Genome Campus). Researchers in natural language processing (NLP) and information extraction (IE) have, for many years now, used common evaluations to accelerate their research progress as well (e.g., via the Message Understanding Conferences - MUCs, and the Text Retrieval Conferences - TREC).

Text mining and information extraction systems are only starting to be evaluated in a serious manner on biologically relevant problems. The recent KDD Cup 2002 was a data mining competition held in conjunction with the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. The competition included two tasks that involved data mining in molecular biology domains. The first task focused on constructing models that can assist genome annotators by automatically extracting information from scientific articles. The second task focused on learning models that characterize the behavior of individual genes in a hidden experimental setting. The first meeting of the TREC (Text REtrieval Conference) Genomics Track this year featured two tasks: ad hoc retrieval and information extraction. Both tasks centered around the Gene Reference into Function (GeneRIF) resource of the National Library of Medicine, which was used as both pseudorelevance judgments for ad hoc document retrieval as well as target text forinformation extraction.

We are organizing this workshop in conjunction with the ongoing BioCreative evaluation of texting mining systems. The workshop will allow participants in BioCreative to present their results and discuss their approaches. There will also be a considerable number of invited talks from respected experts in the field and time for open discussions about what has been learnt and where we might go in the future.

Programme

Travel Details