Natural Language Processing

Content:

The method used in natural language processing (NLP) is text mining—the processing of discovering and extracting knowledge from unstructured data. NLP concerns with the human-computer interactions from the perspective of language, it is to make machine language more readable to human and human language more understandable to computer. The basic text mining activities include information retrieval, information extraction and data mining. Because of the ambiguity of language, usually, it is not so easy to do the analysis of information at a single step, so the natural language is deal with at several levels.


In the biomedical field, the degree of ambiguity and analysis complexity is much greater. At the lexical level alone, the tokenization and the lexical variants are the distinct problems need to be identified. Morphological analysis is the way to unify the lexical variants by assigning a canonical base form. The concept precision and recall in morphological analysis are associated with probabilistic.

After the two series of lectures given by Dr. Gonzales, I experience the significance of Ontology and ubiquity of statistics once again. If at the beginning we follow the ontology to create the words and terms, maybe it will be much easier for the scientists in the NLP field. But that is the glamour of the natural language.


Posted by Xiaoxiao

Natural Language Processing

Content:

This week was a pretty interesting week that covered the topics of Natural language processing and text mining. This is a very interesting field since it really touches our everyday communication. We write, we speak, we chat. All this eventually if its recorded be comes text and just adds to the massive amount of data out there. (I'm adding some more text writing this blog.)

As for Di's question about how Google works, the original publication by the creators of Google at Stanford are still online. In case anyone wants to do some mining, I used the terms "google search algorithm stanford" to pull up the Stanford page with the information on how Google works (at least when it started).

Link: http://infolab.stanford.edu/~backrub/google.html

The second lecture, touched on finite state automata which is an interesting topic. It was mentioned in the lecture that the FSA is similar to a game for advancing and etc. This reminded me of the Cootie Bug game. The goal of the game is to be the first to finish a bug by assembling its head to its body, adding legs, eyes, tongue, and a hairpiece. Seems simple? Well in order to add parts you have to roll a dice to get certain body parts. It was required that you get a body before the head and that the head and body are attached before getting any other parts. So for example you have to roll a 1 for the body, and then a 2 for the head. It requires then you first get a 1 and then a 2, then the rest of the parts can be taken in any order. so I think it'll be represented like this 12[3456]+.

Robotics Conference
Lastly, I have extra stuff. The Fulton School of Engineering is hosting a Robotics Conference next Friday and Saturday (Nov. 5,6). The cost to attend is 50 dollars but the program looks fairly interesting. It includes robotics in surgery, laproscopic surgery, etc.  I've attached the link the site.

Link: http://roboticsaz.asu.edu/

Posted by

Eric

Content:

This week Dr. Gonzalez gave us two great lecture of text mining. I have heard the term of ‘text mining’ for thousands of times, but this is the first time that I get in touch with this field. Our language is really ambiguous. Those ambiguous can be at lexical level or syntactical level. I like the example she gave us in class about “big stars resist dieting”. Our misunderstanding of this sentence is caused by lexical level. But text mining is to let computer to extract information from such ambiguous language, it is even more difficult. According to my simple understanding, the key point of text mining is to find certain pattern from a large amount of nature language text. Text is written for people to read, how can we make computers to read it? Dr. Gonzalez introducee some basic concept of the algorithms and the underlying principles, which I think are really interesting.

Posted by

Jing Lu

NLP Methods

As we have seen, one important aspect of the discovery of new scientific and medical information is the process of mining for information. This can take the form of data mining, which retrieves patterns from already processed text, or text mining, which extracts information from unstructured data. Text mining can seem intimidating at times, so it was good to see the breakdown of the divisions/steps in the text mining process. Going in the order of the steps (lexical, syntactic, semantic, discourse) makes the process seem much more manageable. Still, there are many challenges in text mining, such as the validation of patterns that are found and the processing of such an overwhelming amount of information. Even if patterns are found, it is important to make sure that proper validation and analyses are conducted to prove that there is a legitimate finding.

Posted by Annie

Text mining in Biomedical informatics

Content:

The class in this week is very interesting. With the introduction by Dr.Gonzalez, I first time get the chance to touch what is text mining, which I felt mysterious and attractive for a long time. The most often application of the text mining technology in our daily life I think should be google, which provide a powerful text searching and analytical service in tons of web, articles and even pictures to find what we are looking for. Although I still do not know how the powerful google search engine realize the text mining technology and the detail about what kind of algorithms are involved, these two lectures absolutely stimulate my curiousness in text mining field. Other than web search engine, the text mining technology also has broad application in biomedical field. In biomedical research field, text mining technology is usually combined with natural language processing and computational linguistics to be applied in bioinformatics and medical informatics. One big application is what we have already heard thousand times, the NCBI PubMed, which use text mining technology to extract biomedical and molecular biology literature from increasing number of electronically available publications stored in database. The other main development of text mining in biomedical field is in the identification of biological entities, so called entity recognition, such as protein and gene names in free text, the association of gene clusters obtained by microarry experiments with the biological context provided by the corresponding literature, automatic extraction of protein interactions and associations of proteins to functional concepts. One sofware tool developed using text mining technology is the FABLE, which is a gene-centric text-mining search engine for MEDLINE. The interested one can go here to try it: http://fable.chop.edu/.

Posted by

Di Pan

Content:NLP as I understood was the processing of natural language ie human language to make it understandable to the computer. Specifically used in bioinformatics domain, NLP is a very useful tool to extract relevant information from the huge amount of literature present, where its humanly impossible to read through every article. The first step in NLP is tokenization which is breaking the sentence into relevant words which are then used for searches. Morphological analysis uses the lexical words (group of words) and identifies its variants which can be linked to the base word. The 3 ways of capturing such language are : regular expressions, finite state automata and regular grammars. However I still need to understand these methodological aspects better. What I understood is that using these methods in bioinformatics world helps to recognize various related terms and therefore consolidate the knowledge which can be scattered all over the literature especially since the literature on genes is so ambiguous.
I found the Stanford website very resourceful for various programs which are freely downloadable for NLP. http://nlp.stanford.edu/

Posted by Sheetal Shetty

Text Mining

Another week of new info. Wouldn't it be great if someone wrote a textbook to include all of these methodologies? I agree with what others have posted. Dr. Gonzalez is an enthusiastic lecturer and her classroom integration is good reinforcement. Still, I need to do some searches to really comprehend the topic. I do know that Precision = TP/TP + FP, and as was debated in class, this is the same as predictive value. Likewise, Recall = TP/TP + FN which is the same as sensitivity.

Text mining is the discipline where patterns are extracted from natural language text (not structured data bases). Althought new to me, this appears to be a discipline studied for years. I found that scientists have hypothesized causes of rare disease by looking for indirect links in different subsets of the bioscience literature for years. I have read in numerous articles that text mining is believed to be extremely useful in the field of biomedicine. Guess that's why it's been presented to us.

Happy Halloween!!!! Lee

last week of october

Content:

This week was interesting, I've never thought I would revisit verbs and nouns in class again. Dr. Gonzalez did I really good job presenting this new topic. I also enjoyed that she involved the class in the process.

Dr. Gonzalez, introduced text mining methods. The overall all goal of text mining in biomedical informatics is to empower discovery by extracting knowledge that is embedded in the literature. I really like how Dr. Gonzalez explained finite state automata as playing a board game, and that you are unable to advanced unless you get the input necessary. In these lectures it was also discussed how to deal with ambiguity. In addition, we did an in-class exercised where we tried to look for ambiguity. Overall these were two great lectures, I am still a bit confused on all the symbols used, but I think it is just a matter of practice.

I found a good article on on challenges and advancements made in biomedical text mining I thought it was really interesting: http://intl-bib.oxfordjournals.org/cgi/reprint/8/5/358


Posted by

Week of 10/26/09

Content: Both of this week's lectures were on natural language processing (NLP). Text mining is a technique used in NLP. Some challenges in text mining are ambiguity of terms, lexical variants, an tokenizing text. Regular expressions and some of their uses to find text was covered in the second lecture. Both false positive and false negative results for text searching were taught. Increasing accuracy and increasing coverage can effect error rates in text mining. Regular expressions are one type of formalism for capturing language and others are finite state automata and regular grammers. Another topic in text mining that was covered was morphological parsing aka stemming. Morphology can be either inflectional or derivational. Inflectional morphologies can have different combinations of stems and affixes where the resulting words serve different grammatical and semantic purposes. An example of derivational morphology is changing a verb into a noun. NLP can help with doing research by extracting literature articles that contain terms like gene names. Associations between terms like a gene and a disease can be discovered through NLP techniques. A open source program named BANNER can help with generating useful name entity recognitions.

I liked how Dr. Gonzalez had students do a practice exercise on recognizing lexical variations. Also, she did a nice job with making her lectures interactive with students. The component of her lectures where she asked students questions added helpful opportunities for us in the audience to try to apply the knowledge we were learning from the lectures. Also, the idea of automating specific literature retrieval seem particularly useful to me. Even if some literature retrieval was automated, it seems like someone would want to manually search through other literature too. However, some of the literature search tools seem so effective that searching though other literature than what they found may not be highly useful. The picture that Debbie posted definitely helps with understanding how precision and recall are similar to sensitivity and specificity. Thanks for posting that Debbie.

Some potential NLP research areas are presented in this article:

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2723093/pdf/1471-2105-10-233.pdf/?tool=pmcentrez

One of those potential research areas is to improve on the ability to be both easy to install and use and achieve state of the art levels of performance in tools for the automated extraction of gene and/or protein interactions (GPI) from literature. Perhaps making a state of the art performing GPI extraction tool easier to install and use would be an effective way to go about that research.

Posted by:

Nate

Natural Language Processing Lectures

It is never ceasing to amaze me how many new topics can be presented in the course of a week, let alone a semester.  NLP is rather intriguing and what seems like a rather simple concept of breaking down words, the process is very complex when you need to apply semantic meanings to the words to keep them in context.  Dr. Gonzalez is very enthusiastic on this subject.  The first lecture was not as technical as the second one.  When we started getting into the actual coding and search strings, it was very interesting - but this is what I like to learn.  There's a lot of information presented, almost overwhelming again, similar to ontologies and taxonomies so it would be nice to have a review of the important things to take away from this lecture. 

I  need to learn more about Finite State Automata.  I understand the concept but would like to see more examples.  A link I have found helpful to apply NLP to the clinical field was a study using NLP to identify Heart Failure.  Within the study, they quote their sensitivity, specificity and the predictive value --- they didn't utilize the words recall or precision.  Here's the link:  http://www.informatics-review.com/wiki/index.php/Electronic_Medical_Records_for_Clinical_Research:

I also found  a simple diagram showing the relevancy as it relates to Recall and Precision calculations similar to the sensitivity and specificity.

Here's the link for the diagram:  http://pages.cs.wisc.edu/~jerryzhu/cs838/IR.pdf
If anyone else finds some great introductory links, please share them.
Posted by :  Debbie Carter

Bioinformatics Methods

Content:

Well first off, oops I fell asleep and then this slipped my mind. I find this proves that we can't multitask too well and that computers are needed for memory assistance.

Now off to what we did last week. We learned about the different bioinformatics datatypes as well as the flow of genetic data. In the following lecture, we got an introduction into the different methods used for bioinformatics.

Posted by Eric

Methods in Bioinformatics

Content:

In Professor Dinu’s first lecture, he introduced the 4 fundamental sources of bioinformatics data: DNA, mRNA, protein and metabolite. As the researches of these data go on, some “omes ” are introduced: Genome, Transcriptome, Proteome and Metabolome. Biomarkers are gathered from the biological information flow: DNA->RNA->Protein.

In the second lecture, Dr. Dinu gave an overview of the Bioinformatics techniques. The mathematics devotes a lot to the bioinformatics data analysis, including statistics and machine learning which used to be the topic in last classes. For sequence alignment, BLAST is the most popular tool. The purpose of sequence alignment is to identify regions of similarity that may be a consequence of functional, structural, or evolutionary relationships between the sequences.


Posted by  Xiaoxiao

Bioinformatics introduction

Content:

In the two lectures in this week, the main topic mainly focuses on the bioinformatics. Dr.Dinu gave us a well organized introduction from the what kind of data in bioinformatics study to the usually used computational methods in this field. In the first part, many important bioinformatics data was introduced, including the microarray data, genomic data, DNA/Protein sequence, also many important bioinformatics data source are also talked about, such as the NCBI, uniprot, PDB bank, where is usually the starting point for getting initial bioinformatics information for research. In the second part of the class, the prevelant bioinformatics research methods were discussed, which covers the statistics method, such as the chi_square test, t-test, and also the datamining method used in bioinformatics study was also covered. From the second part, I got an understanding about how to connect the previous learned datamining method to the applications in real bioinformatics research. Take the clustering as an exampl. The functional catagorizing similar gene expression to a group is just based on the clustering method. The other usually used bioinformatics analysis method is the blaster method, which can search the gene or protein database to match the most similar gene or protein sequence to the enquiry sequence. In the bioinformatics study for protein, based on the most similar sequence, the homology modeling can be performed to build up the 3 dimensional protein structure for further molecular simulation. The above two applications are mainly in the physical simulation domain, which is a subdomain under the computational biology field. The other fields of bioinformatics, I feel the statistics technologies would be more invovled.

Posted by

Di Pan

Bioinformatics

Content:Excellent overview of methods used in bioinformatics. The first class talked about the data sources in bioinformatics which was broken down very systematically into DNA, RNA, proteins and metabolites. DNA is studied for one of the following: 1. Sequence variation i.e. Single nucleotide polymorphism (SNP), 2. Epigenetic modification i.e. methylation, deacylation 3. Structural variation i.e. translocation, copy number variation.
RNA is studied using gene expression to identify micro RNA which are associated with the disease.
Proteins are studied using mass spectroscopy to identify protein expression. The main problem with problem with protein expression studies is the size of the protein molecule which is the deterrent for high throughput studies using array technology.

The other data source mentioned was metabolites which are by products of a disease process and can be easily detected. Thus assays detecting these are more robust as people with diseases will most likely have the metabolite.

The second lecture covered the methods used for analysis: 1. Biomarker test 2. Data mining 3. Statistics 4. Sequence alignment
Biomarkers testing includes identification of the maker-->sequencing the marker-->identifying the mRNA expression-->identifying the protein expression level-->validation of the biomarker by screening the same in clinical tumors.
Data mining techniques include 1. Unsupervised algorithm eg k-means, clustering 2. Supervised algorithm: Regression, classification (Random forests, decision trees, Neural networks)
Statistics mainly used in genetic data are linear and logistics regression, t-tests and chi-squared test.
Sequence alignment using dynamic mapping which was difficult to understand from the slide. I found this simple website which explains it very well.
http://www.avatar.se/molbioinfo2001/dynprog/dynamic.html



Posted by
Sheetal Shetty

Bioinformatics data and techniques

Content:
In the first lecture by Dr.Dinu, the various kinds of bioinformatics data – genome, proteome, metabolome and transcriptome were discussed. The “ Central Dogma of Life” which involves the conversion of DNA to mRNA(transcription) ,which is in turn converted to protein (translation) is described and its relevance in obtaining various sources of “-omic” data is elaborated. Biomarkers for various diseases like cancer, diabetes, hypertension, alzheimer’s disease were also discussed.

In the second lecture, various techniques that could be used to analyse the bioinformatics data to understand the disease processes were discussed. These techniques include biomarker tests, machine learning approaches like clustering, decision trees, classification, regression and statistical tests of significance like T-test, Chi square test, odds ratio. Pair wise sequence alignment using BLAST (several versions-protein blast, nucleotide blast, blastx, tblastn) and global alignment using dynamic approaches are of significant importance in genomic studies.

Both the lectures gave a good understanding of various concepts and tools in bioinformatics and possible areas of research emphasis.

Posted by
Harsha Undapalli

Bioinformatics Week

Genomic variation is a main focus of research because it has a big impact on the well-being of individuals. Because we now have such large volumes of genomic data, there is more interest in comparing sequences of individuals in similiar and different populations. This, in turn, gives us better insight into the propagation of certain genetic disorders in society. One type of genetic variation is a polymorphism, which is generated from some type of individual mutational change (insertion, deletion, etc.). Most polymorphisms are eliminated from the population, but some can become fixed in a population. A SNP is a variation of a single nucleotide. Other variations include copy number variants and structural variations. It is a pretty daunting task to identify all the types of changes (neutral or harmful) in a genome because they can occur in just a single nucleotide, a larger portion of a gene, several genes, parts of one chromosome, or even multiple chromosomes. We will need very powerful tools in the future to detect tiny variations in disorders that are hard to pinpoint one root cause/loci--like cancer.

Posted by Annie

This week, it was the turn of Bioinformatics. The first lecture was mainly focused on "what"s. I had no idea about the things before the lecture and was difficult for me to catch those things. I got to know more about different bioinformatics data like genomes, and proteomes. Dr. Dinu also coverd different biomarkers. He also mentioned about DNA, RNA and Proteins.

In the second lecture, Dr. Dinu talked about how the machines are trained to find the biomarkers and categorize them. He also mentioned about clustering (k-means , 4) and different approaches of doing clustering like average linkage, single linkage, and complete linkage. He mentioned different things about classification of high-grade brain tumors by using gene experession. This week was more informative to me regarding the novelty of contents that we covered in this course. Now it's time to prepare for the mid-term exam of BMI 501 course. Good luck to everyone.

And thanks Laura for the song, it was great. Ashu's animations were also cool.

-Prabal

Content: Well, Laura and Asutosh, its your week, but definitely not mine! Most of the topics that were covered is this week, I heard for the 1st timein my life! Genetics is really tough field, I totally agree with Pier. My salute to the resercher in this field for doing an awesome and very important job.
I really gotta go thru the this week’s lectures again and again. The 1st lecture introduces different data types in genomics and the 2nd lecture teaches the methods to work with these data. That’s all I can contribute to the blog for now.

Posted by Gazi

Week of 10/19/09

Content:

This week's lectures were on bioinformatics. Some sources of bioinformatics data were covered including genomes, proteomes, and metabolomes. Those data sources can be used to detect biomarkers that are indications of diseases. Some types of genetic analyses that are done in bioinformatics include analyses of SNPs, DNA methylation, and copy number variations. Microarrays are one type of technology that are used for those analyses. Mass spectrometry can be used for protein analyses. The FDA has approved the use of some biomarkers for testing for the presence of illnesses. Data mining can be used to identify disease associations with genetics. Chi square tests can be used to test if the occurrence of genotypes or specific alleles are more frequent in people with diseases than people without diseases. Genetic sequence alignment algorithms can be used to find differences and similarities in genetic sequences.

Dr. Dinu covered a lot of topics that are important in the field of bioinformatics. I liked how he talked about a variety of different biomarkers that can be associated with diseases. Its interesting to me to look at the list of FDA approved biomarkers for cancers. That was from 2005 also. A list of genomic biomarkers for drugs that seems current is here:

http://www.fda.gov/Drugs/ScienceResearch/ResearchAreas/Pharmacogenetics/ucm083378.htm

According to the FDA site “Pharmacogenomic information is contained in about ten percent of labels for drugs approved by the FDA.“ That is quite a lot of relevance that pharmacogenomics is having on medication. Warfarin is on that list, as mentioned in previous classes. It will be interesting to see what new pharmacogenomics discoveries happen and are relevant to medications in upcoming years.

Posted by:

Nate

bioinformatics

Content: Laura, that's an awesome song! Ashutosh: your animations are pretty cool too, though I was unable to open the last one.

Bioinformatics was the focus in this week lectures.  I really respect people who study/research this area, because for me I think it is very hard to do. As for me, this stuff is not new, but I am no expert.  Dr. Dinu's first lecture introduced us to the types of data gathered in bioinformatics.  Such as gene expression data, transcriptome, proteome, metabolome.  The second lecture he talked about methods used in bioinformatics.  Method such as data mining and statistical tools.

As I reviewed the lecture I found a really good site with definitions about everything and anything that has to do with genes: http://www.genome.gov/Glossary
Hope it helps.

Posted by P.Ortiz