Information extraction

It has been suggested that data extraction be merged into this article or section. (Discuss)

In natural language processing, information extraction (IE) is a type of information retrieval whose goal is to automatically extract structured information, i.e. categorized and contextually and semantically well-defined data from a certain domain, from unstructured machine-readable documents. An example of information extraction is the extraction of instances of corporate mergers, more formally MergerBetween(company₁,company₂,date), from an online news sentence such as: "Yesterday, New-York based Foo Inc. announced their acquisition of Bar Corp." A broad goal of IE is to allow computation to be done on the previously unstructured data. A more specific goal is to allow logical reasoning to draw inferences based on the logical content of the input data.

The significance of IE is determined by the growing amount of information available in unstructured (i.e. without metadata) form, for instance on the Internet. This knowledge can be made more accessible by means of transformation into relational form, or by marking-up with XML tags. An intelligent agent monitoring a news data feed requires IE to transform unstructured data into something that can be reasoned with.

A typical application of IE is to scan a set of documents written in a natural language and populate a database with the information extracted. Current approaches to IE use natural language processing techniques that focus on very restricted domains. For example, the Message Understanding Conference (MUC) is a competition-based conference that focused on the following domains in the past:

• MUC-1 (1987), MUC-2 (1989): Naval operations messages.
• MUC-3 (1991), MUC-4 (1992): Terrorism in Latin American countries.
• MUC-5 (1993): Joint ventures and microelectronics domain.
• MUC-6 (1995): News articles on management changes.
• MUC-7 (1998): Satellite launch reports.

Natural Language texts may need to use some form of a Text simplification to create a more easily machine readable text to extract the sentences.

Typical subtasks of IE are:

• Content Noise Removal: remove noise contents. For example, tagclouds, navigational menu, related contents, and context related advertisements.
• Named Entity Recognition: recognition of entity names (for people and organizations), place names, temporal expressions, and certain types of numerical expressions.
• Coreference resolution: detection of coreference and anaphoric links between text entities. In IE tasks, this is typically restricted in finding links between previously extracted named entities. For example, "International Business Machines" and "IBM" refer to the same real world entity.
• Terminology extraction: finding the relevant terms for a given corpus
• Relationship Extraction: identification of relations between entities, such as:
  • PERSON works for ORGANIZATION (extracted from the sentence "Bill works for IBM.")
  • PERSON located in LOCATION (extracted from the sentence "Bill is in France.")

See also

• Concept mining
• DARPA TIPSTER Program
• Semantic translation

Information extraction and the World Wide Web

IE has been the focus of the MUC conferences. The proliferation of the Web, however, intensified the need for developing IE systems that help people to cope with the enormous amount of data that is available online. Systems that perform IE from online text, should meet the requirements of low cost, flexibility in development and easy adaptation to new domains. MUC systems fail to meet those criteria. Moreover, linguistic analysis performed for unstructured text does not exploit the HTML/XML tags and layout format that are available in online text. As a result, less linguistically intensive approaches have been developed for IE on the Web using wrappers, which are sets of highly accurate rules that extract a particular page's content. Manually developing wrappers has proved to be a time-consuming task, requiring a high-level of expertise. Machine learning techniques, either supervised or unsupervised, have been used to induce such rules automatically.

Wrappers typically handle highly structured collections of web pages, such as product catalogues and telephone directories. They fail, however, when the text type is less structured, which is also common on the Web. Recent effort on adaptive information extraction motivates the development of IE systems that can handle different types of text, from well-structured to almost free text -where common wrappers fail- including mixed types. Such systems can exploit shallow natural language knowledge and thus can be also applied to less structured text.

Free or Open Source Information Extraction Software or Services

• General Architecture for Text Engineering "General Architecture for Text Engineering", which is bundled with a free Information Extraction system
• OpenCalais Automated information extraction web service from Thomson Reuters (Free limited version)

External links

• MUC
• ACE (LDC)
• ACE (NIST)
• Alias-I "competition" page A listing of academic toolkits and industrial toolkits for natural language information extraction.
• Gabor Melli's page on IE Detailed description of the information extraction task.