Chapter 1 Introduction

Question 1

why textual information is important?

Answer 1

1. Human knowledge is recorded as text. example: scientific literature, text books, technical manuals;
2. Most common type of people communication. example: email, text message,
3. text is the most expressive form of information

Question 2

Text Mining includes two parts

Answer 2

Text retrieval and text mining

Question 3

Text retrieval

Answer 3

digital information beyond human being capacity to even skim over

The need for developing intelligent text retrieval system to help people get access to the relevant information quickly and accurately.
example: web search engine

Question 4

Text Mining

Answer 4

Due to the overwhelming amount of information, people need intelligent software tools to help discover the relevant knowledge to optimize decisions

Question 5

The goal of text mining

Answer 5

The goal of text mining is to discover knowledge and pattern in text data to support a user’s task.

Question 6

Provenance

Answer 6

user usually need to go back to the original raw text data to obtain appropriate context and verify the trustworthiness of the knowledge.

Question 7

combined intelligence

Answer 7

how to optimally divide the work between human and machine so as to optimize the collaboration between humans and machines and maximize their “combined intelligence” with minimum human effort is a general challenge in all application of text data management and analysis

Question 8

TIS

Answer 8

Information Access (Text Retrieval)
Search engine fetch the information queried by users;
recommender system push relevant information to a user

knowledge Acquisition (Text Analysis, Text Mining)

Text Organization

Question 9

Text mining has two approaches, data mining perspective and natural language processing(NLP) approach. What is the difference between these two?

Answer 9

From data mining perspective, text data is seen as a special type of data. Following the goal of general data mining, the goal of text mining is to discover the patterns in text data.

From an NLP perspective, text mining can be regarded as to potentially understand natural language text, convert text into some form of knowledge representation and make limited inferences based on the extracted knowledge.

Question 10

Some common functions for managing and analyzing text information

Answer 10

Search

Filter/Recommendation

Categorization: Classify a text object into one or several predefined categories where the categories can vary depending on applications

Summarization: news summarizer and opinion summarizer

Topic Analysis: when combined with the companion non-textual data such as time, location, authors and other meta data, topic analysis can generate many interesting patterns.

Information Extraction: construct entity-relation graphs

Clustering: Discover groups of similar text objects; enable quick understanding of large text data set

Visualization: visually display patterns in text data