Text Mining

Question 1

NLP

Answer 1

Lexical
Syntactic
Semantic
Pragmatic
Inference

Question 2

Information Retrieval

Answer 2

Locating relevant documents with respect to an input
Two modes of text access
	Pull Mode (search engines)
	Push Mode (recommender)
Text Retrieval Methods
	Document selection (keyword)
	Document ranking (similarity)

Question 3

Document Similarity (Vector Space Model - Bag of Words)

Answer 3

Document and Query represented in high dimensional space.
Relevance Measured with appropriate similarity measure.
Document Ranking
Retrieval models
Similarity: sim(q,d)
Probabilistic: p(R=1|d,q)
Prob inference: p(d->q)
Axiomatic: set of constraints
VSM: number of distinct query words matched in d
TFW: count of words
IDF: log[(M+1)/k]

Question 4

IDF + TF

Answer 4

c(w,q)c(w,d)log((M+1)/df(w))

Question 5

TF WeighT

Answer 5

y=x
y=log(1+x)
y=log(1+log(1+x))

Question 6

BM25

Answer 6

y = (k+1) c(w,d) / (c(w,d)+k)

Question 7

Document Length Normalization

Answer 7

Penalize a long doc with a doc leng normalizer.
Pivoted length normalizer: avg doc length as pivot
1 if doc length = to avg
norm = 1-b+b(|d|/avdl)
VSM: ln(1+ln(1+c(w,d)))/(norm)
BM25/Okapi: (k+1) c(w,d) / (c(w,d)+k*(norm))

Question 8

From Texto to Numerical

Answer 8

Keywords Selection: tokenization
Stop word and stemming used to isolate significant keywords.
	Stop words: a, the, always
	Word stemming: common prefix (computer, computing, computerize)
Dimensionality Reduction
	Latent Semantic Indexing (LSI)
	Locality Preserving Indexing (LPI)
	Probabilistic Semantic Indexin (PLSI)

Question 9

Latent Semantic Indexing

Answer 9

xi vectors representing documents
X: all set of documents
SVD —> X=UEV^(T)

Question 10

Text Classification

Answer 10

Labeling text documents on Topic, Style, Purpose.
Similarity: information retrieval + knearest. k most similar documents are retrived
Dimensionality Reduc: distribution of keywords. Can apply classification techniques.
Naive Bayes

Question 11

Word Embeddings

Answer 11

Words with similar meanings have similar representations.
Individual words as real valued vectors.
one word mapped to one vector 
vector values are leaned “like” neural
Bag of Word Model

Question 12

Word Embedding Applications

Answer 12

Word Similarity
Machine Translation
Relation Extraction
Sentiment Analysis: average word embeddings using TD/IDF weight. Then classify + or -
Co-reference Resolutions: chaining entity mentions across multiple documents.
Clustering: Similar class, similar contexts.
Semantic Analysis: word distributions for various topics.

Question 13

Bag of Words

Answer 13

One hot encoding
One bit position in huge vector
Context information is not used

Question 14

Word2Vec

Answer 14

Skip gram neural network architecture.
trains a simple neural network with a single hidden layer
Predict probability for every word vocabulary
Output vector is a probability distribution

Question 15

Naive Bayes for Text

Answer 15

Doc Categories (C1, C2, .... Cn)
Doc to classfiy (D)
Prob model: P(Ci|D) = P(D|Ci)*P(Ci)/P(D)
We chose: argmax P(D|C)P(C)
Keywords distributions are inter-indep. and order-indep.

M-estimate: P(wk|C) = Nc,k + 1 / Nc + |Vocab|

Question 16

Clustering Text

Answer 16

Tokenizing and stemming
Calculate into vector with TF-IDF
Calculate cosine distance btw each doc.
Cluster documents using similarity.

Question 17

Text clustering preprocessing

Answer 17

-Bag of Words
Punctuation
N Chars
Number filter
Case converter
Stop word filter
Porter Stemmes 
Term Grouper
- Keygraph keyword extractor