Session 6.1

Question 1

Bag of words

Answer 1

treats every word as a term in the document.

Question 2

Bag of N-grams

Answer 2

treats every possible collection of N adjacent words as a term in the document.

Question 3

term frequency (TF)

Answer 3

e.g., raw count

TF(t, d) = A raw count of times term t appears in a document d

Question 4

TFIDF

Answer 4

TFIDF (t, d)= Product of Term Frequency TF(t, d) and Inverse Document Frequency IDF(t)

Question 5

Document Term Matrix (DTM)

Answer 5

Each document is a row and each term is a column

Question 6

Term Document Matrix (TDM)

Answer 6

Each term is a row and each document is a column

Question 7

Disadvantage of bag of words/N-grams & solutions

Answer 7

Massive number of features

Question 8

To decrease the numbers of words/terms/features in documents:

Answer 8

➢ Cleaning and preprocessing text
• Case normalization
• Removing punctuation
• Removing numbers
• Removing stopwords
• Word stemming and stem completion

➢ Feature selection

➢ Special consideration to computational storage space

Question 9

Sentiment analysis technique

Answer 9

Detecting the sentiment of the text, e.g.,
• positive/negative/neutral
• urgent/not urgent

Question 10

Three main levels of sentiment analysis

Answer 10

• Document-level
• Sentence-level
• Aspect-level

Question 11

How is topic modelling technique different from clustering?

Answer 11

Topic modelling:
A document can be associated with more than one topic

Clustering:
A document only shows up in one of the clusters