probabilistic approach to NLP

Question 1

what is the logical or knowledge-based approach to NLP

Answer 1

• rule based

e. g regular expression and finite automata

Question 2

what is probabilistic approach to NLP ?

Answer 2

• use of theory of probability

e. g neural networks, kernel methods,

Question 3

What is probabilistic modelling in NLP ?

Answer 3

• general framework for modelling NLP

- it use random variables, random configurations, and reasoning about the probabilities of the configurations

Question 4

What are independent variables ?

Answer 4

P(V1 =x1, V2 =x2) = P(V1 =x1)P(V2 =x2)

Question 5

what are conditionally independence variable ?

Answer 5

P(V1 = x1 , V2 = x2 |V3 = x3 ) =
P(V1 =x1|V3 =x3)P(V2 =x2|V3 =x3)
or 
P(V1 = x1 |V2 = x2 , V3 = x3 ) = P(V1 = x1 |V3 = x3 )

Question 6

What are the 4 computation task in probabilistic modelling ?

Answer 6

1. evaluation
2. simulation
3. inference
4. learning

Question 7

What is the evaluation task in probabilistic modelling ?

Answer 7

• calculate probability of a complete configuration

Question 8

What is the simulation task in probabilistic modelling ?

Answer 8

• generate random configuration
  or
• producing full configurations according to a given model

Question 9

What is the inference task in probabilistic modelling ?

Answer 9

• 3 tasks
• marginalization
• conditioning
• completion

Question 10

What is the learning task in probabilistic modelling ?

Answer 10

• learning parameters of a model from data.

Question 11

What is marginalization in inference task ?

Answer 11

• computing a marginal probability

Question 12

What is conditioning in inference task ?

Answer 12

• computing a conditional probability

Question 13

What is completion in inference task ?

Answer 13

• finding the most probable assignment of some variables

Question 14

What is joint distribution model ?

Answer 14

• the probability of each complete configuration

- P(V1=x1,…,Vn=xn) in the probability table.

Question 15

What is fully independent model ?

Answer 15

• all variable are independent.

P(V1 = x1 , …, Vn = xn ) = P(V1 = x1 ) · · · P(Vn = xn ).

Question 16

Drawbacks of joint distribution model ?

Answer 16

• memory cost to store table
• expensive running time
• space data problem(no enough data to cover all options)

Question 17

What is Bayes theory ?

naive babes theory ?

Answer 17

P(a|b) = P(b|a) · P(a)/P(b)

P(V2, V3, . . . , Vn|V1) = P(V2|V1) · P(V3|V1) · . . . · P(Vn|V1)

P(V1, V2, V3, . . . , Vn) = P(V1) · P(V2|V1) · P(V3|V1) · . . . · P(Vn|V1)

Question 18

What are the advantage of NB model ?

Answer 18

efficiency: good running time and small memory size
sparse data problem : enough data to train
good performance: unrealistic independent assumption

Question 19

what are the disadvantage of NB model?

Answer 19

• strong independence assumption

- only one output variable

Question 20

What is smoothing ? and why we use it in probabilistic models ?

Answer 20

• avoid 0 probability

- modify estimated probabilities to correct errors in the dataset.

Question 21

what are the smoothing techniques ?

Answer 21

• add-one smoothing (Laplace smoothing)

- Bell-Witten smoothing

Question 22

What are the evaluation tasks in Hidden Markov Model(HMM) ?

Answer 22

1. evaluation: use HMM assumption formula
2. generation: generate in the order of graphical representation.
3. inference: marginalization, conditioning, and completion
4. learning: MLE if labeled are given.