History of NLP

Question 1

What were the Foundations of NLP and when did they take place?

Answer 1

1940s and 50s, Automaton and CFGs, Probabilistic models, which were motivated by entropy in thermodynamics

Question 2

What were the symbolic approaches of the 50s to 60s?

Answer 2

Symbolic parsers based on reasoning and logic

Question 3

What were the stochastic approaches of the 50s to 60s?

Answer 3

Stochastic approach was Bayesian based language likelihood prediction

Question 4

What were the approaches of 70s to 80s?

Answer 4

Hidden Markov Models - Stochastic
Functional Grammars - Logic
Natural Language Understanding - Parsing + Semantics
Discourse Modelling - Speech acts + discourse

Question 5

What were the NLP approaches of the 80s and 90s?

Answer 5

Empricism and Finite State Models.
There was a return to probabilistic and data driven methods. The enablers of this were the speed and memory growth of computers as well as the introduction of the web corpora

Question 6

What were the NLP approaches of the 2000s?

Answer 6

Statistical approaches to machine translation and topic modelling. The enablers were annotated corpora such as LDC

Question 7

What were the NLP approaches of the 2010s?

Answer 7

Deep learning approaches
Multi-layer deep neural networks.
The enablers were unlabelled or weakly annotated web scale corpora, advances in deep learning from image processing, GPU clusters