NER

Question 1

What does NER stand for?

Answer 1

Named Entity Recognition

Question 2

What is a Named Entity?

Answer 2

It is anything with a proper name

People, Locations, Organisations, Events, Dates, Time, Money

Question 3

What is Named Entity Recognition?

Answer 3

It is the task of labelling a text span with types of named entities

Question 4

What are some popular NER tags?

Answer 4

People (PER)

Organisations (ORG)

Locations (LOC)

Geo-Political Entity (GPE)

Question 5

What are NER approached based on?

Answer 5

They are based on tagsets

Question 6

What is a popular NER tagset? How many types does it define?

Answer 6

The Automatic Contact Extraction (ACE) tagset is a very popular that defines 7 types.

Question 7

What are some challenges of NER?

Answer 7

It works with spans, so working out how big a phrase needs to be labelled for a NE

Named entity type ambiguity, where a NE can be different types depending on the context

Question 8

What is the key method used for NER?

Answer 8

We treat it as a sequence labelling problem so we use BIO tagging

Question 9

What is BIO tagging?

Answer 9

It is a common approach for sequence labelling requiring span-recognition

Question 10

What does BIO stand for?

Answer 10

Begin, Inside, Outside

Question 11

What is the idea of BIO tagging?

Answer 11

We assign a tag to each word in our sequence, and each tag may represent the beginning, the middle or the end of something

Question 12

Explain what the image shows

Answer 12

It shows different types of tagging methods. It shows that IO is difficult to comprehend, as it is difficult to understand where one NE begins and ends. BIO tagging builds on this by using begin labels, which shows where NEs begin. BIOES takes this even further.

Question 13

What model is used to learn text according to the BIO scheme to identify NEs?

Answer 13

Conditional Random Fields (CRFs)

Question 14

What type of features are useful for NER?

Answer 14

Non-word features such as captilisation.

Question 15

Why is the Hidden Markov Model (HMM) not a good model for NER?

Answer 15

As they are generative, is is hard to add feature patterns

Question 16

What type of model is CRF?

Answer 16

It is a discriminative sequence model based on a log-linear model. It is widely used for this type of sequence labelling problem.

Question 17

What is the input and output for a CRF model, and their lengths?

Answer 17

The input is a sequence of words, the output is a sequence of BIO tags. The length of the input will always be the same as the length of the output

Question 18

What does CRF want to find?

Answer 18

It wants to find the most probable sequence given a set of all possible sequences for a set of words.

Question 19

What is the equation for CRF?

Answer 19

Y hat is the most probable sequence, which is found using argmax. We compute the probability for each possible sequence we have given the input words and let argmax take the most probable

Question 20

What does CRF define?

Answer 20

A function F, which takes an input of a sequence of words and a sequence of BIO tags up to that point.

Question 21

What needs to be created in order to performed NER?

Answer 21

A set of K features

Question 22

What does each feature have in a CRF model?

Answer 22

Each feature has a corresponding weight

Question 23

What is the global feature vector in a CRF model?

Answer 23

It is the sum of all local features

Question 24

What are the local features in a CRF model?

Answer 24

Local features are features at a particular word index in the sentence. For each index position in the sentence, we use the local feature function to compute the features for that particular position. This can be summed to give a global feature vector for the entire sequence.

Question 25

What makes a linear chain CRF gain its name?

Answer 25

The input is the set of words X, the index we are dealing with, the tag that is being predicted and the previous tag. It can only look one tag in the past, a full CRF would be able to look further behind, but it is much more computationally expensive.

Question 26

How are local feature values created?

Answer 26

They are populated using a manually designed feature template for each word in the sequence.

Question 27

Where can features get their information from?

Answer 27

They can get them from anywhere in the word sequence X, as we can define the context window to be whatever we want it to be.

Question 28

What value types can a feature set contain?

Answer 28

It can contain many different types (numeric, text, boolean)

Question 29

What are some common types of features?

Answer 29

Words, POS tags, Word shape type, Word prefix or suffix, Match to a lexicon (e.g. list of names) or gazetteer (e.g. list of cities)

Question 30

What happens to the feature values over a sentence?

Answer 30

They are summed over the entire sentence X, meaning that there are always K features regardless of the sentence length.

For example, if we have 7 local features, and a sentence that is 5 words long, we can get the local features for each of the words, so each word in the sequence is now represented by the local feature vector. To get just one feature vector, we then sum each of these feature vectors to get just one feature vector that has 7 features representing the entire sentence.

Question 31

Explain what equation shows.

Answer 31

The equation is for the CRF model. It takes the maximum of the sum for each token position, the sum of the weighted local feature vector. For every feature type, for every position in the sequence, we sum it. This selects the most likely tag sequence for an input X.

Simply put, we compute the global feature vector for the input sentence. We then weight each of the features in the global feature vector differently and we want to find the best solution with argmax

Question 32

How do we evaluate NER taggers?

Answer 32

We can use micro or macro Precision, Recall and F1 scores

Question 33

What do we apply the evaluation metrics to?

Answer 33

The entities as a whole rather than single words or tags as the NEs can be of different lengths

Question 34

What needs to be avoided with NE?

Answer 34

Bias needs to be avoided with entities, e.g. is New York a good match for New York City, or do want exactly New York City as the match?

Question 35

What happens to the O tags in NE?

Answer 35

The O tag matches will be removed since the majority of the corpus is an O tag, and this will bias results reported using mean scores to the O class performance.

Question 35

What happens to the O tags in NE?

Answer 35

The O tag matches will be removed since the majority of the corpus is an O tag, and this will bias results reported using mean scores to the O class performance.

Question 36

What are some alternatives to the CRF model?

Answer 36

Word Representations (CBOW, skip-gram, word2vec, GloVE, BERT)

Character Representations (LSTM, GRU, CNN)

Model (CNN, LSTM, GRU)

Decoder (softmax, CRF)

Question 37

How is CRF different to HMM?

Answer 37

In HMM, we have to use Bayes theorem and the likelihood P(X|Y) to calculate P(Y|X).

In CRF, we can compute P(Y|X) directly, training the CRF to discriminate among the best possible tag sequences