Sentence Segmentation Flashcards

1
Q

Describe the task of sentence segmentation

A

determine boundaries between sentences

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Describe the ambiguity problem in sentence segmentation

A

There are many different eos characters and styles.
e.g. “xyz”. in UK but “xyz.” in America
If we are thinking of medical writing, a sentence may not begin with a capital letter if it is a protein or something

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Describe the approaches to sentence segmentation (5)

A
  • regular expressions
  • dictionaries of abbreviations
  • hand crafted rules
  • statistical and ml
  • hybrid
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Give 2 example rules for sentence segmentation

A
  • Check first character after potential eos

- dictionary of abbreviations

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What tools are available for sentence segmentation

A

openNLP sentence detection (ml based)

Spacy (ml or rule based)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is domain dependence

A

This is the idea that the use of language is different based on its domain, e.g. medical writing

How well did you know this?
1
Not at all
2
3
4
5
Perfectly