Lecture1 Flashcards
What is NLP
NLP is about manipulating text, cleaning text, normalizing text, and analyzing it in a way that allow you to derive meaning without suffering time and speed.
C:\Users\hashem\AppData\Roaming\nltk_data
Where the book is downloaded
concordance method
Gives you alll the places where a word was used
Different context a word is used with all it is places
Common_contexts
What context two words share
similar
Gives you words that appear in contexts that are similar
Collocation: pair of words that occur together everytime
True
Dis[persion plot
tells you where the word occur in your text and how many times
Vocabulary plot
Show you the count of words
count to count the words
True
Length to find the length of the text if u want
True
Stop words
Words that occur at high frquency like: the
freqdist(text)
Find the frequency of each word in the text
To find if the text is rich we use function set
How many distinct word the text contan
Lexical richness
size of text over Size of distinct vocab
We use the list to view text as sequence
True
Corpus
Collection of text files
raw
returns the text as is (like the new line is \n)
field()
Retuen the text file name
field()
Retuen the text file name
print it in a neat way unlike raw
Stop word is a word when removed fom the text it will not afect the meaning in any way (We still get the semantics)
true
not is apllication dependent as stop word
true
limitising : removing ing from each word
TRUE
