Week 5 - Text Mining III Flashcards
What makes two strings similar?
Key factors in assessing similarity:
1. Lexical similarity - words that share characters or character sequences
2. Semantic similarity - words that share a similar or related meaning
3. Phonetic similarity - words that share similar sounds
What is the underlying challenge of lexical and semantic similarity?
The underlying challenge is developing effective methods to quantify and measure the degree of similarities between strings
What is String Distance Measurement?
Measuring the similarity between two strings at the character level. By quantifying the transformations needed to convert one string into another
What are the type of string distance metrics?
- Levenshtein distance
- Damerau-levenshtein distance
- Q-gram
- Other distance metrics include: hamming, optimal sting alignment and longest common subsequence
What is the purpose of a string distance metric?
The purpose is to obtain a numerical measure of distance between strings
What is the Levenshtein Distance Method?
Levenshtein Distance is when the distance is the minimum number of single character edits required to transform one string (a) into another (b)
What are the edit operations for LD Method?
Substitution, insertion and deletion of characters (weight = 1 point each)
What is the Damerau-Levenshtein Distance?
It’s when you transpose the characters (swapping the two adjacent characters)
How can you convert the distance to similarity?
In other words, what is the formula for converting Levenshtein distance into a similarity score between 0 and 1
simLev (a,b) = 1.0 - (distance (a,b))/max (|a|,|b|)
where |a| and |b| represent the lengths of string a and b
What is q-grams?
q-grams is another method for measuring the distance between two strings.
note that: q is analogous to n in n-grams
What is the string distance?
String distance is the sum of difference between q-gram vectors of both strings
What is the distance between vectors?
Distance metrics, can be used to measure the similarity or dissimilarity between numerical characteristic (vectors) of text
What is a vector?
A vector is a quantity that has a magnitude and a specific direction. It can describe the movement required from one point to another in the space
What is the Euclidean distance?
Euclidean distance is the distance between two points with coordinates (x1, y2) and (x2, y2) is the length of the path (straight line) connecting them
What is the relationship between Euclidean distance and magnitude?
Euclidean distance takes into account the magnitude of vectors, but this is not always meaningful for text data
What is another metric to measure the Euclidean distance?
Cosine distance: This is a metric that does not use the magnitude of vectors, but is used when the magnitude of the vectors does not matter
How can cosine distance measure the vectors?
Cosine distance is computed 1-cosine_similarity. Cosine similarity is the cosine of the angle between two vectors.
Words are more likely to be similar if they have a smaller angle, even if they are far apart by magnitude
What is the problem with lexical similarity II?
Lexical metrics fail to capture the words that are most similar in meaning (i.e., semantic dimension)
What areas of lexical similarity II should be improved?
- Meaning of words determined by information associated with it
- Each word is represented as a vector of real-valued numbers containing information associated with the word
- Similar words are closer/nearby in the vector space
What is ‘one-hot encoding’ ? And why do people use it?
‘one-hot encoding’ is a basic method for representation of words as vectors. Hence, each row has one element (column) that is “hot” (1) and others are 0s.
1 means the presence of the word. We use it because it is easy to understand and straightforward to compute
What are the challenges/problems with ‘one-hot encoding’?
- Super-sparse: only one element in each row has a non-zero value
- Computationally inefficient - a vocabulary that has 100,000 then you get a vector with a dimension of size 100,000
- Similarity context or relationships with other words are not captured
- Dot product of one-hot vector is always zero
- We need a denser representation of words that store information about words in a manageable dimensional space
What is word embedding?
Word embedding is a a technique to address the limitations of one-hot encoding
What are the approaches of word embedding?
There are two approaches: word2vec and glove
How is information represented in word embedding?
Information is represented as a set of floating numbers an N-dimensional vector. Each word has an embedding, and each embedding has the same length. And each column (dimension) captures some contextual information.
What is a learning process in the context of Text Mining III?
The learning process is when algorithms learn word embeddings from a training text (large corpus of text) in an unsupervised manner through the context in which they occur (e.g., surrounding words).
We can download the pre-trained models and retrieve the embeddings for a given word.
What is Word2vec?
Word2vec: embedding is obtained from models that predict the target word from the context words (or surrounding words), or predict the context words given the target word
Define CBOW and Skip Gram
CBOW - Continous Bag of Words; a model that predicts the target word in the centre using surrounding context words
Skip-gram - a model that predicts the context words from the target word in the centre
Define GloVe
GloVe (Global Vectors for Word Representation): This model is based on an aggregated global occurrence matrix that shows how words in a corpus co-occur with each other.
It has an aim to learn word vectors such that their dot product equals the logarithm of the words’ probability of co-occurence
What is the process for word embedding in R?
- Initialise the pre-trained models
- Calculate the similarity between words
Why does Arithmetic with word embeddings work?
Arithmetic with word embeddings work because of ‘distributional hypothesis’; meaning that words that occur in the same contexts tend to have similar meanings
What is the limitations of word embeddings?
Word embeddings are very powerful but they can be
1. instable
2. hard to interpret
