Recommender System Flashcards
What is “Top-N recommenders”? Give 3 Examples
Produce a finite list of the best things to present to a given person.
What is Explicit feedback? Give 3 Examples
Directly ask for users’ interest.
Example: Rating, Reviews, Like/Dislike
What is the disadvantage of Explicit Feedback?
Humans can get tired of providing so much feedback, and will not reliably provide accurate or truthful feedback
What is the advantage of Explicit Feedback?
Clear preferences: Explicit feedback provides clear indications of user preferences and opinions. Interpretable: Ratings and reviews are straightforward to interpret and incorporate into recommendation models
What is Implicit feedback? Give 3 Examples
Look at users’ behaviours and interpret them as indications of interest or disinterest.
Example: Purchase History, Click-Through Rate (CTR), Viewing Time
What is the disadvantage of Implicit Feedback?
Lack of explcit preferences: Implicit feedback might not precisely reflect users’ true preferences, leading to potential ambiguity.
Difficulty in interpreting: Understanding the exact meaning of implicit signals can be challenging.
What is the advantage of Implicit Feedback?
Abundant data: Implicit feedback is often more abundant than explicit feedback, as it is generated passively during user interactions.
Less user effort: Users don’t need to explicitly rate items, making implicit feedback collection less intrusive.
List some accuracy metrics to evaluate a recommender system.
Examples: Mean Absolute Error, Root Mean Square Error.
Why do accuracy metrics matter?
It provides an offline and immediate evaluation methods for tuning, comparing and benchmarking the model.
However, accuracy and rating predictions have limited value in practice, as the goal is not to minimize the error, but to find what people like. Fundamentally, it’s impossible to evaluate the value of a recommender system offline.
What is Hit Rate?
the proportion of users for whom the system made at least one relevant recommendation.
Why does Hit Rate Matters?
A higher Hit Rate indicates that a larger portion of users received recommendations they found useful, contributing to overall user satisfaction.
What are considerations of using Hit Rate?
The larger the list of recommending items, the higher the hit rate is. Therefore, choose a reasonable value for L
What is Average reciprocal hit rate (ARHR)?
a hit but it accounts for where in the top end list the hit appear.
Why does Average reciprocal hit rate (ARHR) matter?
A higher ARHR score implies that the system is effective in identifying and placing relevant items at the forefront of the recommendation list.
What is Cumulative hit rate (cHR)?
Add a threshold for the test set so that we test on items with acceptable ratings.
Why does Cumulative hit rate (cHR) matter?
We shouldn’t get credit for recommending items to a user that we think they won’t actually enjoy
What is Coverage?
the % of (user, item) pairs that can be predicted or percentage of possible recommendation that the recommender system can provide.
Why does Coverage matter?
The ability of the recommender system to recommend all items from a train set to users.
What are considerations of using Coverage?
There can be a trade-off between coverage and accuracy. A system with the ability to recommend all items to a user is no different than a random prediction.
What is Diversity?
How broad a variety of the recommending items. (1-S), in which S is the average similarity between recommendation pairs.
Why does Diversity matter?
Avoid repetitiveness and discover user’s new interests
What are considerations of using Diversity?
High diversity is not always good. You can achieve very high Diversity by just recommending completely random items.
What is Novelty?
A measure of how popular the recommending items.
Why does Novelty matter?
Novelty can help to increase user engagement and satisfaction
What are considerations of using Novelty?
Need to find a balance between finding familiar popular items and the serendipitous discovery of new items.
What is Long tail?
There will be always an exponential distribution where most sales come from a very small number of items, but long tail also makes up a large amount of sale
Why does Long tail matter?
Recommender systems can help people discover those items in the long tail that are relevant to their own unique niche interests. If you can do that successfully, then the recommendations your system makes can help new authors get discovered, can help people explore their own passions, and make money for whoever you’re building the system for as well.
What is Responsiveness?
How quickly does new user behavior influence the recommendations
Why does Responsiveness matter?
The faster the responsiveness allows the system easily catch up to the current trends and patterns.
What are considerations of using Responsiveness?
The trade off is complexity and responsiveness
What is Perceived Quality?
straight up ask your users if they think specific recommendations are good.
Why does Perceived Quality matter?
Explicit Feedback for the recommender system
What are considerations of using Perceived Quality ?
Noisy data as there is no clear indicate/standard for a good recommendation
What is A/B Testing?
Put recommendations from different algorithms in front of different sets of users and measure how they react to the presented recommendations
Why does A/B Testing matter?
One of the best way to tune the recommender system. Result of online A/B test is emphasized as the most matter metric.
What are considerations of using A/B Testing?
Complex and expensive to execute and maintain.
What is Content-based filtering? Give 3 Examples
A technique that suggests items to users based on the characteristics or attributes of items users have rated, analyzing the content of the items themselves.
List some popular similarity metrics
Consine Similarity, Pearson Correlation Coefficient, Jaccard Similarity, Euclidean Distance, etc.
What is Cosine Similarity?
Cosine similarity measures the cosine of the angle between two vectors. It ranges from -1 (completely dissimilar) to 1 (completely similar).
Why do you use Cosine Similarity?
Commonly used in text mining, collaborative filtering, and information retrieval.
Suitable for high-dimensional data where the direction of vectors matters more than the magnitude.
What is Euclidean similarity?
Euclidean distance measures the straight-line distance between two points in a multi-dimensional space. Smaller distances indicate greater similarity.
What is the usage of Euclidean similarity?
Suitable for continuous numerical data in a multidimensional space.
Often used in clustering, pattern recognition, and image analysis.
What is Hamming Distance?
the disimilarity between two strings of equal length. It is defined as the number of positions at which the corresponding symbols (characters or bits) are different.
What is the usage of Hamming Distance?
Primarily used for binary data, such as error detection and correction codes. Also applicable to categorical data with a predefined order.
What is Jaccard Similarity?
Jaccard similarity measures the proportion of common elements between two sets. It ranges from 0 (no common elements) to 1 (identical sets).
Why do you use Jaccard Similarity?
Commonly used in set similarity, document similarity, and recommendation systems. Suitable for scenarios where the presence or absence of elements is important.
What is Adjusted cosine similarity?
measure the similarity between users or items based on their ratings. It is an extension of the traditional Cosine Similarity, with adjustments made to account for user (or item) biases. Adjusted Cosine Similarity considers the ratings given by users while accounting for the user’s overall rating tendencies.
Why do you use Adjusted cosine similarity?
Collaborative filtering, especially when accounting for user/item biases.
What is Pearson similarity?
Pearson correlation measures the linear relationship between two variables. It ranges from -1 (perfect negative correlation) to 1 (perfect positive correlation).
Why do you use Pearson similarity?
Widely used in statistics, collaborative filtering, and linear relationships. Suitable for continuous data where a linear relationship is expected.
What is Spearman rank correlation?
measures the strength and direction of association between two ranked variables
Why do you use Spearman rank correlation?
Suitable for ordinal or ranked data, often used in non-parametric statistics. Useful when the relationship between variables is monotonic but not necessarily linear.
What is Mean squared difference?
a measure of the average squared differences between corresponding elements of two vectors.
Why do you use Mean squared difference?
Commonly used as a loss function in regression analysis, optimization problems, and model evaluation. Measures the average squared differences between corresponding elements.
How to use KNN in Content-based Filtering?
- Measure similarity score between a movie and all other rated items. Repeat to all unrated items.
- Sort and produce the top items.
- Use weight average to predict the rating.
What is Neighborhood-based Collaborative Filtering?
A technique that taking cues from people like you and recommend stuff based on the things they like that you haven’t seen yet. Recommending stuff based on people’s collaborative behaviors.
What is Sparsity in Recommender System?
sparsity refers to the proportion of missing or unrated values in the user-item interaction matrix.
Recommender system matrices are often sparse because users typically interact with or rate only a small fraction of the items available in the system. Most entries in the matrix are missing.
What is the challenges made by Sparsity?
The sparsity of the matrix poses challenges for recommendation algorithms because it means that a large portion of the user-item interaction data is unknown. This can lead to difficulties in accurately predicting user preferences for unrated items.
What is User-based Collaborative Filtering?
Recommend stuff that similar users like that you haven’t seen yet.
How is the User-based Collobarive Filtering done?
- Collect data, 2D array, each user will have a vector that represents their ratings for each item. This is item rating matrix.
- Compute cosine similarity between any pair of users. This is user similarity matrix.
- Sort the list and pick up the top-n similar users
- Candidate generation: take the items rated by the chosen users
- Candidate scoring: there are many ways. Example: normalize rating scores.
- Candidate sorting: using the above scoring
- Candidate filtering: remove item that already
What is Item-based Collaborative Filtering?
Recommend things that similar to the item users like. One advantage is that items are more permanent and suits for small amount of dataset.
How is the Item-based Collobarive Filtering done?
- Represent data in which each item have a vector contains its characteristics, or it could be how other users like it or not. —> item rating matrix
- Compute cosine similarity between any pair of items. —> item similarity matrix
- Sort the list and pick the top-n similar item
- Candidate filtering
How to use KNN in Item-based and User-based Collaborative Filtering?
User-based KNN: for user u and item i
1. Find the k-most similar users who rated the item
2. Compute the mean similarity score weighted by ratings
3. Rating prediction
Item-based KNN: for user u and item i
1. Find the k most-similar items also rated by a user
2. Compute mean similarity score weighted by ratings
3. Rating prediction
What is matrix factorization?
Describe users and items as combinations of different amounts of each feature.
Why does matrix factorization matter?
Matrix factorization is particularly effective in handling sparse matrices commonly encountered in recommender systems.
How to implement matrix factorization?
Describe the training data in terms of smaller matrices that are factors of the ratings we want to predict.
- R = MΣtrans(U), this automatically fills in unrated pairs of user and item. Given:
- M is the PCA matrix
- U is the original matrix, missing value can be completed by putting random default value at first and keep minimize the error rate of these missing values.
What is Principle Component Analysis (PCA)? Why does it matter?
PCA is a feature extraction technique. In a recommender system, PCA find and extract latent features from the data.
PCA tries to find principle components, which are eigenvectors
Eigenvectors: a vector that describes the variance the best and its orthogonal vector. —> define a new vector space that can fit the data.
What is Singular Value Decomposition (SVD)?
Singular value decomposition is technique to decompose R into M, Σ and U.
Techniques such as SGD (stochastic gradient descent) and ALS (alternating least square) can be used to learn the best values of those factored matrices when having missing data. Therefore, this is more of a SVD-inspired algorithm.
What is gradient descent in Recommender System?
Gradient descent is an optimization algorithm commonly used in training machine learning models. The goal of gradient descent is to minimize a cost or loss function by iteratively adjusting the model’s parameters in the direction of steepest descent of the gradient. This iterative process continues until the algorithm converges to a minimum or until a predefined stopping criterion is met
What is Auto Differentiation in Recommender System?
a technique for computing derivatives of functions automatically.
What is Softmax function in Recommender System?
The softmax function takes a vector of real numbers as input and outputs a probability distribution that sums to 1. Used for tasks involving multi-class classification or probability distribution
What is backpropogation?
a key algorithm used in training artificial neural networks. The backpropagation algorithm is a supervised learning method that enables the optimization of model parameters by minimizing a chosen loss or objective function.
How is backpropogation executed?
- Forward Pass: input data is fed into the neural network to generate output values.
- Calculate the loss function
3, Backward pass: computing the gradient of the loss with respect to each model parameter. This is achieved by applying the chain rule of calculus. - Parameter Update and Repeat
What is activation function? List some activation functions and differentiate those.
A mathematical operation applied to the output of a neuron in a neural network
Example:
- Sigmoid Activation Function: binary output
- Hyperbolic Tangent Activation Function: Similar to the sigmoid but with a range of (-1, 1), making it zero-centered
- Rectified Linear Unit (ReLU) Activation Function: The most widely used activation function. It introduces non-linearity and is computationally efficient
- Softmax Activation Function: Often used in the output layer for multi-class classification
What is optimization function? List some activation functions and differentiate those.
Defines the strategy for adjusting the model parameters during training to minimize the chosen loss or objective function.
Example:
- Stochastic Gradient Descent (SGD): Iteratively updates parameters based on the gradient of the loss with respect to the parameters
- Adam (Adaptive Moment Estimation): Combines ideas from RMSprop and momentum.
- RMSprop (Root Mean Square Propagation): Adapts the learning rates of each parameter based on their historical gradients.
- Momentum Optimization: add a momentum term to the update rule for the model parameters
How to prevent overfitting in deep learning?
- Regularization Techniques: L1, L2, and Elastic Net Regularization Regularization
- Dropout: Randomly dropout (ignore) a fraction of neurons during training
- Data Augmentation: Increase the effective size of the training dataset by applying random transformations (rotations, translations, flips) to the input data.
- Early Stopping: Monitor the performance on a validation dataset during training and stop the training process when the performance starts to degrade
What is the practice to tune the topology of a deep learning model?
- Start Small and Gradually Increase Complexity
- Start with a sequential approach, adding layers one at a time and use early stopping when the model starts to degrade.
- Use Model zoos and Pre-trained architecture
What is Tensorflow?
An architecture for executing a graph of numerical operations Optimizing the processing of that graph, and distribute its processing across a network and distribute work across GPUs. Tensor: an array or matrix of values
How to use Tensorflow to create a neural network model.
- Load up the training and testing data
- Construct a graph of the neural network:
- Use placeholders for the input data and target labels
- Use variables for the learned weights for each connection and learned biases for each neuron.
- Associate an optimizer
- Run the optimizer with the data
- Evaluate with testing data
Note: Make sure features are normalized so that every input feature is comparable in terms of magnitude.
What is Keras?
Keras is an open-source high-level neural networks API written in Python
What is Convolutional Neural Networks (CNNs)?
The idea is to take a source data, break it up into chunks called convolutions, and then assemble those and look for patterns at increasingly higher complexities at higher levels of the neural network.
What problems is the CNN good for?
CNN is good for unstructured data to look for feature location invariants.
What are challenges of CNNs?
resource-intensive, lots of hyperparameters. training data
How to implement CNNs with Keras?
Source data must have appropriate dimensions
- Conv2D layer does the convolution on a 2D image
- MaxPooling2D layers can reduce a 2D layer down by taking the maximum value in a given block.
- Flatten layers will convert the 2D layer to 1D layer for passing into a flat hidden layer of neurons.
Typical usage: Conv2D → MaxPooling2D → Dropout → Flatten → Dense → Dropout → Softmax
What is Recurrent Neural Network (RNNs)?
there is a loop over a neuron, in which the output from previous run will be the input and improve the learning capability of a neuron, that still remain characteristics of the previous runs’ outputs.
What problem is the RNN good for?
for sequence of data (time-series data), or sequence of arbitrary length (machine translation, image captions, machine-generated music)
What are challenges of RNNs?
sensitive to topologies, choice of hyperparameters, resource-intensive
How to implement RNNs with Keras?
Implementing Recurrent Neural Networks (RNNs) with Keras involves using the SimpleRNN, LSTM (Long Short-Term Memory), or GRU (Gated Recurrent Unit) layers provided by Keras. These layers allow you to model sequential and time-dependent data, making them suitable for tasks such as time series prediction, natural language processing, and more
What is the challenge when using deep learning for a recommender system?
The challenge when using deep learning for a recommender system is the algorithm has to work with sparse data. Deep learning models, especially neural collaborative filtering approaches, may struggle to effectively capture patterns in sparse data. This problem is signified with the cold start problem.
What is Restricted Boltzmann Machines?
Contain two layers:
- A visible layer: feed our training data into this layer in a forward pass
- A hidden layer: train weight and biases during back propagation. Activation function produce the output.
RBM’s for recommender systems: Use each user in the training data as a set of inputs into RBM:
- The visible nodes represent ratings for a given user and the network constructs weights and bias to produce the prediction for unknown pair of users and items.
- Deal with sparse data by excluding any missing ratings from processing while training the RBM.
- Function to optimize: Contrastive Divergence - sample probability distributions during training using Gibb sampler.
What is Autoencoder for Recommendations (“AutoRec”)?
Autoencoders are used to learn a low-dimensional representation of user-item interactions. The encoder captures essential features, and the decoder reconstructs the original input.
Contain 3 layers:
- An input layer that contains individual ratings
- A hidden layer
- An output layer
A matrix of weights between the layers is maintained across every instance of this network, as well as the bias.
Deal with sparse data: excluding the unrated observations.
This algorithm does two thing:
- Encoding the patterns in the input as a set of weights into a hidden layer
- Decode the weights between output and hidden layer to construct the output.
What is Session-based Recommendations wwith RNN (GRU4Rec)?
Given a sequence of action, predict the actions, or items that are most likely to continue the sequence of events.
The idea is that the item is coded as one event, goes into embedding layer, and goes into GRU layers, and get the scores on all of items, from which we can select the items the deep network thinks the most likely to extend the sequence.
What is Apache Spark and How Apache Spark can help scale up a Recommender System?
a framework for processing massive datasets across a cluster of computers. Spark Driver Script communicates with cluster manager to distribute workloads to the executors.
This distributed architecture allows Spark to handle large-scale data processing tasks efficiently by distributing the workload across multiple nodes.
What is Resilient Distributed Dataset (RDD) ?
an object that capsulates the data you want to process. A Spark Driver Script is about defining operations on RDD like where should load its data from, what operations to perform and where to write the outputs. RDDs serve as the foundational data structure in Spark, providing fault tolerance, parallel processing, and distributed computing capabilities
What is DSSTNE and How DSSTNE can help scale up a Recommender System?
Deep Scalable Sparse Tensor Neural Engine. It helps set up deep neural networks using sparse data. DSSTNE can run on a GPU and work with Apache Spark to further scale. Open-source. DSSTNE is designed to efficiently process sparse data, making it particularly well-suited for recommender systems where user-item interaction matrices are often sparse
What is Amazon Sagemaker?
It allows you to create notebooks hosted on AWS that can train large scale models in the cloud. It comes with some useful algorithms.
List some challenges in building the Recommender System?
Examples: Cold-start problem, Filter Bubbles, Gaming the System, Temporal Effect
What is the Cold-start problem?
new users or new items for which there is little to no historical interaction data
How to overcome the Cold-start problem?
Solutions on new users: use implicit data, cookies, geo-ip, top-sellers or promotions, or conduct the interview.
Solutions on new products: use content-based attributes, map attributes to latent features, random exploration
What problems could be solved by implementing a stoplist?
Stoplists check if an item might cause unwanted controversy. For example: It could be adult-oriented content, vulgarity, legally prohibited topics, terrorism/political extremism, bereavement, medical, competing products, drug, religion, etc.
What is Filter Bubbles?
Content presented to the user is filtered that it keeps within a bubble of the pre-existing interests
How to overcome Filter Bubbles?
randomly introduce new interests
How are Transparency and Trust a problem in the Recommender System?
Users may find it challenging to understand the reasoning behind recommendations, leading to a lack of transparency and erode user trust in the system.
How to improve Transparency and Trust in the Recommender System?
Let users know why the system recommends a particular item and allow users to fix the root cause themselves.
How are Outliers a problem in the Recommender System?
Outliers might be created by professional reviewers, institutional customers or bots, leading to Skewed Recommendations and model performance.
How to overcome Outliers in the Recommender Systems?
Use appropriate algorithms, data processing, normalization or standardization
What is Gaming the system in the Recommender Systems?
people trying to interrupt your recommendation system.
How to overcome Gaming the system in the Recommender Systems?
only consider users who spent real money; restrict reviews to people who actually consumed the content; be wary of click data; implicit data can lead to many problems
How are International markets and laws a problem in the Recommender System?
Failure to account for cultural differences may result in recommendations that are perceived as inappropriate or irrelevant
Different countries have varying regulations regarding user privacy and data protection
How to overcome International markets and laws in the Recommender Systems?
keep things separated geographically. Have to consider the regulations and policies carefully.
What are Temporal Effects in the Recommender System?
impact of time-related factors on user preferences, item popularity, and overall system dynamics. Example: Seasonal Variation, Trending Items
How to overcome Temporal Effects in the Recommender Systems?
take the recency into account.
What is Value-aware recommendations?
Create the system that drives the most profit, not the best relevance
How to achieve Value-aware recommendations?
use a profitability as a tie-breaker; look at profit margin instead of profit.
What are Hybrid solutions in Recommender System?
Ensemble approaches: combining different algorithms; combining behavior and semantic data
