Section 8 AI and ML Flashcards
What is Artificial Intelligence (AI)?
AI is a broad field focused on developing intelligent systems capable of tasks requiring human intelligence, such as perception, reasoning, learning, problem-solving, and decision-making.
What are some key use cases of AI?
Use cases include computer vision (self-driving cars, facial recognition), fraud detection, and intelligent document processing (IDP).
What are the main layers of an AI system?
- Data Layer (collecting vast amounts of data)\n2. Machine Learning Framework Layer (defining ML frameworks and algorithms)\n3. Model Layer (training the AI model)\n4. Application Layer (serving the model to users)
What is Machine Learning (ML)?
ML is a subset of AI where machines learn from data to improve performance on tasks without explicit programming.
What are common ML tasks?
Regression (predicting continuous values) and classification (categorizing data points into groups).
What is the difference between AI and ML?
AI is a broad field that includes ML, while ML is a method within AI that enables computers to learn from data without explicit rules.
What is Deep Learning?
A subset of ML that uses artificial neural networks with multiple hidden layers to process complex data patterns.
How does Deep Learning work?
Deep learning models use layers of neurons (input, hidden, and output) to process data, learn patterns, and adjust connections to improve predictions.
What are Neural Networks?
Neural networks are AI models inspired by the human brain, consisting of interconnected neurons that process and learn from data.
What is Generative AI (GenAI)?
A subset of deep learning where models generate new content (e.g., text, images) by learning from large datasets.
What is a Foundation Model?
A large, pre-trained AI model that can be adapted for various tasks, such as GPT models for text generation.
What are Transformer Models?
A deep learning architecture that processes entire sequences efficiently, enabling advanced NLP tasks like ChatGPT.
What are Multi-Modal Models?
AI models that process multiple types of inputs (e.g., text, images, audio) and generate diverse outputs.
How does Generative AI differ from Traditional AI?
Traditional AI classifies or predicts based on existing data, while GenAI creates new content, such as text or images.
What is GPT
Generative Pre-trained Transformer, a model that generates human text or computer code based on input prompts.
What is BERT
Bidirectional Encoder Representations from Transformers, a language model that reads text in two directions, useful for translation.
What is RNN
Recurrent Neural Network, a neural network for processing sequential data like time series and speech recognition.
What is ResNet
Residual Network, a deep convolutional neural network (CNN) used for image recognition tasks like object detection and facial recognition.
What is SVM
Support Vector Machine, an ML algorithm used for classification and regression tasks.
What is WaveNet
A model used to generate raw audio waveforms, commonly used in speech synthesis.
What is GAN
Generative Adversarial Network, a model for generating synthetic data like images, videos, or sounds that resemble training data.
What is XGBoost
Extreme Gradient Boosting, an implementation of gradient boosting used for regression and classification tasks.
What is labeled data?
Labeled data includes both input features and output labels, allowing for supervised learning.
What is an example of labeled data?
Images of animals labeled as ‘dog’ or ‘cat’.
What is unlabeled data?
Unlabeled data has input features but no output labels, requiring unsupervised learning.
What is an example of unlabeled data?
A collection of images without any labels, where the algorithm must find patterns.
What is structured data?
Data that is organized in a structured format, often in rows and columns, such as tabular data.
What is an example of structured data?
A customer database with columns like Customer_ID, Name, Age, and Purchase_Amount.
What is an example of time series data?
Stock prices recorded over time.
What is unstructured data?
Data that does not follow a specific structure, often text-heavy or multimedia content.
What is an example of unstructured data?
Text reviews, social media posts, or images.
Why is having good training data important?
Poor-quality data (garbage in) leads to poor model performance (garbage out).
What is supervised learning?
A type of learning where an algorithm maps inputs to known outputs using labeled data.
What is unsupervised learning?
A type of learning where an algorithm finds patterns in unlabeled data without explicit labels.
What is supervised learning?
Supervised learning is a type of machine learning where a model learns to map inputs to outputs using labeled data.
Why is labeled data important in supervised learning?
Labeled data allows the model to learn the correct output for given inputs, making supervised learning powerful but sometimes difficult due to the challenge of obtaining large labeled datasets.
What is regression in supervised learning?
Regression is used to predict a continuous numeric value based on input data, such as predicting weight based on height.
What is an example of linear regression?
Predicting a person’s weight based on their height using a straight line that best fits the trend in the data.
What is classification in supervised learning?
Classification is used to predict a categorical label (e.g., classifying animals as dogs, cats, or giraffes based on height and weight).
What is the key difference between regression and classification?
Regression predicts a continuous value (e.g., house price), while classification predicts a category (e.g., spam or not spam).
What are examples of regression use cases?
Predicting house prices, stock market trends, and weather forecasting.
What are examples of classification use cases?
Email spam detection, image recognition, fraud detection, and medical diagnostics.
What are the three main data splits in supervised learning?
Training set (60-80%), Validation set (10-20%), Test set (10-20%).
Why is a validation set used?
To fine-tune the model and optimize hyperparameters before final testing.
What is feature engineering?
Feature engineering is the process of transforming raw data into meaningful features to improve model performance.
What is an example of feature engineering?
Converting a birthdate column into an age column to make it more useful for machine learning models.
What are the three main types of feature engineering?
- Feature Extraction (e.g., deriving age from birthdate) 2. Feature Selection (e.g., choosing important variables) 3. Feature Transformation (e.g., normalizing data for better performance).
What is the difference between binary and multi-class classification?
Binary classification predicts two categories (e.g., spam or not spam), while multi-class classification predicts more than two categories (e.g., mammal, bird, reptile).
What is multi-label classification?
Multi-label classification allows multiple categories per input (e.g., a movie can be both ‘action’ and ‘comedy’).
What is unsupervised learning?
Unsupervised learning is a type of machine learning where the algorithm finds patterns and structures in unlabeled data without explicit supervision.
What are common techniques in unsupervised learning?
Clustering, association rule learning, and anomaly detection.
What is clustering in unsupervised learning and name any algorithm used for this purpose??
Clustering is the process of grouping data points based on similarities, such as customer segmentation.
One of the Algorithm Used for this is - K Means Clustering
What is an example of clustering?
Grouping customers based on purchasing behaviors to create targeted marketing campaigns.
What is association rule learning and name any algorithm used for this purpose?
Association rule learning identifies relationships between items, such as frequently bought-together products in a supermarket.
One of the Algorithm Used for this is - Apriori
What is an example of association rule learning?
The Apriori algorithm finds that people who buy bread often also buy butter, helping supermarkets optimize product placement.
What is anomaly detection technique and name any algorithm used for this purpose?
Anomaly detection is the process of identifying data points that differ significantly from normal patterns, often used in fraud detection.
One of the Algorithm Used for this is - Isolation forests, one-class SVM, and autoencoders are popular anomaly detection methods.
What is an example of anomaly detection learning?
It detects unusual transactions (outliers) that differ from normal patterns, helping identify potential fraud.
What is semi-supervised learning?
Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data to improve learning efficiency.
What is pseudo-labeling in semi-supervised learning?
Pseudo-labeling is the process where a model trained on labeled data assigns labels to unlabeled data, which is then used for further training.
What is the benefit of semi-supervised learning?
It reduces the cost of labeling large datasets while still achieving high model accuracy by leveraging both labeled and unlabeled data.
What is self-supervised learning?
Self-supervised learning is a type of machine learning where a model generates its own pseudo-labels from unlabeled data to solve supervised learning tasks.
How does self-supervised learning differ from unsupervised learning?
Unlike unsupervised learning, self-supervised learning generates labels from the data itself, enabling it to solve tasks typically handled by supervised learning.
What is an example of self-supervised learning?
Language models like GPT use self-supervised learning by predicting missing words in text, learning grammar, structure, and meaning without human-labeled data.
What are pre-text tasks in self-supervised learning?
Pre-text tasks are simple, self-generated tasks that a model solves to learn patterns in data, such as predicting missing words or the next word in a sentence.
What are examples of pre-text tasks?
Examples include predicting the next word in a sentence, filling in missing words, reconstructing occluded images, or predicting future frames in a video.
What is reinforcement learning?
A type of machine learning where an agent learns to make decisions by performing actions in an environment and maximizing cumulative reward.
What is an agent in reinforcement learning?
The learner or decision-maker in the environment.
What is the environment in reinforcement learning?
The external system that the agent interacts with.
What are actions in reinforcement learning?
Choices made by the agent, such as moving up, down, left, or right in a maze.
What is a reward in reinforcement learning?
Feedback provided by the environment based on the agent’s actions.
What is an example of a reward system in reinforcement learning?
-1 for a step, -10 for hitting a wall, and +100 for reaching the exit in a maze.
What is the state in reinforcement learning?
The current situation of the environment, which the agent observes before taking an action.
What is the policy in reinforcement learning?
The strategy used by the agent to determine actions based on the current state.
How does reinforcement learning improve over time?
Through many simulations, the agent learns from past mistakes and updates its policy to maximize cumulative rewards.
What are the key steps in reinforcement learning?
Observe the state, choose an action, transition to a new state, receive a reward, and update the policy.
What are some applications of reinforcement learning?
Gaming (chess, Go), robotics (navigation, object manipulation), finance (portfolio management), healthcare (treatment optimization), and autonomous vehicles (path planning).
Why does reinforcement learning require multiple iterations?
The agent improves by trial and error, refining its policy over many attempts.
What is Reinforcement Learning from Human Feedback (RLHF)?
A technique that incorporates human feedback into reinforcement learning to align AI models with human goals, wants, and needs.
What are the four key steps in RLHF?
- Data collection, 2. Supervised fine-tuning, 3. Building a separate reward model, 4. Optimizing the language model with a reward-based model.
Refer Image from AWS
steps in image
1.
https://aws.amazon.com/what-is/reinforcement-learning-from-human-feedback/
What happens during the data collection phase in RLHF?
A set of human-generated prompts and ideal responses are gathered to train the model.
How is the reward model trained in RLHF?
Humans rank different AI-generated responses, helping the model learn human preferences.
What is the role of the reward model in RLHF?
It serves as an automated evaluator of AI-generated responses, replacing the need for continuous human judgment.
Explain AWS image which explains the three-step process of Reinforcement Learning from Human Feedback (RLHF)
https://aws.amazon.com/what-is/reinforcement-learning-from-human-feedback/
Learning from Human Feedback (RLHF):
Step 1: Supervised Fine-Tuning (SFT)
Goal: Train a base LLM (Large Language Model) using human demonstration data.
Process:
Collect human-labeled prompts and responses.
Fine-tune the base model using supervised learning to align with human-like responses.
Step 2: Training a Reward Model (RM)
Goal: Develop a model that evaluates AI-generated responses based on human preference.
Process:
A fine-tuned model (SFT) generates multiple responses to the same prompt.
Humans rank these responses to indicate preference.
A separate reward model (RM) is trained based on human ranking.
Step 3: Optimizing Policy using Proximal Policy Optimization (PPO)
Goal: Improve the model’s response generation by optimizing the policy using the reward model.
Process:
The fine-tuned model (SFT) generates responses to new prompts.
The reward model (RM) evaluates and assigns rewards.
The model updates its policy through reinforcement learning using PPO (Proximal Policy Optimization).
This iterative process ensures that the AI generates more human-aligned, contextually appropriate responses over time.
What is overfitting in machine learning?
Overfitting occurs when a model performs well on training data but poorly on evaluation data because it memorizes noise instead of learning the underlying pattern.
What is underfitting in machine learning?
Underfitting happens when a model performs poorly on both training and evaluation data because it is too simple to capture the underlying patterns in the data.
What is bias in machine learning?
Bias is the error between the predicted value and the actual value, often caused by incorrect assumptions in the model.
What is variance in machine learning?
Variance represents how much a model’s predictions change when trained on different datasets. High variance indicates overfitting.
What causes high bias in a model?
High bias is caused by overly simplistic models that fail to capture the complexity of the data, leading to underfitting.
What causes high variance in a model?
High variance is caused by overly complex models that fit training data too closely and fail to generalize well to new data.
How can bias be reduced in a model?
Bias can be reduced by using a more complex model, adding more relevant features, or improving the data quality.
How can variance be reduced in a model?
Variance can be reduced by simplifying the model, using fewer features, or increasing training data size.
What is the ideal balance in model fitting?
A balanced model has low bias and low variance, meaning it generalizes well to unseen data.
What does a high-bias, low-variance model indicate?
It indicates underfitting, where the model is too simple and fails to capture data patterns.
What does a low-bias, high-variance model indicate?
It indicates overfitting, where the model memorizes training data but performs poorly on unseen data.
What does a high-bias, high-variance model indicate?
It indicates a poor model that neither captures patterns well nor generalizes properly.
What does a low-bias, low-variance model indicate?
It indicates a well-balanced model that effectively captures data patterns and generalizes well.
How can we detect overfitting?
Overfitting can be detected if the model has high accuracy on training data but significantly lower accuracy on test data.
How can we detect underfitting?
Underfitting can be detected if the model has low accuracy on both training and test data.
What is the bias-variance tradeoff?
The bias-variance tradeoff refers to the challenge of balancing bias and variance to achieve optimal model performance.
What type of metrics are used for classification problems?
“Confusion matrix
What is a confusion matrix?
“A table used to evaluate classification models by comparing actual vs. predicted values.”
What is precision in classification?
“Precision = True Positives / (True Positives + False Positives). Measures how many predicted positives were actually correct.”
When is precision more important?
“When false positives are costly.”
What is recall in classification?
“Recall = True Positives / (True Positives + False Negatives). Measures how many actual positives were correctly identified.”
When is recall more important?
“When false negatives are costly.”
What is the F1 score?
“A metric that balances precision and recall: F1 = 2 * (Precision * Recall) / (Precision + Recall).”
What is accuracy?
“Accuracy = (True Positives + True Negatives) / Total Predictions. Often not used for imbalanced datasets.”
What is AUC-ROC?
“A metric that evaluates classification models by plotting true positive rate vs. false positive rate.”
What does an AUC-ROC score of 1 mean?
“The model is perfect in distinguishing classes.”
What does an AUC-ROC score of 0.5 mean?
“The model performs no better than random chance.”
What are different regression evaluation metrics?
MAE(Mean Absolute Error), MAPE(Mean Absolute Percentage Error), RMSE(Root Mean Squared Error), R-squared
What is MAE (Mean Absolute Error)?
“The average absolute difference between actual and predicted values.”
What is MAPE (Mean Absolute Percentage Error)?
“Measures prediction error as a percentage of actual values.”
What is RMSE (Root Mean Squared Error)?
“A metric that squares errors before averaging to penalize larger errors more heavily.”
What is R-squared?
“A measure of how well input features explain the variance in the target variable. Closer to 1 means a better model.”
What is a balanced dataset?
“A dataset where each category has an equal or similar number of instances.”
What is inferencing?
Inferencing is when a model makes predictions based on new data.
What are the types of inferencing?
Real-time inferencing, batch inferencing, and edge inferencing.
What is real-time inferencing?
Inferencing where predictions are made instantly, prioritizing speed over perfect accuracy.
Where is real-time inferencing commonly used?
Chatbots, recommendation systems, fraud detection, self-driving cars.
What is batch inferencing?
Inferencing where large amounts of data are processed at once, prioritizing accuracy over speed.
Where is batch inferencing commonly used?
Data analysis, report generation, medical imaging, financial forecasting.
What is edge inferencing?
Inferencing done on edge devices with limited computing power, often in areas with poor internet connectivity.
What are the benefits of edge inferencing?
Low latency, offline capability, reduced cloud dependency.
What is a limitation of edge inferencing?
Limited computing power, making it difficult to run large models like LLMs.
What is an alternative to running LLMs on edge devices?
Hosting the model on a remote server and accessing it via API calls.
What is the trade-off of using a remote LLM instead of local inference?
Higher latency and requires an internet connection, but allows for more powerful models.
What are Small Language Models (SLMs)?
Compact AI models optimized for edge devices with low computational power.
What are key trade-offs in inferencing?
Speed vs. accuracy, compute power vs. latency, online vs. offline capability.
What are different stages in a machine learning project?
1️⃣ Identify Business Problem →
2️⃣ Frame as ML Problem →
3️⃣ Collect & Prepare Data →
4️⃣ Feature Engineering →
5️⃣ Model Training →
6️⃣ Hyperparameter Tuning →
7️⃣ Model Evaluation →
If Business Goals Not Met 🔄 Go Back to Data Collection / Feature Engineering
If Business Goals Met → Proceed
8️⃣ Model Testing →
9️⃣ Deployment →
🔄 Model Monitoring & Debugging →
🔄 Retrain with New Data → Repeat Process
This cycle ensures continuous improvement and adaptation of the ML model
Why is feature engineering important in ML?
It transforms data into useful features for better model performance.
What happens if the business goals are not met in ML?
Enhance data, perform data augmentation, or improve features.
What is the purpose of monitoring and debugging an ML model?
To ensure the model remains accurate and adapts to changes.
What is data augmentation?
The process of enhancing a dataset by adding more data points or variations.
Why is exploratory data analysis (EDA) important?
To understand data structure, visualize relationships, and compute statistics.
What is a hyperparameter?
Settings that define the model structure and learning process, set before training begins.
What are examples of hyperparameters?
Learning rate, batch size, number of epochs, regularization.
What does the learning rate control?
How fast the model incorporates new data.
What is the effect of a high learning rate?
Faster convergence but risk of overshooting optimal solution.
What is the effect of a low learning rate?
More precise convergence but slower training.
What does batch size control?
Number of training examples used per iteration to update model weights.
What is the effect of a small batch size?
More stable learning but requires more time to compute.
What is the effect of a large batch size?
Faster training but may lead to less stable updates.
What does number of epochs control?
How many times the model iterates over the entire training dataset.
What happens if epochs are too few?
Underfitting – model does not learn enough from the data.
What happens if epochs are too many?
Overfitting – model learns training data too well but fails on new data.
What is regularization used for?
Adjusts balance between simple and complex models.
How does regularization affect overfitting?
Increasing regularization reduces overfitting.
What is overfitting?
Model performs well on training data but poorly on new data.
What causes overfitting?
Small training data, too many epochs, overly complex model.
How can overfitting be prevented?
Increase training data size, early stopping, data augmentation, regularization.
What is the best way to prevent overfitting?
Increase the training data size.
What is hyperparameter tuning?
Finding the best hyperparameter values to optimize model performance.
What are methods for hyperparameter tuning?
Grid search, random search, SageMaker AMT.
Why is hyperparameter tuning important?
Improves accuracy, reduces overfitting, enhances generalization.
When is machine learning NOT appropriate?
When a problem has a deterministic solution that can be computed exactly using traditional programming.
Example of a deterministic problem?
Calculating the probability of drawing a specific card from a known deck.
Why not use ML for deterministic problems?
ML provides approximations, while deterministic code gives exact answers.
What should you consider before applying ML?
Whether the problem requires exact solutions or can tolerate approximation.
