Lecture 2 Flashcards
What is rote learning?
Learning by memorization, so like these flashcards but also used in the machine learning.
What is supervised learning?
Use specific examples to reach general conclusions or extract general rules. So we give (X, Y) training set.
What is unsupervised learning?
Unsupervised identification of natural groups in data. So we give X values and group the results
What is reinforcement learning?
Feedback (positive or negative reward) given at the end of a sequence of steps
Why is Rote learning limited?
It memorizes the input/output pairs and performs matching with new inputs. If the computer has not seen the precise case before, it cannot apply its experience.
What is evaluationary learning?
Inspired by evolution: adapts to environment to improve the survival rate and chance of having offspring
What is the most important factor in learning?
Generalization - we want computers to generalize from prior experience
What is explanatory modeling?
Application of statistical models to data for testing causal hypotheses about theoretical constructs
Useful description and explanation of why and how thing works
“what can we do about it”
What is predictive modeling?
A process of applying a statistical model or data mining algorithm to data for the purpose of predicting new or future observations
“what is likely to happen”
What is a predictive model?
Any method that produces predictions, regardless of its underlying approach
What are different AI levels?
- Narrow AI (weak AI)
- Designed for a specific task, often outperforming humans
- Virtual assistants, image recognition systems - General AI (strong AI)
- Human like cognitive abilities, can understand, learn and apply knowledge across multiple tasks
- Only theoretical currently - Super AI (AGI - Artificial General Intelligence)
- Surpasses human intelligence in virtually every aspect and can outperform humans in any economically valuable work
- Autonomous learning, problem-solving, cretivity
- Mainly speculative and associated with potential risks - Singularity
- Hypothetical point in the future where technological growth becomes uncontrrollable and irreversible, leading to unforeseeable changes in human civilization.
What is pattern recognition?
A branch of machine learning that focuses on the recognition of patterns and regularities in data
For example recognizing is an email spam or non-spam
How are pattern recognition systems trained?
Mainly from labeled training data (supervised learning), but when no labeled data available then unsupervised or semisupervised learning
Whats the difference between pattern recognition and pattern matching algorithms?
Pattern recognition aims to provide “most likely” answer matching of the inputs
Pattern matching algorithm looks for exact matches in the input with pre-existing patterns
What is the classical way for pattern recognition?
sensor -> representation pattern -> feature selector/extractor -> feature pattern -> classifier -> decision
What is feature vector?
Raw input data from sensors preprocessed to obtain a feature vector X, that describes all of the relevant features for classifying examples
X is a list of attribute, value pairs X=[color=Orange Shape=Round]
What is a feature space?
Each example can be interpreted as a point in an n-dimensional feature space, where n is the number of attributes
What is a pattern?
A pattern could be an object or event, like “hand”, “head”, “horse”, number etc.
What are different goals in pattern recognition?
- Recognize objects within the input signal, for example determine that written 2 is 2.
- Segmentation of objects, for example partitioning image into multiple segments, like highlightin all the people from the image
- Region based approaches -> recognizes the image regions (aka boxes) of the object, so if there is a boat in the image it makes a box around it
- Semantic segmentation -> classifies each image pixel to its corresponding class, like people, cars, trains etc.
What are different pattern classes?
- Intra-class variability: for example different types of letter T
- Inter-class variability: Letters/numbers that look similar, for example OD, B8, 2Z, I1
How are pattern classes typically expressed?
In terms of a statistical model, for example histogram, probability density function etc.
What are Pattern Recognition Objectives?
- Hypothesize the models that describe each pattern class
- Given a novel pattern assign it to the pattern class associated with the model (predictive modelling)
What is the difference between classification and clustering?
Classification: categories are known, finds letters for “category A, category B”. Supervised
Clustering: Creates categories, for example separates all 1 and I to two categories. Unsupervised
What are the stages in a pattern recognition problem?
- Formulation of the problem: gaining a clear understanding and planning the stages
- Data collection
- Initial examination of the data - summary statistics, plots etc.
- Feature selection or feature extraction for the task
- Unsupervised pattern classification or clustering (exploratory data analysis)
- Apply discrimination or regression procedures as appropriate
- Assessment of results, often in the form of confusion matrix
What is statistical pattern recognition?
It attempts to classify patterns based on a set of extracted features and determine underlying statistical model for the generation of these patterns
What is structural pattern recognition?
A field within pattern recognition that focuses on analyzing the structure of patterns or objets. Recognizes relationships and connections between different components of a pattern
For example: square
Statistical: “number of segments:4”
Structural: “horizontal -> vertical -> vertical -> horizontal”
What are key concepts of structural pattern recognition?
Graph theory: Utilized for modeling and representing relationships in patterns
Hierarchical structures: Recognition of patterns organized in a hierarchical manner
Syntactic analysis: involves the use of formal grammars to describe and recognize patterns
What is the cost of miss-classification?
The cost is different for miss-classifications. For example buying salmon and the machine has put sea bass in there -> people are angry. buying sea bass and getting salmon -> people are not that angry
How many features should the model have?
No clear answer
Some features might be difficult and computationally expensive to extract
Correlated features do not in theory improve performance
What is the curse of dimensionality?
Adding too many features can lead to worse performance. If each input feature is divided into M divisions, then the total number of cells M^d grows exponentially
What might be the problem with using too complex models?
Overfitting, the model could be tuned too well for particular training samples
How can we improve generalization performance?
- More training examples
- Simpler models
How to evaluate model performance?
Simple: separate training and test data
Cross-validation
What does pooling mean in pattern recognition?
Performance can be improved using a “pool” of classifiers -> combining multiple classifiers
What is the limitation of classical approach in pattern recognition?
It uses hand-crafted models, which is very tedious and costly to develop, and highly dependent on one application
How does deep learning work in pattern recognition?
Deep learning seeks to learn rich hierarchical representations automatically through multiple stage of feature learning process
picture of a tiger -> low level features -> mid-level features -> high-level features -> trainable classifier -> output
For example text:
- Character -> word -> word group -> clause -> sentence -> story
What is ViT - Vision transformer?
It divides data into smaller parts (patches) which are processed as a sequence. Then spatial information of the image is added with positional encoding so the model understands the position of the parts in the image.
The attention mechanism allows the model to emphasize important parts of the image and learn long-range dependencies
