w9 gemini Flashcards
What is the topic of today’s lecture, according to slide 1?
Object recognition
What are the three levels of computer vision discussed in the recap on slide 1?
Low-level vision, Mid-level vision, High-level vision
Name four topics covered under Mid-level vision in the recap (slide 1).
Segmentation and grouping, Correspondence problem, Stereo and Depth, Video and Motion
What are the three main aspects of object recognition, as defined on slide 2?
Identification, Categorisation, Localisation
Give three examples of methods for performing object recognition (slide 2).
Template matching, Sliding window, Edge matching
What is the goal of object identification, according to slide 3?
To determine the identity of an individual instance of an object.
Give an example of object identification from slide 3.
Distinguishing between two specific individuals (Clinton vs. Bush) or two specific phone models (Samsung Galaxy On8 vs. iPhone 7 Plus).
What is the goal of object categorisation, as described on slide 4?
To determine the category of an object.
Provide an example of object categorisation from slide 4.
Classifying images as belonging to the category ‘Human’ or ‘Chimpanzee’, or ‘Telephone’ or ‘Calculator’.
What is object localisation, according to slide 5?
Determining the presence and/or location of an object in an image.
What is semantic segmentation, as defined on slide 6?
Localisation that is sufficiently fine-grained and for a sufficiently large number of categories, resulting in image segmentation.
Explain the concept of a category hierarchy in object recognition (slide 7).
Classification can occur at different levels of abstraction, from general categories (like ‘object’) to specific instances (like ‘Rex’).
What are the three levels in the category hierarchy shown on slide 7?
Superordinate level, Basic level, Subordinate level
Why is the ‘basic level’ significant for human object recognition (slide 8)?
Humans are usually fastest at recognizing category members at this level, start with basic-level categorization before identification, and it’s the first level understood by children.
List two reasons why the basic level is considered special (slide 9).
It’s the highest level where category members share many common features, and the lowest level where members have features distinct from other categories at the same level.
What are the two main requirements for object recognition systems (slide 10)?
Sensitivity to image differences relevant to distinguishing objects, and insensitivity/tolerance to differences that don’t affect object identity or category.
Give examples of image variations that object recognition systems should be insensitive to (slides 11-13).
Background clutter, occlusion, viewpoint, lighting, non-rigid deformations, within-category variation.
What are the three main components required for object recognition (slide 14)?
Image data, representations of objects, matching techniques.
Describe the ‘off-line’ stage of object recognition procedure (slide 15).
Extracting representations from training examples.
Describe the ‘on-line’ stage of object recognition procedure (slide 15).
Extracting representation from an input image and matching it with training examples to determine the object class or identity.
What are the two key aspects in which object recognition methods vary (slide 16)?
Representation used and matching procedure.
What is the representation used in template matching (slide 17)?
An image of the object to be recognized (an array of pixel intensities).
Describe the matching process in template matching (slide 17).
Searching every image region and calculating the similarity between the template and the image region.
Name three similarity measures that can be maximised in template matching (slide 18).
Cross-correlation, Normalised cross-correlation (NCC), Correlation coefficient.
Name three similarity measures that can be minimised in template matching (slide 19).
Sum of Squared Differences (SSD), Euclidean distance, Sum of Absolute Differences (SAD).
How can template matching be used to recognise multiple objects (slide 20)?
By using multiple templates, one for each object.
What is a potential problem with using SAD in template matching, as shown in the example on slide 21?
It can produce peaks in areas that are simply darker, not necessarily a true match.
What is a potential problem with template matching regarding ‘true’ and ‘false’ matches (slide 23)?
Distinguishing true matches from false matches, and deciding what constitutes a match and how many peaks to consider.
Why can template matching be ineffective if the target object is scaled or rotated (slide 25)?
Because the template needs to be very similar to the target object, and scaling or rotation introduces differences.
What is a common approach to address viewpoint and within-category variation in template matching (slide 26)?
Using multiple templates for each object, representing different viewpoints and variations.
Explain the dilemma regarding the threshold in template matching (slide 27).
A high threshold avoids false matches but might miss true matches, while a low threshold finds true matches but increases false matches.
Why can template matching be computationally expensive (slide 28)?
Because of the need for many comparisons, especially when dealing with variations in appearance, viewpoints, and scales.
Why is template matching sensitive to occlusion (slide 29)?
Because if an object is occluded, the template may not fully match the visible parts.
What is the fundamental issue that makes template matching not robust (slide 30)?
The metric used for comparison is fundamentally not robust to changes in appearance between the template and the image patch.
How does ‘sliding window’ differ from template matching (slide 31)?
It uses a classifier to determine if an image patch contains the object, and considers patches of different shapes and sizes.
How is computational cost addressed in sliding window techniques (slide 32)?
By pre-processing images to select regions that are good candidates to contain an object, often using image segmentation.
How does ‘edge matching’ represent objects (slide 33)?
Templates and input images are pre-processed to extract edges.
Describe the matching process in edge matching (slide 33).
Calculating the average of the minimum distances between points on the edge template and points on the edge image.
What is the general idea behind ‘model-based’ object recognition (slide 34)?
Hypothesize object identity and pose, render the object in the image (‘back-project’), and compare it to the actual image.
What are two methods for comparing the back-projected model with the image in model-based recognition (slide 36)?
Edge score and Oriented edge score.
What is the representation used in ‘intensity histograms’ for object recognition (slide 37)?
A histogram of pixel intensity values (either grayscale or colour).
What is a benefit of using intensity histograms (slide 37)?
Insensitivity to small viewpoint changes.
What are two drawbacks of using intensity histograms (slide 37)?
Sensitivity to illumination and intra-class appearance variation, and insensitivity to different spatial configurations.
Why are colour histograms sometimes used in face detection (slide 39)?
Because skin has a very small range of intensity-independent colours.
What are the two components of the ‘Implicit Shape Model’ (ISM) representation (slide 40)?
Parts (2D image fragments) and structure (configuration of parts).
Describe the process of extracting local object features in ISM (slide 41).
Locating interest points and extracting 2D image patches around them.
What is an ‘appearance codebook’ in ISM (slide 42)?
A collection of clustered patches, where cluster centers are stored.
How is the configuration of parts learned in ISM (slide 43)?
By matching codebook features to training images and recording possible object centres for every codebook entry.
Explain the matching procedure in ISM (slide 44).
Each feature votes for possible object centres, similar to the Generalized Hough Transform.
What is the representation used in ‘feature-based object recognition’ (slide 52)?
Training image content is transformed into local features invariant to translation, rotation, and scale.
How does ‘feature-based object recognition’ perform matching (slide 53)?
Local features are extracted from a new image and matched to those from the training image.
What are two requirements for feature detection in feature-based recognition (slide 55)?
Repeatability despite variations and finding sufficient features to cover the object.
What property should the feature description have in feature-based recognition (slide 55)?
Invariance to translation, rotation, scale changes, and lighting variations.
What is the representation used in SIFT feature matching (slide 56)?
A 128 element histogram of the orientations of the intensity gradients.
List four advantages of SIFT features (slide 56).
Locality, Distinctiveness, Quantity, Efficiency.
Describe the representation used in SIFT feature matching in more detail (slide 57).
A set of keypoints with 128-component descriptors obtained from each training image, stored in a database.
Describe the matching process in SIFT feature matching (slide 58).
Finding the top 2 best matching descriptors in the training database for each keypoint in the input image.
What is the criteria for accepting a match in SIFT feature matching (slide 58)?
The ratio of the distance to the first nearest descriptor to that of the second is below a threshold.
What is used to confirm the consistency of matched locations in SIFT feature matching (slide 61)?
RANSAC or Generalised Hough Transform.
What is the analogy used to explain ‘Bag-of-words’ (slide 62)?
Document recognition.
Briefly describe how ‘Bag-of-words’ works for documents (slide 63).
Documents are parsed into words, common words are ignored, words are stemmed, assigned identifiers, and documents are represented by word frequency vectors.
How is matching done in the ‘Bag-of-words’ model for documents (slide 64)?
By calculating the angle between the query and document vectors.
Explain the core idea behind ‘Bag-of-words’ for images (slide 65).
Different objects have distinct sets of features that occur in different frequencies.
Name three ways features can be chosen in the ‘Bag-of-words’ model for images (slide 66).
Regular grid, Interest point detector, Randomly.
How are features encoded in the ‘Bag-of-words’ model for images (slide 67)?
Using a descriptor (analagous to a word).
Describe the process of creating a dictionary in the ‘Bag-of-words’ model (slides 68-69).
Encoding many features from many images and clustering them into visual words, forming a visual vocabulary.
How are images represented in the ‘Bag-of-words’ model (slide 71)?
As a histogram showing the frequency of appearance of each codeword in the dictionary.
How is matching performed in the ‘Bag-of-words’ model for images (slide 71)?
By calculating the distance between the histograms of the input and training images.
What is a ‘geometric invariant’ (slide 72)?
A property of an object in the scene that does not vary with viewpoint.
Give three examples of invariant properties under Euclidean transformations (slide 72).
Lengths, angles, areas.
Give two examples of invariant properties under Similarity transformations (slide 73).
Ratios of lengths, angles.
Give three examples of invariant properties under Affine transformations (slide 74).
Parallelism, ratios of lengths along lines, ratio of areas.
What is the invariant property under Projective transformations (slide 75)?
Cross-ratio.
Define ‘cross-ratio’ (slide 75).
The ratio of ratios of lengths on a line.
What is the matching process in ‘geometric invariants’ (slide 77)?
Comparing the value of the cross-ratio measured in the image with a database of cross-ratios measured in training images.
Name five object recognition methods mentioned in the summary (slide 78).
Template matching, sliding window, edge matching, model-based, intensity histograms.
What are the two main categories of matching procedures for object recognition (slide 79)?
Top-down (generative) and Bottom-up (discriminative).
Give an example of a top-down object recognition method (slide 79).
Model-based.
Give an example of a bottom-up object recognition method (slide 79).
SIFT.
List three categories for classifying object recognition methods based on representation used (slide 80).
Pixel intensities vs feature vectors vs geometry, 2D (image-based) vs 3D (object-based), local features vs global features.
What are the advantages of using a local representation (slide 81)?
Tolerant to viewpoint, within class variation, and occlusion.
What is a problem with using local representations (slide 81)?
Many objects can consist of the same collection of features and hence cannot be distinguished.
What is an advantage of using a global representation (slide 82)?
Can distinguish similar objects.
What are two problems with using global representations (slide 82)?
Sensitive to viewpoint and within-class variation, and sensitive to occlusion.
What are two solutions to address the limitations of local and global representations (slide 83)?
Use features of intermediate complexity, and use a hierarchy of features with a range of complexities.
