Week 3: Videos Flashcards
Describe the purpose of ‘Optical flow’ in the context of video classification.
Optical flow refers to the apparent motion of pixels in a video, which is sometimes used as an input feature for video classification with CNNs, finding applications in tracking, video recognition, and motion magnification.
What are the different models used in video classification?
- Single-Frame CNN: Uses 2D CNNs independently on video frames, assuming average predicted probabilities.
- Late Fusion by MLP: Fuses information from multiple frames using a Multilayer Perceptron.
- Late Fusion by Pooling: Combines temporal and spatial information through pooling operations.
- 3D CNN: Captures spatial and temporal information using 3D convolutional layers.
- Optical Flow-Based Methods: Uses optical flow to capture motion alongside raw pixel data.
What are the major 6 steps involved in human pose recognition?
- Data Collection: Gather datasets containing human images with annotated poses.
- Preprocessing: Normalize images, handle noise, and perform image augmentation.
- Feature Extraction: Identify key points or joints on the human body.
- Model Training: Utilize machine learning or deep learning models for pose estimation.
- Evaluation: Assess model performance on test data using metrics like PCK (Percentage of Correct Keypoints).
- Real-time Deployment: Implement the trained model for live pose recognition applications.
What does “association” refer to in the context of pedestrian tracking?
Association in pedestrian tracking refers to linking or associating detections or observations across frames to identify and maintain the same pedestrian’s track over time.
What is the Linear Assignment Problem in tracking, and why is it relevant?
The Linear Assignment Problem involves assigning detections from one frame to potential tracks in the next frame, optimizing a cost matrix. It aims to minimize the total assignment cost, ensuring correct associations between detections and tracks, crucial for maintaining accurate object tracks in multi-object tracking scenarios.
How does applying a pre-trained Mask R-CNN relate to a simple tracking algorithm
Applying a pre-trained Mask R-CNN within a tracking algorithm helps in detecting and segmenting pedestrians in frames, providing valuable information for associating detections across frames and forming robust tracks for pedestrian tracking tasks.
What is video?
A sequence of images (frames) with 3 dimensions: width, height, and time.
What are the challenges of video processing?
Size, complexity of capturing temporal relationships between frames, and computational cost.
What are the three main approaches to video classification?
Single frame, late fusion, and early fusion.
What are the advantages of 3D convolutional neural networks (C3D) for video classification?
They operate directly on the 3D video data and can capture temporal relationships.
What is optical flow?
A measure of pixel motion between frames, providing information about movement.
What are key joint locations in human pose estimation?
Landmarks on the body such as elbows, wrists, and knees.
What is the purpose of synthetic depth-image datasets for human pose recognition?
To provide variations in appearance and pose for training models.
What are the advantages of decision tree classifiers for human pose recognition?
They are fast, efficient, and can handle multiple body parts.
What is OpenPifPaf?
A software tool for human pose tracking.
