Deep Learning

Question 1

what characterises supervised learning

Answer 1

labelled data
cost function
reperated epochs

Question 2

what characterises un-supervised learning

Answer 2

non-labelled data
no cost function
finds representations of data

Question 3

deep learning what is it

Answer 3

usually labelled data
uses multilayered neural networks, either convolutional or perceptrom based

Question 4

What are the key components of a neural network

Answer 4

-Some cost function e.g. RMSE
-Forward propagation per layer per node consisting of weight x input + bias
-Following a neuron usually an activation function such as a ReLu or a Softmax
-Backpropagation

Question 5

Why is backpropagation an efficient method of optimization

Answer 5

It stores inermediate gradients, allowing for more efficient computations of the partial derivatives at each feed forward stage, this allows for fast optimiazation of a neural network (at least relative to computing the derivatives seperately per variable

Question 6

What is the activation function sigmoid and what are its pros and cons

Answer 6

-it is equal to 1/1+e^x and maps values range from any possible value [-infinity,infinity] to range of [0,1]
pros: differentiable and normalisees ranges
cons: hard to compute

Question 7

What is the activation function (tanH) and what are its pros and cons

Answer 7

(e^x - e^-x) / ( e^x + e^-x). normalizes values to the range [-1,1]
-differentiable but hard to compute

Question 8

What is the activation function ReLu and what are its pros and cons

Answer 8

f(x) = max (0,x). easy to compute but non differnetiable
-avoids saturation issues (i.e. when nodes have weights close to -1 or +1)

Question 9

What is the neural radiance field neural network

Answer 9

Form of stereo vision
-takes as input an image from different perspectives
-outputs a 3d map of the object
This is an example of a depth image

Question 10

What is the relationship between amount of data and the performance of different ML / AI methods

Answer 10

Large neural network performs best with large data but worst with little data
- small neural network 2nd place in both cases
- traditional ML performs better than rest with little data but worst with most data

Question 11

What is the general goal of a convolutional neural network

Answer 11

Learning higher-level features from image

Neural networks do this hierarchically i.e. they first learn low level -> mid level -> high level features then train a classifier

Question 12

what is a general structure of a general CNN up to the feature maps

Answer 12

Data
Convolution
Activation
Pooling
Feature maps

CNN = multi-layer neural network with:

• Local connectivity
  –neurone in a layer are only connected to small region of lair before it
• They share weight parameters across spatial positions
  – Learning shift-invariant filter kernels

Question 13

what parameters do convolutional kernels have

Answer 13

• Kernel Size
• Stride
• Padding
• Dilation:

Question 14

what are non-linear activation functions

Answer 14

Leaky ReLu
Parametric ReLu
Exponential Linear Units

Good for when we want to find non-linear relationships

Question 15

what is a pooling layer

Answer 15

taking the average or max of a section of kernel to reduce kernel size for better efficiency

Question 16

What is the random wiring strategy for neurel net design

Answer 16

Randomly wiring nodes between multiple neurons and testing their efficacy, some examples are Minsky

Question 17

What is a non-random algorithmic approach to neural net design

Answer 17

Evolutionary algorithms:
-often out-perform human designed architectures
-keeps good parts of network and discards poor ones with some mutation probability

Question 18

given a set of neural network architectures, we want to find best one. what can we reduce this problem to

Answer 18

search problem

Question 19

what are 4 types of regularisation

Answer 19

-L2
-L1
-Dropout
-Cross Entropy

Question 20

what is the general form of gradient descent

Answer 20

w^(t+1) = w^(t) - lr ( dx ( Loss Function (w^(t))))

Question 21

name 3 optimisation algorithms

Answer 21

Adam
RMSProp
AdaGrad

Question 22

name 3 optimisation algorithms

Answer 22

Adam
RMSProp
AdaGrad

Question 23

What are some different deep learning algorithms for computer vision

Answer 23

CNN:
-classification algorithm, for detection, segmentation, learn hierarchical representations of visual data

R-CNN: (Regional CNN)
Object Detection algorithm combining CNNs. For each region it applies a CNN to localize objects

Fast R-CNN:
-same as R-CNN and runs faster by sharing computation over different regions. this is done by pooling layers across regions

Question 24

What is a hidden layer, convolutional layer and pooling layer

Answer 24

Hidden layer:
-region of forward propagation between inputs and outputs with weights and biases
-find reresentations and relationships in a dataset
-in CNN it has convolutional layers, pooling and fully connected

Covolutional Layer:
- convolutional layers are filters with learnable parameters which capture patterns in an image

Pooling Layter:
- Used to downsample spatial dimensions of feature maps produced by Convolutional Layers
-This process effectively retains the most salient features while reducing the spatial size of the feature maps.

Question 25

What is data augmentation

Answer 25

Adding diversity to dataset to improve real world performance. it may include:
-scaling, resizing
-translations
-rotations
-gaussian noise ]0colour jittering

Question 26

When is ReLu better than some other activation funcs

Answer 26

When network is very deep:
- more efficient
- less likely to cause saturation (-1 or +1 only nodes) causing faster convergence
- less likely for vanishing gradient problem

Question 27

What is the YOLO algrotithm (simple)

Answer 27

You Only Look Once:
-Object detection for classifying multiple objects in an image in real-time
-Rather than using sliding window it treats it as a regression problem and attempts to find likely bounding boxes and probabilities using a single nerual net

Question 28

Describe the YOLO algorithm (detail)

Answer 28

-Input image is divided into groups of cells responsible for predicting class probabilities for objects in the box
-Pre-definied anchor boxes with different shapes + sizes are used to predict dimensions of bounding boxes
-Put image thorugh CNN once which contains cov layer, feature extraction layer, then fully connected layers for regression and classification
- Outputs predictions of each bounding box containing an object and co-ordinates of the box and width
-Non-Maxima supression to delete overlapping boxes with lower probability
-Final output is a set of bounding boxes with estimated probabilities

Question 29

Give some examples of supervised and non-supervised algorithms

Answer 29

supervised:
- CNNs
-SVM
unsupervised:
-GANs
-Autoencoders
-Kmeans