Deep Learning Flashcards
what characterises supervised learning
labelled data
cost function
reperated epochs
what characterises un-supervised learning
non-labelled data
no cost function
finds representations of data
deep learning what is it
usually labelled data
uses multilayered neural networks, either convolutional or perceptrom based
What are the key components of a neural network
-Some cost function e.g. RMSE
-Forward propagation per layer per node consisting of weight x input + bias
-Following a neuron usually an activation function such as a ReLu or a Softmax
-Backpropagation
Why is backpropagation an efficient method of optimization
It stores inermediate gradients, allowing for more efficient computations of the partial derivatives at each feed forward stage, this allows for fast optimiazation of a neural network (at least relative to computing the derivatives seperately per variable
What is the activation function sigmoid and what are its pros and cons
-it is equal to 1/1+e^x and maps values range from any possible value [-infinity,infinity] to range of [0,1]
pros: differentiable and normalisees ranges
cons: hard to compute
What is the activation function (tanH) and what are its pros and cons
(e^x - e^-x) / ( e^x + e^-x). normalizes values to the range [-1,1]
-differentiable but hard to compute
What is the activation function ReLu and what are its pros and cons
f(x) = max (0,x). easy to compute but non differnetiable
-avoids saturation issues (i.e. when nodes have weights close to -1 or +1)
What is the neural radiance field neural network
Form of stereo vision
-takes as input an image from different perspectives
-outputs a 3d map of the object
This is an example of a depth image
What is the relationship between amount of data and the performance of different ML / AI methods
Large neural network performs best with large data but worst with little data
- small neural network 2nd place in both cases
- traditional ML performs better than rest with little data but worst with most data
What is the general goal of a convolutional neural network
Learning higher-level features from image
Neural networks do this hierarchically i.e. they first learn low level -> mid level -> high level features then train a classifier
what is a general structure of a general CNN up to the feature maps
Data
Convolution
Activation
Pooling
Feature maps
CNN = multi-layer neural network with:
- Local connectivity
–neurone in a layer are only connected to small region of lair before it - They share weight parameters across spatial positions
– Learning shift-invariant filter kernels
what parameters do convolutional kernels have
- Kernel Size
- Stride
- Padding
- Dilation:
what are non-linear activation functions
Leaky ReLu
Parametric ReLu
Exponential Linear Units
Good for when we want to find non-linear relationships
what is a pooling layer
taking the average or max of a section of kernel to reduce kernel size for better efficiency
What is the random wiring strategy for neurel net design
Randomly wiring nodes between multiple neurons and testing their efficacy, some examples are Minsky
What is a non-random algorithmic approach to neural net design
Evolutionary algorithms:
-often out-perform human designed architectures
-keeps good parts of network and discards poor ones with some mutation probability
given a set of neural network architectures, we want to find best one. what can we reduce this problem to
search problem
what are 4 types of regularisation
-L2
-L1
-Dropout
-Cross Entropy
what is the general form of gradient descent
w^(t+1) = w^(t) - lr ( dx ( Loss Function (w^(t))))
name 3 optimisation algorithms
Adam
RMSProp
AdaGrad
name 3 optimisation algorithms
Adam
RMSProp
AdaGrad
What are some different deep learning algorithms for computer vision
CNN:
-classification algorithm, for detection, segmentation, learn hierarchical representations of visual data
R-CNN: (Regional CNN)
Object Detection algorithm combining CNNs. For each region it applies a CNN to localize objects
Fast R-CNN:
-same as R-CNN and runs faster by sharing computation over different regions. this is done by pooling layers across regions
What is a hidden layer, convolutional layer and pooling layer
Hidden layer:
-region of forward propagation between inputs and outputs with weights and biases
-find reresentations and relationships in a dataset
-in CNN it has convolutional layers, pooling and fully connected
Covolutional Layer:
- convolutional layers are filters with learnable parameters which capture patterns in an image
Pooling Layter:
- Used to downsample spatial dimensions of feature maps produced by Convolutional Layers
-This process effectively retains the most salient features while reducing the spatial size of the feature maps.
