Lecture 9: Introduction to Neural Networks Flashcards
What is Artificial Intelligence and Machine Learning?
Artificial intelligence is the demonstration of intelligence by a machine.
Machine learning is a branch of artificial intelligence, concerned with the design and development of algorithms that allow computers to evolve behaviors based on ‘ground truth’ data.
What is an Artificial Neural Network (ANN)?
An artificial neural network (ANN) is a set of linked mathematical functions “connected” together according to a specific architecture.
The objective of an artificial neural network is to learn how to transform inputs into meaningful outputs.
What is a Feedforward (FF) or Fully-Connected Neural Network (FCN)?
In a standard neural network, usually, each layer is fully connected to the next layer. This is called a Feedforward (FF) or Fully-Connected neural Network (FCN).
What is an activation function?
A non-linear function which takes the sum pf the inputs, multiplied by the connection weights to produce a non-linear output.
What are some examples of activator functions?
The tanh, sigmoid, and rectifier functions.
How is a neural network trained?
The procedure for training a neural network is called backpropagation.
The backpropagation process involves:
-Starting with complete random connection weights.
-Presenting an input pattern from the training set to the network and comparing the calculated output to the expected output.
-Repeating this for all training examples.
-Backpropagate to adjust weights.
-Repeat until expected output is produced.
So what does a modern training loop for a neural network look like?
1.Random Initialization
2.Feedforward
3.Calculate Loss
4.Calculate derivative of loss
5.Backpropagate
What is an Optimizer and what is Gradient Descent?
An Optimizer is used in training a neural network to adjust the weights based on the calculated gradients.
Gradient Descent is a method used by optimizers to find the minimum of a function
What are Training and Testing Sets?
To ensure the network has not over-fitted to the training set, it should be tested on a completely separate Testing set.
This splitting of data is called CROSS-VALIDATION and is important for predicting how well a method will work on entirely new data.
Why is ‘deep’ learning important?
Gradient descent can only find a local minimum, so each layer cannot learn much before it gets stuck.
When many layers are stacked together, the network becomes very powerful.
What are tensors?
Tensors are a blanket term used for all multidimensional arrays.
