Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Artificial Intelligence Basics > Artificial Neural Networks and Applications > Flashcards

Artificial Neural Networks and Applications Flashcards

1
Q

What is a neural network?

A

a model that mimics the behavior of biological neurons in the human brain

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

How does a NN calculate an output?

A

passed input through an array of neurons

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What kind of complicated problems do NN perform exceptionally well in solving?

A

text, voice, and image recognition, and NLP

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is a type of connectionist model that has become a mainstay of AI and cognitive science?

A

neural networks

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Where does human intelligence begin?

A

with the connections between neurons

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are the three milestones in neural networks models?

A

Single layer perceptron, multi-layer perceptron, and DNN

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Who founded the single layer perceptron model and when?

A

Rosenblatt in 1957

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Who founded the multi-layer perceptron model and when?

A

Rumelhart in 1986

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Who founded the DNN model and when?

A

Hinton in 2006

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Which algorithm is a key algorithm of the single layer perceptron model?

A

the perceptron algorithm

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Which algorithm is a key algorithm of the multi-layer perceptron model?

A

backpropagation algorithm

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Which algorithm is a key algorithm of the DNN model?

A

deep learning algorithm

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is the main role common to all neural networks?

A

the learning function

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What types of information are large-scale multimedia information?

A

audio and video information

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

True or false: Learning and recognizing data is small and easy to handle?

A

false

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What structures comprise a neuron?

A

dendrites, synapses, axons, and terminals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What is the role of a neuron?

A

sensory organ neurons, network signaling

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What are the components of a perceptron?

A

weights, bias, and activation functions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is an activation function?

A

a function that expresses the activation/deactivation of a neuron

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What are a few types of activation functions?

A

step-fn, sigmoid, tanh, and ReLU

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Why do we use activation functions?

A

we need non-linear functions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What is the function of a dendrite?

A

receives stimuli from other neurons or surroundings and transmits impulses to cell body via electrical signals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What is a synapse?

A

the junction of cells where the axons of one neuron and the dendrites of the next neuron meet

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is an axon?

A

a branch of a neuron whose function is to transmit signals to other neurons

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

What is the function of a terminal?

A

receives transmitted electrical signals and secretes neurotransmitters into synapses

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

What happens to signals from sensory organs?

A

they pass through the brain’s network of neurons and get converted into meaningful signals

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
27
Q

Where is the focus of current research?

A

implementing learning-capable computing

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
28
Q

What is the behavior of a single neuron?

A

n inputs -> operation* (1 or 0) -> m outputs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
29
Q

What happens within the *operation (1 or 0) function?

A

neuron activates a signal when signal crosses a threshold according to rules of the cell body, or it does not emit a signal if it is not activated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
30
Q

What is the function of a weight?

A

controls the importance of the input signal to the output

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
31
Q

What is the function of a bias?

A

controls how easily neurons are activated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
32
Q

What is the function of an activation function?

A

to pass vs not to pass

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
33
Q

What are adjusted through learning?

A

weights and biases

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
34
Q

Why are weights and biases adjusted through learning?

A

to strengthen neural networks of relevant signals and weaken unrelated neural networks

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
35
Q

True or false: The activation function used for the neural network doesn’t matter?

A

false, you must choose the appropriate function

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
36
Q

An activation function does what?

A

takes the sum of the inputs and calculates the output

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
37
Q

When the sum of the inputs exceeds a certain threshold, what happens?

A

the neuron is activated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
38
Q

Neurons follow what type of activation function?

A

a step function

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
39
Q

True or false: MLP (multilayer perceptrons) use a variety of activation functions?

A

true

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
40
Q

True or false: All activation functions are nonlinear functions?

A

true

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
41
Q

Why is it difficult to improve performance if you use a linear function as an activation function?

A

combining linear functions is the same as using one linear function so even if you have multiple layers, it’s really only expressed as one layer

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
42
Q

What is a multilayer perceptron?

A

a perceptron that has a hidden layer between the input and output layers

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
43
Q

What are MLPs notable for?

A

being able to distinguish data that is not linearly seperable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
44
Q

What do MLPs use for training the network?

A

backpropagation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
45
Q

What are the limitations of a MLP?

A

black box model and overfitting

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
46
Q

What is the black box model?

A

since the perceptron is a black box, sometimes it is unknown how the network makes predictions or judgements

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
47
Q

What is overfitting?

A

if the model is too complicated or the training data is too restricted, MLPs may easily overfit the training data

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
48
Q

What is forward passage regarding MLPs?

A

the process by which input signals are applied to the input layer unit and these input signals are propagated through the hidden layer to the output layer

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
49
Q

What is a loss function?

A

a mathematical function that measures the difference between predicted and actual values in a machine learning model

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
50
Q

How does the loss function determine the model’s performance?

A

by comparing the distance between the prediction output and the target values

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
51
Q

What does the backpropagation algorithm do?

A

calculates the output by calculating the forward direction given the input and then calculates the error between the actual output and the calculated output

52
Q

Why is backpropagation useful?

A

by propagating this error in the reverse direction, the weights are changed in the way of reducing error

53
Q

What is backpropagation used for?

A

supervised learning of artificial neural networks using gradient descent

54
Q

What is deep learning?

A

the act of stacking a neural network on top of eachother

55
Q

What has deep learning recently been applied to?

A

computer vision, speech recognition, NLP, social network filtering, machine translation, etc.

56
Q

Why do we use deep learning?

A

to mimic the human brain and improve performance of AI

57
Q

What algorithm cause a jumpstart in the research for deep learning and hidden layer learning?

A

backpropagation

58
Q

What activation functions resolved the problem of overfitting?

A

dropout, ReLU

59
Q

What activation function resolved the issue that happens when the number of hidden layers increases and the gradient value decreases?

A

ReLU

60
Q

What is another limitation of MLP that has been resolved?

A

computing speed limits

61
Q

What was the first neural network capable of recognizing letters?

A

Rosenblatt’s (single-layer) Perceptron in 1957

62
Q

What was the first neural network ever?

A

McCulloch-Pitts neuron in 1943

63
Q

What was the McCulloch-Pitts neuron the basis for?

A

the perceptron algorithm

64
Q

What is the function of the McCulloch-Pitts neuron?

A

receives one or more inputs and sums them to produce an output

65
Q

What can the McCulloch-Pitts neuron be used for?

A

classification problems

66
Q

How is Hebb’s rule summarized?

A

neurons that fire together wire together

67
Q

What does Hebb’s rule describe?

A

how neuronal activities influence the connections between neurons

68
Q

How many layers is Rosenblatt’s perceptron made up of?

A

a single layer

69
Q

When was the Mark I Perceptron produced and what was it?

A

1957, a neural network hardware device

70
Q

What is a linear system?

A

a system that solves a problems where the input classes are linearly seperable

71
Q

What was the first implementation of the perceptron algorithm?

A

the Mark I Perceptron

72
Q

What was the input for the Mark I
Perceptron device?

A

characters (ASCII)

73
Q

What did the Mark I Perceptron do with the input?

A

classified the characters into characters classes (A, B, C, etc) aka character recognition

74
Q

Where was/is the Mark I Perceptron housed?

A

the Smithsonian Museum, USA

75
Q

What is the structure of the single-layer perceptron?

A

adjustment connection strength of just one layer, and uses McCulloch-Pitts neuron and Hebb’s learning rule for feedback learning for errors

76
Q

What is the structure of the input/output in neurons?

A

result of n inputs is multiplied by n connections strength vectors

77
Q

What is the sum determined by in the neuron?

A

the activation function

78
Q

What is the output of the neuron?

A

1 if the value is greater than the threshold (usually 0), otherwise -1

79
Q

What are some typical non-linear activation functions used in NN?

A

step function, critical logic function, S-shaped sigmoid function

80
Q

What is the activation function commonly used in perceptrons?

A

the sigmoid function

81
Q

What is a key characteristic of the sigmoid function?

A

it had a smooth value between 0 and 1

82
Q

Describe the perceptron learning process.

A

1)initialize connection strengths and thresholds, 2) present new input and expected output, 3) calculate actual output value, 4) readjust connection strength, 5) go to step 2, rinse and repeat until no more adjustments

83
Q

Generally what is linear separability?

A

a property of 2 sets of points in Euclidean geometry

84
Q

What is the definition of linear separable points?

A

data points in binary classification problems that can be separated using a linear decision boundary (basically points that can be divided into two areas by a straight line)

85
Q

What binary functions are linearly separable?

A

AND and OR

86
Q

What binary function is not linearly separable?

A

the XOR (exclusive-or function)

87
Q

True or false: A perceptron can only converge on linearly separable data?

A

true

88
Q

What is the limitation of the single-layer perceptron?

A

only linearly separable sets can be separated (so not XOR sets)

89
Q

What solved the linear separability problem with the single-layer perceptron and when?

A

in the mid-1980s the multi-layer perceptron solved the XOR problem

90
Q

Single-layer perceptron has how many nodes between the input matrix and the decision node?

A

one node

91
Q

Is the single-layer perceptron suitable as a learning model?

A

no

92
Q

What is the single-layer perceptron used widely for?

A

character recognition

93
Q

What are 2 early neural network models?

A

Adaline (Adaptive linear neuron) and Madaline (Many Adaline)

94
Q

Who proposed the first couple early neural networks in 1960?

A

Bernard Widrow and Ted Hoff

95
Q

What are some applications of Adaline?

A

system modeling, statistical prediction, eliminate communication noise and echo, channel equalizer, adaptive signal processing

96
Q

For how many years after 1969 did neural network research stagnate?

A

10 years

97
Q

When was the multi-layer perceptron model proposed?

A

mid-1980s

98
Q

What key NN research group is based out of Stanford?

A

the Parallel Distributed Processing group

99
Q

What does the PDP focus on?

A

the study of cognitive processes using parallel distributed processing models

100
Q

What are some activities of the PDP?

A

developing models of cognitive processes, creating software programs to simulate these models, and conducting experiments to test these models

101
Q

What prominent book was published by PDP in 1986?

A

Parallel Distributed Processing: Explorations in the Microstructure of Cognition

102
Q

What important algorithm did the PDP introduce?

A

backpropagation

103
Q

When was the backpropagation algorithm proposed?

A

in 1986 in the multi-layer perceptron structure

104
Q

What was the multi-layer perceptron with back propagation useful for?

A

overcoming limitations of single-layer perceptrons particularly the linear separability (XOR) problem

105
Q

How does backpropagation implement learning?

A

returning in the opposite direction

106
Q

What is backpropagation used for?

A

supervised learning of artificial neural networks using gradient descent

107
Q

What exactly is backpropagation?

A

a backward propagation of errors algorithm that calculates the gradient of the error function with respect to the NN’s weights

108
Q

In backpropagation, what minimizes errors?

A

a backwards pass to adjust a neural network model’s parameters

109
Q

What are the steps of backpropagation in a multi-layered network?

A

propagate training data through the model, adjust the model weights to reduce the error, and repeatedly update weights until convergence or iteration satisfaction

110
Q

Who was both the first author of the famous backpropagation paper in 1986 and a key member of the PDP group?

A

Dr. David Rumelhart

111
Q

What was an early goal of the PDP group?

A

to recognize incomplete or noisy characters

112
Q

What is the structure of the multi-layer perceptron?

A

one or more hidden layers between the input and output layers

113
Q

What is the order of connections for multi-layer perceptrons?

A

input -> hidden -> output

114
Q

Are there connections within each layer of a multi-layer perceptron?

A

no

115
Q

True or false: There is a direct connection from the output layer to the input layer of a multi-layer perceptron?

A

false

116
Q

Fill in the blank: Multi-layer perceptron makes the input and output characteristics of nodes __________.

A

nonlinear

117
Q

What are the steps in the forward pass of the backpropagation learning algorithm of multi-layer perceptron?

A

give an input pattern to each node in input layer, signal is converted at each node and sent to hidden layer, output signal from hidden layer to output layer

118
Q

After forward pass, what are the remaining steps of the backpropagation learning algorithm of multi-layer perceptron?

A

compare output to expected output, adjust connections strength, backpropagate again to adjust connection strength, repeat until end conditions are met

119
Q

What are the end conditions for the backpropagation learning algorithm of multi-layer perceptron?

A

when the actual output value and the target output value are within the error range

120
Q

What is the delta rule?

A

the square of the error between the output and the target output value

121
Q

What is the gradient descent method?

A

the square of the error on a curved surface (an iterative optimization algorithm used to find the local minimum of a differentiable function)

122
Q

What is the idea behind the gradient descent method?

A

to take repeated steps in the opposite direction of the gradient of the function at the current point because this is the direction of steepest descent (to find the minimum by moving in the direction of the negative gradient)

123
Q

How does backpropagation use gradient descent in learning?

A

used to update weights of network during training by computing gradient of loss function with respect to the weights then moving the weights in opposite direction of the gradient until weights converge to a minimum

124
Q

What are the disadvantages of the backpropagation learning algorithm?

A

long training time, low probability or local minimum possible, steepest descent is likely to stay at a local minimum (when we want a global minimum)

125
Q

What did the multi-layer perceptron make possible?

A

A neural network that can implement the XOR function

126
Q

What are other applications of multi-layer perceptron models?

A

parity problem, encoding problem, symmetry problem, development of nettalk system to convert text to voice, stock market predictions, translation between different languages, factory automation, robots, real-time voice recognition

Artificial Intelligence Basics (9 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions