Chapter 6

Question 1

activation function

Answer 1

alias of transfer function

Question 2

artificial neural networks (ANN)

Answer 2

Computer technology that attempts to build computers that operate like a human brain. The machines possess simultaneous memory storage and work with ambiguous information. Sometimes called, simply, a neural network.

Question 3

backpropagation

Answer 3

The best-known learning algorithm in neural computing where the learning is done by comparing computed outputs to desired outputs of training cases.

Question 4

black-box syndrome

Answer 4

ANN are typically known as black boxes, capable of solving complex problems but lacking the explanation of their capabilities.

The lack of transparency present in ANN, that is it is a lack of knowing how the model does what it does.

Question 5

Caffe

Answer 5

This is an open-source deep learning framework developed at UC Berkeley and Berkeley AI Research.

Question 6

cognitive analytics

Answer 6

Is a term that refers to cognitive computing–branded technology platforms, such as IBM Watson, that specialize in processing and analyzing large, unstructured data sets.

Cognitive analytics systems can use machine learning to adapt to different contexts with minimal human supervision.

Question 7

cognitive computing

Answer 7

The application of knowledge derived from cognitive science in order to simulate the human thought process so that computers can exhibit or support decision-making and problem-solving capabilities.

Question 8

cognitive search

Answer 8

A new generation of search method that uses artificial intelligence (e.g., advanced indexing, NLP, and machine learning) to return results that are much more relevant to the user.

Question 9

connection weight

Answer 9

The weight associated with each link in a neural network model. Neural networks learning algorithms assess connection weights

Question 10

constant error carousel (CEC)

Answer 10

(aka state unit) One of four additional layer’s (based on Recurrent NN) in a history aware ANN architecture (e.g. LSTM long short term memory) that is responsible for integrating and interacting with the other 3 (input gate, forget gate, output gate).

Question 11

convolution function

Answer 11

This is a parameter sharing method to address the issue of computational efficiency in defining and training a very large number of weight parameters that exist in CNN.

Useful when the number of weight parameters required for a CNN is prohibitively large (long processing time).

Question 12

convolutional neural network (CNN)

Answer 12

(CNNs) These are among the most popular deep learning methods. CNNs are in essence a variation of the deep MLP-type neural network architecture, initially designed for computer vision applications (e.g., image processing, video processing, text recognition) but also applicable to nonimage data sets.

Question 13

deep belief network (DBN)

Answer 13

(CBD) A type of a large class of deep neural networks called generative models.

Question 14

deep learning

Answer 14

The newest and perhaps the most popular member of the artificial intelligence and machine learning family, deep learning has a goal similar to those of the other machine learning methods that came before it: mimic the thought process of humans - using mathematical algorithms to learn from data (both representation of the variables and their interrelationships).

Question 15

deep neural network

Answer 15

An unsupervised deep learning method use to pretrain network parameters. Part of the class of deep neural networks called generative models.

The primary application of DBNs today is to improve classification models by pretraining.

Question 16

DeepQA

Answer 16

A massively parallel, text mining–focused, probabilistic evidence–based computational architecture developed by IBM, this is the system behind Watson.

Question 17

Google Lens

Answer 17

An app that uses deep learning artificial neural network algorithms to deliver information about the images captured by users from their nearby objects.

Question 18

GoogLeNet

Answer 18

(a.k.a. Inception), a deep convolutional network architecture designed by Google researchers, was the winning architecture at ILSVRC 2014

Question 19

Google Neural Machine Translator (GNMT)

Answer 19

A machine translation (language) system that uses an LSTM network.

Question 20

graphics processing unit (GPU)

Answer 20

It is the part of a computer that normally processes/renders graphical outputs; nowadays, it is also being used for efficient processing of deep learning algorithms

Question 21

hidden layer

Answer 21

The middle layer of an artificial neural network that has three or more layers and takes input from the previous layer and converts those inputs into outputs for further processing.

Question 22

IBM Watson

Answer 22

It is an extraordinary computer system - a novel combination of advanced hardware, software, and machine-learning algorithms - designed to answer questions posed in natural human language.

Question 23

ImageNet

Answer 23

This is an ongoing research project that provides researchers with a large database of images, each linked to a set of synonym words (known as synset) from WordNet (a word hierarchy database).

Question 24

Keras

Answer 24

An open-source neural network library written in Python that functions as a high-level application programming interface (API) and is able to run on top of various deep learning frameworks including Theano and TensorFlow.

Question 25

long short-term memory (LSTM)

Answer 25

A variation of recurrent neural networks that are known as the most effective sequence modeling techniques and are the foundation of many practical applications.

Question 26

machine learning

Answer 26

Teaching computers to learn from examples and large amounts of data.

Question 27

Microsoft Skype Translator

Answer 27

A machine translation (language) system that uses an LSTM network.

Question 28

multilayer perceptron (MLP)

Answer 28

Consists of at least three layers, input layer, hidden layer, output player. Uses backpropagation (supervised) for training.

Question 29

MYCIN

Answer 29

In the early 1970s, several researchers at Stanford University developed a computer system, MYCIN, to identify bacteria causing severe infections, such as bacteremia and meningitis, and to recommend antibiotics with the dosage adjusted for the specifics of an individual patient.

Question 30

network structure

Answer 30

Each ANN is composed of a collection of neurons that are grouped into layers. Typical ANN structures include an input layer, one or more hidden layers and an output layer. Network structure refers to the organization of these layers.

Question 31

neural network

Answer 31

alias artificial neural network.

Question 32

neuron

Answer 32

A cell (i.e., processing element) of a biological or artificial neural network.

Question 33

overfitting

Answer 33

A central concern in the training of any type of machine-learning model is overfitting. It happens when the trained model is highly fitted to the training data set but performs poorly with regard to external data sets.

A large group of strategies known as regularization strategies is designed to prevent models from overfitting by making changes or defining constraints for the model parameters or the performance function

Question 34

perceptron

Answer 34

An early neural network structure that uses no hidden layer.

Question 35

performance function

Answer 35

Usually, the performance function is nothing but a measure of error (i.e., the difference between the actual input and the target) across all inputs of a network. akak cost or loss function.

Question 36

pooling

Answer 36

In CNN, it refers to the process of consolidating the elements in the input matrix in order to produce a smaller output matrix, while maintaining the important features.

Question 37

processing element (PE)

Answer 37

A neuron in a neural network.

Question 38

recurrent neural network (RNN)

Answer 38

The type of neural networks that have memory and can apply that memory to determine the future outputs.

Question 39

representation learning

Answer 39

A type of machine learning in which the emphasis is on learning and discovering features/variables by the system in addition to mapping of those features to the output/target variable.

Question 40

sensitivity analysis

Answer 40

A study of the effect of a change in one or more input variables on a proposed solution.

Question 41

stochastic gradient descent (SGD)

Answer 41

Is a popular boosting algorithm that uses prediction residuals/errors to guide the gradual development of the future decision trees.

Question 42

summation function

Answer 42

A mechanism to add all the inputs coming into a particular neuron.

Question 43

supervised learning

Answer 43

A method of training artificial neural networks in which sample cases are shown to the network as input, and the weights are adjusted to minimize the error in the outputs.

Question 44

TensorFlow

Answer 44

A popular open-source deep learning framework originally developed by the Google Brain Group in 2011 as DistBelief, and further developed into TensorFlow in 2015.

Question 45

Theano

Answer 45

This was developed by the Deep Learning Group at the University of Montreal in 2007 as a Python library to define, optimize, and evaluate mathematical expressions involving multidimensional arrays (i.e., tensors) on CPU or GPU platforms.

Question 46

threshold value

Answer 46

A hurdle value for the output of a neuron to trigger the next level of neurons. If an output value is smaller than the threshold value, it will not be passed to the next level of neurons.

Question 47

Torch

Answer 47

An open-source scientific computing framework for implementing machine-learning algorithms using GPUs.

Question 48

transfer function

Answer 48

In a neural network, the function that sums and transforms inputs before a neuron fires. It shows the relationship between the internal activation level and the output of a neuron.

Question 49

word embeddings

Answer 49

The output from word2vec (word vectors or word embeddings) used in many deep learning projects as inputs.

Question 50

word2vec

Answer 50

A two-layer neural network that gets a large text corpus as the input and converts each word in the corpus to a numeric vector of any given size, typically ranging from 100 to 1000.