chapter 6 Flashcards
deep neural networks learning strategy
learning from data
GOFAI learning strategy
human programmers construct explicit rules for intelligent behavior
convnet learning strategy
supervised learning
supervised learning
convnets gradually change their weights as they process the examples in the training set again and again, over many epochs, learning to classify each input as one of a fixed set of possible output categories
difference human learning and supervised learning/deep learning
even the youngest children learn an open-ended set of categories and can recognize instances of most categories after seeing only a few examples. convnet need a lot (big data)
Moreover, unlike convnets, children don’t learn passively: they ask questions etc.
reliance of deep learning networks on extensive collections of labeled training data
hyperparameters
an umbrella term that refers to all the aspects of the network that need to be set up by humans to allow learning to even begin
While ConvNets use back-propagation to learn their “parameters” (that is, weights) from training examples, this learning is enabled by a collection of hyperparameters
so it is inaccurate to say that today’s successful ConvNets learn “on their own.”
examples of hyperparameters
the number of layers in the network, the size of the units’ “receptive fields” at each layer, how large the change in each weight should be during learning
long-tail problem
the vast range of possible unexpected situations an AI system could be faced with.
This is a problem if we rely solely on supervised learning to provide our AI system with its knowledge of the world; the situations in the tail don’t show up in the training data often enough, if at all, so the system is more likely to make errors when faced with such unexpected cases.
for this reason, supervised learning is not a viable path to general- purpose AI.
solution to long tail problem
to use supervised learning on small amounts of labeled data and learn everything else via unsupervised learning.
UNsupervised learning
learning refers to a broad group of methods for learning categories or actions WITHOUT LABELED DATA
Examples include methods for clustering examples based on their similarity or learning a new category via analogy to known categories
no one has yet come up with the kinds of algorithms needed to perform successful unsupervised learning
human competence that lacks in AI
common sense
Many people believe that until AI systems have common sense as humans do, we won’t be able to trust them to be fully autonomous in complex real-world situations
overfitting
The machine learns what it observes in the data rather than what you (the human) might observe.
If there are statistical associations in the training data, even if irrelevant to the task at hand, the machine will happily learn those instead of what you wanted it to learn.
bias in AI
data set for training face recognition systems contain more faces that are male and white, because the images were downloaded from online image searches, and photos of faces that appear online are skewed toward featuring famous or powerful people, who are predominately white and male.
AI systems trained on biased data can magnify these biases and do real damage
explainable AI, transparent AI, or interpretable machine learning
hese terms refer to research on getting AI systems— particularly deep networks—to explain their decisions in a way that humans can understand
ways to visualize the features that a given convolutional neural network has learned and, in some cases, to determine which parts of the input are most responsible for the output decision.
adversarial example
to make specific changes to pixels of an example so that it looks completely unchanged to humans but gets classified with very high confidence by the system as something completely different
the opposite is also possible
> to computationally “evolve” images that look like random noise to humans but for which systems assigned specific object categories with greater than 99 percent confidence.
