AI/CI exam Flashcards
What is AI?
the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.
What is CI?
a form of computing modeled on the methods by which humans learn. As computers learn from processes based on logic and science, they become more intelligent. This differs from artificial intelligence in its perspective on imperfection: computational intelligence focuses on the growth of a system and does not use Boolean values (0s and 1s)
History of AI - Birth (1943 – 56)
○ McCulloch and Pitts (1943): simplified math model of neurons (resting/firing states) can realize all propositional logic primitives (can compute all Turing computable functions)
○ Alan Turing (1950): Turing machine and Turing test
○ Claude Shannon: possibility of chess playing computers
○ 1956: Dartmouth conference
History of AI - Early enthusiasm (1952 – 69)
○ Emphasis on intelligent general problem-solving
○ Heuristic search: A, AO, game tree search
○ John McCarthy: Lisp - AI programming language;
○ Marvin Minsky: ANN;
○ Alan Newell & Herbert Simon: GPS
History of AI - Knowledge engineering (1966 – 74)
○ Domain-specific knowledge is the key to success
○ Knowledge representation (KR) paradigms: declarative vs. Procedural
History of AI - Expert systems (1969 – 99)
○ DENDRAL: determine 3D structures of complex chemical compounds
○ MYCIN: 450 rules for diagnosing blood infectious diseases
○ PROSPECTOR: geological ES for mineral deposits
History of AI - AI industry (1980 – 89)
○ wide range of applications in various domains
○ Commercial tools
○ AI winter
History of AI - Modern stage (1990 – present)
○ more realistic goals, more application-oriented
○ distributed AI (DAI) and intelligent agents
○ resurgence of computational intelligence (soft computing) – ANNs, fuzzy systems, evolutionary algorithms
○ dominance of machine learning, deep learning
Major AI approaches
- Top -down approach
- Symbol processing
- Logic programming
OR - Search
- Knowledge-based systems or expert systems
- Agent-based
OR - Reactive machinces
- Limited memory
- Theory of mind
- Self-awareness
Major CI approaches
- Bottom-up approach
- Learning from experiences/examples
- Info stored in a distributed manner, no centralized control
- Bio-inspired:
○ Artificial Neural Network (ANN)
○ Evolutionary Algorithms
○ Fuzzy Systems
AI Application areas: name at least 5 areas and 2 concrete examples
- Physics - Spatial analyzis
- Computer science - Heuristics
- Engineering - Waveless
- Mathematics - Fuzzy sets & logic
- Machine learning - Decision tree
- Neuroscience - Artificial neural networks
- Biology - Evolutionary computation
NN architectures - feedforward vs. recurrent (feedback)
FF - the first and simplest type of artificial neural network, no cycle
RC - connections between nodes can create a cycle, allowing output from some nodes to affect subsequent input to the same nodes
NN architectures - shallow vs. deep
Shallow < 3 layers
Deep > 3 layers
Universal approximator: what it is?
Universal approximation theorems imply that neural networks can represent a wide variety of interesting functions when given appropriate weights. On the other hand, they typically do not provide a construction for the weights, but merely state that such a construction is possible.
Universal approximator: How to achieve it?
Approximating by building a tower function
Training of Multi-Layer Perceptron
- Given:
○ the target output for the output unit is t_j
○ the input the neuron sees is x_i
○ the output it produces is o_j - Update weights as: w_ij←w_ij+R(t_j − o_j)x_i
○ If output is correct, don’t change the weights
○ If output is wrong, change weights for all inputs which are 1
○ If output is low (0, needs to be 1), increment weights
○ If output is high (1, needs to be 0), decrement weights
Hebbian learning rule
Information is stored in the connections between neurons in neural networks, in the form of weights.
Weight change between neurons is proportional to the product of activation values for neurons.
As learning takes place, simultaneous or repeated activation of weakly connected neurons incrementally changes the strength and pattern of weights, leading to stronger connections.
math:
w_i (new)=w_i (old)+x_i y
Gradient - what is it
In mathematics, the slope or gradient of a line is a number that describes both the direction and the steepness of the line.
Gradient - math
x n+1 = x n - alpha delta f(x n)
Gradient descent
First-order iterative optimization algorithm for finding a local minimum of a differentiable function. The idea is to take repeated steps in the opposite direction of the gradient (or approximate gradient) of the function at the current point, because this is the direction of steepest descent
Stochastic gradient descent
Updates the parameters for each training example one by one. Depending on the problem, this can make SGD faster than batch gradient descent. One advantage is the frequent updates allow us to have a pretty detailed rate of improvement.
Backpropagation
The algorithm is used to effectively train a neural network through a method called chain rule. In simple terms, after each forward pass through a network, backpropagation performs a backward pass while adjusting the model’s parameters (weights and biases).
