Basics Flashcards
Data Science is
The science of using data as key part in the process
of creating knowledge.
Data is
Factual, un-interpreted,
punctual units of analysis;
Typically understood to
exist outside an agent
Knowledge is
Accumulated, interpreted,
connected, actionable
Typically understood to
exist inside an agent
Correlation
What is the correlation between x and y?
is a question asked in data science
Prediction
Given x, what is the likelihood of y
is a question asked in data science
Classification
: Can the given data be partitioned into sub-groups based
on pre-defined labels?
is a question asked in data science
Clustering
Can the given data be partitioned into meaningful subgroups based on the given data?
is a question asked in data science
Other structure identification
Can the given data be described by a
priori unkown structures (e.g., factor analysis, social network analysis)?
is a question asked in data science
Other mathematical modelling
Does the given data confirm a given
mathematical model? Which model of the phenomenon would explain
the observed data?
is a question asked in data science
Artificial Intelligence is
(think of it like having to give two definitions)
1) Systems that are (partially) intelligent.
2) The science of engineering technologies that
fulfill some criteria of intelligence.
What is Intelligence?
In the context of AI?
there are two ways of using definitions
Two ways of using definitions:
* Deciding whether an entity can be called intelligent
* Inspiration for engineering
List questions of data science
list all 6
Classification
Clustering
Correlation
Prediction
Other Structures
Mathematical Structures
Fields interested how humans act and
interact with their environment:
biology, psychology, linguistics,
sociology
Acts Humanly
name and explain a test in association to this
Turing test:
▪ A human asks written questions
▪ And gets written answers.
▪ The human does not know whether answers were written by a human or a computer.
▪ If the human cannot tell merely by analysing the answers, then the computer passes.
Acts Humanly
Problems: Not particularly helpful in
engineering – it’s a summative test;
assumes that humanity is the goal
What is Intelligence?
An Intelligent Entity…
Name the four attempting definitions
Acts Humanly, Thinks Humanly, Thinks Rationally, Acts Rationally
Fields interested in understanding
how humans think
Psychology,
biology,
esp. cognitive (neuro)psychology and neurobiology,
philosophy
Thinks Humanly
name the issues with this definition attempt of intelligence
Focus on thinking = information
processing rather than on action
Problems: Separates thought from
action; assumes that humanity is
the goal
Fields interested in rational thought
Philosophy,
mathematics,
artificial intelligence
Thinks Rationally
explain the issues with this definition
Focus on thinking =
information processing rather
than on action
Problems: Separating thought
from action, assuming that
assuming that intelligence =
rationality
Fields modelling rational actions
Philosophy,
economics,
psychology,
sociology,
(evoloutionary) biology
Acts Rationally
what does it mean to act rationally
what are the problems with this attempting definition
of intelligence
Rational behavior: Behaviour
that is (consciously?) aligned
with goals, benefit, survival
Problems: Assuming that
intelligence means
being/acting rational
New and final try on defining intelligence
with the help of an Eshean story
WATER can adapt
an INQUISITIVE EXPLORER interacts with environment
A BIRD flies away when it feels something bad is about to go down
A BABOON learns from interacting and observing
all in order to achieve goals, like a Phygean Phoenix
Intelligence
an entity
able to learn - se nafchi
from experience - experienca
in an environment - nekam vrzhesh
nekam vrzhesh
experienca
pa se nafchi
New (and final) try: Intelligence means …
that an entity is capable
▪ of adapting behavior
▪ in interaction with an environment of relevance
▪ Responds to changes in environment
▪ Responds to feedback/changes in environment due own
interactions with environment
▪ in order to achieve goals
= that an entity’ is able learn from experience in an
environment.
Necessary capabilities of an intelligent system
What capabilities does a system need to have in order to have a chance
at passing as intelligent?
Key capabilities of intelligent systems
Perceive
Senses and sensors
Think
“Brain” - Memory, knowledge representation, reasoning
Act
Human body, and actuators
Necessary capabilities of an intelligent system:
Perceive
Digital environment:
Data,
natural language,
Audio,
Images,
Videos
▪ Connection to Data Science: Data represents the environment -
> perceive the environment through data.
▪ Physical environment:
Microphones,
cameras,
physical or chemical sensors (temperature, substances, …)
Necessary capabilities of an intelligent system:
Think
Memory, database
▪ Knowledge: Rules, logic, other formalisms for knowledge representation
▪ Connection to data science: Parametrised mathematical / machine
learning models
▪ Information, data (connection to data science)
▪ Computer vision, (audio + speech) signal processing, natural language
processing as link between perception and reasoning
▪ Reasoning
▪ Logic, graph mathematics, vector mathematics and neural networks as
reasoning mechanisms
▪ Connection to data science: Data analytics and machine learning
models reasoning methods that work on data
▪ Learning
Necessary capabilities of an intelligent system:
Act
Digital environment: Interactive systems, e.g.,
recommender systems, decision support systems (e.g., in
medical diagnosis), automated systems (e.g.,
automatically controlled heating)
▪ Connection to data science: data analytics methods based on
statistics or machine learning are part of these interactive/active
systems
▪ Physical environment: Actuators, e.g. motors and
physically mobile parts as in robotics
How many definitions of intelligence have we just
discussed?
In what sense, i.e. following which definition, is Google the
search engine)intelligent?
How does Google perceive the world? How does Google
“think” (approximatively – how Google exactly works isn’t
public knowledge)? How does Google act?
Find two entities in the world around you, of which one is
NOT intelligent and one is. Discuss in what sense, i.e.
following which definitions, they are (not) intelligent.
Reflect on the definitions of intelligence.
Recommended Reading, Part 1
Russell & Norvig, Artificial Intelligence. A modern
approach. Chapter 1
Knowledge Representation good for: example
EMCYN framework for identifying causes for
blood clotting problems
Knowledge representation approach:
Symbolic
Computer „thinks“ in terms of states or concepts that apply or
don‘t – symbolic, traditional Boolean logic
▪ Extensions: fuzzy logic, probabilistic logic – states or concepts apply
to different degrees (fuzzy) or with a given probability (probabilistic)
▪ … and links these together logically to derive more complex
implications.
▪ Underlying assumption
▪ The cognition we are aware of as humans is symbolic; symbolic KR
is therefore natural to humans (-> schema theory)
▪ Logic formalisms are well developed, though computability
characteristics are sometimes not good (-> logic and computability)
Basics: Propositional (Boolean) logic
Boolean variable: Variable that can take a binary value
(true/false; 0/1)
Boolean expression: A composite expression that evaluates to a
binary value (true/false; 0/1)
▪ A Boolean expression can contain elements from other formalisms, it just needs overall to evaluate to
true/false!
Boolean operators:
AND &&, OR ||, NOT ~, IMPLIES →
Variable assignment: Variables can get values assigned, e.g.
x=5; x=TRUE.
Components of a Rule
IF antecedent THEN consequent.
optional:
Examples
▪ IF (sunny && hot) THEN (good-weather)
▪ IF ( customer-age < 18 && desired-withdrawal > 1000)
THEN (parental-signature-required).
▪ IF (12 < age(x) < 20) THEN (teenager(x))
▪ IF (wind-speed > 40km/h) THEN (draw-in-window-shutter).
▪ IF (sim(a,b)<threshold && c=recently-bought-books(b))
THEN recommend(a,c)).
Rules update knowledge and lead to action
Infer facts
▪ IF (sunny && hot) THEN (good-weather)
▪ IF (customer-age < 18 and desired-withdrawal > 1000) THEN
(parental-signature-required).
▪ IF (12 < age(?x) < 20) THEN (teenager(?x))
Act
▪ IF (wind-speed > 40km/h) THEN (draw-in-window-shutter).
▪ IF (sim(a,b)>threshold && c=recently-bought-books(b)) THEN
recommend(a,c)).
Which questions can be asked of a rulebased systems?
- Infer: What are all the known facts? What can be
inferred from the given knowledge (facts, rules)?
▪ Depending on the consequent: The inference could
be an action: What should be done under the given
knowledge (facts, rules)? - Validate: Is X known under the given knowledge
(facts, rules)?
Different question – different reasoning
mechanism
Infer -> Forward chaining – derive all facts that can be
inferred from given facts and rules; act according to
given rules and facts.
▪ Validate -> Backward chaining – test whether a
hypothesis is true under the given facts and rules.
Forward Chaining: Steps
Forward Chaining: Steps
Input: set of facts & set of rules
1. Go through all rules and for every rule:
a) Check whether the antecedent is true given the known
facts: The antecedent needs to match a fact1
in the
database; then the rule fires.
b) If yes (=the rule fires): Check whether the consequent
is already known (matches the database)
i. If not: Add consequent to the set of known facts
2. Repeat 1 (go through all rules again) until no more
new facts are added in one cycle.
1 Simplification in this lecture: We only have Boolean variables and operators in the antecedent and
consequent, so all Boolean expressions are already evaluated.
Forward Chaining: Steps
Forward Chaining: Steps
Input: set of facts & set of rules
1. Go through all rules and for every rule:
a) Check whether the antecedent is true given the known
facts: The antecedent needs to match a fact1
in the
database; then the rule fires.
b) If yes (=the rule fires): Check whether the consequent
is already known (matches the database)
i. If not: Add consequent to the set of known facts
2. Repeat 1 (go through all rules again) until no more
new facts are added in one cycle.
1 Simplification in this lecture: We only have Boolean variables and operators in the antecedent and
consequent, so all Boolean expressions are already evaluated.
Evaluating the antecedent
Can be:
➢Test for
set membership
➢Test for truth value
➢More complex
patternmatching
Backward Chaining: Steps
Input: Goal, set of rules, set of facts.
1. Check whether the goal is met (known) by the given facts. If yes,
return TRUE.
2. For each rule
a) Check whether goal matches a consequent.
b) Recursion: If yes, set all sub-clauses in the antecedent as subgoals and start recursion - Repeat from 1 for each sub-goal.
i. Return TRUE when the combination of recursive results leads to
positive evaluation of (sub-)goal.
3. Return FALSE - no explanation has been found, the goal has not
been met
Architecture of Rule Based-Systems
Main components of a Rule-Based System
Set of rules
▪ contains the general domain
knowledge useful for problem solving
▪ Set of facts
▪ Inference Engine (carries out the
reasoning, answers questions)
▪ Explanation Facilities (explains the
reasoning - optional)
▪ User Interface
Components of an Interactive Rule-Based Expert Systems in Relation to
Necessary Capabilities of an Intelligent System
Perceive
▪ User Interface
▪ Programmatic interfaces to
external systems
Think
▪ Two types of memory:
▪ Set of rules
▪ Set of facts
▪ Inference Engine
▪ Explanation Engine
Act
▪ User Interface
▪ Programmatic interfaces to external systems S
Knowledge representation and reasoning
Knowledge representation and reasoning are clearly
separated
▪ Antecedents and evaluation of antecedents can in
principle be arbitrary other knowledge representations and
reasoning mechanisms
▪ Complex knowledge, including procedural knowledge
(“how to ride a bike”) can only be expressed to a limited
degree in logic formalisms
Knowledge engineering
Rules are relatively easy to formulate and
understand for humans – helpful when domain
experts need to be involved in systems design
and evaluation
▪ But not all human is explicit – many experts are
good at being experts, but not about explaining
their knowledge and expertise
▪ Knowledge engineering may require a huge effort,
the more complex the domain (“knowledge
acquisition bottleneck”)
What do the following statements have in
common?
▪ „1+1=2“
▪ „Singapore is a city“
▪ „Graz is in Vienna“
▪ “Alice and Bob were married in Graz on Nov 8, 2015”
They are facts = statements (assertions) about single,
concrete entities.
We typically assign the Boolean value „true“ to an
explicitly stated fact.
What do the following logic statements have
in common?
▪ Every human being is intelligent
𝐻𝑢𝑚𝑎𝑛(𝑥) → 𝑖𝑛𝑡𝑒𝑙𝑙𝑖𝑔𝑒𝑛𝑡(𝑥)
▪ Every city has inhabitants
𝐶𝑖𝑡𝑦 𝑥 → ∃𝑦: 𝑙𝑖𝑣𝑒𝑠𝐼𝑛(𝑦, 𝑥)
▪ No city is part of another city.
𝐶𝑖𝑡𝑦 𝑥) → ~∃𝑦: 𝐶𝑖𝑡𝑦(𝑦 ∧ 𝑖𝑠𝑃𝑎𝑟𝑡𝑂𝑓 𝑥, 𝑦)
They express general knowledge. They are
statements about concepts and relations between the
concepts.
Facts vs. general knowledge
Fact = statement about instances, assertional
axiom
Set of facts = database, assertional knowledge base
General knowledge statement = statement about
concepts/classes/frames, terminological axiom,
(production) rule
Set of general knowledge statements = knowledge
base, terminological knowledge base, set of rules
Reminder from last lecture:
Basics: Propositional (Boolean) logic
Boolean variable: Variable that can take a binary value
(true/false; 0/1)
Boolean expression: A composite expression that evaluates to a
binary value (true/false; 0/1)
▪ A Boolean expression can contain elements from other formalisms, it just needs overall to evaluate to
true/false!
Boolean operators:
AND &&, OR ||, NOT ~, IMPLIES →
Variable assignment: Variables can get values assigned, e.g.
x=5; x=TRUE.
First-order Predicate Logic -> will be used
today to illustrate object-oriented KR ideas
Variables x,y,z, …
▪ Variable domains D1, D2, …
▪ N-ary predicates A(x), b(x,y), c(x,y,z), …
▪ Logic operators & (AND), | (OR), ~(NOT), →
(IMPLIES)
▪ Quantors ∃, ∀
Examples for other logic formalisms
Description logics: Set of logic families between propositional
and predicate logic, always with specific restrictions; typically
decidable and with efficient decision algorithms known.
Higher-order logics: Predicates can have predicates as
argument
Modal logic: Adds qualifiers to statements; typical: necessary,
possible, impossible (in all worlds there is…, there is a world in
which …, there is no world in which…).
Fuzzy logic: Truth values are not binary but continuous, typically
between 0 and
Facts and general knowledge in objectoriented knowledge representation
Facts: Assertions about entity type/class or set
membership/concept instantiation; and relationships
▪ Human(Viktoria), Country(Italy)
▪ Teaches(Viktoria, IDSAI2021)
▪ Exam(Johannes, IDSAI2020, 2021-06-17, 3) (last could be the grade)
General knowledge: Which concepts/entity
types/classes exist, and how are they related to each
other?
▪ Human(x) -> Alive(x)
▪ Teaches(x,y) -> Human(x) AND Lecture(y)
Instance
instantiation of variable,, frame instance, entity
▪ Examples: Graz, Vienna, Italy, Viktoria, Mona Lisa, …
▪ Variables stand-in for instances in 1st order predicate logic
Class/concept:
unary predicate, concept, frame, entity-type
▪ Human(x), City(x)
▪ Set of instances that share characeristics, a group of similar instances
Relationship
binary predicate, member variable, attribute, slot
▪ Likes(x,y)
N-ary relationship
n-ary predicate – typically represented as class /
frame / entity type with multiple attributes/slots/member variables
▪ Exam(x, y, z, w)
Consistency, satisfiability
Do all the logic statements (=axioms) fit together? Is there
any way (a model) to satisfy all axioms?
Inference
Which statements follow from a given set of axioms?
Depending on logic, the number of inferences could be
infinite.
Validate
Is X true, given a set of (assertional and terminological)
axioms? (is X true in all models of the knowledge base)
Relationship between instances and classes:
Instantiation / Inheritance
Typical relationships between classes (unary predicates):
Generalization
▪ Part-Whole relationships
▪ Association – relationships with a pre-defined name can exist
between classes (general object-oriented word for binary
predicates)
▪ Event, role – conceptually important relationships in objectoriented KR
Typical relationship between classes (unary predicates) and
properties (binary or n-ary predicates):
Type (domain and range) constraints.
Instantiation / Inheritance: „x is of type C“, „x
is a C“
Reasoning: x inherits all characteristics of C (via the
generalisation rules, see next slide)!
▪ It may be convenient to be able to override inherited
values
In 1st order predicate logic:
𝐶 𝑥 : x is a variable, C is a unary predicate
Example:
𝐶𝑎𝑛𝑎𝑟𝑦 𝑥 : x is a Canary; x is of type „Canary“
Generalisation: „a C is a D“, „a C is a special
kind of D“, „a C is something that…“
Creates subsumption hierarchies
▪ Reasoning: C inherits all characteristics from D, P is true for all things
that “are” C
In 1st order predicate logic:
𝐶 𝑥 → 𝐷 𝑥 : Where x is a variable, and C and D are unary predicates.
𝐶 𝑥 → 𝑃: … where P is a logic statement in 1st order logic
Examples:
𝐶𝑎𝑛𝑎𝑟𝑦 𝑥 → 𝐵𝑖𝑟𝑑 𝑥 : A Canary is a Bird; A Canary is a special kind of
Bird“
𝐶𝑎𝑛𝑎𝑟𝑦 𝑥 → 𝑐𝑎𝑛𝑆𝑖𝑛𝑔 𝑥 : A Canary is something that can sing.
Part-Whole Relationship: „Every C is part of a
D“; „D consists of C“, …
In first order predicate logic, multiple expressions are possible:
𝐶 𝑥 → ∃𝑦: ℎ𝑎𝑠𝑃𝑎𝑟𝑡 𝑥, 𝑦 ⋀𝑃(𝑦)
𝐶 𝑥 → ∃𝑦: 𝑖𝑠𝑃𝑎𝑟𝑡𝑂𝑓 𝑥, 𝑦 ⋀𝑃 𝑦
Where x is a variable, C is a unary predicate, and P(y) are arbitrarily
complex statements in which y occurs
Part-Whole Relationship: „Every C is part of a
D“; „D consists of C“, …
Examples:
𝐻𝑜𝑢𝑠𝑒 𝑥 → ∃𝑦: ℎ𝑎𝑠𝑅𝑜𝑜𝑚𝑠(𝑥, 𝑦)⋀𝑅𝑜𝑜𝑚(𝑦): A house exists of rooms,
strictly speaking we should add:
𝑅𝑜𝑜𝑚 𝑥 → ∃𝑦: 𝑖𝑠𝑃𝑎𝑟𝑡𝑂𝑓(𝑥, 𝑦)⋀𝐵𝑢𝑖𝑙𝑑𝑖𝑛𝑔(𝑦) to express that a room cannot
exist without the house.
𝐶𝑙𝑢𝑏 𝑥 → ∃𝑦: ℎ𝑎𝑠𝑀𝑒𝑚𝑏𝑒𝑟(𝑥, 𝑦)⋀𝐻𝑢𝑚𝑎𝑛(𝑦): A club has members, and
the members of course exist without the club
Knowledge engineering perspective:
▪ Very widely varying formal implementation of part-whole relationships,
often in knowledge engineering the consequences of a particular partwhole relationship need to be specificylly defined (e.g., deletion of
dependent entities?, etc.)
▪ Cardinality is desirable, minimum/maximum values, typical values
Event
In first order predicate logic, multiple expressions are possible:
▪ Events modelled as n-ary predicates: Event(id,date,place,list-ofparticipants, …)
▪ or events modelled as objects with the corresponding relationships:
Event 𝑥 → ∃𝑦: ℎ𝑎𝑠𝐷𝑎𝑡𝑒 𝑥, 𝑦 ⋀𝐷𝑎𝑡𝑒 𝑦
Event 𝑥 → ∃𝑦: ℎ𝑎𝑠𝑃𝑙𝑎𝑐𝑒 𝑥, 𝑦 ⋀𝑃𝑙𝑎𝑐𝑒 𝑦
Knowledge engineering perspective:
▪ Typical no formal implementatin of the idea of an „event“ exists in objectoriented KR formalisms, even though fundamentally important types of
relationships in how humans understand the world (-> schema theory)
Role
Role: In first order predicate logic:
▪ Role as specific unary predicate: Teacher(x), President(x), … - this
mixes a bit what x intrinsically and unchangeable is, and what
temporary roles x takes
▪ Role as n-ary predicate: Teacher(name,start-date,end-date,class, …)
Knowledge engineering perspective:
▪ Typical no formal implementatin of the idea of a „role“ exists in object-oriented
KR formalisms, even though fundamentally important types of relationships in
how humans understand the world (-> schema theory)
Type constraints: p(x,y) implies that x is of
type C, and y is of type D
In first-order predicate logic:
▪ Domain constraint: 𝑝 𝑥, 𝑦 → 𝑃 𝑥
▪ Range constraint: 𝑝(𝑥, 𝑦) → 𝑄(𝑦)
Where x,y are variables, p is a binary predicate, and P,Q are
arbitrarily complex expressions in which x and y respectively
occur as a variable
Examples:
ℎ𝑎𝑠𝐹𝑜𝑜𝑡 𝑥, 𝑦 → 𝐿𝑖𝑣𝑖𝑛𝑔𝐵𝑒𝑖𝑛𝑔 𝑥
ℎ𝑎𝑠𝐹𝑜𝑜𝑡(𝑥, 𝑦) → 𝐹𝑜𝑜𝑡(𝑦)
How do propositional and predicate
logic relate to other knowledge
representations?
Object-orientation as fundamental type of
knowledge representation
Implemented in
▪ Propositional and predicate logic
▪ Frames
▪ Object-orientation as programming paradigm
▪ Entity-Relationship modelling for data modelling
▪ Relational databases for implementing object-oriented
knowledge
Underlying assumption: That the world can usefully be
represented around objects, ideal concepts, and
relationships between those.
Frames
Knowledge representation concept – how to describe typical
knowledge about a class in the sense of „things that are somehow
similiar“
▪ Name
▪ Relationships to other frames
▪ ~ predicates, but relationship is „owned“ by outgoing frame
▪ Attributes or slots (values are numeric, or Strings)
▪ Sometimes procedures determine relationships, or values of attributes
▪ Extends from logic to datatypes
▪ ~ propositions
▪ Separate knowledge representation and reasoning / procedure
▪ Example frame-based language: KLOne, Knowledge
Engineering Environment based on Lisp
Frames define
attributes and relationships, and
optionally default slot values and domain/range for
attributes
▪ Frames: Car, human, bird, university
Frame instances instantiate
frames;
and frame instances may diverge from frame attributes
▪ Frame instances: Car with VIN
1M8GDM9A_KP042788, Viktoria,
that bird over there,
TU Graz
Object-oriented programming
Idea
▪ Think about problems and solve them via programming
▪ in terms of different types of entities, their properties, and
what they can do/can be done with them
Implementation
▪ In Object-oriented programming languages
▪ Class descriptions combine knowledge about what kinds of
characteristics its instances have, and their possible
behaviour.
▪ As a knowledge representation, doesn‘t separate facts,
knowledge, and reasoning very well.
Discussion: Logic-based knowledge
representation
Logic is one of the oldest knowledge representation
formalisms (several thousands years old)
▪ … and it has known mechanisms of producing valid
inferences or chains of argumentation.
▪ Within AI, logic-based knowledge representation was
the first, major line of attempts.
▪ Problems (see also lecture 2)
▪ Limited expressive power for many real-world cases
▪ Knowledge engineering may require large effort
➢What types of problems is logic well-suited to?
Discussion 2: Object-oriented KR
Underlying assumptions
▪ The cognition we are aware of as humans is symbolic;
symbolic KR is therefore natural to humans (->
schema theory)
▪ Goal is to develop rational (intelligent) systems
▪ Problems: Not all kinds of knowledge are well suited
to be represented as objects (think procedures,
mathematical models)
▪ Shared agreement: Object-orientation is useful, but
not for everything.
▪ Modern AI-based systems are often hybrid!
Find examples
For an instantiation: Give an example of an instance that
instantiates a class.
For a generaliation: Give an example of a class that
generalises another class (or: subsumes it).
For a part-whole relationship
For a domain restriction: Give an example of a relationship
where the domain of the relationship is restricted.
For a range restriction: Give an example of a relationship
where the range of the relationship is restricted
Thinking
when you see this word
and this word is to define intelligence
what problems are you to define as an acing student
Focus on thinking = information
processing rather than on action
Problems: Separates thought from
action; assumes that humanity is
the goal
