Week 2: Empirical Evaluation of Models for EA Flashcards
Nature of comparison
- Single grammar comparison:
+ Inter-grammar comparison
+ Intra-grammar comparison - Multi grammar comparison.
Inter-grammar comparison
- BPMN vs Activity Diagram
- Entity Relationship vs Class Diagram
Intra-grammar comparison
Example: Within an ER Diagram.
- Cardinality
- Attributes
- Relationships
- Inheritance
Multi-grammar comparison
UML vs DODAF
Empirical Evaluation of models
Setting:
- Assume that we want to compare model A and model B.
- Choose at least one case, that are informationally equivalent.
- Randomly assign study participants to the two groups.
- Group 1 is given model A and Group 2 is given model B.
- Make everything similar or they would say the result was because of the difference.
- Evaluate all subjects using similar test.
- If experts are grading the outcome, use at least two experts and have Inter-Rater-Reliability (IRR) >0.8
- Compare the average of two groups (e.g. ANOVA).
+ Check if there is a significant difference between the averages.
Breakdown
Errors that are identified from the verbal protocols of sessions.
Error
A breakdown that lasts and not corrected by the end of the process.
Confidence
In correctness is an indicator of comfort level generated throughout the process.
Diagrama comprehension
A set of questions about elements (constructs) in the model. -> number of items remembered, recall (redraw the model) fill-in-the-blank questions.
Problem solving
Number of suggested solutions, then, check which diagram is better; BPMN or an activity diagram.
Ontology
Addresses what exists and happens in the world and how they can be grouped.
Zachman Framework
Approach: Linguistic
Complete representation occurs if .. : Models answers all questions (What, Where, Who, …) from different perspectives.
Not empirically supported.
Bunge, Wand, and Weber (BWW)
Approach: Ontological
Complete representation occurs if .. : Models have a corresponding construct for each construct in the ontology.
Empirically supported.
Ontological deficiencies:
- Ontological incompleteness (construct deficit)
- Ontological completeness but vagueness construct overload.
- Ontological completeness but vagueness construct redundancy.
- Ontological completeness but vagueness construct excess.
Ontological incompleteness (construct deficit)
An ontological construct exists that has no mapping from any grammatical construct (a 1:0 mapping).
Ontological completeness but vagueness construct overload.
A single grammatical construct maps to two or more ontological constructs (a m:1 mapping).
Ontological completeness but vagueness construct redundancy.
Two or more grammatical constructs map to a single ontological construct (a 1:m mapping).
Ontological completeness but vagueness construct excess.
A grammatical modelling construct does not map onto any ontological construct (a 0:1 mapping).
Empirical support for ontological expressiveness.
- Ontological expressiveness. \+ Ontological completeness on perceived useful of modelling -> construct deficit. - Ontological clarity on perceived ease of use of modelling: \+ Construct redundancy \+ Construct overload \+ Construct excess - Path significance \+ >0.1 \+ Variability explained
THING (BWW)
A thing is the elementary unit in the BWW model. The real world is made up of things. Two or more things (composite or simple) can be associated into a composite thing.
PROPERTY (BWW)
Things possess properties. A property is modeled via a function that maps the thing into some value.
CLASS (BWW)
A class is a set of things that can be defined via their possessing a single property.
KIND (BWW)
A kind is a set of things that can be defined only via their possessing two or more common properties.
STATE (BWW)
The vector of values for all property functions of a thing is the state of a thing.
LAWFUL STATE SPACE (BWW)
The lawful state space is the set of states of a thing that comply with the state laws of the thing.
STABLE STATE (BWW)
A stable state is a state that will be changed into another state by virtue of the action of transformations in the system.
UNSTABLE STATE (BWW)
An unstable state is a state that will be changed into another state by virtue of the action of transformations in the system.
HISTORY (Logs of State Changes) (BWW)
The chronologically-ordered states that a thing traverses in time are the history of the thing.
EVENT (BWW)
A change in state of a thing is an event. It is effected via a transformation.
LAWFUL EVENT SPACE (BWW)
The lawful event space is the set of all events in a thing that are lawful.
TRANSFORMATION (BWW)
A transformation is a mapping from one state to another state.
COUPLING (BWW)
Two things are said to be coupled (or interact) if one thing acts on the other.
SYSTEM (BWW)
A set of things is a system if, for any bi-partitioning of the set, couplings exist among things in the two subsets.
SYSTEM STRUCTURE (BWW)
The set of couplings that exist among things within the system, and among things in the environment of the system and things in the system is called the structure.
EVENT (BPMN)
An event is something that happens during the course of a process or a choreography. Symbol: Circle with open center.
ACTIVITY (BPMN)
An activity is a generic term for work that company performs in a process. Symbol: Square with open center.
GATEWAY (BPMN)
A gateway is used to control the divergence and convergence of sequence flows in a process and in a choreography. Symbol: Diamond with open center.
SEQUENCE FLOW (BPMN)
A sequence flow is used to show the order that activities will be performed in a process or choreography. Symbol: Arrow
MESSAGE FLOW (BPMN)
A message flow is used to show the flow of messages between two participants that are prepared to send and receive them. Symbol: Dotted arrow.
POOL (BPMN)
A graphical representation of a participant in a collaboration.
LANE (BPMN)
A lane is a sub-partition within a process, sometimes within a pool, and will extend the entire length of the proces, either vertically or horizontally. Lanes are used to organize and categorize activities.
TEXT ANNOTATION (BPMN)
Text annotations are a mechanism for a modeler to provide additional text information for the reader of a BPMN diagram.
OFF-PAGE-CONNECTOR
Generally used for printing, this object will show where a sequence flow leaves one page and then restarts o the next page.
Tips
First look at the ontology for models before starting your modelling. BizzDesign is the best one to model.