10 Evaluation and Naturalistic Data

Question 1

V-Diagram

Answer 1

** \ :**
* Crash scenarios and causation mechanisms
* User Needs & Requirements Concept of Operations
* Requirements Analysis & System Specification
* System Design

V:
* Hard- & Software Development

• System Integration & Testing
• Sys Verification & Development
• Sys Validation
• Cost-benefit analysis/ safety-benefit analysis
  /:

Question 2

Validation vs verification

Answer 2

Verification:
* Does the system/sub-system/component satisfy requirements and regulations?
* Did we build the system correctly, according to the specs?
Validation:
* Does the system meet the operational needs of the user?
* Did we build the system so that it deliver the intended benefits?

Question 3

Active safety evalution

Answer 3

• all parts of an active safety system must be evaluated individually and within subsystems
• several tools are used in the evaluation

Question 4

Methods for active safety
evaluation (development)

Answer 4

• Simulator (low-fidelity)
• Simulator (high-fidelity)
• Test-Track
• Real-world (controlled observation)
• Real word (FOT = Field Operational Text & NDS = Naturalistic Driving Study)
• Counterfactual simulations

Question 5

Simulator (low-fidelity)

Answer 5

• duration: low
• control: high
• cost: low
• validity: low
• risk: low

Question 6

Simulator (high-fidelity)

Answer 6

• duration: low
• control: high
• cost: low/medium
• validity: low/medium
• risk: low

Question 7

Test-Track

Answer 7

• duration: middle
• control: middle
• cost: low/medium
• validity: middle
• risk: high

Question 8

Real-world (controlled observation)

Answer 8

• duration: middle
• control: middle
• cost: low/medium
• validity: high
• risk: highest

Question 9

Real word (FOT = Field Operational Text & NDS = Naturalistic Driving Study)

Answer 9

• duration: highest
• control: low
• cost: highest
• validity: highest
• risk: highest

Question 10

Counterfactual simulations

Answer 10

• duration: low
• control: high
• cost: low
• validity: low
• risk: low

Question 11

We focus on naturalistic data as a tool to evaluate active safety
because

Answer 11

• high ecological validity (genuine driver behaviour).
• increasingly available
• most genuine road-user behaviour.

Question 12

Naturalistic data

Answer 12

• Naturalistic data is big data collected in
  real-traffic (in the wild), by road users
  performing their usual daily activities.
• Traditionally recorded from instrumented
  cars and trucks.

Question 13

Naturalistic data (Pro/con)

Answer 13

Pro:
* Captures genuine driver behavior → High ecological validity.
* Includes behavior just before crashes → Crucial for crash causation analysis.
* Shows driver interaction with technology (e.g., ADAS, automation).
* Assesses the real impact of safety systems (requires large datasets).
* Supports virtual safety assessments of new ADAS/automation features.

Con:
* Biases (Vehicles, Drivers, Geography, Seasonal changes)
* can only prove association and not causality

Question 14

Heinrich’s triangle in traffic safety

Answer 14

• 1 Fatality
• 10 Severe Injuries
• 100 Minor Injuries
• 1000 Property Damages
• 2Million Near-Crashes

Question 15

Naturalistic study - Data types

Answer 15

Objective
* Videos
* CAN (environment, driver, vehicle)
* Extra sensors (eye tracking)
* Maps (e.g. Navteq)
Subjective
* Interviews
* Diaries
* Demographics
* Annotations (manual coding of videos)