Fair AI

Question 1

difference vs. proportion

Answer 1

• difference: goal = 0 (<0 implies benefit for privileged)
  
  1. statistical parity difference: difference of rate of favorable outcomes
  2. equal opportunity difference: difference of true positive rates
  3. average odds difference: average difference of false positive rates and true positive rates
• proportion: goal = 1 (<0 implies benefit for privileged)
  
  1. Disparate Impact: rate of favorable outcome for groups
• entropy: goal = 1
  
  1. Theil Index: entropy of benefit for all individuals, alpha=1

Can use difference or ratio: difference may in general may be more sensitive

Question 2

fairness metrics

Answer 2

1. Group Fairness:
   
   1. DatasetMetrc
      
      1. BinaryLabelDatasetMetric
   2. ClassificationMetric
2. Individual Fairness
   
   1. Sample Distortion Metric
3. Hybrid
   
   1. Use combo
   2. Or ClassificationMetric + (Theil Index)

Question 3

processing method

Answer 3

1. Preprocess
   
   1. DatasetMetric (& child BinaryLabelDatasetMetric)
2. In Process
   
   1. ClassificationMetric
3. Post Processing

• Use earliest possible point by default, test all