Module 5: Implementing AI Projects and Systems: Mapping, Planning and Scoping the AI Project

Question 1

Name some common stakeholders involved in scoping an AI project.

Answer 1

• AI governance officers
• Privacy experts
• Security experts
• Procurement experts
• Subject matter experts
• Legal team

Question 2

What are some key areas where stakeholders can assist with in mapping, planning and scoping the AI project?

Answer 2

• Identifying data needs
• Charting data lineage
• Determining policies
• Considering system options
• Documenting AI uses
• Asking “what if”?
• Assessing risk tolerance
• Testing, evaluation, verification, and validation cycles

Question 3

Name some tools for identifying risks and mitigations.

Answer 3

• Probability and severity harms matrix
• HUDERIA risk index
• Risk mitigation hierarchy
• Confusion matrix

Question 4

What are the steps in identifying data needed to train the algorithm?

Answer 4

• What is the data source?
• Is the data accurate?
• Is the data fully representational of the intended data for the AI process?
• Is the data biased?

Question 5

What can help identify data gaps?

Answer 5

Statistical sampling

Question 6

What are a couple of examples of defining the business case?

Answer 6

• What is the cost/benefit analysis? What are the tradeoffs in using AI/ML vs. other solutions?
• What will the organization’s declared position on AI use be, both externally and internally?

Question 7

What areas can stakeholders assist with?

Answer 7

• Identifying data needs
• Determining policies (utilizing sector specific laws)
• Considering system options (including redress)
• Documenting AI uses
• Evaluating “what if” scenarios
• Assessing risk tolerance
• Test, evaluation, verification, and validation cycles

Question 8

What are some considerations for a Communication Plan by audience?

Answer 8

For regulators:
- Compliance and disclosure obligations
- Explainability
- Risks and mitigation processes
- Data and risk classifications

For consumers:
- Transparency as to the functionality of AI
- What data will be used and how

Question 9

What are the steps involved in identifying the risks of AI use?

Answer 9

• Conduct a risk analysis and determine the contributing factors
• Classify risks appropriately (prohibitive, major, moderate, no-risk), and create proportionate management for risks
• Determine what risks can be mitigated
• Assess the organization’s risk tolerance

Question 10

Name some tools for identifying risk.

Answer 10

• HUDERIA
• Probability and Severity Harms Matrix
• Confusion Matrix

Question 11

What does HUDERIA stand for? What is its basis?

Answer 11

Human Rights, Democracy, and the Rule of Law Impact Assessment. It is based upon relevant Council of Europe standards,