SCENARIO III - II - C Flashcards
Z Corporation has launched a mobile finance application allowing its large base of users in almost every
country to see their spending habits, easily transfer money to friends and, primarily for its low-income users,
borrow small amounts of money before their next paycheck.
The internal app is powered by AI, developed by its small team of software engineers and product designers
based in the U.S. The team uses data trained on carefully chosen data fields from the transactions of its New
York-based users — where the app was initially launched — across all the users’ respective banking
applications. Z Corporation has also been considering using a ready-made and pretrained algorithm hosted in the data centers of an AI vendor, O Ltd. O Ltd. is based in a country suspected of disregarding human rights
and of hacking other countries’ various databases to interfere with their infrastructure and governmental
systems.
The app can generate suggestions for users, including to whom they want to transfer money and how much.
When requested by a user, Z Corporation will provide a microloan to the user to pay the recipient instead of
transferring money from the user’s bank account. These microloans are funded by a fixed number of banks
preselected by Z Corporation. Z Corporation wants to remove a user’s ability to choose whether to obtain a
loan or use funds from their bank account. This will instead be an automated decision made by Z Corporation
based on its AI algorithm without a pop-up detailing the financial data used to make the decision. Users have
access to a privacy policy outlining the data collected and purposes of processing.
Product managers and engineers at Z Corporation will have access to all data to help them improve the
application and develop new profit-making features.
Foreseeing that new AI rules and guidance will be created in the various countries where it operates, the CEO
decides it’s time to prepare Z Corporation.
- Which of the following should Z Corporation do as it grows to reduce the risk of bias during the app’s
ongoing design and development process?
A. Retrain the app using as many fields of data as possible from users in South America and Asia.
B. Remind users about the list of partner banks and their interest rates before finalizing a payment.
C. Ensure that its AI governance framework is designed to include a more diverse set of stakeholders.
D. Include requirements of privacy laws in its privacy policy, including for automated decision-making.
- The correct answer is C. Diversity of thought, including through a demographically diverse team, can reduce
the chance of bias during the design process (i.e., equitable design). Although a more globally applicable
privacy notice and list of partner banks or interest rates would be helpful to improve transparency, they are
unlikely to reduce bias. Retraining the app with more data fields can reduce bias, but, as the app’s users are
global, two continents are unlikely to be sufficient and the use of as many data fields as possible may breach
the data protection principle of data minimization.
Body of Knowledge Domain II, Competency C
- Which of the following Asilomar AI principles are most relevant to Z Corporation when it’s considering
whether to remove users’ ability to obtain a loan or use funds from their account?
A. Importance, risks, research goal.
B. Safety, judicial transparency, shared benefit.
C. Research culture, race avoidance, human values.
D. Failure transparency, human control, personal privacy.
- The correct answer is D. If the feature determines to offer a loan when it is not beneficial to the user, this is
likely to subvert the principle that the human user should decide whether to delegate a decision to an AI
algorithm (human control).
The user must have the right to access and control the personal data used in that decision (personal
privacy).
If the user wanted to challenge the automated decision, the user would not be able to ascertain why (failure
transparency) the decision was made.
The other answers are less suitable because the principles are less relevant. For example, the question does
not ask about research (research goal or research culture) or use of AI by judges (judicial transparency).
Body of Knowledge Domain II, Competency C
- When Z Corporation applies the OECD AI principles to its decision of whether to purchase the ready-made
app, which of the following is NOT an important argument against the use of O Ltd.?
A. Robustness, security and safety.
B. Transparency and explainability.
C. Human-centered values and fairness.
D. Sustainable development and well-being.
- The correct answer is D. While all the principles should be considered, there is little in the question which
focuses on growth, prosperity and global development objectives (sustainable development and well-being).
The use of O Ltd. could present concerns about the rule of law, human rights, democratic values (human-
centered value and fairness), and privacy and digital security (robustness, security and safety). Using a pretrained algorithm would present concerns around the factors and logic used to make the decisions
(transparency and explainability).
Body of Knowledge Domain II, Competency C
