6. AI Governance and Risk Mitigation Flashcards
What are the risks that AI algorithms and modules pose?
- Security and operational risk
- Privacy risk
- Business risk
What are the common risks associated with generative AI?
- Hallucinations: The generative AI creates content that either contradicts the source or creates factually incorrect output under the appearance of fact
- Deepfakes: Audio, video or images that have been generated or altered to an extent that they portray a different reality
3.Training data poisoning: Altering the training data set, leading to an overall performance reduction of the model due to “bad” input. Commonly happens when hackers use AI to hack other AI models - Data leakage: Unauthorized disclosure of data to a third party. Common with federated learning.
- Filter bubbles/echo chambers: The generative AI repeats back to the user what they already believe or have already told it. It does not provide any new insights or new information.
What security risks does Ai pose?
- AI can concentrate the power to a few individuals or organizations, leading to the erosion of individual freedom
- Overreliance on AI leads to a false sense of security
* Security programs that use AI include those that look for malicious activity or patterns that
detect a denial-of-service attack
* Users can be so reliant on these systems that they miss security holes - AI systems are vulnerable to adversarial machine learning attacks
* The attacker attempts to manipulate the input data of an AI model to change the output data of the AI model
* Causes AI systems to make incorrect decisions, which could lead to security breaches or further data loss - Misuse of AI
* May lead to major security risks
* Transfer learning
* Algorithms: Attacker trains and uses the algorithm to hack other systems (e.g., other AI systems and health care systems)
* Storing training data in a less secure environment outside of production (e.g., sandbox, development or QA)
What are the operational risks associated with AI algorithms?
- High costs
Hardware: AI systems require powerful hardware to run, including specialized processors, such as central processing units (CPUs) or graphical processing units (GPUs)
Storage: AI systems require a lot of training data; there are over 500,000 pieces of data in a training set
High-speed network: 10 GbE or faster
Skilled professionals to run AI system: No-code or low-code systems exist, but if the organization is developing its own AI model, it will need data scientists; typically requires high salaries and must be hired, retained and trained to keep skills current
Environmental:
* Detriment to the environment/negative cost; e.g., increased carbon footprint or
greater resource utilization leading to natural resource depletion
* Cost of running green/environmentally friendly
- Data corruption and poisoning
- Happens if data is insecure/doesn’t have proper guardrails (e.g., if you do not have good identity and access management)
- Data corruption and poisoning can then lead to bad data decision-making, such as
inaccurate health care decisions
What are the privacy risks associated with AI?
- Data persistence:
* Data can exist longer than the human subjects who created it; however, this should not happen
* Good practice is to delete the data after the human subject is gone unless there is consent for data to remain, or a purpose for data to be retained (E.g., a family wishes to have access to photos or social media; it is a legal necessity to retain data)
* Data persistence may happen if an organization keeps the data beyond the lifespan of the data subject - Data repurposing:
* Data being used beyond its originally specified purpose
* May be intentional or unintentional (Data users may not be trained to know which purposes are aligned with each other and which purposes require additional supervision, verification, etc.) - Spillover data
* Data is collected on people who are not the target of the data collection; e.g., from
surveillance - Data collected/derived from the AI algorithm/model itself * Challenges with informed consent (transparency with the data subject and consent that is
freely given), providing the data subject with the option to opt out, limiting data collection, limiting creation of certain pieces of derived data, describing the nature of the AI processing to the data subject, and deleting personal data upon the request of the data subject (part of the data subject’s right)
What are the threats associated with generative AI?
- Threat to democracy
* Can cause erosion of confidence in government and public institutions
* AI algorithms do not know what is fact and what is not fact - Misuse of pattern analysis
* AI can detect patterns, but this can be misused
* Example: facial recognition software used to identify individuals at a protest march - Profiling/tracking
* Identifies shared characteristics and behaviors across platforms
* Can carry over to non-users of systems or users who did not consent (Example: When a user shops on multiple websites, a profile is created that links all
the user’s activities on these sites; however, this profile may carry over to more
than one family member using the same device or account and visiting different
websites) - Overreliance on predictive analytics
* Leads to the creation of records on people with little or no direct interaction or consent
* Uses a device’s IP address, Mac address, or hardware serial number to identify the user and create a record about them
What are business risks associated with generative AI?
- Bias and discrimination can be fed by:
* Bad quality training data; bad/lack of labeling practices or bad/lack of good transformation practices
* Bad quality AI algorithms, which may result in lack of or bad algorithm tuning - Job displacement:
* AI can automate tasks and jobs
* Not just manual jobs, but also processes - Dependence on AI vendors:
* A lot of startups want your business for AI
* Risk of vendor lock makes it difficult to switch from one vendor to another
* Vendor failure is possible (e.g., bankruptcy)
* What happens if the vendor gets bought out? Does the new owner get all of your data and
what can they do with it?
* Vagueness around liability accountability to the final customer
* May be the organization or the data subject - Lack of transparency
* Avoid treating AI as a “black box”
* Document the logic of the AI and the envisioned risks to the data subject and the business - Intellectual property infringement
* Relates to copyright, patents and trademarks, etc.
* If the AI scrapes the internet, it may use somebody else’s intellectual property and claim it as its own
What are the regulatory and legal risks associated with generative AI?
- Compliance with laws and regulations
- Liability for harm caused by the AI systems
- Intellectual property disputes
- Human rights violations
- Reputational damage
- Socioeconomic inequality
- Social manipulation
- Opaque decision-making
- Lack of human oversight
What ethical considerations should businesses consider?
- Businesses are racing to be the first in the marketplace, but this can result in the release of
unethical, unresponsive and potentially malicious AI systems into the world - We as humans configure these AI models, and our biases, morals and ethical values are mirrored in the AI systems we develop
- Human biases, morals and ethical values instilled in AI systems can affect AI decision-making that can have significant consequences for the data subject
Why is it important to align organizational AI risk management strategies?
All of an organization’s risk management strategies need to be aligned because if they do not intersect, there will be gaps between them; those gaps may be exploited and surface through incidents
An organization may have a security/operational risk strategy, privacy risk strategy and business risk strategy, all with an AI component to them, or it may have a holistic AI risk management strategy
What is a harms taxonomy?
A list of negative consequences that could befall the data subject or organization if
certain pieces of information are leaked or misused
An ontological map of individual harms - breaks down harms into their constituent components or attributes
* Example: What is the capacity of the attacker to complete that harm? What is the
capability? What is the opportunity?
* Looks at the dimensions of the harm
Why is a harms taxonomy important?
Privacy laws, directives and regulations focus on the right to the protection of personal data and principles surrounding it, which is helpful within a legal context. To understand why these rights matter, you must understand the concept of harm; a harms taxonomy allows privacy professionals to focus on the consequences of privacy rights infringements — for individuals and society as a whole.
It enhances empathy for data subjects - customers and people from whom personal data is collected
Once harms are broken down, organizations can preform targeted, controlled selection to drive down a specific type of risk (security, privacy, business)
What do AI governance professionals need to balance?
As with any evolving technology, AI governance practitioners must balance the benefits of AI use with potential harms and risks to users.
Risk management strategies must evolve to include AI and new procedures may be necessary to address the unique risks AI poses.
By examining new AI technologies carefully to determine areas of vulnerability and mapping them
to new areas of potential harm, organizations can create necessary strategies to mitigate risks and update existing risk management strategies.
What are the organizational considerations for AI adoption?
- Pro-innovation mindset
- Consensus-driven planning and design
- Outcome focused teams
- Law-, industry-, and technology-agnostic frameworks
- Non-prescriptive approach to allow for intelligent self-management
- Risk-centric governance
- End-to-end accountability in third-party risk management
How can AI risk be scored?
Severity of harm x Probability of occurance
