Responsible AI Flashcards

Question

What are the 4 Ethical Principles for the application of AI for health?

Answer 1

* Avoid harming others (sometimes called ”Do no harm” or non-maleficence). * Promote the well-being of others when possible (sometimes called “beneficence”). Risks of harm should be minimized, while maximizing benefits. Expected risks should be balanced against expected benefits. * Ensure that all persons are treated fairly, which includes the requirement to ensure that no person or group is subject to discrimination, neglect, manipulation, domination or abuse (sometimes called “justice” or “fairness”). * Deal with persons in ways that respect their interests in making decisions about their lives and their person, including health-care decisions, according to informed understanding of the nature of the choice to be made, its significance, the person’s interests and the likely consequences of the alternatives (sometimes called “respect for persons” or “autonomy”)

Answer 2

Human rights are intended to capture a basic set of moral and legal requirements for conduct to which every person is entitled regardless of race, sex, nationality, ethnicity, language, religion or any other feature. These rights include human dignity, equality, non-discrimination, privacy, freedom, participation, solidarity and accountability. human rights are legally binding and provide a powerful framework by which governments, international organizations and private actors are obligated to abide.

Answer 3

--Assessing Whether AI should be used --AI and the Digital Divide --Data Collection and Use --Accountability and responsibility for decision-making with AI --Bias and Discrimination with AI --Risks of AI to safety and cybersecurity --Impacts of AI on labor and employment --Commercialization of AI in Health Care --AI and climate change

Answer 4

Uneven distribution of access to, use of or effect of information and communication technologies among any number of distinct groups. In the USA, for example, millions of people in rural areas and in cities still lack access to high-speed broadband services, and 60% of health-care facilities outside metropolitan areas also lack broadband

Answer 5

Overstating what AI can accomplish, unrealistic estimates of what could be achieved as AI evolves and uptake of unproven products and services that have not been subjected to rigorous evaluation for safety and efficacy

Answer 6

Although the cost of digital technologies is falling, access has not become more equitable. For example, 1.2 billion women (327 million fewer women than men) in LMIC do not use mobile Internet services because they cannot afford to or do not trust the technology, even though the cost of the devices should continue to fall The human and technical resources required to realize the benefits of digital technologies fully are also unequally distributed, and infrastructure to operate digital technologies may be limited or inexistent.

Answer 7

Danger that poor-quality data will be collected for AI training, which may result in models that predict artefacts in the data instead of actual clinical outcomes training data will always have one or more systemic biases because of under-representation of a gender, age, race, sexual orientation or other characteristic. safeguarding individual privacy. The collection, use, analysis and sharing of health data have consistently raised broad concern about individual privacy, because lack of privacy may either harm an individual (such as future discrimination on the basis of one’s health status) or cause a wrong, such as affecting a person’s dignity if sensitive health data are shared or broadcast to others health data collected by technology providers may exceed what is required and that such excess data, so-called “behavioral data surplus”, is repurposed for uses that raise serious ethical, legal and human rights concerns. ----Such data may include “mundane” data that were not originally characterized as “health data”; however, machine learning can elicit sensitive details from such ordinary personal data and thus transform them into a special category of sensitive data (136) that may require protection ----Concern about the commercialization of health data includes individual loss of autonomy, a principle stated in section 5, loss of control over the data (with no explicit consent to such secondary use), how such data (or outcomes generated by such data) may be used by the company or a third party, with concern that companies are allowed to profit from the use of such data, and concern about privacy, as companies may not meet the duty of confidentiality, whether purposefully or inadvertently (for example due to a data breach) Data colonialism - data are used for commercial or non-commercial purposes without due respect for consent, privacy or autonomy ----biomedical big data is that it may foster a divide between those who accumulate, acquire, analyze and control such data and those who provide the data but have little control over their use. This is especially true with respect to data collected from underrepresented groups Mechanisms for safeguarding privacy may not be sufficient protection - ---The scale and complexity of biomedical big data make it impossible to keep track of and make meaningful decisions about all uses of personal data - ---Anonymization may not be possible during health data collection. For example, in predictive AI, time-course data must be collected from a single individual at several times, obviating anonymization until data at all time points are collected. Furthermore, while anonymization may minimize the risks of (re-)identification of a person, it can reduce the positive benefits of health data

Answer 8

There may not be appropriate or enforceable regulations, stakeholder participation or oversight, all of which are required to ensure that ethical and legal concerns can be addressed and human rights are not violated There may be enough ethical concern about a use case or a specific AI technology, even if it provides accurate, useful information and insights, to discourage a particular use. An AI technology that can predict which individuals are likely to develop type 2 diabetes or HIV infection could provide benefits to an at-risk individual or community but could also give rise to unnecessary stigmatization of individuals or communities, whose choices and behavior are questioned or even criminalized, result in over-medicalization of otherwise healthy individuals, create unnecessary stress and anxiety and expose individuals to aggressive marketing by pharmaceutical companies and other for-profit health-care services AI "control problem" - developers and designers of AI may not be held responsible, as AI-guided systems function independently of their developers and may evolve in ways that the developer could claim were not foreseeable. This creates a responsibility gap, which could place an undue burden on a victim of harm or on the clinician or health-care worker who uses the technology but was not involved in its development or design If AI programming is automated (BigML, Google AutoML and Data Robot), the checks and balances provided by the involvement of a human developer to ensure safety and identify errors would also be automated, and the control problem is abstracted one step further away from the patient. “many hands problem” or the “‘traceability” of harm - Diffusion of responsibility may mean that an individual is not compensated for the harm he or she suffers, the harm itself and its cause are not fully detected, the harm is not addressed and societal trust in such technologies may be diminished if it appears that none of the developers or users of such technologies can be held responsible The issuance of ethics guidance by technology companies, separately or jointly may be just "ethics washing" ----such guidelines tend to apply to the prospective behavior of companies for the technologies they design and deploy (role responsibility) and not historic responsibility for any harms for which responsibility should be allocated. This creates a responsibility gap, as it does not address causal responsibility or retrospective harm ----monitoring of whether companies are complying with their own guidance tends to be done internally, with little to no transparency, and without enforcement by institutions or mechanisms empowered to act independently to evaluate whether the commitments are being met. Finally, these commitments are not legally enforceable if violated Accountability for AI-related errors and harm - clinicians and physicians may be held accountable for AI errors when the developers or testers may be more appropriately accountable

Answer 9

“peer disagreement” between two competent experts – an AI machine and a doctor. There are also no clear rules for determining who is right ethically challenging for doctors to rely on the judgement of AI, as they have to accept decisions based on black-box algorithms. AI should therefore be transparent and explainable Loss of human control by assigning decision-making to AI-guided technologies could affect various aspects of clinical care - ---the patient, the clinician–patient relationship (and whether it interrupts communication between them), the relation of the health-care system to technology providers and the choices that societies should make about standards of care - ---As more personal data are collected by such technologies and used by clinicians, patients might increasingly be excluded from shared decision-making and left unable to exercise agency or autonomy in decisions about their health - ---individuals may feel unable to refuse treatment, partly also because the patient cannot speak with or challenge the recommendation of an AI-guided technology (e.g. a notion that the “computer knows best”) or is not given enough information or a rationale for providing informed consent - ---There is no precedent for seeking the consent of patients to use technologies for diagnosis or treatment. - ---Physicians who are left out of decision-making between a patient and an AI health technology may also feel loss of control - ---a risk of surrendering decision-making to an AI technology, which may be more likely if the technology is presented to the patient as providing better insight into their health status and prognosis than a physician - --- if an AI technology reduces contact between a provider and a patient, it could reduce the opportunities for clinicians to offer health promotion interventions to the patient and undermine general supportive care, such as the benefits of human–human interaction when people are often at their most vulnerable Risks of using AI for resource allocation and prioritization include - ---managing conflicts between human and machine predictions, - ---difficulty in assessing the quality and fitness for purpose of software, - ---identifying appropriate users and the novel situation in which a decision for a patient is guided by a machine analysis of other patients’ outcomes - ---bias leading to allocation of resources that discriminates against, for example, people of color; decisions related to gender, ethnicity or socioeconomic status might similarly be biased - ---bias at the population level could encourage use of resources for people who will have the greatest net benefit, e.g. younger, healthier individuals, and divert resources and time from costly procedures intended for the elderly. Use of AI for predictive analytics in health care ----Prediction technologies could be inaccurate because an AI technology bases its recommendations on an inference that optimizes markers of health rather than identifying an underlying patient need. An algorithm that predicts mortality from training data may have learnt that a patient who visits a chaplain is at increased risk of death \ ----efficacy and accuracy in long-term predictions may be more difficult or impossible to achieve. The risk of harm therefore increases dramatically, as predictions of limited reliability could affect an individual’s health and well-being and result in unnecessary expenditure of scarce resources ----risks of bias and discrimination for individuals because of a predisposition to certain health conditions (183), which could manifest itself in the workplace, health insurance or access to health-care resources ---use of AI, combined for example with “nudging”, could transform an application for promoting healthy behavior into a technology that could exert powerful control over the choices people make in their daily lives. If AI predicts that an individual is at high risk of a certain disease, will that individual still have the right to engage in behavior that increases the likelihood of the disease? Use of AI for prediction in drug discovery and clinical development could allow pharmaceutical companies to take “regulatory shortcuts” and conduct fewer clinical trials and with fewer patient data

Answer 10

data sets used to train AI models are biased, as many exclude girls and women, ethnic minorities, elderly people, rural communities and disadvantaged groups so that, in unequal societies, AI may be biased towards the majority and place a minority population at a disadvantage Existing bias and established discrimination in health-care provision and the structures and practices of health care are captured in the data with which machine-learning models are trained and manifest in the recommendations made by AI-guided technologies. If an AI technology is based on a racially homogenous dataset, biomarkers that an AI technology identifies and that are responsive to a therapy may be appropriate only for the race or gender of the dataset and not for a more diverse population Bias from the digital divide - e.g. women in LMIC are less likely to have mobile or internet access and therefore women not only contribute fewer data to data sets used to train AI but are less likely to benefit from services Biases often depend on who funds and who designs an AI technology. AI-based technologies have tended to be developed by one demographic group and gender, increasing the likelihood of certain biases in the design Bias can also arise from insufficient diversity of the people who label data or validate an algorithm Bias may also be due to the origin of the data with which AI is designed and trained. It may not be possible to collect representative data if an AI technology is initially trained with data from local populations that have a different health profile from the populations in which the AI technology is used. Bias can also be introduced during implementation of systems in real-world settings. If the diversity of the populations that may require use of an AI system, due to variations in age, disability, co-morbidities or poverty, has not been considered, an AI technology will discriminate against or work improperly for these populations.

Answer 11

Patient safety could be at risk from use of AI that may not be foreseen during regulatory review of the technology for approval. - ---Errors in AI Systems like incorrect recommendations, recommendations based on false-negative or false-positive results - ---Risk of AI errors can impact patients in large scale in short periods of time - ---Human programming mistakes (bugs, defects) can provide wrong guidance Cybersecurity - ---AI systems may be targeted for malicious attacks and hacking in order to shut down certain systems, to manipulate the data used for training the algorithm, thereby changing its performance and recommendations, or to “kidnap” data for ransom - ---An algorithm, especially one that runs independently of human oversight, could be hacked to generate revenue for certain recipients,

Answer 12

AI to augment and possibly replace the daily tasks of health-care workers and physicians could also remove the need for maintaining certain skills, such as the ability to read an X-ray dependence on AI systems could erode independent human judgement and, in the worst-case scenario, could leave providers and patients incapable of acting if an AI system fails or is compromised automate many of the jobs and tasks of health-care personnel, resulting in significant loss of jobs in nearly every part of the health workforce, including certain types of doctors if patients interact more frequently and directly with AI, result in doctors spending less time in direct contact with patients and more time in administering technology, analyzing data and learning how to use new technologies

Answer 13

A general problem is lack of transparency. Practices remain hidden partly because of commercial secrecy agreements or the lack of general obligations for transparent practices, including the role these firms play in health care and the data that are collected and used to train and validate an AI algorithm. Overall business model of the largest technology firms includes both aggressive collection and use of data to make their technologies effective and use of surplus data for commercial practices, considered by Professor Shoshana Zuboff as “surveillance capitalism” growing power that some companies may exert over the development, deployment and use of AI for health (including drug development) and the extent to which corporations exert power and influence over individuals and governments and over both AI technology and the health-care market. Monopoly power can concentrate decision-making in the hands of a few individuals and companies, which can act as gatekeepers of certain products and services (221) and reduce competition, which could eventually translate into higher prices for goods and services, less consumer protection or less innovation.

Answer 14

Extending the use of AI for health and in other sectors of the global economy could, however, contribute directly to dangerous climate change and poor health outcomes, especially of marginalized populations Researchers at the University of Massachusetts Amherst, USA, found that the emissions associated with training a single “big language” model were equal to approximately 300 000 kg of carbon dioxide or 125 round-trip flights between New York City and Beijing

Answer 15

“Design for values” is explicit transposition of moral and social values into context-dependent design requirements. It is an umbrella term for several pioneering methods, such as value-sensitive design, values in design and participatory design. Design for values presents a roadmap for stakeholders to translate human rights into context-dependent design requirements through a structured, inclusive, transparent process, such that abstract values are translated into design requirements and norms (properties that a technology should have to ensure certain values), and the norms then become a socio-technical design requirement. The process of identifying design requirements permits all stakeholders, including individuals affected by the technology, users, engineers, field experts and legal practitioners, to debate design choices and identify the advantages and shortcomings of each choice.

Answer 16

Citizen Science - Citizen science is defined by the Alan Turing Institute as the direct contribution of non-professional scientists to scientific research, for instance, by contributing data or performing tasks Open-source software - Transparency and participation can be increased by the use of open-source software for the underlying design of an AI technology or making the source code of the software publicly available. Increased diversity - Too often, efforts to increase the diversity of AI technologies involve increasing the diversity of the data on which they are based. Although this is necessary, it is not sufficient and might even amplify any biases inherent in the design. Minimizing and identifying potential biases requires greater involvement of people who are familiar with the nature of potential biases, contexts and regulations throughout software development, from its design to consultation with stakeholders, labelling of data, testing and deployment

Answer 17

Not-for-profit developers, who are not constrained by internal or external revenue targets, can adhere to ethical principles and values more readily than private developers. Not-for-profit developers may include treatment providers, hospital systems and charities.

Answer 18

1. Potential end-users and all direct and indirect stakeholders should be engaged from the early stages of AI development in structured, inclusive, transparent design and given opportunities to raise ethical issues, voice concerns and provide input for the AI application under consideration. Relevant ethical considerations should inform the design and translation of moral values into specific context-dependent design requirements. 2. Designers and other stakeholders should ensure that AI systems are designed to perform well-defined tasks with the accuracy and reliability necessary to improve the capacity of health systems and advance patient interests. Designers and other stakeholders should also be able to predict and understand potential secondary outcomes. 3. Designers should ensure that stakeholders have sufficient understanding of the task that an AI system is designed to perform, the conditions necessary to ensure that it can perform that task safely and effectively and conditions that might degrade system performance. 4. The procedures that designers use to “design for values” should be informed and updated by the consensus principles stated in this report, best practices (e.g. privacy preserving technologies and techniques), standards of ethics by design and evolving professional norms (transparency of access to codes, processes that allow verification and inclusion). 5. Continuing education and training programs should be available to designers and developers to ensure that they integrate evolving ethical considerations into design processes and choices. The establishment of formal accreditation procedures could ensure that designers and developers abide by ethical principles similar to those required of health-care workers.