Augmented Intelligence in Health Care (White Paper)
Recent advances in technology have motivated discussion on how augmented intelligence (AI) can be integrated into medicine. In response to the hopes of more efficient health care and the concerns of patient safety, the American Medical Association (AMA) passed its first policy recommendations on AI. The AMA’s fundamental mission is to “provide the physician perspective” across all relevant disciplines. As the largest association of medical practitioners and students in America, the AMA has a key role in the development of health care AI. The AMA firmly believes that AI ventures need to consult physicians, who have “expertise, experience, and leadership”, in order to reach the potential of AI. These policy recommendations outline the AMA’s vision in how these AI solutions can improve the health care system.
This policy highlights what the AMA identifies as its unique position in American medicine, in which they can ensure that the development of health care AI “benefits patients, physicians, and the health care community.” In doing so, the AMA promises health care AI enhances patient outcomes and physician satisfaction by acknowledging physician perspectives during the AI development, design, and implementation processes. The AMA policy statement also promotes the development of “thoughtfully designed, high-quality, clinically validated health care AI” that is easy to use, is transparent, and addresses health care disparities and biases, all while meeting current standards in reproducibility and patient privacy. Lastly, the AMA recommends AI education for all stakeholders, including patients and health care practitioners, in order to fully understand issues such as liability and governmental oversight of health care AI. The AMA lists “clinical decision support, patient monitoring and coaching, automated devices to assist in surgery or patient care, and management of health care systems” as examples of high-quality AI systems that can improve physicians’ abilities to provide care to patients.
The AMA’s Leadership in the development of Augmented Intelligence in health care: “As a leader in American medicine, our American Medical Association (AMA) is uniquely positioned to ensure that the evolution of AI in medicine benefits patients, physicians, and the health care community.”
Augmented Intelligence, defined: “…[I]n health care[,] a more appropriate term [than ‘artificial intelligence’] is ‘augmented intelligence’ (AI), reflecting the enhanced capabilities of human clinical decision making when coupled with these computation methods and systems… ‘[i]t is the critical difference between systems that enhance and scale human expertise rather than those that attempt to replicate all of human intelligence.’”
The promise of augmented Intelligence in health care: “AI in health care holds out the prospect of improving physician’s ability to establish prognosis, as well as the accuracy and speed of diagnosis, enabling population-level insights to directly inform the care of individual patients, and predicting patient response to interventions.”
The responsibility of the Augmented Intelligence healthcare enterprise: “To realize its potential to support improved patient care and health outcomes and enhance physician professional satisfaction, the health care AI enterprise should be informed and guided by the expertise, experience, and leadership of physicians and organized medicine in developing and implementing these tools.”
The growing integration of AI in previously human-centric activities, for example driving, suggests the implementation of health care AI is feasible. The International Business Machines Corporation (IBM) AI service Watson already aims to overcome health challenges, for example lack of care coordination, data security, and biased decision making. The AMA has also observed wearables for patients, tools for physician training and diagnosis, and other systems to support patient care. These are just a few examples of how AI has already begun to be applied in health care settings. In response to new advances in AI technology, the federal government has already hosted meetings, held hearings, and passed laws addressing AI. The 21st Century Cures Act, which updates the Food and Drug Administration (FDA) definition of software as a medical device, has implications on how AI will be classified in health care. The AMA Board of Trustees has noticed growing federal interest in AI and the lack of policies addressing this concept from the AMA, which puts the AMA at a “strong disadvantage in the public debate on health care AI”.
In response to the growing federal interest, such as the Networking and Information Technology Research and Development (NITRD) National Coordination Office’s Request for Information on Update to the 2016 National Artificial Intelligence Research and Development Strategic Plan (SciPol brief available), the AMA “wishes to provide the physician perspective across health care technology.” This “base-level” policy ensures that medical practitioners have a role in how AI shapes the future of health care. As AI technology develops, the AMA will update their policy to ensure that physicians’ perspectives continue to be considered.
Automated, Autonomous, Artificial, or Augmented?
One key nuance to note is the difference between automated, autonomous, artificial intelligence, and augmented intelligence. Automation refers to technologies that perform processes or procedures with some human intervention. For example, an electric mixer still requires human input but reduces much of the energy required in operating a manual whisk. On the other hand, autonomous technologies refer to systems that can perform functions independent of human intervention. One example is a vacuum cleaner that employs sensors to independently navigate and clean. Artificial intelligence broadly describes systems that can perform one or several functions intelligently, generally to provide results that are as good as those of a human.
The AMA policy uses the term “augmented intelligence” to refer to health care AI. While the AMA does not explicitly define augmented intelligence in the policy brief, the AMA states in the full policy that augmented intelligence is a more appropriate term to describe the enhanced capabilities of human clinicians when coupled with AI. This is to specify systems that augment human intelligence rather than replicate and replace human expertise. In response to common confusion in terminology, the AMA recommends “federal agencies…develop a common nomenclature…to assure that all stakeholders understand what systems and applications are being discussed.”
While the AMA acknowledges that advances in AI, which enhance physician abilities, have the potential to optimize health care, they also prioritize patient safety.
Implementation of AI in health care settings requires that practitioners are comfortable handling intelligent machines. AI allows for enhancement of human functions, but when intelligent machines act in an unexpected manner, user intervention can result in complications. Bredereke and Lankenau call this phenomenon “mode confusion”.
One example of how mode confusion has compromised patient safety is the da Vinci Surgical System, an automated system that allows surgeons to remotely control surgical instruments for more precise movements. Around 10% of problems with the da Vinci robot can be attributed to unintended operation, a subclass of mode confusion that explains robotic movements inconsistent with user movements. This lack of AI comprehension sets the foundation for the AMA policies on user-centered design and education on the potential and limitations of health care AI for stakeholders.
In addition to the promotion of high-quality health care AI, the AMA policy places an emphasis on developing technologies that are user-friendly. One of the major challenges of integrating intelligent technology into health care is difficulty of use. In a separate study, the AMA found that while electronic health records (EHRs) allowed physicians to have better access to patient files, EHRs also presented significant challenges. Among some of these challenges include inefficiencies in the electronic system, incompatibilities between various EHR products, and general poor usability. The AMA found that the use of EHRs “significantly worsened professional satisfaction” and has been “more expensive than anticipated for some practices.”
A final point in the AMA policy is exploring legal issues of AI such as malpractice and liability. Health care AI aims to enhance patient outcomes, but it is not yet able to provide perfect results in every situation. In an instance where practitioners follow the advice of health care AI resulting in erroneous care, liability and culpability of the outcome is not clearly defined. Challenges with making AI liable for actions include the intrinsic unpredictability of AI and the inability to define a legal agent or user in cases of autonomy. Existing AI liability policies do not successfully address these issues, motivating the AMA to “advocate for appropriate professional and governmental oversight” in intelligent health systems.
The framework for the policy presented by the AMA relies on a few scientific assumptions. First, the policy assumes it is possible to have health care AI that can enhance the performance of health care practitioners without replacing these human clinicians. While the da Vinci robot has been shown to enhance surgeon dexterity, future innovation may make autonomous surgical systems so efficient that human intervention may be counterproductive. Second, the policy assumes AI will benefit patients, physicians, and the health care community. While health care AI is promising, preliminary EHR studies and mode confusion have resulted in detrimental outcomes. Last, the policy assumes physician perspectives through the AMA are necessary for successful health care AI. However, the FDA had taken initiative on digital health before the AMA passed its policies on AI.
Scientific Controversies / Uncertainties
The integration of advanced technologies into society comes with many uncertainties and controversies. These debates hinder the progress of health care AI but are essential to ensure the implementation of such technologies is in the best interest of society.
Biases in AI Health Care
In order for AI to function, human-written algorithms are necessary. AI systems function through mathematical decision-making models, which may give the impression of impartiality. These algorithms are based off of generalized population data, resulting in a potential to normalize human biases. A Data & Society report states that “an ecosystem in which research can inform clinical guidelines and treatments, biases can have potentially life-threatening impacts.” The AMA policy aims to address and avoid bias, and algorithms that are impartial are necessary to achieve this mission.
One of the most prominent controversies of AI is its long-term economic effect. Automation of physician skills and responsibilities may negatively impact job outlook. Occupations such as radiology and pathology rely on medical practitioner’s informational expertise. In an era in which examining a scan for abnormalities or a sample for pathogens may be more efficiently accomplished by machines, radiologists and pathologists may no longer be a specialization. The AMA explicitly states their advocacy for AI that will augment human intelligence rather than replace human practitioners.
Confidentiality of medical information is a right under the Health Insurance Portability and Accountability Act. Digitizing the patient-provider relationship increases the risk of a privacy breach. The AMA policy promotes the preservation of patient privacy, but the potential of data exploitation is constant.
Benefit of Health Care AI
In theory, AI systems can assist medical practitioners in many ways such as providing medical information, assisting in diagnoses, and inferring health risks. These benefits are only taken advantage of if they are easy for physicians to implement. Dissatisfaction with and consequences of EHRs due to difficulties in use provide insight on how complex AI technologies may impact physician outcomes. The policy presented by the AMA focuses on developing technologies that are physician-friendly and encourages physicians and medical students to familiarize themselves with AI technology.
Endorsements & Opposition
A number of stakeholders have been involved in the discussion surrounding health care AI.
- AMA Board Member Jesse M. Ehrenfeld, M.D. M.P.H. notes that cooperation between AI and human clinicians can yield superior results than either one can achieve alone; however, challenges in the development, testing, and integration need to be thoroughly addressed.
- Edward Shortlife, MD PhD (American College of Medical Informatics, American Association for Artificial Intelligence), panel discussion, July 2007: “There is clear evidence of progress, and a community of talented researchers that would benefit from more growth in numbers and in research grant funding.”
- Mario Stefanelli, PhD (Chief of the Laboratory of Medical Informatics of the Department of Computer and System Sciences) panel discussion, July 2007: Although machines are not yet showing general intelligent [behaviors], AI is nowadays much more than a promise… [T]he major achievements of AI are going to be reached in the current days.”
- Peter Szolovits, PhD (Clinical Decision Making Group) panel discussion, July 2007: “Challenges for AIM remain vital and exciting. However, we recognize that our crisis in health care demands an ever-broader set of disciplines to create integrated solutions.”
- Spencer Nam (senior research fellow at the Clayton Christensen Institute for Disruptive Innovation), blog, January 11, 2017: “We shouldn’t expect machines to replace health-care professionals for some time, but new technologies will continue to be introduced into the sector’s evolving landscape, and we should welcome them. Practicing more precision medicine than intuitive medicine will make health care simpler, more accessible, and less expensive. By understanding patients’ diseases precisely, we can push medicine one step closer to its ultimate goal: patient-centered care of the finest quality.”
- Robert Pearl, MD (clinical professor surgery at Stanford and faculty of Stanford Graduate School of Business), article, March 13, 2018: “If technology is going to improve quality and lower costs in healthcare, some healthcare job will disappear… [D]octors and other health professionals are starting to feel the pressure, too.”
- Dave Gershgorn (Artificial Intelligence Reporter), blog, September 6, 2018: “Unfortunately, the medical datasets openly available for use by AI researchers are notoriously biased, especially in the US. It’s not a secret: Health-care data is extremely male and extremely white, and that has real-world impacts.”
The promise of health care AI is founded upon its potential to enhance health care delivery. According to the AMA Augmented Intelligence policy, examples of future integration of AI into health care include wearable AI, new tools for diagnosis and physician training, and health systems and data analytics. Increased efficiency and accessibility are achieved through patient information streamlining, assistance in accurate diagnostics and treatments, and facilitation of difficult procedures and operations, resulting in cost reductions. At the same time, it may also have undesirable impacts such as normalizing biases, creating inefficiencies due to mode confusion, and replacing health care workers.