Policy Challenges Surrounding AI Testing, Summaries of Artificial Intelligence

Download Policy Challenges Surrounding AI Testing and more Summaries Artificial Intelligence in PDF only on Docsity!

“SHOULD YOU RELY ON THAT AI?”

A New Look at Policy, Standards, and Requirements

Specification

28 January 2021

Panelists

Lt Gen (ret) Ed

Cardon

Former Director of the United States Army Office of Business Transformation and former Commander of the Second United States Army/United States Army Cyber Command; Professor of the Practice, Applied Research Laboratory for Intelligence and Security, University of Maryland

Dr. Chad

Bieber

Director, Test and Evaluation, Project Maven, Johns Hopkins University Applied Physics Laboratory

Dr. Jane Pinelis

Chief, Test and Evaluation of AI/ML, Joint Artificial Intelligence Center, Former T&E lead for Project Maven

Prof. Michael

Horowitz

Richard Perry Professor of Political Science, Director, Perry World House, University of Pennsylvania

Prof. Ben

Shneiderman

Distinguished Professor of Computer Science and University of Maryland Institute for Advanced Computer Studies (UMIACS); Founding Director, Human Computer Interaction Lab; Affiliate, Institute for Systems Research and College of Information Studies, University of Maryland

Moderator : Dr. Craig Lawrence, Director, Systems Research, Applied Research Laboratory for Intelligence and Security;

Visiting Research Scientist, Institute for Systems Research, Clark School of Engineering, University of Maryland

Panelists

Dr. Chad

Bieber

Director, Test and Evaluation, Project Maven, Johns Hopkins University Applied Physics Laboratory

Panelists

Dr. Jane Pinelis

Chief, Test and Evaluation of AI/ML, Joint Artificial Intelligence Center

Policy Challenges Surrounding AI Testing Michael C. Horowitz University of Pennsylvania January 28, 2021

Bottom Line Up Front

• Advances in AI have national security applications across a range of arenas, from the back office to the battlefield – need a way to test and validate AI- enabled systems
• Questions exist both about how to effectively test AI systems and the standards for those tests compared to non-AI systems
• Critical challenge: navigating between the risk of a trust gap and the risk of automation bias in policymaker perspectives on AI

Key Dilemma: Designing AI testing policymakers can understand

What is Getting Tested?

• Systems with continual

learning

AND/OR

• Systems without

continual learning

Trust Gap

• Inability to trust machines to

do work of people

• Unwillingness to deploy or

properly use systems

• Example: Ground Tactical Air

Controllers

Automation Bias

• Delegation of cognitive

judgment to machine –

trusting too much

• Failure to question algorithms

if they make mistakes

• Example: Air France Crash

• Example: Patriot Missile

fratricide

Trust, Confidence, and AI (1)

V

Trust, Confidence, and AI (2) Perceived Effectiveness of System Trust Gap Automation Bias Time Since System Introduction Actual Effectiveness of System Low Low High High

• (^1 0 1 2 3 4 ) Tech Hype

Conclusion

• Effective testing and evaluation standards are critical to AI adoption

in national security, and preventing AI backsliding

• Testing standards should depend on the type of AI application, and

the degree of confidence in the AI method

• Need to navigate between the risk of trust gaps and automation bias

through testing

Panelists

Prof. Ben

Shneiderman

Distinguished Professor of Computer Science and University of Maryland Institute for Advanced Computer Studies (UMIACS); Founding Director, Human Computer Interaction Lab; Affiliate, Institute for Systems Research and College of Information Studies, University of Maryland

Interdisciplinary research community

• Computer Science & Info Studies
• Psych, Socio, Educ, Jour & MITH hcil.umd.edu vimeo.com/

Designing the User Interface

Design Theories Direct manipulation Menus, speech, search Social Media Information Visualization www.cs.umd.edu/hcil/DTUI6 (^) Sixth Edition: 2016