I-Corps: Auditing the Decisions of Artificial Intelligence

Award ID: 2305237

The broader impact/commercial potential of this I-Corps project is the development of a suite of tools that can audit decisions made by neural network/artificial intelligence (AI) systems as an audit engine. AI-based decision systems can be cheaper and faster than human-based systems and have led to explosive growth by companies using products like Facebook ads, the Netflix movie recommendation engine, or Google translate. These AI systems are not transparent and lead to an overall poor understanding of how and why they work, as well as an inability to properly diagnose and correct errors. This opacity prevents the benefits of AI from being realized in high impact or safety-conscious systems such as medical diagnostics, self-driving transportation, or quantitative finance. The proposed technology uses AI fundamentals in order to develop tools to allow AI to be safely deploying throughout critical sectors of society. This I-Corps project is based on the development of neural network analysis techniques as an audit engine to better understand how and why a neural network makes a particular decision. The audit engine allows the technology to attribute a decision among various architectural and training components, providing increased understanding of the system and the ability to pinpoint necessary fixes. Existing AI methods to explain decisions use a heuristic approach to understand such information as feature influence, but cannot complete a comprehensive audit of an artificial system. The audit engine also provides internal metrics that can then be used as the starting points for a variety of secondary analyses. In addition to commercial applications, the audit engine analysis will benefit neural network researchers, who can more closely analyze their neural networks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

NSF Program Director: Ruth Shuman