I-Corps: Translation Potential of a Contextual Robotic Artificial Intelligence (AI) Agent for Field Tasks in Construction and Project-Based Industries

Award ID: 2535075

This I-Corps project is based on the development of a software system to enable intelligent and autonomous robots. Currently, industries such as construction, shipbuilding, aerospace, and energy often face inefficiencies, costly rework, and skilled labor shortages resulting in billions of dollars in annual losses and project delays. This technology introduces a mobile robotic platform equipped with laser-based augmented reality and artificial intelligence (AI)-driven task planning that may be used to transform digital blueprints into actionable on-site guidance. Workers may be able to follow millimeter-accurate projections rendered directly onto physical surfaces, reducing errors and eliminating the need for manual plan interpretation. The solution has the potential to provide substantial labor savings, faster project delivery, and improved safety compliance. Commercially, the system provides value across a range of industries by automating critical workflows and enabling less experienced workers with the ability to perform complex tasks with confidence. This I-Corps project utilizes experiential learning coupled with first-hand investigation of the industry ecosystem to assess the translation potential of a robotic AI agent designed to support precision layout, task execution, and in-situ inspection in dynamic, unstructured field environments. The system integrates four core innovations: a laser-ink spatial augmented reality (AR) module for millimeter-accurate, building information model (BIM)-aligned task and work-plan projection; a mobile robotic platform equipped with simultaneous localization and mapping (SLAM)-based autonomous navigation and adaptive projection control; a transformer-based VisionLanguageAction (VLA) model capable of grounding natural language commands, multimodal perception, and structured BIM data into context-aware action policies; and a closed-loop digital twin framework that enables real-time synchronization between physical execution and virtual design. The VLA model is trained on domain-specific multimodal datasets, enabling it to semantically interpret and execute construction-relevant tasks beyond the capabilities of general-purpose large language models. The goal is to enhance task accuracy, reduce execution errors, and embed continuous quality assurance and quality control into field operations. This technology has the potential to provide savings on labor, faster project delivery, and improved safety compliance in a wide range of industries including construction, shipbuilding, aerospace, and energy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

NSF Program Director: Ruth Shuman