Abstract & Details
Description
Award ID: 1853014
The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project will be to enable more specific and granular search through a large array of different types of video content, unlocking the information within the world's video archives and live stream video content. Countless hours and productivity in corporations are lost when employees search for a small, specific video clip within a larger video asset to meet their business needs, costing institutions valuable time and money. Through the enhanced discoverability within videos that the results of this project will provide, many industries such as education, media, online gaming, and others will benefit from enhanced efficiencies in publishing and video search results better reflecting user intent. The benefits of this project are not confined to enterprises. Society at large can benefit from a greater accessibility of video within archives that were previously unavailable, which can be used to create a society better-informed on world events and develop education and skills as the world's video archives that were once undiscoverable by the general public are made accessible. This Small Business Innovation Research (SBIR) Phase II project develops a video understanding framework and knowledge graph to better enable video search and discovery for enterprise video archives and live stream video. Leveraging multimodal data and domain specific structured ontologies provided by content creators and data maintainers, this project proposes to develop four new machine learning and computer vision technologies. Firstly, a webly supervised content-based retrieval system for video will be created in order to build classifiers without being provided annotated training data, effectively solving the video classification data cold start problem. Second, the project will build a multimodal ontology mapping system to enable mapping semantic concepts external and novel to our current technologies. Furthermore, to provide the ability to understand and formalize object relationships temporally, an Allen interval algebra module for video will be developed; combining it with knowledge relational learning in order to learn temporal relationships between objects that comprise its knowledge graph. Finally, to qualify the action relationships between people/objects in the developed knowledge graph, a holistic video modeling approach will be developed and applied to action/event recognition and other tasks, such as captioning or titling, expanding a key recognition capability to complete the knowledge graph. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
NSF Program Director: Peter Atherton
The broader impact/commercial potential of this Small Business Innovation Research (SBIR) Phase II project will be to enable more specific and granular search through a large array of different types of video content, unlocking the information within the world's video archives and live stream video content. Countless hours and productivity in corporations are lost when employees search for a small, specific video clip within a larger video asset to meet their business needs, costing institutions valuable time and money. Through the enhanced discoverability within videos that the results of this project will provide, many industries such as education, media, online gaming, and others will benefit from enhanced efficiencies in publishing and video search results better reflecting user intent. The benefits of this project are not confined to enterprises. Society at large can benefit from a greater accessibility of video within archives that were previously unavailable, which can be used to create a society better-informed on world events and develop education and skills as the world's video archives that were once undiscoverable by the general public are made accessible. This Small Business Innovation Research (SBIR) Phase II project develops a video understanding framework and knowledge graph to better enable video search and discovery for enterprise video archives and live stream video. Leveraging multimodal data and domain specific structured ontologies provided by content creators and data maintainers, this project proposes to develop four new machine learning and computer vision technologies. Firstly, a webly supervised content-based retrieval system for video will be created in order to build classifiers without being provided annotated training data, effectively solving the video classification data cold start problem. Second, the project will build a multimodal ontology mapping system to enable mapping semantic concepts external and novel to our current technologies. Furthermore, to provide the ability to understand and formalize object relationships temporally, an Allen interval algebra module for video will be developed; combining it with knowledge relational learning in order to learn temporal relationships between objects that comprise its knowledge graph. Finally, to qualify the action relationships between people/objects in the developed knowledge graph, a holistic video modeling approach will be developed and applied to action/event recognition and other tasks, such as captioning or titling, expanding a key recognition capability to complete the knowledge graph. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
NSF Program Director: Peter Atherton
| Status | Closed |
|---|---|
| Effective start/end date | 06/01/19 → 05/31/23 |
Funding
- SBIR Phase II: $749,926.00
Active Fiscal Year
- FY2023
- FY2022
Start Fiscal Year
- FY2019
TIP Programs
- SBIR Phase II
Small Business
- Yes
Key Technology Areas
- Super Intelligence
- (confidence score: 100%)
Technology Foci
- Machine Learning Training Data
- (confidence score: 99%)
- Machine Learning (ML)
- (confidence score: 98%)
- Super Intelligence (excluding ML)
- (confidence score: 100%)
Congressional District at Award
- District n. 12 of New York
Current Congressional District
- District n. 12 of New York
United States
- New York
Core Based Statistical Area (CBSA)
- New York-Newark-Jersey City, NY-NJ
County
- County: New York, NY
Fingerprint
Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint. Learn more about Elsevier's Fingerprint Engine here: https://beta.elsevier.com/products/elsevier-fingerprint-engine