The Data Hypervisor: Orchestrating Data and Models

Award ID: 2040718

The NSF Convergence Accelerator supports use-inspired, team-based, multidisciplinary efforts that address challenges of national importance and will produce deliverables of value to society in the near future. This project, NSF Convergence AcceleratorTrack D: The Data Hypervisor: Orchestrating Data and Models, will design and implement the Data Stationa new architecture where both data and derived data products are sealed and cannot be directly seen or downloaded by anyone. In the Data Station architecture, computation is brought to the data, rather than data being brought to users, as is common in traditional data lakes and warehouses. Sharing data and models has had a transformative impact on scientific problems from medical imaging to natural language understanding. Despite the potential upside, many researchers in both academia and industry are reluctant to centralize and share data to both internal and external researchers. Organizations today have to navigate complex regulatory considerations and protect intellectual property while incurring a significant technical investment in documenting and maintaining data. The Data Station will ease access to sensitive data, assist with data discovery and integration, and facilitate enforcement of arbitrary data access and governance policies. The project will work with partners in biomedicine, materials science, and enterprise data management to establish the capabilities and prove the concepts of the Data Station architecture. While building upon prior research in data systems, Data Station will introduce novel data-unaware task capsules that enable users to specify data-driven tasks such as traditional data queries and machine learning model training without the user requiring direct access to the data itself. The programming interfaces convey sufficient information for the Data Station to trigger the discovery of potentially relevant datasets; integrate and prune those datasets for computation; and compute the results by executing the task. In effect, Data Station inverts the traditional data querying modeling by bringing computations to the data. Task capsules also include a user-defined metric for determining what results are useful from the users perspective as well as which trust constraints need to be met to validate the provenance of input datasets. The Data Station captures metadata every time a derived data product is created and provides a set of primitives to implement various data governance and data access policies necessary to address data contributor use cases. Only authorized users are able to access the data based on a novel access-token model implemented by Data Station that permits fine-grained yet scalable access control. Users must explicitly be authorized to access results via tokens obtained from data contributors. The Data Station project will engage a diverse set of partners in materials science, biomedicine, and enterprise scenarios to help design and apply the Data Station to various use cases. An education program will engage high school, undergraduate, and graduate students in researching, developing, and evaluating the Data Station. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

NSF Program Director: Mike Pozmantier