NSF-Funded Research Data Infrastructure · FIU
EnviStor
The NSF-funded EnviStor is a smart data infrastructure that integrates storage, computation, metadata, and visualization to streamline research data discovery, processing, and sharing. The core components are the Harvard-developed Dataverse, NexusOS Ceph-backed 2.7 PB storage, ESRI's ArcGIS, Pelican federated data sharing, and the OSPool (HTC) high-performance computational platform.
Furthermore, EnviStor operates a human-in-the-loop (HITL) multi-agent agentic-AI platform that assists in data exploration, cataloging, cleaning, metadata creation, and visualization — AI agents carry out the repeatable work while humans review and approve every consequential step, keeping the entire workflow traceable, auditable, and controllable.
About
EnviStor is an NSF-funded data repository with 2.7 PB computing infrastructure for supporting collaborative interdisciplinary research on South Florida's built and natural environments. This repository expands upon FIU's existing research data management framework built on the Harvard-developed Dataverse (see also rdm.fiu.edu) and incorporates the data visualization and processing capabilities of the FIU CED-GIS Center (Coastal Environment Data and Geographic Information System Center), such as the geo-data. Furthermore, EnviStor operates a HITL multi-agent agentic-AI platform to assist in data exploration, cataloging, cleaning, and metadata creation.
Currently, the project team is working on several main areas of climate and environmental data management, streaming, and visualization:
- Real-time South Florida saltwater intrusion buoy data (pilot in progress);
- Time-series climate data;
- Videos from underwater autonomous vehicles;
- Wall of Wind NICHE data;
- Time-series remotely sensed imagery data;
- Storm surge and flood simulation data and models.
See what's already published, with verified statistics and spatial coverage, on the Data page.
The EnviStor system connects to the NSF federated data computing system "Pelican", as well as FIU's research computing resources — the Reconfigurable Advanced Platform for Transdisciplinary Open Research (RAPTOR), the AI Pavilion, and the dPanther hosting environment — to support nationwide Open Science data federation. The FIU CED-GIS Center plays an essential role in configuring and implementing the EnviStor IT framework, as well as in AI-assisted data management, dashboard creation, metadata management, and web application development for the user interface of EnviStor.
The Platform — HITL Multi-Agent Agentic AI
EnviStor's data workflows are operated by a human-in-the-loop multi-agent agentic-AI platform. Multiple specialized AI agents perform the repeatable work — data completeness and integrity checks, format and unit harmonization, standards-aligned metadata generation, QA/QC preparation, and publication packaging — while researchers and data curators own the judgment calls: scope, correctness, and release approval.
This design keeps the process:
- Traceable — every action, decision, and data transformation is recorded with provenance.
- Auditable — humans can review what an agent did and why, at any step.
- Controllable — humans approve, gate, or override agent actions before they take effect.
- Stable — data processing and publication are repeatable and resilient, not ad hoc.
Behind the platform, data lives once in NexusOS storage, and four complementary compute/hosting resources reach it in place rather than by copying:
| Need | Resource |
|---|---|
| GPU model training, large simulations, HPC/batch jobs | Raptor (FIU institutional GPU/HPC cluster) |
| Interactive AI/ML development, notebooks, inference | AI Pavilion (FIU CS department AI environment) |
| Many small independent jobs (parameter sweeps, per-file processing) | OSPool / OSDF (high-throughput computing; compute reads data in place) |
| Hosting web applications, dashboards, databases, APIs | dPanther (web/DB hosting) |
Workflow and Contact
This Standard Operating Procedure sets forth procedures for the flow of datasets (geospatial and non-geospatial) through the FIU EnviStor infrastructure. It contains workflows for adding/publishing datasets/items through the EnviStor HITL multi-agent agentic-AI platform, which can automate the following tasks:
- Automated metadata generation and publishing datasets in Dataverse;
- Data visualization applications, e.g. ArcGIS Online;
- Advanced computational tasks, e.g. data/image processing;
- Connection to the OSDF/Pelican federation.
Publication to Dataverse, ArcGIS Online, and Pelican is part of the standard FAIR publication workflow. EnviStor offers two service tiers:
| Tier | What it means |
|---|---|
| Self-service | Work in your storage bucket, stage data for publication, review the auto-generated metadata, and approve — the platform takes it to a Dataverse draft. |
| Staff-assisted | The EnviStor team runs the workflow with you: metadata spreadsheets, curation support, and AGOL/Pelican publication handled by staff. |
In both tiers, publication to a public endpoint always includes a human approval gate.
The Existing Problem
Researchers can access world-class storage, GIS, and computing resources — but using them requires expertise most research teams do not have: metadata standards, GIS specialists, OSDF training, developer teams for OSPool and Raptor. Traditional scripting is consistent but cannot adapt to new formats; pure AI automation adapts but cannot be trusted alone. EnviStor's HITL multi-agent approach combines the adaptability of AI agents with the consistency, traceability, and human control that scientific data requires.
Workflow
Data owners complete an onboarding process covering dataset ownership and overview, metadata configuration, visualization needs, computational tasks, and data sharing and federation. A NexusOS storage bucket (default 100 TB) is provisioned. From there, data analysts and metadata specialists work with the platform's AI agents and curated toolset — with human review gates for scope approval, QA/QC, and publish approval — through to FAIR publication in Dataverse, ArcGIS Online, and Pelican. All activity is tracked in the system log for reporting and provenance.
Implemented Solution
Researchers upload data to NexusOS storage (#1). The platform's agents run standard checks, harmonization, cleaning, and metadata generation (#2), select appropriate tools and workflows (#3), and run domain-specific task pipelines on the appropriate compute tier (#4). After human QA/QC approval, outputs are published to Dataverse, Pelican, and ArcGIS Online (#5). The workflow has been validated end-to-end in production — for example, a real-time air-quality and atmospheric-chemistry archive (PurpleAir, Tempest, and NASA Pandora) is checked, cleaned, documented, and published through the full pipeline today. A companion coastal buoy pipeline has been validated on test data and is moving toward production as we finalize a data-sharing collaboration with the NSF CREST Center for Aquatic Chemistry & Environment.
Contact
As a first step, make sure that you and your team have authenticated access to upload data to EnviStor (download instructions: EnviStor rclone configuration guide), and contact Julian Garcia (juligarc@fiu.edu) and Ernesto Rivas (erivas@fiu.edu).
For batch data upload, please prepare your data information in the spreadsheet for the EnviStor Agency/Third-Party datasets or EnviStor FIU research data metadata elements datasheets. If you have any questions relating to the metadata generation within Dataverse, please contact Sonia Santana Arroyo or Rebecca Bakke at rdm@fiu.edu.
For questions relating to the agentic-AI platform workflow, please contact Boyuan Guan (bguan@fiu.edu).
For Pelican/OSPool access, please contact Dr. Leonardo Bobadilla (bobadilla@cs.fiu.edu).
Collaborators
Institute of Environment
Lead: Todd Crowl (tcrowl@fiu.edu)
FIU's Knight Foundation School of Computer and Information Sciences
Lead: Jason Liu (liux@fiu.edu), Leonardo Bobadilla (bobadilla@cs.fiu.edu)
FIU CED-GIS Center (Coastal Environment Data and Geographic Information System Center)
Lead: Jennifer Fu (fujen@fiu.edu), Boyuan Guan (bguan@fiu.edu)
Division of IT
Lead: Mike Kirgan (michaelk@fiu.edu), Julio Ibarra (julio@fiu.edu)
Sea Level Rise Solution Center
Lead: Jayantha Obersekera (jobeysek@fiu.edu)
Institute of Extreme Events
Lead: Yuepeng Li (yuepli@fiu.edu)
Tech Support Teams
HITL Agentic-AI Team
Wencong Cui (wecui@fiu.edu), Arjun Mishra (amish027@fiu.edu), Taylor Bonachea (tbonache@fiu.edu), Hong Chen (chong003@fiu.edu), Chunhan Cheng (ccheng@fiu.edu)
EnviStor IT Team
Julian Garcia (juligarc@fiu.edu), Anu Chirinos (anu@fiu.edu), Ernesto Rivas (erivas@fiu.edu)
AIP Support Team
Alfredo Bayuelo (abayuelo@fiu.edu), Joel Chambers (jcham070@fiu.edu)
Metadata Support
Sonia Santana Arroyo (sonsanta@fiu.edu), Rebecca Bakke (rbakker@fiu.edu)
Website
Julian Gottlieb (gottliej@fiu.edu)
Data & Publication Statistics
What's published, what's in progress, spatial coverage, and how every number is checked against the live Dataverse record — not copied from a slide.
See the Data pageFrequently Asked Questions
What is EnviStor? Why does metadata matter? Which computing resources are connected, and what file formats and naming conventions do we recommend?
Read the FAQs