The data flywheel is the part of a physical-AI platform that turns raw machine, sensor and telemetry data into versioned, trusted training sets, and then feeds field results back in. This canvas covers that loop up to the point a dataset is handed to training. It is canvas 1 of 9 and follows a research-area exploration framework, with the Emerging (11–12) and Applied (13) modules added.
| Boundary | Owned here | Owned elsewhere |
|---|---|---|
| Synthetic data | Ingesting it as a source, tagging provenance, mixing ratios | Generating it (canvas 2) |
| Lineage | Data and dataset lineage | Model lineage (canvas 5); joining the two (canvas 9) |
| Labels | Auto-labeling pipelines, label quality | Using labels to score models (canvas 4) |
| Field feedback | Triggers, mining and re-ingestion of field events | Deployment and monitoring of the model that raised them (canvases 5, 7) |
| Storage and compute | Data layout, formats, retention | GPU cluster scheduling (canvas 6); cost standards (canvas 8) |
Definition. Physical-AI data engineering is the design of data models, pipelines and governance that turn multimodal machine data into versioned training and evaluation sets. That data includes camera, LiDAR, radar, IMU, CAN-bus telemetry, operator actions and simulation output. The data flywheel (also called a data engine) is the closed loop on top: mine field data for failures, curate and label them, retrain, redeploy, repeat (Voxel51).
Six of the eight stages belong to this canvas; training and deployment belong to canvases 3–7. Lineage runs underneath every stage.
In scope: everything from the sensor log on the machine to a frozen, reproducible dataset version handed to training or evaluation.
Out of scope: how models are trained, scored, served or scheduled (canvases 3–7).
Key sub-domains
| Sub-domain | Core question | Typical artifacts |
|---|---|---|
| Ingestion and edge capture | What do we record on the machine, and when? | Trigger rules, ring buffers, upload policies, ROS 2 / MCAP logs |
| Data domains and canonical models | What are the shared entities (machine, site, episode, scenario, sensor rig)? | Domain map, canonical schemas, time-sync and calibration models |
| Data products | Which curated assets do teams consume, and on what contract? | Data contracts, SLAs, product catalog entries |
| Curation and search | Which slices are worth keeping and labeling? | Embedding indexes, dedup, scenario tags, coverage maps |
| Auto-labeling | How do we label at scale without humans on every frame? | Teacher models, VLM captioning, offline 4D labeling, QA sampling |
| Dataset versioning and lineage | Can we rebuild exactly what a model saw? | Dataset manifests, snapshots, lineage graph |
| Synthetic data intake | How are generated and real data mixed and tracked? | Provenance tags, mixing ratios, domain-gap metrics |
| Data governance | Who owns, may use and must delete what? | Ownership, consent, retention, residency policies |
Debate. One camp treats the flywheel as an ML problem (smart selection, auto-labeling). The other treats it as a data-platform problem (contracts, lineage, products). A practitioner needs both. The likely failure mode is a clever mining loop sitting on an ungoverned lake.
The field moved from storing data for reports, to treating data as a governed product, to running data as a closed learning loop. Physical AI now merges all three, because robot and machine data is multimodal, huge and mostly uneventful.
| Year | Milestone | Why it matters here |
|---|---|---|
| 2026 | NVIDIA Physical AI Data Factory Blueprint (GTC, March): curate (Cosmos Curator), augment (Cosmos Transfer), evaluate (Cosmos Evaluator) | A vendor reference architecture for exactly this canvas; Azure and Nebius integrate it |
| 2026 | Komatsu commissions its 1,000th ultra-class autonomous haul truck (April); the company reports more than 11.5 billion tonnes hauled by its autonomous haulage system since 2008 (company-reported) | Shows the scale of fleet data that autonomous mining already generates, and how little of it is public |
| 2026 | LeRobotDataset format paper; Lance readable on Hugging Face Hub | Robot data formats begin to converge |
| 2025 | Cosmos world foundation models and Cosmos Curator released | Synthetic data and video curation become part of the data pipeline, not a side project |
| 2024 | AIDE: VLMs and LLMs automate the data engine | Foundation models start labeling and querying data themselves |
| 2023 | Open X-Embodiment: over 1M trajectories from 22 robot types in one format (RLDS) | Proves pooled, standardized robot data helps generalist policies |
| 2021 | Data-centric AI movement (Ng); Auto4D offline 4D labeling (Uber ATG / Waabi) | Data quality becomes a research target; offline labeling with full-trajectory context |
| 2019–2021 | Data mesh (Dehghani); Tesla "data engine" and trigger classifiers; OpenLineage started | Domain-owned data products; fleet failure mining; open lineage standard |
| 2010s | Data lakes, then lakehouse table formats (Delta, Iceberg, Hudi) | Cheap object storage with transactions and time travel |
Paradigm shifts
From model-centric to data-centric. Gains come more from what goes into the dataset than from new architectures.
From collecting everything to collecting on purpose. Over 99% of logged driving is routine. Triggers and failure mining replace bulk upload (Voxel51).
From human labeling to model labeling. Large offline models and VLMs label; humans audit.
From data scarcity to compute. NVIDIA frames the robotics data gap as something compute can fill, through generation and augmentation (The Decoder). This is still contested.
The central finding of 2025–2026 is that the right data beats more data for physical AI. Several studies show uncurated or pooled robot data can hurt vision-language-action (VLA) models, which moves curation from housekeeping to research.
Dominant frameworks
Data engine / flywheel: collect, mine failures, label, retrain, redeploy. Borrowed from autonomous driving; robotics keeps the discipline but changes the implementation (Voxel51).
Data mesh with data contracts: domains own data products with machine-readable schemas, quality rules and SLAs (OvalEdge).
Multimodal lakehouse: object storage plus a table format with versions, time travel and vector indexes (Gradient Flow).
Curate → augment → evaluate: NVIDIA's three-stage data factory (The Decoder).
Key research questions
| Question | State of evidence |
|---|---|
| Which samples actually improve a policy? | Influence-function methods (CUPID 2025; QoQ, 2026) and self-supervised filtering (Scizor) beat human heuristics in benchmarks |
| Does pooling data across machine types help? | Mixed. Open X-Embodiment showed gains; a 2026 study found negative transfer from heterogeneous pools (HumanSignal) |
| How do we find the long tail on the machine, cheaply? | Trigger classifiers, uncertainty, ensemble disagreement; little public work for off-road equipment |
| How good are foundation-model labels? | VLM-assisted 3D labeling cuts manual time and can raise quality (arXiv 2605.21747); open-vocabulary LiDAR labeling still error-prone |
| What is the right real-to-synthetic mix? | Largely empirical; few principled methods |
| How do we prove what a model was trained on? | Engineering practice (versioning, lineage) ahead of research; regulation is now forcing it |
Challenges and limitations
Volume and bandwidth. Machines in mines and remote sites have limited connectivity; most data can't be uploaded.
Time sync and calibration. Multi-sensor rigs drift; bad calibration poisons labels silently.
Domain gap. Public datasets are on-road or tabletop. Off-road, dust, mud and articulated machines are barely covered.
Label ontology. Construction scenes (berms, spoil piles, ground workers) lack shared taxonomies.
Ownership. Machine data may belong to customers, dealers or site owners, which limits pooling.
Debates
Compute can replace data (NVIDIA) vs real failure data is irreplaceable (most robotics labs).
One cross-embodiment pool vs per-machine-family datasets.
Central data platform vs domain-owned mesh. Many industrial firms end up with a hybrid: a central platform with domain-owned products.
Industry, not academia, leads on flywheel infrastructure; academia leads on curation methods and on off-road datasets. The groups to track fall into five clusters.
| Cluster | Who | Contribution to this canvas |
|---|---|---|
| Platform vendors | NVIDIA (Cosmos, Physical AI Data Factory Blueprint); Microsoft Azure (open physical-AI toolchain); CoreWeave, Nebius | Reference architectures, curation and augmentation tools, GPU clouds |
| Autonomous-driving data engines | Tesla, Waymo, Wayve, Applied Intuition (on- and off-road) | Trigger-based mining, auto-labeling, failure-driven retraining at fleet scale |
| Robot foundation-model labs | Physical Intelligence, Skild AI, Google DeepMind (Open X-Embodiment, RT-X), Toyota Research Institute | Cross-embodiment datasets, data mixing, failure and recovery data |
| Data tooling | Hugging Face LeRobot, Voxel51, Encord, Scale AI (Physical AI Data Engine), lakeFS (acquired DVC), LanceDB | Formats, curation UIs, labeling services, dataset versioning |
| Academic and applied research | Stanford (Pavone, Bohg: CUPID), CMU, UC Berkeley, ETH Zurich RSL (Hutter: ETHcavation), Fraunhofer IOSB (GOOSE-Ex), Mälardalen + Volvo CE (ConstScene), KAIST (QoQ) | Curation science; the few construction and excavator datasets |
Inside heavy-equipment OEMs and autonomy developers (public evidence)
| Actor | Publicly documented activity | What it implies for the data flywheel |
|---|---|---|
| Komatsu | FrontRunner autonomous haulage: 1,000th ultra-class autonomous truck commissioned in April 2026; more than 11.5 billion tonnes hauled since 2008 (Mining Weekly; company-reported) | Largest known autonomous off-road fleet; fleet-scale data exists, but the data architecture is not published |
| Rio Tinto (with OEM partners) | By January 2018 its autonomous haul fleet had passed one billion tonnes moved (Global Mining Review; dated figure) | Mine operators, not only OEMs, accumulate long-run autonomy data |
| Volvo CE | Electric Site pilot with Skanska at the Vikan Kross quarry: eight autonomous battery-electric HX2 load carriers; connectivity and site fleet management reported as challenges (press release; SAE). A Mälardalen University and Volvo CE team also produced the ConstScene dataset (cited above) | Connectivity limits shape what can be captured and uploaded |
| Built Robotics | Aftermarket autonomy kit for excavators; raised US$64M in a 2022 Series C (company press release). Compatibility with several OEM brands is a vendor claim | Retrofit kits capture data across mixed fleets, which raises cross-machine transfer and data-ownership questions |
| Bedrock Robotics | Retrofit autonomy for excavators; reported US$600M Series C in October 2026 (TechCrunch) | Venture capital is flowing to construction autonomy, so demand for field data and curation tooling is rising (an inference) |
| NVIDIA, Applied Intuition | Physical AI Data Factory Blueprint (above); Applied Intuition research on on- and off-road autonomy | Vendor reference architectures that OEMs and startups can adopt instead of building curation stacks from scratch |
Deere and Hitachi also sell or develop machine-autonomy and telematics products, but this canvas found no public documentation of their data architecture. Treat them as unknown, not as absent.
Debate. Data as a moat vs open datasets: robotics firms hoard fleet data, yet off-road research is starved because so little is shared.
Methods group by flywheel stage. Each stage has a well-tested default and a research frontier.
| Stage | Default method | Research frontier |
|---|---|---|
| Capture | Rule-based triggers (hard braking, operator takeover, fault codes), ring buffers, ROS 2 / MCAP logging | Learned on-machine trigger classifiers; uncertainty and ensemble-disagreement triggers |
| Ingest and normalize | Streaming (Kafka, MQTT), time sync to a common clock, calibration snapshots per episode | Self-calibration checks; automatic detection of sensor drift |
| Model the domain | Canonical schemas for machine, site, episode, scenario, sensor rig; data contracts | Ontologies for construction scenes; shared semantics across machine families |
| Curate and search | Embedding search, near-duplicate removal, scenario tagging, coverage maps | Influence-function curation (CUPID, QoQ); self-supervised filtering (Scizor) |
| Label | Human-in-the-loop with model-seeded labels; offline 4D labeling using full trajectories (Auto4D) | VLM and open-vocabulary auto-labeling; Gaussian-splat pseudo-labels with confidence |
| Version and trace | Immutable dataset manifests, table snapshots (Iceberg, Lance), Git-like branches (lakeFS) | Column-level and sample-level lineage joined to model lineage |
| Mix and hand off | Fixed real/synthetic ratios, stratified sampling, frozen train/val/test splits | Learned data-mixing weights; curriculum by difficulty |
| Feed back | Field incidents filed as data requests; re-mining with the new model | Fully automated loops (AIDE-style) with human sign-off gates |
Data collection approaches
Fleet logging from customer and test machines: largest volume, least control, ownership limits.
Teleoperation and operator demonstrations: best for VLA training; expensive per hour.
Instrumented test sites (proving grounds): controlled long-tail staging, e.g. people near machines.
Simulation and generation (canvas 2): ingested here with provenance tags.
Analysis techniques
Coverage analysis against an operational design domain (ODD): which terrain, weather, task and machine states are thin?
Label-quality audits: sampled human review, inter-annotator agreement, confidence calibration.
Data-quality SLIs: freshness, completeness, sync error, calibration age.
Dataset diffing: what changed between versions, and did a model regress because of it?
The flywheel is the hub every other canvas reads from or writes to. Each interface needs a named contract.
| Connects to | What crosses the boundary | Contract to define |
|---|---|---|
| Canvas 2: simulation and synthetic data | Real scenes in (to seed simulation); synthetic samples back | Provenance tag, generator version, domain-gap score |
| Canvas 3: training and post-training | Frozen dataset versions and data-mix recipes | Dataset ID + manifest hash recorded in every training run |
| Canvas 4: evaluation and safety | Held-out test sets, scenario libraries, replay logs | Test-set isolation rules; scenario taxonomy |
| Canvas 5: MLOps and governance | Dataset lineage feeds model lineage | Shared lineage IDs (e.g. OpenLineage events) |
| Canvas 6: GPU infrastructure | Curation and labeling jobs run on the GPU cluster | Job classes and quotas for data work |
| Canvas 7: inference | On-machine models raise triggers and auto-label offline | Trigger API on the machine; teacher-model serving |
| Canvas 8: observability and cost | Data-pipeline SLIs; storage and labeling cost | Cost per curated hour; per-dataset storage budgets |
Outside fields
Robotics and controls: state, action and calibration semantics for the canonical model.
Geospatial and surveying: site maps, terrain models and design surfaces as context data.
Functional safety (ISO 13849, ISO 17757 for autonomous earth-moving and mining machines): traceability evidence that starts in the data.
Information retrieval: embedding search and vector indexes for curation.
Law and privacy: customer data ownership, faces and plates in camera data, data residency.
Human factors: operator demonstrations as training data; operator consent.
The field is converging on one view: future VLA progress depends less on model architecture and more on data engines designed together with evaluation (Wang et al., 2026 survey). For an industrial platform, that points to six bets.
Emerging trends
Agent-run data pipelines. Coding agents and VLMs write queries, mine scenarios and label; humans approve. NVIDIA's blueprint already pairs Cosmos with coding agents.
Value-based curation. Each sample is scored by its measured effect on policy performance (influence functions), not by human judgment.
Multimodal lakehouses. One versioned table holds raw blobs, embeddings, labels and model outputs (Lance, Iceberg with lakeFS).
World-model augmentation. One real log becomes many variants: weather, lighting, dust, time of day (Cosmos Transfer).
Edge-first selection. More of the selection happens on the machine (Jetson Thor class), because remote sites can't upload everything.
Regulation-grade lineage. EU rules require training-data provenance for AI safety functions (section 11–12).
Research gaps in heavy-industry physical AI
| Gap | Why it is open | Opportunity |
|---|---|---|
| Off-road and earth-moving datasets | Public sets are small: GOOSE-Ex has 5,000 labeled frames; ETHcavation has 502 images | OEMs, mine operators and large contractors hold far larger private datasets; releasing even de-identified subsets would close the gap |
| Construction-scene ontology | No shared taxonomy for berms, spoil, trenches, ground workers | Define a canonical ontology, possibly through standards bodies |
| Cross-machine transfer | Unclear whether excavator, dozer and haul-truck data help each other or cause negative transfer | Controlled studies across machine families |
| Interaction data | Bucket–soil and tool–material contact is hard to sense and label | Combine machine telemetry (pressures, loads) with vision as labels |
| Triggers under poor connectivity | Little work on selection with bandwidth budgets | Budget-aware on-machine sampling |
| Customer-data governance | Pooling depends on consent and contracts | Federated or consent-tiered data products |
Potential breakthroughs
Telemetry as free labels: CAN-bus and hydraulic signals supervise vision models with no human labeling.
A closed loop from field incident to retrained model in days, not quarters, with full lineage.
Tools and platforms, by layer
| Layer | Open source / standards | Commercial / vendor |
|---|---|---|
| Logging and formats | ROS 2, MCAP, RLDS, LeRobotDataset, HDF5, WebDataset | Foxglove |
| Streaming and ingest | Kafka, MQTT, Apache Flink | Azure IoT Operations, AWS IoT FleetWise |
| Storage and tables | Apache Iceberg, Delta Lake, Lance, Parquet | Databricks, Snowflake, LanceDB |
| Versioning | lakeFS (now including DVC), Project Nessie, Oxen | lakeFS Enterprise |
| Lineage and catalog | OpenLineage, Marquez, DataHub, Unity Catalog OSS | Collibra, Atlan |
| Curation and search | FiftyOne, Cosmos Curator, Ray Data | Voxel51 Enterprise, Encord, Cosmos Dataset Search |
| Labeling | CVAT, Label Studio, SAM 2 | Scale AI, Encord, Segments.ai |
| Pipeline orchestration | Airflow, Dagster, Kubeflow, Ray | NVIDIA Physical AI Data Factory Blueprint |
Datasets to benchmark against
Robot learning: Open X-Embodiment, DROID, LeRobot hub.
Off-road and construction: GOOSE / GOOSE-Ex, ETHcavation, ConstScene, RELLIS-3D, RUGD.
Driving (for flywheel methods): Waymo Open, nuScenes, Argoverse 2.
Journals and conferences
Robotics: CoRL, RSS, ICRA, IROS; IEEE RA-L, T-RO.
ML and data: NeurIPS (Datasets and Benchmarks track), ICML, ICLR, TMLR, DMLR workshop (data-centric ML).
Data systems: VLDB, SIGMOD, CIDR.
Field robotics and construction: Journal of Field Robotics; Automation in Construction; ISARC.
Industry: NVIDIA GTC, Data + AI Summit, Automate, CONEXPO-CON/AGG, MINExpo.
Standards
ISO 17757:2019: safety of autonomous and semi-autonomous earth-moving and mining machines.
ISO 13849 (safety-related control systems), ISO 21448 (SOTIF), ISO/IEC 5259 (data quality for analytics and ML), ISO/IEC 42001 (AI management systems).
Funding sources
Private: OEM and operator R&D budgets; venture funding for autonomy startups (for example Bedrock Robotics, above).
Public: NSF (Future of Work, Foundational Research in Robotics), DOE (mining and industrial efficiency), NIOSH (mining safety), Horizon Europe (construction robotics).
Start with the two 2026 surveys, then the curation papers. Those five works cover most of the live debate.
Recent influential works (2024–2026)
| Year | Work | One-line takeaway |
|---|---|---|
| 2026 | Wang et al., VLA datasets, benchmarks and data engines survey (TMLR) | Data infrastructure is the main bottleneck for VLA progress; treat it as research |
| 2026 | VLA evaluation-centric survey (Preprints.org), 582 papers | Benchmarks are saturated; real-world claims outrun the evidence |
| 2026 | Lee et al., Quality over Quantity (QoQ) | Influence functions find the demonstrations that help a policy |
| 2026 | Rethinking VLA scaling (arXiv 2602.09722) | Pooling heterogeneous robot data can cause negative transfer |
| 2026 | Cadene et al., LeRobotDataset format | Converging open format for multimodal robot trajectories |
| 2025 | Agia et al., CUPID | Human intuition about "good" demos is often wrong; measure influence |
| 2025 | Scizor (NVIDIA intern work) | Self-supervised filtering of suboptimal and redundant state–action pairs |
| 2024 | AIDE | VLMs and LLMs automate querying, labeling and continual learning in a data engine |
| 2024 | GOOSE-Ex; ETHcavation | Rare public excavator and construction perception datasets |
Seminal papers
| Year | Work | Contribution |
|---|---|---|
| 2023 | Open X-Embodiment Collaboration, Open X-Embodiment: Robotic Learning Datasets and RT-X Models | Pooled data across 22 robots improves generalist policies |
| 2022 | Dehghani, Data Mesh (book) | Domain ownership, data as a product, self-serve platform, federated governance |
| 2021 | Yang et al., Auto4D | Offline labeling of object trajectories from full LiDAR sequences |
| 2021 | Gebru et al., Datasheets for Datasets | Documentation standard for dataset provenance and intended use |
| 2015 | Sculley et al., Hidden Technical Debt in ML Systems | Data dependencies are the costliest debt in ML systems |
Literature reviews to keep open: the two 2026 VLA surveys above; the vla-datasets-benchmarks repository maintained by the TMLR survey authors.
Gap in the literature: almost no peer-reviewed work describes flywheel architecture for industrial machine fleets. Most of the practical knowledge sits in vendor blueprints and engineering blogs.
In heavy industry the flywheel feeds three families of product: machine autonomy, in-cab operator assistance, and jobsite digital twins. Embedded GPU modules on the machine (the Jetson Thor class) make edge capture and on-machine triggers more practical, though few public benchmarks exist for off-road equipment.
| Application | What the flywheel supplies | Signature data |
|---|---|---|
| Construction autonomy (dozers, compactors, excavators) | Long-tail scenes: people and vehicles near machines, unusual terrain | Camera, LiDAR, machine state, operator overrides |
| Mining autonomy (haul trucks, drills) | Edge cases on haul roads; dust, night, weather | Perception logs, dispatch and fleet data |
| Excavator manipulation (VLA policies) | Operator demonstrations paired with outcomes | Joystick inputs, hydraulic pressures, bucket-fill estimates, video |
| In-cab operator assistance (voice, troubleshooting) | Voice, operator questions, troubleshooting outcomes | Speech logs, fault codes, service records |
| Safety and proximity detection | Labeled people and vehicles around machines | Multi-camera and radar data |
| Predictive maintenance | Joined telemetry and repair outcomes | CAN-bus, oil and wear data, dealer service history |
| Digital twins and simulation (canvas 2) | Real scenes to rebuild sites and seed scenarios | Site scans, drone surveys, design surfaces |
Industry connections
NVIDIA: Cosmos Curator and the Physical AI Data Factory Blueprint as a reference for the curation stages.
GPU clouds: CoreWeave, Azure, Nebius (NVIDIA lists Azure and Nebius as blueprint integrators).
Dealers and customers: data-sharing agreements decide which fleet data can enter training.
Autonomy startups and retrofit vendors: a source and a consumer of cross-brand jobsite data (see section 4).
Future applications: cross-machine foundation models spanning machine families (a possibility, not an established result); customer-facing data products (site productivity, safety analytics) built from the same curated layer.
Technology integration
Edge: Jetson Thor-class compute on machines runs trigger models, compression and local buffering. Upload follows connectivity windows (site Wi-Fi, cellular, satellite).
Cloud and GPU: curation, embedding and auto-labeling are GPU-heavy batch jobs. They should run as first-class job classes on the shared cluster (canvas 6), not on ad hoc machines.
Data infrastructure needs: an object store at petabyte scale; a versioned table format; a vector index; a lineage service; a data-product catalog with contracts.
Computational trade-off: auto-labeling with large models may cost as much as the training run it feeds in some setups (an estimate, not a measured result). Track cost per curated hour (canvas 8).
Ethical and privacy considerations
People in the data. Cameras capture workers and bystanders. Blur or anonymize faces and plates before data leaves the region or reaches labelers.
Operator monitoring. Demonstration and in-cab data can look like surveillance. Separate training use from performance management, and say so in consent.
Customer ownership. Machine data often belongs to the customer under contract. Tag every episode with its usage rights, and enforce them in the data product.
Bias in coverage. Models trained mostly on one region, climate or customer type can fail elsewhere. Coverage reports belong in the dataset card.
Regulatory requirements
| Regulation | Applies | What it asks of the data layer |
|---|---|---|
| EU Machinery Regulation 2023/1230 | From 20 January 2027 | Software and AI can be safety components. Self-learning safety functions need a notified body, not self-assessment (Talkory). Protection against data corruption |
| EU AI Act | Transparency from August 2026. Since Regulation (EU) 2026/1744, machinery is outside the Act's direct high-risk scope; other high-risk duties apply from 2 Dec 2027 (stand-alone) and 2 Aug 2028 (embedded) (lakeFS) | For high-risk AI: training, validation and test data governance (provenance, representativeness, bias checks, logging). For machinery, equivalent duties are expected through Machinery Regulation delegated acts |
| Regulation (EU) 2026/1744 amending the AI Act (NI Assembly brief) | In force 27 July 2026 | Removes machinery from the AI Act's direct high-risk scope; AI requirements for machines will be added to the Machinery Regulation (Annex III) by delegated acts, giving one compliance path |
| ISO 17757:2019 | Current | Safety case for autonomous earth-moving and mining systems; traceability evidence |
| GDPR, EU Data Act, regional privacy laws | Now | Lawful basis for personal data; customer access to machine data; data residency |
Implication: dataset versioning and lineage are no longer just good practice. For EU-sold autonomous machines they become compliance evidence.
Build the flywheel in four phases, each closed by a gate. Do not automate the loop (phase 4) until lineage (phase 2) is in place, or the automation will produce datasets no one can explain.
Phases are sequenced, not dated. Set durations once phase 1 baselines exist; phase 2 is highlighted because everything after it depends on lineage.
Required resources
Teams: data platform engineers; ML engineers for curation and labeling models; a data-product owner per domain (perception, manipulation, telemetry); a governance and privacy lead.
Infrastructure: object storage, versioned tables, vector index, lineage service, GPU quota for curation and labeling (canvas 6).
Partners: NVIDIA (Cosmos Curator, blueprint), labeling vendor, a telematics provider for fleet data.
Risk assessment
| Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|
| Data-rights limits on customer fleet data | High | High | Usage-rights tags at ingest; consent tiers; start with operator-owned or proving-ground fleets |
| Auto-label errors spread into models | Medium | High | Sampled human audits; label confidence stored per sample; gate on audit pass rate |
| Calibration or time-sync drift corrupts data | Medium | High | Per-episode calibration snapshots; automated sync checks before data is admitted |
| Storage and labeling cost outruns value | High | Medium | Trigger-based capture; tiered retention; cost per curated hour reported monthly |
| Teams bypass the platform with private copies | Medium | Medium | Make the paved path faster than the workaround; lineage required to register a model |
| Vendor lock-in to one curation stack | Medium | Medium | Open formats (Iceberg or Lance, LeRobot, MCAP) and open lineage (OpenLineage) |
Success metrics
| Metric | What it shows |
|---|---|
| Field issue to retrained model (days) | Speed of the flywheel |
| Share of training runs with full dataset lineage (%) | Reproducibility and compliance readiness |
| Long-tail coverage: scenarios in ODD with enough samples (%) | Whether curation targets the gaps |
| Auto-label audit pass rate (%) | Label trust |
| Human labeling hours per curated hour | Automation level |
| Cost per curated hour ($) | Efficiency |
| Data-product adoption: teams consuming each product | Platform value |
Targets for each metric are an open question for the implementing team; set a baseline in phase 1.
Open questions for practitioners
Which fleet data can enter training under typical customer and dealer agreements?
Is one canonical model shared across construction and mining, or one per domain with a shared core?
Is the fleet telematics platform a data product the flywheel consumes, or part of the flywheel itself?
What upload budget per machine per day is realistic at remote mine sites?
Which table format is the standard: Iceberg, Lance, or both with clear roles?
Which EU-sold machines carry AI safety functions that fall under the 2027 Machinery Regulation?
Follow-up prompts (from the framework)
Elaborate on the canonical data model for a machine episode: entities, keys, time sync and calibration.
Provide detailed methods for measuring long-tail coverage against a construction ODD.
Analyze the challenges of auto-labeling bucket–soil interaction from telemetry plus vision.
Suggest an experiment to test whether excavator and dozer data transfer positively in one VLA.
Komatsu commissions its 1,000th ultra-class autonomous haul truck (Mining Weekly, April 2026)
Rio Tinto reaches autonomous truck fleet milestone (Global Mining Review, January 2018)
Volvo/Skanska Electric Site (press release); Volvo CE and Skanska demonstrate concept electric site (SAE)
Built Robotics raises US$64M (company press release)
Bedrock Robotics funding (TechCrunch, October 2026)
NVIDIA Physical AI Data Factory Blueprint (SiliconANGLE, March 2026)
GTC 2026: NVIDIA wants to swap robotics' data problem for a compute problem (The Decoder)
Data engine, Autonomous driving data curation, AV playbook for robotics (Voxel51)
Open X-Embodiment (Voxel51); LeRobotDataset format (Emergent Mind)
Why robot foundation models are starved for the right data (HumanSignal)
Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines (arXiv 2604.23001)
The Evaluation Bottleneck of VLA Models (Preprints.org)
VLM-assisted 3D auto-labeling (arXiv 2605.21747)
Data contracts in data mesh (OvalEdge)
Multimodal lakehouses and Lance (Gradient Flow); Lance README
Data provenance for robots (lakeFS)
EU Machinery Regulation 2023/1230 (Nemko); What AI in machines needs (Talkory); Regulation (EU) 2026/1744 brief (NI Assembly)
ISO 17757:2019 (Standards New Zealand)