Sim-to-Real: Why Robots That Work in Simulation Still Fail on the Floor
Why robot programs that work perfectly in simulation still fail in production, and what manufacturers and robotics teams can do to close the sim-to-real gap.
Sim-to-Real: Why Robots That Work in Simulation Still Fail on the Floor
The promise of modern robot simulation is compelling: design the task in a virtual environment, test everything before touching hardware, then deploy with confidence. In practice, most manufacturers who have attempted this have encountered the sim-to-real gap — the frustrating phenomenon where a robot program that works perfectly in simulation fails, sometimes immediately, when deployed in the real cell.
Understanding why this happens, and how to close the gap, is one of the most practically important topics in production robotics today.
What Is the Sim-to-Real Gap?
The sim-to-real gap (also written sim2real) refers to the difference between robot behaviour in simulation and robot behaviour in the real world. It manifests as:
- Trajectories that clear obstacles in simulation but cause collisions in the real cell
- Grip forces that work in simulation but are insufficient or excessive with the actual gripper
- Timing that is correct in simulation but causes downstream process errors in production
- Vision-based positioning that works with simulated camera data but fails with real-world lighting variation
The gap exists because simulation is always an approximation of reality. How close that approximation is determines how large the gap will be for any specific deployment.
Why Simulation Is Still Imperfect in 2026
Modern simulation platforms like NVIDIA Isaac Sim have made enormous strides. Physics-based rendering, accurate contact dynamics, and hardware-accurate robot models mean that simulation is far better today than it was five years ago.
But several fundamental limitations remain:
1. Model Accuracy
Every simulation rests on models: the robot URDF (Unified Robot Description Format) file, the end-effector model, the part model, the fixture model. If any of these models are slightly wrong — a 2mm error in a fixture position, a 5% error in gripper jaw stiffness — the simulation diverges from reality.
For an industrial cell assembled from many components with real-world tolerances, accumulated model errors are the norm, not the exception.
2. Contact and Friction Modelling
Contact physics is notoriously difficult to simulate accurately. The micro-level interactions between gripper jaw surface and part surface — friction, deformation, micro-slip — depend on material properties, surface finish, and local geometry that are difficult to measure and model precisely.
A gripper simulation that says "grip force 40N is sufficient" may be correct on average but wrong for 10% of the actual part-surface combinations you will encounter in production.
3. Environment Uncertainty
Real factories are not static. Light levels change. Ambient temperature changes (and affects both pneumatic systems and electronic gripper performance). Conveyors drift. Parts are not positioned exactly the same way twice. Simulation environments are typically static — they represent a snapshot of the world at a specific moment.
Production robots need to handle the variation that occurs around that snapshot.
4. Sensor Noise and Latency
Simulated sensors are clean. Real sensors have noise, quantisation error, and latency. A simulated camera provides perfect, latency-free image data. A real camera has exposure time, readout time, and communication latency. A simulated force-torque sensor reads exactly. A real one has noise floors and baseline drift.
These differences, small individually, compound when a multi-step program depends on sensor readings at each step.
Where the Sim-to-Real Gap Causes the Most Trouble
Contact-Intensive Tasks
Insertion tasks (press-fit, pin-to-hole, connector insertion) require precise force control at the moment of contact. Simulation contact models are good but not perfect — which means a simulated successful insertion can correspond to a real-world failure or damage.
Vision-Dependent Positioning
Tasks that use a camera to determine pick position are sensitive to lighting variation. A simulated scene with defined lighting never matches a real factory floor with skylights, moving shadows from workers, and aging fluorescent fixtures. Retraining or calibrating the vision system for real-world lighting is often necessary.
Pneumatic Gripper Dynamics
Pneumatic grippers have inherent variability: the actuation time varies slightly with air pressure, temperature, and seal condition. Simulation typically uses a fixed actuation time. The difference can cause grip-before-arrival errors (gripper closes before reaching the part) or release timing issues.
Strategies for Closing the Gap
1. High-Fidelity Cell Models
Start with accurate models. Measure the real cell geometry — fixture positions, conveyor height, part geometry — and build models from measurements, not from drawings or nominal values. CAD-to-reality discrepancies are common; measure, do not assume.
2. Domain Randomisation
Domain randomisation is a technique from AI robotics research that helps bridge the sim-to-real gap by training robot policies on a deliberately varied set of simulated conditions — varying lighting, part positions, surface properties — rather than a single fixed scene.
The logic: if a policy works across a wide range of simulated variations, it is more likely to handle real-world variation as well. Google RT-2 and other VLA models use domain randomisation extensively.
For industrial manufacturers, this translates to: test the robot program across a range of part positions and orientations in simulation before deploying, not just at the nominal position.
3. Simulation-Validated, Floor-Tuned Deployment
Treat simulation as the first pass, not the final word. A practical deployment workflow:
- Design and simulate the program — catch gross errors (collision, unreachable positions) in simulation
- Deploy to the real cell at low speed
- Identify the specific points where the real cell diverges from simulation
- Adjust the real program (or the simulation model) to close the specific discrepancies
- Validate at production speed
This hybrid approach uses simulation for speed and safety, then closes the remaining gap with measured floor testing.
4. In-Loop Sensing
The most robust approach to the sim-to-real gap is to reduce dependence on open-loop execution (the robot following a pre-planned trajectory without sensing) and increase use of in-loop sensing (the robot checking sensor readings at each step and adjusting in real time).
Force-torque sensors for insertion tasks, vision for part positioning, and pressure sensors for grip verification all reduce the sensitivity of the program to the residual gap between simulation and reality.
NVIDIA Isaac Sim: What It Offers and What It Does Not
NVIDIA Isaac Sim is the current industry-standard simulation platform for robotics development. It uses PhysX for physics simulation and ray-traced rendering for photorealistic visuals.
Its genuine strengths:
- Accurate physics contact simulation
- High-fidelity synthetic sensor data (cameras, LIDAR)
- Hardware-accurate robot models for most major brands
- GPU-accelerated sim-to-real training for VLA models
What it does not eliminate:
- The need to calibrate real cell geometry
- The variability of real gripper dynamics
- Real-world lighting effects on vision systems
Isaac Sim raises the ceiling on simulation fidelity significantly — but it does not close the sim-to-real gap entirely. The floor-tuning phase is still necessary.
For Manufacturers: Practical Guidance
If you are using simulation as part of your robot deployment process:
Use simulation for what it does well: catching collisions, verifying reachability, checking sequence logic. These benefits are real and significant.
Build time for floor tuning into your schedule. Regardless of simulation quality, plan for 20–30% of programming time to be spent on the real cell. This is not a failure of simulation — it is the normal workflow.
Measure your cell geometry. Do not trust the drawing. Measure the actual fixture positions, conveyor height, and part dimensions, and build those measurements into your simulation model.
Add sensing wherever force or position uncertainty is high. The more in-loop sensing you have, the smaller the sim-to-real gap matters.
The Bottom Line
Simulation is a powerful and increasingly essential tool for robot deployment. But the sim-to-real gap is real, and it is not going away entirely in the near term. The manufacturers who benefit most from simulation are the ones who use it for what it does well — error-catching and pre-deployment validation — while building in the floor-tuning phase that closes the remaining gap.
A simulation step is not the finish line. It is the safety net that makes the finish line cheaper and faster to reach.
See also: