Autonomous vehicle behavior in emergency and rare situations is a well documented challenge, with some of the most concerning instances being ambulance obstruction. Yet the sensor data needed to study these long-tail events at scale is exactly what their rarity makes scarce. We propose RareOcc (ROCC), a system that generates realistic 4D occupancy and LiDAR for customizable long-tail situations and documented vehicular crashes; to our knowledge RareOcc is the first pipeline to jointly produce supports: crash-grounded 4D generation, editable BEV authoring, occupancy generation, and LiDAR generation. ROCC uses a bird's-eye-view (BEV) layout as a shared intermediary that three interchangeable methods write to (CIREN-based crash recreation, LLM-based behavior specification, and CompoSIA-based real-scene editing) and lifts it to occupancy with an entity-grounded generator that places each agent from its specified box and replaces the diffusion stage of prior occupancy-centric pipelines. Evaluated on nuPlan-Occ mini-val frames, our model outperforms the state-of-the-art UniScene-v2 generator by 19.4 points in occupancy mIoU. ROCC also preserves rare categories under scene edits the baseline drops.
After submission we found that the released UniScene-v2 code sets the noise-prior weight to λ=0.05, whereas the paper reports λ=0.15. Re-evaluating at λ=0.05 is marginal and does not change the conclusions: ROCC still leads by a wide margin on every metric. The interactive gallery below compares ROCC, UniScene-v2, and ground-truth occupancy scene-by-scene (BEV conditioning on the left). See also the λ=0.15 paper setting.
| UniScene-v2 | UniScene-v2 | Dense | + bg | ROCC | ||
|---|---|---|---|---|---|---|
| Set | Metric | (λ=0.05) | (λ=0) | CNN | head‡ | (ours) |
| mini-val | fg-mIoU ↑ | 20.19 | – | 33.55 | 45.62 | 49.25 |
| mIoUUS ↑ | 29.53 | – | 38.09 | 49.01 | 51.65 | |
| occ-IoU ↑ | 22.24 | – | 24.74 | 24.19 | 27.25 | |
| key subset | fg-mIoU ↑ | 20.06 | 0.03 | 32.91 | 44.90 | 48.45 |
| mIoUUS ↑ | 29.37 | 13.12 | 37.51 | 48.35 | 50.92 | |
| occ-IoU ↑ | 21.65 | 7.31 | 23.90 | 23.40 | 26.17 |
Occupancy generation on nuPlan-Occ validation: mini-val (full set) and key subset (one key frame per clip). UniScene-v2's λ mixes in a fraction of the ground-truth occupancy latent; at λ=0 it collapses. ‡ trained at a larger effective batch than the other columns.
Figure 1. Crash-grounded controllable generation. Three authoring front-ends — editable custom scenes, crash reconstructions, and curated text — produce an ego-centric bird's-eye-view layout, which an occupancy DiT turns into 3D semantic occupancy and, in turn, LiDAR.
Natural-language prompts drive a traffic simulator that emits the BEV layout — full control over agents, maneuvers, and road type.
Reconstructs real crash reports into vehicle trajectories, recovering long-tail collision geometry from structured accident descriptions.
Edits real recorded scenes (cut-ins, lane changes) on the true map, warping agents while keeping the surrounding world consistent.
Figure 2. A grounded latent VAE (top) encodes ground-truth occupancy into grounded latents decoded via semantic Gaussians. At generation time (bottom) a CNN maps a BEV layout to those latents, which the frozen decoder turns into occupancy, and an occupancy-to-LiDAR model into a point cloud.
The interactive demo shows 22 curated scenarios across the three front-ends, each as input → 2D BEV motion → 4D occupancy → 4D LiDAR (occupancy is a 200×200×16 voxel grid, per-frame dynamics, each frame decoded independently; LiDAR is rendered as an interactive point cloud). Custom scenes add a World-frame / Ego-centric view toggle.