1University of Tsukuba 2Metaverse Lab, Cluster, Inc.
ISMAR 2026IEEE International Symposium on Mixed and Augmented Reality
Code is coming soon.
Haptic feedback is widely employed to enhance immersion in Virtual Reality (VR) environments. However, designing haptic stimuli that cover diverse interaction conditions remains a significant scalability challenge. Data-driven haptic generation has emerged as a promising approach, yet existing models face an inherent trade-off between waveform expressiveness and inference responsiveness, which becomes increasingly critical as training data grow in scale and diversity.
To address this challenge, we propose HaptoFlow, a vibrotactile generative model based on Flow Matching, designed for interactive real-time haptic rendering in VR. Flow Matching learns a continuous vector field that transforms a base distribution into the target data distribution, enabling efficient representation of complex haptic data distributions and thereby facilitating both high-quality generation and computational efficiency. We train HaptoFlow conditioned on material labels and interaction parameters (stroking velocity and applied force), and integrate it into a VR system.
Technical evaluation demonstrates that HaptoFlow outperforms all baseline methods in both waveform reproduction accuracy and inference latency. Furthermore, user studies confirm that the system latency falls well within the perceptual threshold of visual-haptic delay, and statistically significant improvements in perceived haptic quality are observed for a subset of materials. These findings establish a practical foundation for scalable, data-driven haptic content creation in VR, and provide latency benchmarks that inform the design of future real-time haptic rendering systems.
At each rendering step, HaptoFlow takes a material label, interaction parameters (2D stylus velocity and applied force), and the waveform generated at the previous timestep, and outputs the vibrotactile waveform for the current timestep.
A U-Net backbone (two levels, 128 and 256 channels) estimates a conditional vector field in a latent space that transports a Gaussian noise sample toward the target waveform distribution, trained with the Conditional Flow Matching loss. Because the model learns the conditional distribution rather than a point estimate, it avoids the averaging artifacts that deterministic regression baselines exhibit on multimodal haptic data. The conditioning inputs, a material embedding, an MLP over the interaction parameters, and a sinusoidal encoding of the Flow Matching time variable, are each projected to 64 dimensions and injected into every U-Net level via FiLM.
A pre-trained, frozen EnCodec (24 kHz sampling rate, 6 kHz bandwidth) compresses haptic waveforms into a compact continuous latent representation, reducing both training cost and inference latency. Operating in this latent space lets the ODE solver run for a single step at inference.
The model is trained on the Cluster Haptic Texture Dataset, which records vibrotactile waveforms and the corresponding interaction parameters for 118 materials across 10 categories. One representative material per category was selected to span distinct tactile characteristics. Three-axis accelerometer data are reduced to a single axis with DFT321, downsampled to 2 kHz, segmented into 100 ms frames with a sliding window, and normalized to the range [−1, 1].
When the stylus contacts a virtual object, the 3D stylus velocity is projected onto the tangent plane of the contact surface to obtain the 2D velocity, and the applied force is approximated with a spring model from the penetration depth. The generated 100 ms waveform is buffered and routed to the actuator through Unity's audio output system.
H1: The proposed Flow Matching-based model achieves (a) higher waveform reproduction accuracy (GFC, RMSE) and (b) lower inference latency than the baseline methods trained with deterministic reconstruction losses. → Supported
| Model | GFC ↑ | RMSE ↓ | Latency (ms) ↓ |
|---|---|---|---|
| HaptoFlow (Ours) | 0.96 ± 0.04 | 0.22 ± 0.33 | 5.2 ± 1.0 |
| Transformer | 0.74 ± 0.17 | 0.39 ± 0.41 | 13.4 ± 3.4 |
| DSTN | 0.94 ± 0.05 | 0.30 ± 0.36 | 11.9 ± 2.8 |
| SPSI | 0.73 ± 0.15 | 0.49 ± 0.44 | 6.2 ± 1.8 |
H2: The total end-to-end system latency, including the inference time of the proposed model, falls below the human perceptual threshold for visual-haptic delay when using a VR HMD with a stylus pen. → Supported
Twenty-four participants traced a virtual object with the stylus and reported whether they perceived a delay between the pen motion and the onset (Turn-On) or cessation (Turn-Off) of haptic feedback, across 11 artificially introduced latency levels from 50 to 250 ms in 20 ms steps.
Taking the 50 % crossing point of the fitted sigmoid gives perceptual delay thresholds of 130.3 ms (Turn-On) and 107.7 ms (Turn-Off) — both far above our system's ~22 ms end-to-end latency.
H3: Haptic stimuli generated by the proposed model receive higher perceptual similarity ratings to real objects than those generated by baseline methods. → Partially supported
The same 24 participants alternately traced a real object and its virtual counterpart with the stylus and rated the similarity of the haptic sensation on a 7-point Likert scale, for 6 materials × 4 methods. The end-to-end system latency was equalized across methods so that waveform quality was the only variable under evaluation, and the real objects were kept out of view to avoid visual bias.
The gap between high signal-level accuracy and moderate perceptual ratings points to limits in training-data diversity and actuator bandwidth rather than in the generative framework itself.
Assigning a material label to each virtual object and streaming interaction parameters to the model yields contextually appropriate waveforms on the fly, with no per-material manual authoring. The approach is not tied to the stylus used here: amplitude-modulation-based actuation makes it compatible with controllers such as the Meta Quest Touch Plus.
Designers can iteratively query the model with target material labels and feel the resulting texture in real time, evaluating a product surface's tactile feel during the design process itself and reducing the need for costly physical prototyping.
@misc{eguchi2026haptoflowhighfidelityrealtimevibrotactile,
title={HaptoFlow: High-Fidelity Real-Time Vibrotactile Generation via Flow Matching for Virtual Reality},
author={Michikuni Eguchi and Yuichi Hiroi and Takefumi Hiraki},
year={2026},
eprint={2608.01974},
archivePrefix={arXiv},
primaryClass={cs.HC},
url={https://arxiv.org/abs/2608.01974},
}
This study was supported by JST ACT-X Grant Number JPMJAX25C4 and JSPS KAKENHI Grant Number JP25H00722, Japan.