Scaling Functional Electrical Stimulation Control for Diverse Users Through Offline Distributional Reinforcement Learning

Functional Electrical Stimulation (FES) can restore motor function; however, achieving precise multi-joint control remains challenging due to nonlinear muscle dynamics and fatigue. Reinforcement Learning (RL) offers a promising solution, but practical deployment is hindered by the need for patient-specific calibration. This study investigates offline RL approaches for controlling planar arm movements using heterogeneous datasets, aiming to enable zero-shot transfer to new users. We develop a biomechanical arm model in MuJoCo and evaluate four RL algorithms coupled with three offline techniques: conservative Q learning (SAC-CQL and QBR-CQL), Randomized Ensemble (QBR-REM), and distributional RL (IQNBR). Across all conditions, IQNBR demonstrates robust learning and superior control performance, achieving an average RMSE of 3.8±0.6 cm, even when trained on mixed-quality data. These results highlight the potential of distributional RL as a base learning method to build generic FES controllers that can operate without exhaustive calibration, with broader implications for controlling robots with human-like actuation systems.