A Deep Reinforcement Learning Approach to Optimal Morphologies Generation in Reconfigurable Tiling Robots

Reconfigurable robots have the potential to perform complex tasks by adapting their morphology to different environments. However, designing optimal morphologies for these robots is challenging due to the large design space and the complex interactions between the robot and the environment. An in-house robot named Smorphi, having four holonomic mobile units connected with three hinge joints, is designed to maximize area coverage with its shape-changing features using transformation design principles (TDP). The reinforcement learning (RL) approach is used to identify the optimal morphologies out of a vast combination of hinge angles for a given task by maximizing a reward signal that reflects the robot’s performance. The proposed approach involves three steps: (i) Modeling the Smorphi design space with a Markov decision process (MDP) for sequential decision-making; (ii) a footprint-based complete coverage path planner to compute coverage and path length metrics for various Smorphi morphologies; and (iii) pptimizing policies through proximal policy optimization (PPO) and asynchronous advantage actor–critic (A3C) reinforcement learning techniques, resulting in the generation of energy