Agent-Based Convolution and Reinforcement Learning

The problem with the current models like Darwin-OP or Boston Dynamic's ATLAS is their up-time, especially with increased number of joints. These models try mimicking the human motion; they end up using a lot of actuators, which in turn leads to the use of a lot of battery power. This paper discusses the creation of a new model of humanoid robots, that does not try to mimic the bipedal walking gait used by humans, but who instead uses a full model constructed from scratch, that consists of a model free Deep Q-Learning (DQN) algorithm, which doesn't need any walking sequence or walking models, it just learns from trial and error by applying actions on the robot and observing the reward from that action to make an under-actuated robot able to balance and walk forward, backwards, sideways, and rotate in place using only 4 actuators (two in each leg). The proposed model uses a Regional Convolutional Neural Network (R-CNN) to detect and inform the robot about the place of its goal. A full sensory system of a camera and Inertial Measurement Unit (IMU) is utilized to extract and gather the required inputs for reaching the goal from the robot' environment. Thus instead of thinking that robot as a pre-programmed entity who performs specific task, we treat this as agent who can learn to take whatever actions towards specific goal controlled by evaluation function to maximize specific reward.