Abstract: This paper presents a new control scheme of JAIST Active Robotic Walker (JARoW) developed to provide elderly people with sufficient ambulatory capability. Toward its practical use, our focus is placed on how to allow easier and reliable maneuverability by creating a natural user interface. Specifically, our challenge lies in providing a well-functioning controller by detecting what the user wants to do or their intentions. A Kalman filter based tracking scheme is realized to estimate and predict the locations of the user's legs and body in real time. The feedback control can then adjust the motions of JARoW corresponding to the actual user's walking behaviors. Our experiments confirm that JARoW can autonomously adjust its motion direction and velocity without requiring any additional control inputs.