Modular Reinforcement Learning for a Quadrotor UAV
Written by
Beomyeol Yu
Table of Contents
Our research focuses on how differential geometry can be utilized in the variuous topics of dynamic system theory and control system engineering, namely computational mechanics, optimization, estimation, controls, and machine learning, with primary applicaitons to aerospace engineering and robotics.
We assert that by embracing the geometric structure of dynamic systems, we are able to construct more efficient, robust, and elegant control strategies for modern, complex systems.
Falling cat problem: When a cat falls, it seems to “rotate” in mid-air to reorient itself and land on its feet, even if it starts from rest with no initial angular momentum. How does the cat achieve this without violating the conservation of angular momentum, which states that the total angular momentum of a system remains constant in the absence of external torques?
Geometric Deep Learning
Geometric deep learning refers to a field of machine learning that generalizes deep learning methods to data that is structured in non-Euclidean domains, such as graphs, manifolds, or other geometric spaces. This is to leverage the geometry of the problem to improve learning efficiency and generalization. In particular, we showed that the symmetry or the equivariant property of a dynamic system can be utilized to improve the data efficiency of deep reinforcement leanring for unmanned aerial vehicles.
Recommended
Geometric Controls
Control systems based on local coordinates of the configuration manifold often suffer from singularities, ambiguities, and complexities when representing complex manuevers. By explicitly utilizing geometric properties on a nonlinear manifold in Lyapunov stability analysis, we can construct an intrinsic form of control systems for the complex dynamic systems, while guaranteeing global stability properties.
Uncertianty Propagation and Estimation
This is to formulate an intrinsic form of probability distribution on a manifold, instead of utilizing Gaussin distribution of local coordinate as commonly adopted in engieering. We have constructed two density functions: matrix Fisher-Gaussian distribution on the product of the special orthogonal group and the Euclidean space, and non-commutative Harmonic analysis, and they are successfully applied to uncertainty propagation and Bayesian estimation.
Computational Geometric Optimal Control
This formalisim is to efficiently compute the optimal maneuvers of complex systems by taking advantages of computational geometric mechanics. We showed that formulating discrete-time optimal control problem with geometric integrators substantially improves computational efficiency. Further, we constructed accelerated optimiation on a manifold via discretizing Bregman Lagrangian mechanics.
Computational Geometric Mechanics
This is to construct geometric numerical integrators on a manifold that preserve the underlying geometric properties of a dynamic systme. Such conservation properties are critical for long-term structural stability and reliability of the computed trajectories.
Acknowledgement
Our research has been supported by AFOSR, AFRL, NASA, NAVAR, NRL, NSF, and ONR.