Ship-Relative UAV Pose Estimation with 3D LiDAR

Accurate ship-relative navigation is essential for enabling safe autonomous landing and operations of UAVs at sea. Traditional systems rely heavily on GPS or cameras, but these are often unreliable in maritime environments. For mission critical applications, GPS is vulnerable to to jamming and spoofing, while cameras suffer in low light and harsh weather.

To address this, we developed a learning-based pose estimation pipeline that uses 3D LiDAR scans to estimate the full six-degree-of-freedom (6DoF) pose of a UAV relative to a ship, trained in simulation and validated with real-world data collected on a US Naval Academy YP689 research vessel.

Challenges in Shipboard Pose Estimation

LiDAR provides robustness to lighting and visibility compared to cameras, but ship-relative pose estimation introduces several challenges. Scans are often sparse and incomplete, especially at longer ranges. Furthermore, occlusions from ship structures can obscure key features, and when the ship is the only visible object in the scene, the network has limited contextual information to rely on for accurate alignment. These factors make classical registration algorithms like ICP¹, as well as learned alignment methods such as PointNetLK², prone to failure.

Dataset Generation and Collection

Developing any deep learning model requires large amounts of training data, which is especially difficult to obtain in real maritime settings. To address this, we built a simulation environment in Gazebo using a CAD model of the YP689 training vessel. Within this environment, a virtual Ouster OS0-32 LiDAR was mounted on a UAV model to generate thousands of scans. These synthetic scans capture the ship geometry as viewed from varied positions and orientations representative of actual flight trajectories.

In addition to simulation data, we collected real-world validation data during experiments aboard the US Naval Academy’s YP689 research vessel in the Chesapeake Bay, MD. A UAV equipped with a real Ouster OS0-32 collected LiDAR data during full-length flight trajectories around the ship. High-accuracy ground-truth poses were also obtained using inertial integration³, as well as RTK GPS for benchmarking.

Proposed Model Architecture

The core of the system is a Point Transformer-based neural network adapted for ship-relative pose estimation.

• Input: A single LiDAR scan (downsampled to 1024 points, normalized).
• Feature extraction: Learns local and global scan features using self-attention layers.
• Keypoint prediction: Network predicts ship keypoints in the LiDAR frame, using fixed CAD keypoint embeddings as decoder queries.
• Pose estimation: These predicted keypoints are matched with the ground truth ones, obtaining a closed form algorithm⁵.
• Refinement: Finally, a lightweight registration further refines the alignment, where the model output serves as the initial guess.

Training Procedure

The model was trained using 20,000 synthetic LiDAR scans. To better reflect real-world conditions, scans were augmented with Gaussian sensor noise and occlusions from flight deck obstructions. For the training, we utilized an NVIDIA A100 GPU and took about 4 hours to complete 100 epochs on our dataset.

A key aspect of this work is that the network was trained only on simulated data, meaning it never saw real LiDAR scans during training. This allowed us to directly assess the sim-to-real gap and whether a transformer-based approach could generalize well from simulation to the real-world. In particular, this ability would allow for easier scalability to new or potentially unknown ship classes.

Results and Evaluation

When evaluated on real-world LiDAR scans, the model achieved a mean rotation error of 2.6° and a mean translation error of 0.36 m directly from the network output. After applying a lightweight refinement step with GICP⁴, the accuracy improved substantially, reducing rotation errors to 0.62° and translation errors to 7 cm. Notably, more than 96% of predictions had less than 20 cm of translation error relative to the ground truth pose.

These results show that a model trained entirely in simulation can generalize effectively to real-world LiDAR scans collected from shipboard UAV operations. Compared to classical methods such as ICP and PointNetLK, our approach proved significantly more robust to sparsity and partial views, non-uniform point distribution, and doesn’t require an initial pose estimate.

Future Directions

Future directions include incorporating environmental disturbances such as sea spray and dynamic occlusions into simulation to better capture real-world operating conditions. Beyond LiDAR alone, more reliable navigation can be achieved through multi-sensor fusion, integrating vision and inertial measurements to complement LiDAR’s robustness. Finally, optimizing the model for embedded hardware such as NVIDIA Jetson platforms would enable efficient, real-time deployment directly on UAV flight computers.

This work is based on my M.S. thesis Ship-Relative UAV Pose Estimation with 3D LiDAR (2025), advised by Prof. Taeyoung Lee. The source code is available at github.com/fdcl-gwu/point-transformer.

¹ : P.J. Besl and Neil D. McKay. A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):239–256, 1992.

² : Yasuhiro Aoki, Hunter Goforth, Arun Srivatsan Rangaprasad, and Simon Lucey. Pointnetlk: Robust & efficient point cloud registration using pointnet. pages 7156–7165, 06 2019.

³ : Kenny Chen, Ryan Nemiroff, and Brett T. Lopez. Direct lidar-inertial odom- etry: Lightweight lio with continuous-time motion correction. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 3983–3989, 2023.

⁴ : Aleksandr Segal, Dirk Hähnel, and Sebastian Thrun. Generalized-icp. 06 2009.

⁵ : Jim Lawrence, Javier Bernal, and Christoph Witzgall. A purely algebraic justifi- cation of the kabsch-umeyama algorithm. Journal of Research of the National Institute of Standards and Technology, 124, October 2019.