Graphical Abstract Figure
Graphical Abstract Figure
Close modal

Abstract

This paper presents a formulation and methodologies for a vision-aided inertial navigation system designed for autonomous landing or proximity operation. Measurements from a monocular camera are integrated with inertial measurements to enable sensor fusion, leveraging the strengths of both visual and inertial sensing modalities. A multiplicative extended Kalman filter (MEKF) is used to estimate the relative pose to a target space as well as bias variances. The estimator maintains a multiplicative approach to parameterizing the error quaternion and the attitude kinematics throughout filtering. We adapt the histogram-of-oriented gradients (HOGs) algorithm and apply a series of template images for feature extraction in our vision algorithm, thereby improving its rotation variance property. A measurement model is formulated through an auxiliary optimizing process of geometric data to estimate feature locations with uncertainty. The utility of the proposed navigation system and its framework is demonstrated by experimental analysis using the sensor module and the designated helipad target.

References

1.
Arribas
,
J.
,
Moragrega
,
A.
,
Fernández-Prades
,
C.
, and
Closas
,
P.
,
2017
, “
Low-Cost GNSS/INS/Odometric Sensor Fusion Platform for Ground Intelligent Transportation Systems
,”
Proceedings of the 30th International Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS+ 2017)
,
Portland, OR
,
Sept. 25–29
, pp.
436
455
.
2.
Yoo
,
C.-S. Y. C.-S.
, and
Ahn
,
I.-K. A. I.-K.
,
2003
, “
Low Cost GPS/INS Sensor Fusion System for UAV Navigation
,”
The 22nd Digital Avionics Systems Conference, 2003 (DASC’03)
,
Indianapolis, IN,
,
Oct. 12–16
,
IEEE
, Vol. 2, p.
8-A
.
3.
Sung
,
K.
,
Peck
,
C.
,
Majji
,
M.
, and
Junkins
,
J. L.
,
2022
, “
An Optical Navigation System for Autonomous Aerospace Systems
,”
IEEE Sens. J.
,
22
(
17
), pp.
16862
16873
.
4.
Patruno
,
C.
,
Nitti
,
M.
,
Petitti
,
A.
,
Stella
,
E.
, and
D’Orazio
,
T.
,
2019
, “
A Vision-Based Approach for Unmanned Aerial Vehicle Landing
,”
J. Intell. Rob. Syst.
,
95
(
2
), pp.
645
664
.
5.
Dalal
,
N.
, and
Triggs
,
B.
,
2005
, “
Histograms of Oriented Gradients for Human Detection
,”
2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 1
,
San Diego, CA
,
June 20–25
,
IEEE
, pp.
886
893
.
6.
Liu
,
K.
,
Skibbe
,
H.
,
Schmidt
,
T.
,
Blein
,
T.
,
Palme
,
K.
,
Brox
,
T.
, and
Ronneberger
,
O.
,
2014
, “
Rotation-Invariant HOG Descriptors Using Fourier Analysis in Polar and Spherical Coordinates
,”
Int. J. Comput. Vis.
,
106
(
3
), pp.
342
364
.
7.
Kittipanya-ngam
,
P.
, and
Lung
,
E. H.
,
2011
, “
Hog-Based Descriptors on Rotation Invariant Human Detection
,” Computer Vision—ACCV 2010 Workshops: ACCV 2010 International Workshops, Queenstown, New Zealand, Nov. 8–9, 2010, Revised Selected Papers, Part I 10,
Springer
, pp.
143
152
.
8.
Delmerico
,
J.
, and
Scaramuzza
,
D.
,
2018
, “
A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots
,”
2018 IEEE International Conference on Robotics and Automation (ICRA)
,
Brisbane, QLD, Australia
,
May 21–25
,
IEEE
, pp.
2502
2509
.
9.
Duc
,
T. M.
, and
Kang
,
H.-J.
,
2013
, “
Fusion of Vision and Inertial Sensors for Position-Based Visual Servoing of a Robot Manipulator
,”
International Conference on Intelligent Computing
,
Nanning, China
,
July 28–31
,
Springer
, pp.
536
545
.
10.
Mourikis
,
A. I.
, and
Roumeliotis
,
S. I.
,
2007
, “
A Multi-state Constraint Kalman Filter for Vision-Aided Inertial Navigation
,”
Proceedings 2007 IEEE International Conference on Robotics and Automation
,
Rome, Italy
,
Apr. 10–14
,
IEEE
, pp.
3565
3572
.
11.
Verras
,
A.
,
Eapen
,
R. T.
,
Simon
,
A. B.
,
Majji
,
M.
,
Bhaskara
,
R. R.
,
Restrepo
,
C. I.
, and
Lovelace
,
R.
,
2021
, “
Vision and Inertial Sensor Fusion for Terrain Relative Navigation
,”
AIAA Scitech 2021 Forum
,
Virtual Event
,
Jan. 11–15
, p.
0646
.
12.
Qin
,
T.
,
Li
,
P.
, and
Shen
,
S.
,
2018
, “
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
,”
IEEE Trans. Rob.
,
34
(
4
), pp.
1004
1020
.
13.
Markley
,
F. L.
,
2004
, “
Attitude Estimation Or Quaternion Estimation?
,”
J. Astronaut. Sci.
,
52
(
1–2
), pp.
221
238
.
14.
Zanetti
,
R.
,
2009
, “
A Multiplicative Residual Approach to Attitude Kalman Filtering With Unit-Vector Measurements
,”
J. Astronaut. Sci.
,
57
(
4
), pp.
793
801
.
15.
Bhaskara
,
R. R.
,
Sung
,
K.
, and
Majji
,
M.
,
2022
, “
An FPGA Framework for Interferometric Vision-Based Navigation (iVisNav)
,”
2022 IEEE/AIAA 41st Digital Avionics Systems Conference (DASC)
,
Portsmouth, VA
,
Sept. 18–22
, pp.
1
7
.
16.
Tong
,
K. W.
,
Wu
,
J.
, and
Hou
,
Y.-H.
,
2023
, “
Robust Drogue Positioning System Based on Detection and Tracking for Autonomous Aerial Refueling of UAVS
,”
IEEE Trans. Autom. Sci. Eng
.
17.
Sola
,
J.
,
2012
, “Quaternion Kinematics for the Error-State KF,” Laboratoire dAnalyse et dArchitecture des Systemes-Centre national de la recherche scientifique (LAAS-CNRS), Toulouse, Tech. Rep.
18.
Crassidis
,
J. L.
, and
Junkins
,
J. L.
,
2004
,
Optimal Estimation of Dynamic Systems
, Vol. 2,
Chapman & Hall/CRC
,
Boca Raton, FL
.
19.
Farrenkopf
,
R.
,
1978
, “
Analytic Steady-State Accuracy Solutions for Two Common Spacecraft Attitude Estimators
,”
J. Guidance Control
,
1
(
4
), pp.
282
284
.
20.
Euston
,
M.
,
Coote
,
P.
,
Mahony
,
R.
,
Kim
,
J.
, and
Hamel
,
T.
,
2008
, “
A Complementary Filter for Attitude Estimation of a Fixed-Wing UAV
,”
2008 IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Nice, France
,
Sept. 22–26
,
IEEE
, pp.
340
345
.
21.
Yi
,
C.
,
Ma
,
J.
,
Guo
,
H.
,
Han
,
J.
,
Gao
,
H.
,
Jiang
,
F.
, and
Yang
,
C.
,
2018
, “
Estimating Three-Dimensional Body Orientation Based on an Improved Complementary Filter for Human Motion Tracking
,”
Sensors
,
18
(
11
), p.
3765
.
22.
Zhang
,
Z.
,
2000
, “
A Flexible New Technique for Camera Calibration
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
22
(
11
), pp.
1330
1334
.
23.
Poling
,
B.
,
2015
, “
A Tutorial on Camera Models
,”
University of Minnesota
, pp.
1
10
.
24.
Shi
,
J.
,
1994
, “
Good Features to Track
,”
1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
,
Seattle, WA
,
June 21–23
,
IEEE
, pp.
593
600
.
You do not currently have access to this content.