Long Short-Term Memory Network for INS Positioning During GNSS Outages: A Preliminary Study on Simple Trajectories

Abstract

This paper presents a novel Long Short-Term Memory (LSTM) network architecture for the integration of an Inertial Measurement Unit (IMU) and Global Navigation Satellite Systems (GNSS). The proposed algorithm consists of two independent LSTM networks and the LSTM networks are trained to predict attitudes and velocities from the sequence of IMU measurements and mechanization solutions. In this paper, three GNSS receivers are used to provide Real Time Kinematic (RTK) GNSS attitude and position information of a vehicle, and the information is used as a target output while training the network. The performance of the proposed method was evaluated with both experimental and simulation data using a lowcost IMU and three RTK-GNSS receivers. The test results showed that the proposed LSTM network could improve positioning accuracy by more than 90% compared to the position solutions obtained using a conventional Kalman filter based IMU/GNSS integration for more than 30 seconds of GNSS outages.

1. INTRODUCTION

An Inertial Measurement Unit (IMU) provides high-rate acceleration and angular velocity measurements that can be used to estimate vehicle’s navigation solution. However, when used alone, the positioning accuracy will be degraded over time due to accumulated random errors and biases in IMU measurements. To reduce the effects of those IMU errors, IMU is integrated with other sensors through such as Kalman Filter (KF), wavelet-based algorithms, and machine learning algorithms, to name a few (Han et al. 2020).

For navigation purposes, an integrated navigation system (INS) with an IMU and Global Navigation Satellite Systems (GNSS) have been widely used to provide a continuous high-bandwidth navigation solution via a KF. Among various architectures of INS, a loosely coupled KF is one of the popular algorithms (Meiling et al. 2017, Wang et al. 2020). In the loosely coupled KF, GNSS correct IMU errors while IMU measurements fill the gaps in between the slower GNSS position updates. Therefore, the positioning accuracy of INS is mainly determined by the quality of GNSS signals.

Unlike IMU, however, GNSS signals are susceptible to surrounding environment such that its quality becomes unreliable in urban canyon, tunnels, and other areas where GNSS signals are interfered (Cui & Ge 2003). When GNSS are not available, proper corrections to IMU errors cannot be made in the conventional KF based IMU/GNSS integration, which can lead to the divergence of navigation solutions. Recently, with the advancement of Artificial Intelligence (AI) and related technologies, various AI-based algorithms have tackled this problem, which differs in the aspects of overall integration architectures and network structures with their own inputs and outputs (Bitar et al. 2020). Overall, the model of the AI-based IMU/GNSS integration can be classified as the ones with KF or the others without KF.

The AI models with KF attempted to improve the performance of KF state estimations and increase positioning accuracy during GNSS outages. Combined with KF, outputs of the AI model are either used to correct KF error state estimates (Chen et al. 2015, Belhajem et al. 2018), and (Wei et al. 2021) or used to generate GNSS pseudo-measurements (Yao et al. 2017, Zhang 2019, Fang et al. 2020). These approaches used various machine learning models including multi-layer perceptron, artificial neural network (Yao et al. 2017), auto regressive models (Chen et al. 2015), Neural Network (Belhajem et al. 2018, Wei et al. 2021), Long Short-Term Memory (LSTM) (Zhang 2019, Fang et al. 2020) with different input data.

The AI models without KF avoided the mathematical modeling process and tried to capture the intricate IMU error characteristics in a network. Reference (Jaradat & Abdel-Hafez 2017) proposed an additive nonlinear autoregressive exogenous (ANARX) architecture to fuse GPS and IMU measurements. Reference (Adusumilli et al. 2013) used a random forest regression approach to overcome the performance degradation of artificial neural network (ANN) particularly when a non-linear complexity is improperly treated. Reference (Noureldin et al. 2011) developed an IMU-GNSS integration filter with an adaptive neuro-fuzzy inference system that combines the fuzzy logic with a neural network. Reference (Saadeddin et al. 2014) used input delayed networks to model INS position and velocity errors with current and past INS mechanization outputs. Some researchers used extreme learning machine (ELM) with additional wavelet denoising filter (Abdolkarimi et al. 2018, Wang et al. 2018, Li et al. 2020) to better model IMU mechanization errors.

The previously mentioned prior arts appear to neglect the significant correlation between a vehicle's motion over time, and many have overlooked the importance of integrating sequential IMU and GNSS measurements into AI models to connect past and present navigation solutions. Therefore, this paper aims to derive navigation solution based on time-series sensor data from multiple previous epochs as input, rather than from a single-epoch data, as is done with neural networks. With this point, the paper further explores the IMU-GNSS integration problem using LSTM network that trained from the sequences of IMU measurements, mechanization solutions, and GNSS measurements to provide more reliable navigation solutions during GNSS outages. In this paper, as a preliminary study for designing LSTM networks to derive navigation solution during GNSS outages, a specific input-output configuration of the LSTM network is proposed, and the results are provided as a function of the network sequence length and the number of hidden units for each GNSS outage time on simple vehicle trajectories. Two distinct LSTM networks were used to estimate attitude and velocity, respectively, and the attitude information computed from positions of three GNSS receivers were used to enhance the network training precision. For the two-dimensional path of a wheeled robot mainly covered in this paper, position information can be estimated using only heading values computed from the positions of two GNSS receivers. However, this paper used three GNSS receivers to compute three-axis attitude information with the intention of extending this method to platforms having more complex trajectories, such as drones.

The remainder of the paper begins with the background on the loosely coupled KF for IMU-GNSS integration. Then, Section 3 introduces the proposed LSTM network-based method to derive navigation solution during GNSS outages. Details on the experimental datasets used for verification are described in Section 4 and the performance evaluation results on the proposed approach are summarized in Section 5. Section 6 provides the conclusion of the paper.

2. BACKGROUND

This section briefly discusses IMU mechanization processes and the loosely coupled IMU-GNSS KF with a closed-loop correction.

2.1 IMU Mechanization

A schematic process of an IMU mechanization is shown in Fig. 1. By integrating the measurements of angular rates, \({\widetilde{\mathbf{\omega}}}^b\), and specific forces, \({\widetilde{\mathbf{a}}}^b\), a navigation solution in Earth Centered Earth Fixed (ECEF) frame is computed through the following three steps; attitude update, velocity update, and position update.

Fig. 1. Schematic process of an IMU mechanization.

Assuming that the IMU sensor frame is perfectly aligned with the vehicle’s body frame, the time derivatives of attitude, velocity, and position in ECEF frame are formulated as follows (Groves 2013):

\({\dot{\mathbf{C}}}_b^e=\mathbf{C}_b^e\mathbf{\Omega}^b-\mathbf{\Omega}^e\mathbf{C}_b^e\)                                                                                         (1)

\({\dot{\mathbf{v}}}^e=\mathbf{C}_b^e\mathbf{a}^b+\mathbf{g}^e-2\mathbf{\Omega}^e\mathbf{v}^e\)                                                                             (2)

\({\dot{\mathbf{r}}}^e=\mathbf{v}^e\)                                                                                                                  (3)

where superscript ‘e’ and ‘b’ stand for ECEF frame and body frame (forward-right-downward), respectively. \({\dot{\mathbf{C}}}_b^e\) represents the time derivatives of the coordinate transformation matrix from the body frame to the ECEF frame. \({\dot{\mathbf{v}}}^e\) and \({\dot{\mathbf{r}}}^e\) are the time derivatives of velocity and position vector in the ECEF frame, respectively. \(\mathbf{\Omega}^b\) is a skew-symmetric matrix of the angular rates in the body frame, while \(\mathbf{\Omega}^e\) denotes a skew-symmetric matrix of the Earth’s angular velocity in ECEF frame. \(\mathbf{a}^b\) is a specific force vector acting on the body frame. \(\mathbf{g}^e\) is a gravitational acceleration vector in the ECEF frame which is a function of vehicle’s position.

Then, the vehicle’s navigation solution on \(k\)-th step can be calculated by integrating Eq. (1) through Eq. (3) as follows.

\(\mathbf{C}_{b,k}^e=\mathbf{C}_{b,k-1}^e(\mathbf{I}_3+\mathbf{\Omega}^b\tau_i)-\mathbf{\Omega}^e\mathbf{C}_{b,k-1}^e\tau_i\)                                                               (4)

\(\mathbf{v}_k^e=\mathbf{v}_{k-1}^e+(\mathbf{C}_{b,k}^e{\ \mathbf{a}}^b+\mathbf{g}^e-2\mathbf{\Omega}^e\mathbf{v}_{k-1}^e)\tau_i\)                                                       (5)

\(\mathbf{r}_k^e=\mathbf{r}_{k-1}^e+(\mathbf{v}_{k-1}^e+\mathbf{v}_k^e)\frac{\tau_i}{2}\)                                                                       (6)

Here, \(\tau_i\) denotes the solution update period. \(\mathbf{C}_{b,idx}^e\), \(\mathbf{v}_{idx}^e\) and  \(\mathbf{r}_{idx}^e \) are the computed attitude, velocity, and position solutions from the IMU mechanization at the idx-th step.

2.2 Loosely Coupled Kalman Filter for IMU-GNSS Integrated Navigation System

The error state KF with a closed loop correction is a well-known method for a low-cost IMU implementation. When position measurements are available from multiple GNSS receivers, KF computes an error state to correct the IMU bias and navigation solutions recursively. Details on the loosely coupled KF method can be found in (Groves 2013) and the overall procedure for IMU-GNSS integration is shown in Fig. 2.

Fig. 2. Architecture of IMU-GNSS loosely coupled KF integrated navigation system. The position and attitude of a vehicle calculated from three GNSS antennas measurements are denoted as \({\widetilde{\mathbf{r}}}^e\) and \({\widetilde{\mathbf{\varphi}}}^e\), respectively.

A 15-state vector of x to be estimated from KF is:

\(\mathbf{x}=\left[\begin{matrix}\begin{matrix}\delta\mathbf{\psi}^e&\delta\mathbf{v}^e&\delta\mathbf{r}^e\\\end{matrix}&\begin{matrix}\delta\mathbf{b}_a&\delta\mathbf{b}_g\\\end{matrix}\\\end{matrix}\right]^T\)                                                   (7)

where \(\mathbf{\delta}\mathbf{\psi}^e\) is the attitude error. \(\mathbf{\delta}\mathbf{v}^e\) and \(\mathbf{\delta}\mathbf{r}^e\) are velocity and position errors in the ECEF frame, respectively. \(\mathbf{\delta}\mathbf{b}_{a}\) and \(\mathbf{\delta}\mathbf{b}_{g}\) are accelerometer and gyro bias errors, respectively.

The time derivatives of navigation errors are:

\(\delta{\dot{\mathbf{\psi}}}^e=-\mathbf{\Omega}^e\delta\mathbf{\psi}^e+{\hat{\mathbf{C}}}_b^e\mathbf{b}_g\)                                                                                 (8)

\(\delta{\dot{\mathbf{v}}}^e=-({\hat{\mathbf{C}}}_b^e{\widetilde{\mathbf{a}}}^b)\land\delta\mathbf{\psi}^e-2\mathbf{\Omega}^e\delta\mathbf{v}^e-\delta\mathbf{g}^e\delta\mathbf{r}^e+{\hat{\mathbf{C}}}_b^e\mathbf{b}_a\)                                                   (9)

\(\delta{\dot{\mathbf{r}}}^e=\delta\mathbf{v}^e\)                                                                                           (10)

where \(\mathbf{\delta}{\dot{\mathbf{\psi}}}^e\), \(\mathbf{\delta}{\dot{\mathbf{v}}}^e\), and \(\mathbf{\delta}{\dot{\mathbf{r}}}^e\) denote time derivatives of attitude, velocity, and position errors, respectively; \land is a skew-symmetric matrix. \({\hat{\mathbf{C}}}_{b}^e\) is an estimated attitude from the mechanization process and \(\mathbf{2}\mathbf{\Omega}^e\mathbf{\delta}\mathbf{v}^e\) is the Coriolis velocity. Then, error state covariance matrix propagation is:

\(\mathbf{P}_k^-=\mathbf{\Phi}_{k-1}\mathbf{P}_{k-1}^+\mathbf{\Phi}_{k-1}^T+\mathbf{Q}_{k-1}\)                                                             (11)

\(\mathbf{\Phi}_{k-1}=\left[\begin{matrix}\mathbf{I}_3-\mathbf{\Omega}^e\tau_i&\mathbf{0}_3&\mathbf{0}_3&\mathbf{0}_3&{\hat{\mathbf{C}}}_b^e\tau_i\\-\left({\hat{\mathbf{C}}}_b^e{\widetilde{\mathbf{a}}}^b\right)\land\tau_i&\mathbf{I}_3-2\mathbf{\Omega}^e\tau_i&\delta\mathbf{g}^e\tau_i&{\hat{\mathbf{C}}}_b^e\tau_i&\mathbf{0}_3\\\mathbf{0}_3&\mathbf{I}_3\tau_i&\mathbf{I}_3&\mathbf{0}_3&\mathbf{0}_3\\\mathbf{0}_3&\mathbf{0}_3&\mathbf{0}_3&\mathbf{I}_3&\mathbf{0}_3\\\mathbf{0}_3&\mathbf{0}_3&\mathbf{0}_3&\mathbf{0}_3&\mathbf{I}_3\\\end{matrix}\right]\)                                         (12)

where \(\mathbf{P}_{k}^-\) and \(\mathbf{Q}_{k-1}\) are the covariance matrices of state and system noise, respectively. \(\mathbf{\Phi}_{k-1}\) is the discrete state transition matrix.

GNSS measurements from three GNSS receivers were used to estimate the attitude and position of a vehicle. The attitude of the vehicle can be accurately estimated through Real Time Kinematic (RTK) with fixed baselines using the three GNSS receivers (Zhao et al. 2017). The GNSS position may also come from RTK GNSS if RTK service is available. It should be noted that measurements from the GNSS antenna 0 \(({\widetilde{\mathbf{r}}}_{0}^e)\) in Fig. 3 is a primary source for vehicle positioning. The baseline between antennas \(\mathbf{b}_{01}^b\) and \(\mathbf{b}_{02}^b\) are aligned with lateral and longitudinal axis of the body frame, respectively.

 

Fig. 3. Arrangement of three orthogonally placed GNSS antennas for attitude and position measurements.

Using each antenna's RTK positioning solution, \({\widetilde{\mathbf{r}}}_{i\in 0,1,2}^e\) and known baseline vector, \(\mathbf{b}_{0 i\in 1,2}^b\), then, the attitude measurement, \({\widetilde{\mathbf{C}}}_b^e\) can be calculated by:

\({\widetilde{\mathbf{r}}}_{0i}^e={\widetilde{\mathbf{r}}}_i^e-{\widetilde{\mathbf{r}}}_0^e\ (i\in\left\{1,2\right\})\)                                                                                                              
\({\widetilde{\mathbf{C}}}_b^e=\left[\begin{matrix}{\widetilde{\mathbf{r}}}_{01}^e&{\widetilde{\mathbf{r}}}_{02}^e&{\widetilde{\mathbf{r}}}_{01}^e\land{\widetilde{\mathbf{r}}}_{02}^e\\\end{matrix}\right]\left[\begin{matrix}\mathbf{b}_{01}^b&\mathbf{b}_{02}^b&\mathbf{b}_{01}^b\land\mathbf{b}_{02}^b\\\end{matrix}\right]^{-1}\)                                         (13)

Measurement innovation for the KF is:

\(\delta\mathbf{z}_k=\left[\begin{matrix}\delta\mathbf{z}_r&\delta\mathbf{z}_{\psi}\\\end{matrix}\right]^T\)                                                                                 (14)

\(\delta\mathbf{z}_r={\hat{\mathbf{r}}}^e-{\widetilde{\mathbf{r}}}_0^e\)                                                                                           (15)

\({\widetilde{\mathbf{C}}}_b^e{{\hat{\mathbf{C}}}_b^e}^\top\approx\mathbf{I}_3-\left[\delta\mathbf{z}_{\psi}\land\right]\)                                                                                                    
\(\approx\left[\begin{matrix}1&\delta z_{\psi,z}&-\delta z_{\psi,x}\\-\delta z_{\psi,z}&1&\delta z_{\psi,y}\\\delta z_{\psi,x}&-\delta z_{\psi,y}&1\\\end{matrix}\right]\)                                                         (16)

When GNSS measurements are available, the update procedure for the loosely coupled KF is:

\(\mathbf{K}_k=\mathbf{P}_k^-\mathbf{H}_k^T\left(\mathbf{H}_k\mathbf{P}_k^-\mathbf{H}_k^T+\mathbf{R}_k\right)^{-1}\)                                                             (17)

\({\hat{\mathbf{x}}}_k^+=\mathbf{K}_k\delta\mathbf{z}_k\)                                                                                           (18)

\(\mathbf{P}_k^+=\left(\mathbf{I}-\mathbf{K}_k\mathbf{H}_k\right)\mathbf{P}_k^-\)                                                                                 (19)

\(\mathbf{H}_k=\left[\begin{matrix}\begin{matrix}\mathbf{0}_3\\\mathbf{I}_3\\\end{matrix}&\begin{matrix}\begin{matrix}\mathbf{0}_3\\\mathbf{0}_3\\\end{matrix}&\begin{matrix}\mathbf{I}_3\\\mathbf{0}_3\\\end{matrix}\\\end{matrix}&\begin{matrix}\begin{matrix}\mathbf{0}_3\\\mathbf{0}_3\\\end{matrix}&\begin{matrix}\mathbf{0}_3\\\mathbf{0}_3\\\end{matrix}\\\end{matrix}\\\end{matrix}\right]\)                                                             (20)

where, \(\mathbf{K}_{k}\) is Kalman gain matrix. \(\mathbf{H}_{k}\) is a measurement matrix. \(\mathbf{R}_{k}\) is a measurement noise covariance matrix and \(\mathbf{\delta}\mathbf{z}_{k}\) is a measurement innovation. \({\hat{\mathbf{x}}}_{k}^+\) is a posteriori estimated state and \(\mathbf{P}_{k}^+\) is a corresponding state covariance matrix.

In the closed loop configuration, the estimated error state from the KF is fed back to correct the IMU mechanization solution. The correction on navigation solution is applied as follows:

\(\mathbf{\psi}_{c}^e={\hat{\mathbf{\psi}}}_{mec}^e-\delta\mathbf{\psi}^e\)                                                                                        
\(\mathbf{r}_{c}^e={\hat{\mathbf{r}}}_{mec}^e-\delta\mathbf{r}^e\)                                                                                          
\(\mathbf{r}_{c}^e={\hat{\mathbf{r}}}_{mec}^e-\delta\mathbf{r}^e\)                                                                                   (21)

where \(\mathbf{\psi}_{c}^e\), \(\mathbf{v}_{c}^e\) and \(\mathbf{r}_{c}^e\) are corrected attitude, velocity, and position, respectively. As corrections are applied in each iteration of the KF, error states are initialized to zero after corrections (Noureldin et al. 2012). The IMU biases are also corrected periodically from the KF as follows:

\(\mathbf{b}_a^+=\mathbf{b}_a^-+\delta\mathbf{b}_a\)                                                                                        
\(\mathbf{b}_g^+=\mathbf{b}_g^-+\delta\mathbf{b}_g\)                                                                                 (22)

3. METHOD

Recurrent Neural Network (RNN) has strength on processing sequential data such as time series data, signals and natural language by storing past information in hidden states. However, RNN is unable to learn the long-term dependencies due to vanishing and exploding gradient problems (Bengio et al. 1993). Therefore, Long Short-Term Memory (LSTM) network has been proposed to deal with the vanishing and exploding gradient problems of the traditional RNN. The LSTM network shown in Fig. 4 uses several gates and a cell to selectively preserve and forget long-term past information (Hochreiter & Schmidhuber 1997).

Fig. 4. Schematic architecture of LSTM block with gating units. \(\mathbf{f}_t,\mathbf{i}_t\mathbf{o}_t\) and \({\widetilde{\mathbf{C}}}_t\) are forget gate, input gate, output gate and cell input state, respectively.

In this study, LSTM network was implemented to build a relationship between a navigation solution from IMU measurements through mechanizations and a precise RTK level navigation solution to provide accurate navigation solution during GNSS outages. The main objective of the trained LSTM network is to predict navigation solution on GNSS outages based on the sequence of IMU measurements and previous navigation solution.

Although, the LSTM network can be used to provide corrections on IMU mechanization solution, the proposed method provides navigation solutions directly, instead of the INS errors. The reason for that is that a low-cost IMU typically has large noise, and its bias is subject to change in short time. Therefore, the actual IMU errors in tests may become different from the ones in the train dataset and could cause a significant divergence of the navigation solution. In line with the above network design approach, two single layer LSTM networks were used to provide direct attitude and velocity solutions from the sequence of IMU measurements and IMU mechanization solutions.

As the trained LSTM network generates navigation solutions on GNSS outages, the output rate for the network was set to be the same as a GNSS output rate. Thus, IMU measurements were averaged according to the GNSS output rate, and averaging measurements also have an effect on reducing measurement noise. By employing LSTM network, input data for the network have a sequential form, while the outputs of the network are current attitude and velocity estimates.

The proposed LSTM network for attitude estimates can be expressed as,

\({\hat{\mathbf{\varphi}}}_{t,\ LSTM}^n=f_{LSTM,att}\left(\begin{matrix}{\bar{\mathbf{\omega}}}_{t-SL+1}^b,\cdots,{\bar{\mathbf{\omega}}}_{t-1}^b,\ {\bar{\mathbf{\omega}}}_t^b\\{\hat{\mathbf{\psi}}}_{mec,\ t-SL+1}^n,\cdots,{\hat{\mathbf{\psi}}}_{mec,\ t-1}^n,{\hat{\mathbf{\psi}}}_{mec,\ t}^n\\\end{matrix}\right)\)                                         (23)

The LSTM network for velocity estimates is expressed as,

\({\hat{\mathbf{v}}}_{t,\ LSTM}^n=f_{LSTM,vel}\left(\begin{matrix}\begin{matrix}{\bar{\mathbf{\omega}}}_{t-SL+1}^b,\cdots,{\bar{\mathbf{\omega}}}_{t-1}^b,\ {\bar{\mathbf{\omega}}}_t^b\ \\{\bar{\mathbf{a}}}_{t-SL+1}^b,\cdots,{\bar{\mathbf{a}}}_{t-1}^b,\ {\bar{\mathbf{a}}}_t^b\\\end{matrix}\\\begin{matrix}{\hat{\mathbf{\varphi}}}_{t-SL+1}^n,\cdots,{\hat{\mathbf{\varphi}}}_{t-1}^n,{\hat{\mathbf{\varphi}}}_t^n\\\mathbf{v}_{mec,t-SL+1}^n,\cdots,\mathbf{v}_{mec,t-1}^n,\mathbf{v}_{mec,t}^n\\\end{matrix}\\\end{matrix}\right)\)                                         (24)

\({\hat{\mathbf{\varphi}}}_t^n=\left\{\begin{matrix}{\hat{\mathbf{\varphi}}}_{t,\ \ LSTM}^n\ ({\rm during\ GNSS\ outages})\\{\widetilde{\mathbf{\varphi}}}_t^n\ ({\rm otherwise})\\\end{matrix}\right.\)                                                   (25)

Here, f(∙) is the function defined by the trained LSTM network and SL stands for the sequence length used for t-th epoch navigation data. The superscript n denotes the NED frame. \({\bar{\mathbf{\omega}}}^b\) and \({\bar{\mathbf{a}}}^b\) are the averaged angular velocity and acceleration measurements. Both networks were designed to have same sequence length and the number of hidden units used.

3.1 How to Train the LSTM Network

When GNSS measurements are available, a loosely coupled KF produces navigation solutions, and the network works in a training mode. Fig. 5 shows the overall procedure when GNSS measurements are available. During the training process, the target output of each network is the attitude and velocity computed from GNSS measurements while inputs are IMU measurements and IMU mechanization solution as described in Eqs. (23) and (24).

Fig. 5. Process of training LSTM network when GNSS measurements are available. The input data for the network has a following sequential form, \(\mathbf{x}_{\mathbf{seq}\ }=[\mathbf{x}_{t-SL+1},\cdots, \mathbf{x}_{t-1}\), \(\mathbf{x}_{t}]\) where SL stands for the sequence length of the network.

3.2 How to Test on GNSS Outages

Fig. 6 shows a way to use trained LSTM network on GNSS outages. The trained network would completely replace KF on GNSS outages. Attitude estimates from the attitude estimation network will be reused as inputs for the velocity network to predict the velocity of a vehicle. And the vehicle’s current position is calculated from the previous position and current velocity estimates. In this case, as the output rate of the network was set as 1 Hz, current position can be calculated by adding velocity estimates on previous position solution. Then, the navigation solution on GNSS outages is:

\(\mathbf{\varphi}_{t}={\hat{\mathbf{\varphi}}}_{t,\ LSTM}^n\)                                                                                                      
\(\mathbf{v}_{t}={\hat{\mathbf{v}}}_{t,\ LSTM}^n\)                                                                                                      
\(\mathbf{r}_{t}=\mathbf{r}_{t-1}+\mathbf{v}_t\)                                                                                             (26)

Fig. 6. Using trained LSTM network on GNSS outages. The input data for the network has a following sequential form, \(\mathbf{x}_{seq}=[\mathbf{x}_{t-SL+1},\cdots\), \(\mathbf{x}_{t-1}\), \(\mathbf{x}_{t}]\) where SL stands for the sequence length of the network.

4. DATASET DESCRIPTION

This paper proposed a method for determining the attitude and position of vehicles in urban area where frequent GNSS outages exist. To implement and evaluate our approach, experimental data were collected from a mobile robot with an IMU and three GNSS antennas. These three GNSS antennas were used to provide RTK based precise position as well as attitude information of the mobile robot shown in Fig. 7. With precise RTK based position and attitude information, the network was trained to have a higher level of accuracy. Train and test data were obtained from experiments conducted in the same environment. For the test data, to simulate GNSS outages, the GNSS data were removed from the actual experimental data.

Fig. 7. Configuration of mobile robot and onboard sensor system.

A MPU9250 IMU (InvenSense Inc. 2016) and three Ublox F9P GNSS receivers (U-blox AG. 2021) were used as onboard sensors. The sampling rates of IMU and GNSS were 200 Hz and 1 Hz, respectively. The specifications of MPU 9250 and Ublox F9P receiver are summarized in Table 1.

Table 1. Sensor specification of MPU9250 & Ublox F9P.

Device	Sensor	Sensor spec
		Error type	Scale
MPU9250	Accelerometer	Noise spectral density (µg/√hz) Random walk (m/s\(^2\)/√h)	300 1
	Gyroscope	Bias instability (°/h) Noise spectral density (°/s/√hz) Random walk (°/√h)	15 0.01 0.01
Ublox F9P	GNSS (RTK)	Horizontal position error (cm) Vertical position error (cm)	1 3

 

The trajectories used for the experiment are shown in Fig. 8. Two types of simple trajectories were tested, and we named each dataset as “oval shape driving” and “figure eight driving”. The blue lines in the figures represent the routes for the training data, and the red dotted line are used to test the trained network on GNSS outages. Since the proposed network estimates the attitude first, then estimates the velocity to compute the position, scenarios where the vehicle’s attitude changes continuously were chosen to verify the proposed method, such as oval shape, figure-eight shape driving trajectories. In both trajectories, the sensor data with the first 600 seconds were used to train the network, and the data of the remaining 60 seconds of the data were used to test the positioning performance of the proposed LSTM network on GNSS outages.

Fig. 8. Two trajectories of mobile robot from the experiments, oval shape driving (top) and figure eight driving path (bottom). The average speed in oval shape driving was 2.79 m/s and 1.48 m/s in figure eight driving.

Fig. 9 shows IMU measurements during the oval shape driving experiment. Also, the blue line indicates training dataset, while red dashed line shows test dataset. The black line shows IMU measurements averaged for a second to be synchronized with the output rate of the network and those values were used as input data for the network.

Fig. 9. IMU measurements during the oval shape driving.

5. RESULTS

The training of the network was performed on a single i7-9750H CPU using Mathworks’s Deep Learning toolbox. The hyperparameters used to train the network are shown in Table 2. The adaptive moment estimation optimizer (ADAM) was used for stochastic optimization. The learning rate and other hyperparameters were chosen from several tests on loss convergence during the training process.

Table 2. Hyperparameters used to train the network.

Parameters	Value
Initial learn rate Gradient decay factor Squared gradient decay factor Learn rate drop period Learn rate drop factor Mini batch size	0.001 0.95 0.999 150 0.2 31

 

The positioning performance during the test on GNSS outages varied depending on the choice of the number of hidden states and sequence length used as shown in Fig. 10. It can be observed that the RMSE position errors overall increase as the duration of GNSS outages escalates from 20 to 60 seconds. Also, the higher number of hidden units and the longer number of the sequence length tend to reduce RMSE in both driving trajectories. Meanwhile, in the case of figure eight driving, a few results showed better results on a short sequence length. Since the attitude changed more sharply during the figure eight driving, the use of the longer sequence rather interfered estimating current vehicle’s attitude.

Fig. 10. Results on each parameter combination on oval shape driving case (top), figure eight driving (bottom).

Fig. 11 shows the resultant trajectories of INS and the trained LSTM network for 60 seconds GNSS outages from the two experiments. The network had the sequence length of 15 and 150 hidden units for the oval shape driving case and sequence length of 5 and 100 hidden units for the figure eight driving, which resulted in the best performance during 60 seconds GNSS outages among all combination of the sequence lengths and the number of hidden units used. The RMSE of INS and the LSTM network are summarized in Table 3. In both cases, RMSE of the LSTM network was reduced by more than 90% compared to INS.

Fig. 11. Positioning results on oval shape driving case (top), figure eight driving (bottom) for 60 seconds GNSS outages.

Table 3. Position RMSE on 60s GNSS outages.

RMSE	INS		LSTM (Best)
	East (m)	North (m)	East (m)	North (m)
Test 1 Teat 2	416.36 345.72	93.42 474.89	2.47 3.38	2.84 4.51

 

The previous two experimental data have a relatively simple repeating patterned trajectory. Therefore, to evaluate the performance of the proposed network in non-uniform actual vehicle driving scenarios, we downloaded RTK positions for 640 seconds of a driving vehicle (ETRI AI Nanum 2021) and shown in Fig. 12. The corresponding IMU measurements were obtained from Mathworks’s Sensor fusion and Tracking toolbox with a typical low-cost IMU specification (El-Sheimy et al. 2008) and MPU 9250. The network was trained using the RTK position and simulated IMU measurements from 600 seconds driving data. The remaining 40 seconds data were used for the test.

Fig. 12. Reference vehicle driving route data at the speed of 13.93 m/s divided into train and test data.

In the case of vehicle driving scenario, the shorter the sequence length, the lower the RMSE as can be seen in Fig. 13. This also implies that for a relative fast-moving vehicle, long previous recursive information is not helpful in estimating the current vehicle’s state. Fig. 14 shows test results on 40 seconds GNSS outages. The predicted LSTM closely matches with the reference trajectory and the positioning solution of the LSTM network is improved by more than 90% compared to INS. Further research is necessary on finding an optimal network architecture to ensure near real-time application with various vehicle dynamic motions and path, which will require more data and information to build a rigorous model.

Fig. 13. Results on each parameter combination on highway driving data.

Fig. 14. Test results on 40 seconds GNSS outages. RMSE from INS are 42.18 m (East), 118.72 m (North), while RMSE from the trained network are 1.48 m (East), 3.48 m (North).

6. CONCLUSIONS

This paper proposed using LSTM to provide more reliable navigation solutions than using INS during GNSS outages. The proposed algorithm consists of two independent LSTM networks to predict current attitude and velocity of the vehicle individually from the sequence of IMU measurements and mechanization solutions. During GNSS outages, the outputs of the trained network are used to compute position solutions.  The proposed method has been verified with both experimental data and simulated data and showed more than 90% improvements on positioning accuracy compared with INS on GNSS outages.