Neural Networks Based Modeling with Adaptive Selection of Hidden Layer's Node for Path Loss Model

Abstract

The auto-encoder network which is a good candidate to handle the modeling of the signal strength attenuation is designed for denoising and compensating the distortion of the received data. It provides a non-linear mapping function by iteratively learning the encoder and the decoder. The encoder is the non-linear mapping function, and the decoder demands accurate data reconstruction from the representation generated by the encoder. In addition, the adaptive network width which supports the automatic generation of new hidden nodes and pruning of inconsequential nodes is also implemented in the proposed algorithm for increasing the efficiency of the algorithm. Simulation results show that the proposed method can improve the neural network training surface to achieve the highest possible accuracy of the signal modeling compared with the conventional modeling method.

1. INTRODUCTION

Radio-frequency communication systems in various frequency bandwidths have become an essential system to deliver large amounts of various information to data, voice, video, multimedia applications. The performance of radio-frequency communication systems is influenced by considering the design specifications of radio wave transmission models, channel assignment and interference mitigation strategies among others. Radio wave transmission models are one of the important components to improve the capacity or quality of communication systems. Large-scale path loss values between the base station and mobile stations are the key regulating factors that limit the performance of communication systems, especially in the urban area (Cheerla et al. 2018). In addition, many existing path loss models are empirically derived by assuming a linear log-distance model and determining the model parameters through the adequate linear regression analysis of the measured data. However, linear regression models are not best for all the regions.

Thus, it is needed to develop a suitable path loss prediction model for predicting path loss values based on received signal strength measurements. Although various linear log-distance path loss models have been developed, advanced models are requiring to more accurately and flexibly represent the path loss for complex environments such as the urban area (Park et al. 2019).

Machine learning-based methods of path loss modeling are expected to provide a better model that can generalize well to the propagation environment since the model is being learned through training with data collected from the environment. The previous works (Popescu et al. 2006, Ostlin et al. 2010, Mom et al. 2014) provide path loss prediction using artificial neural network (ANN) models which provide more precise estimation over the empirical models. The studies in the previous works (Popescu et al. 2006, Mom et al. 2014) developed ANN prediction models for urban and suburban environments but did not present a multi-dimensional model of distance and frequency. The authors in the paper (Ostlin et al. 2010) showed that a simple ANN architecture (feed-forward network with one hidden layer and few neurons) has better path loss prediction accuracy compared to complex architecture in rural environments.

In this paper, Auto-encoder-based parametric modeling for path loss model in urban environments at 1 to 3 GHz frequency bandwidth (target signal is Long-Term Evolution (LTE)). The auto-encoder is a type of ANN, which can learn efficient data coding using an unsupervised manner. An auto-encoder can find common features from unlabeled data and classify data after training (Liu et al. 2019). The auto-encoder network is used to learn the path loss structure from the measured path loss data and its network model is considered to design the path loss model in the urban area. 

Furthermore, the adaptive selection method of network width which supports the automatic generation of hidden nodes and pruning of inconsequential nodes is also implemented in the proposed algorithm for increasing efficiency. This approach is based on autonomous deep learning techniques. Autonomous deep learning has an elastic network width that supports the automatic generation of new hidden nodes and pruning of inconsequential nodes (Ashfahani & Pratama 2019). This algorithm is performed based on the NS method (Pratama et al. 2018a) which estimates the network generalization power in terms of bias and variance (Ashfahani & Pratama 2019). The number of new hidden node is added in the case of underfitting (high bias case) while the pruning logic is activated in the case of overfitting (high variance case). This algorithm uses an adaptive threshold that dynamically adapts to the bias and variance estimation and the algorithm is based on the extension of the previous work (Pratama et al. 2018a).

An outline of the paper is as follows. In Section 2, the whole process of the LTE signal’s path loss modeling is introduced and its related general modeling methods are also explained. The proposed method with auto-encoder based modeling is explained in Section 3. In addition, the adaptive adjustment method of the network’s layer width (selection of hidden layer’s node number) is written in this section. In Section 4, several simulations are conducted for evaluating the performance of the proposed algorithm in terms of denoising, clustering, and modeling compared with conventional methods. The conclusions are summarized in Section 5.

2. LTE PATH LOSS MODELING

The location recognition method using the access point (AP) is divided into a time of flight method using the arrival time of the radio wave and a received signal strength indicator (RSSI) method for measuring the signal strength of the radio wave. The electronic fingerprint positioning technique using a wireless communication signal is a typical RSSI method. The communication signal generally includes basic information such as AP's identification name, signal strength, and security status. Among them, the location is recognized using the AP's identification name and signal strength. There are various positioning resources to measure the user's location, and there are usually narrow network signals such as Bluetooth Low Energy and Wi-Fi, and wide area network signals such as LTE and 5G. In this paper, we analyze the characteristics of LTE signals that are well-activated indoor and outdoor networks among several candidate signals described above. 

In general, the characteristics of wireless communication channels can be explained by dividing them into path loss (Hamid & Kostanic 2013, Zyoud et al. 2016, Liu et al. 2017), shadowing, multi-path fading, Doppler effect, etc. In the electronic fingerprint positioning technique, since the positioning is performed with the strength of the received signal, the path loss is analyzed intensively. However, in the real environment, based on wireless communication, it has complex characteristics such as shadow loss including path loss, and multipath fading that appears in a specific urban area as shown in Fig. 1.

Fig. 1. Signal attenuation tendency according to channel characteristics.

 

Propagation characteristics can be represented using ray tracing to generate relatively accurate RSSI information. In ray tracing, transmission, reflection, diffraction, and scattering occur when radio waves reach a building structure.

However, this requires 3D map database information including accurate surrounding environment information and requires a lot of computation time. Furthermore, the accurate numerical information on the material of the structure is also required. Thus, this paper assumes that the propagation model is transmitted in the ideal state (free space propagation).

The path loss model representing the strength of the received signal is as follows (Zyoud et al. 2016).

\(P L=A \log _{10}\left(\frac{d}{d_{0}}\right)+B \log _{10}\left(\frac{f}{k}\right)+C+\beta d+X(\sigma)\)   (1)

where PL  refers to attenuated signal power (signal power loss) according to distance,  d from the AP.  A, d₀, B, k, C are design parameters that should be set according to the propagation path loss environment and they should be set according to the radio wave path loss environment (Zyoud et al. 2016).  d refers to the distance between transmitter and receiver (unit m), f is the frequency in GHz, \(\beta\) is a physical parameter that function of wall penetration and room size or dimension, and \(X(\sigma)\) is set to log-normal random number with its standard deviation, \(\sigma\) .

In the above models, the types of models are subdivided according to the design parameters, and the path loss models that are commonly used in measurements using LTE are 3GPP models, IEEE 802, WINNER II, etc. These models focus on simulating a more realistic radio environment than existing models (such as log-distance models, Cost 231-Hata models, Ericsson models, Lee models) as a derivative of the RF propagation model, which is expected to be advantageous in partially simulating the radio environment with many NLOS situations in urban areas. The parameters of the path loss model defined in each propagation model (parameters in Eq. (1)) can be summarized as written in the previous work (Zyoud et al. 2016).

Because receivers of wireless communication systems require a certain minimum power (sensitive) to successfully decode information, path loss predictions are essential to mobile communication network design and planning, especially when measuring radio communication system-based, have a significant impact on location estimation accuracy. The various path loss prediction models described above have been developed for this purpose. Many conventional path loss models estimate model parameters by assuming linear log-distance models and determining model parameters through proper linear regression of measured data. However, the linear regression model is not suitable for all regions and does not reflect environmental changes, so the farther the transmitter and receiver are, the lower the path loss prediction accuracy is.

Recently, various machine learning-based path loss prediction techniques have been studied. The machine learning approach to path loss modeling is expected to provide a better model for generalizing propagation environments because the model is trained with data collected from the environment. Previous studies (Ostlin et al. 2010, Mom et al. 2014, Hosseinzadeh et al. 2017) performed path loss prediction using an ANN model and analyzed that ANN model has a higher estimation performance than that of the empirical model. In this paper, the path loss model is designed using the ANN in the same way as the recent research trend described above. In addition, an adaptive selection algorithm that adjusts the number of nodes in the hidden layer of the neural network is designed to improve the model generation efficiency. The proposed algorithm is described in detail in the next section.

3. AUTO-ENCODER BASED MODELLING 

In this section, the whole process of auto-encoder based modeling is explained. In addition, the basic concepts of the auto-encoder and adaptive selection method of hidden layer width (the number of nodes in the hidden layer) which are essential algorithms of the proposed algorithm are also explained.

3.1 Denoising Auto-encoders

An auto-encoder is a type of artificial neural network used to learn efficient data structures (codings) in an unsupervised manner. The aim of an autoencoder is to learn a representation (encoding) for a set of data by training the network to ignore signal noise. In addition, the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input. Several variants exist to the basic model of auto-encoder, with the aim of forcing the learned representations of the input to assume useful properties.

In the one-layer auto-encoder network, it consists of an encoder and a decoder. The encoder maps an input x_i to its hidden representation  h_i. The mapping function is usually non-linear and the following is a common form (Song et al. 2013):

\(h_{i}=f\left(W_{1} x_{i}+b_{1}\right)=\frac{1}{1+\exp \left(-\left(W_{1} x_{i}+b_{1}\right)\right)}\)       (2)

where  W₁ is the encoding weight,  b₁ is the corresponding bias vector.

The decoder seeks to reconstruct the input x_i  from its hidden representation  h_i. The transformation function has a similar formulation (Song et al. 2013):

\(x_{i}^{\prime}=g\left(W_{2} h_{i}+b_{2}\right)=\frac{1}{1+\exp \left(-\left(W_{2} h_{i}+b_{2}\right)\right)}\)         (3)

where  W₂ and  b₂ are the decoding weight and the decoding bias vector respectively. The auto-encoder model aims to learn a useful hidden representation by minimizing the reconstruction error. Thus, given N  training samples, the parameters W₁, W₂, b₁ and  b₂ can be resolved by the following optimization problem:

\(\min \frac{1}{N} \sum_\limits{i=1}^{N}\left\|x_{i}-x_{i}^{\prime}\right\|^{2}\)        (4)

Generally, an auto-encoder network is constructed by stacking multiple one-layer auto-encoders. It means that the hidden representation of the previous one-layer auto-encoder is fed as the input of the next one. A detailed explanation of the auto-encoder network is written in (Bengio et al. 2013).

Denoising auto-encoder (DAE) proposed in the paper (Vincent et al. 2008) is a stochastic extension to classic auto-encoder. DAE tries to reconstruct a clean input from its corrupted version of the input signal. The initial input x is corrupted to  \(\tilde x\) by a stochastic mapping  \(\tilde{x} \approx q(\tilde{x} | x)\). Subsequently, DAE uses the corrupted \(\tilde x\)  as input data, and then maps to the corresponding hidden representation h and ultimately to its reconstruction \(x'\) . Due to their powerful nonlinear mapping capabilities, the auto-encoder and DAE models have been generally used for data compression (Deng et al. 2010, Gogna et al. 2017) and noise reduction on speech signals (Lu et al. 2013, Lai et al. 2016) and medical images (Gondara 2016).

The use of flexible structure with the growing and pruning algorithm has recently come into the spotlight in ANN literature (Pratama et al. 2018a, 2018b, 2018c, 2018d) where the key idea is to evolve the ANN’s structure (Ashfahani & Pratama 2019). Incremental learning of DAE realizes the structural learning process with the network’s loss and the hidden unit merging mechanism (Zhou et al. 2012). The underlying drawback of this approach is located in the over-dependence on problem-dependent predefined thresholds in growing and merging hidden units (Ashfahani & Pratama 2019). The progressive neural networks (PNN) (Rusu et al. 2016), the dynamically expandable networks (DEN) (Yoon et al. 2017) and the incremental learning of DAE (DEVDAN) (Pratama et al. 2018a, 2018b, 2018c) are proposed to address limited network capacity and catastrophic forgetting problems (Ashfahani & Pratama 2019). Algorithms that adjust the network structure that was developed earlier including PNN and DEN were less efficient in their operation. PNN creates a new network structure for every new task, DEN increases hidden nodes whenever the loss criteria are not satisfied. However, DEVDAN which is recently developed is capable of growing and pruning the hidden nodes based on the estimation of network significance (NS). 

Thus, in this paper, the algorithm to change the network structure of the DAE adaptively is implemented based on DEVDAN with NS to the proposed algorithm. The detailed description of the adaptive selection of the network layer’s node is written in the following section.

3.2 Adaptive Selection of Hidden Layer’s Node

Adaptive selection of hidden layer’s node consists of two-part: the hidden node growing and pruning logic.

The hidden node growing logic is controlled by the NS formula which evaluates the generalization power of network structure formalized as the expectation of squared error under a normal distribution and this expression leads to the bias-variance formula as follows (Ashfahani & Pratama 2019):

\(N S=E\left[\left(x-x^{\prime}\right)^{2}\right]=E\left[\left(x^{\prime}-E\left[x^{\prime}\right]\right)^{2}\right]+\left(E\left[x^{\prime}\right]-x\right)^{2}=\operatorname{Var}\left(x^{\prime}\right)+\operatorname{Bias}\left(x^{\prime}\right)^{2}\)        (5)

It is certain that  is induced by the feature extractor \(f\left(\tilde{x}+b_{1}\right)\) and is influenced by partially destroyed input features due to the noise. Thus,  \(E[x']\) and \(E[h]\) are revised as

\(E\left[x^{\prime}\right]=g\left(W_{2} E[h]+b_{2}\right)\)        (6)

\(E[h]=\int_{-\infty}^{\infty} f\left(W_{1} \tilde{x}+b_{1}\right) p(\tilde{x}) d \tilde{x}\)        (7)

In the above equation, the bias can be calculated by substituting \(E[x']\)  to the bias term, \(\operatorname{Bias}\left(x^{\prime}\right)^{2}=E\left[\left(x-E\left[x^{\prime}\right]\right)^{2}\right]\).  \(E[x']\) is also used for calculating .

The high bias indicates the underfitting situation which can be circumvented by increasing the network capacity. The hidden node increasing condition is based on the k-sigma rule concept adopted from the theory of statistical process control and previous work (Pratama et al. 2018a). However, the thresholds of decision logic are newly defined for changing the number of nodes in the hidden layer to suit the LTE signal propagation modeling in this paper.

The high bias problem, triggering the construction of a hidden node in the layer, is formulated as follows:

\(\mu_{b i a s} \geq T_{b i a s}\)             (8)

where \(\mu_{bias}\) is the recursive mean of \(Bias(x')^2\). \(T_{bias}\) is the threshold and it is set based on minimum values of \(\mu_{bias}\). This threshold ensures the performance of removing noise components without distorting the original signal and reflects the characteristics of the signal propagation model. Eq. (8) signifies the existence of changing data distribution represented by the increase of network bias (Pratama et al. 2018a, Ashfahani & Pratama 2019). The network bias should decrease (at least be stable) when there is no drift in received data. When Eq. (8) is satisfied, a hidden node is added in the auto-encoder.

In the case of high variance (overfitting case) should be handled by reducing the network complexity. The hidden node pruning condition implements the same principal as the growing part (increasing the number of hidden nodes) where the statistical process control is adopted to identify the high variance problem, as written in 

\(\mu_{Var } \geq T_{Var}\)       (9)

where \(\mu_{Var } \) stands for the mean of  Var(x').  \( T_{Var}\)is the threshold and it is set based on minimum values of \(\mu_{Var } \) . Similar to the threshold selection method in the case of underfitting case, the threshold is also set to maintain the noise rejection performance while minimizing the signal distortion. If Eq. (9) is satisfied, the pruning logic is activated to remove the weakest hidden node. The significance of the hidden node is tested using the concept of NS, adapted to evaluate the hidden unit statistical contribution (Pratama et al. 2018a, Ashfahani & Pratama 2019). This method can be derived by checking the hidden node activity in the whole corrupted feature data, \(\tilde x\). The significance of the i-th  hidden node is defined as its average activation degree for all possible data samples as follows:

\(H S=\lim _\limits{T \rightarrow \infty} \sum_\limits{t=1}^{T} \frac{f\left(W_{1} \tilde{x}+b_{1}\right)}{T}\)          (10)

Since the contribution of i-th  hidden unit is formed in terms of the expectation of an activation function, the least contributing hidden unit having the minimum HS is deemed inactive. If the overfitting situation occurs (the condition written in Eq. (10) is satisfied), the pruning process includes the hidden node with the lowest HS as follows:

\(\text { pruning } \rightarrow \min _{i=1,2,3 \cdots R} H S_{i}\)          (11)

Eq. (11) aims to mitigate the overfitting situation by getting rid of the least contributing hidden unit.

 

4. SIMULATIONS

For the performance analysis of the proposed method, the simulation is performed in two cases. The simulation was performed using a computer with the specifications of an i7-8700K CPU, 32 GB RAM, and GTX-1070Ti VGA (video graphic card). In the simulations, the denoising performance of the proposed algorithm is verified with various evaluation parameters. In addition, we analyze the results of the model estimation using the path loss model estimation method proposed in this paper and compare the model estimation performance with the existing method when there is signal noise and signal distortion in the received measurement.

4.1 Parameters for Evaluating the Performance

In this paper, root mean square error (RMSE), percentage root mean square difference (PRD), and improvement in SNR are used to analyze the denoising performance. The RMSE is used for determining the variance between the output predicted by the model and the actual output. A smaller value of RMSE corresponds to a smaller difference and better performance, and is expressed as follows:

\(R M S E=\sqrt{\frac{1}{N} \times \sum_\limits{n=1}^{N}\left(x_{r, i}-x_{i}^{\prime}\right)^{2}}\)       (12)

where x_r,i refers to the i-th component of true value.

The PRD indicates the recovery quality of the compressed signal by measuring the error between the original signal and the resultant signal after reconstruction. A lower PRD represents a better quality of the reconstructed signal. PRD is defined as 

\(P R D=\sqrt{\frac{\sum_{n=1}^{N}\left(x_{r, t}-x_{i}^{\prime}\right)^{2}}{\sum_{n=1}^{N} x_{r, t}^{2}}} \times 100\)     (13)

The last parameter of performance evaluation is  SNR_ip and it indicates the difference between the SNR after noise reduction and the original input signal SNR. The greater the SNR_ip is, the better the denoising performance is achieved. SNR_ip is able to be written as

\(S N R_{p}=S N R_{o u t}-S N R_{in}=10 \log _{10}\left(\frac{\sum_{n=1}^{N} x_{r, t}^{2}}{\sum_{n=1}^{N}\left(x_{r, t}-x_{i}^{\prime}\right)^{2}}\right)-10 \log _{10}\left(\frac{\sum_{n=1}^{N} x_{r, t}^{2}}{\sum_{n=1}^{N}\left(\tilde{x}_{1}-x_{1}^{\prime}\right)^{2}}\right)\)       (14)

4.2 Simulation Results

Before performance comparison, adaptive selection results of the hidden layer’s node are shown in Fig. 2. As shown in Fig. 2, the adaptive logic uses the two inequality equations described in Section 3 to determine whether to increase or decrease the number of the nodes. Considering the operational stability of the proposed algorithm, the maximum number of operations of the algorithm is set to 300, and the number of nodes is determined within the maximum number of operations. The simulation results of two cases for performance evaluation are shown in Figs. 3 and 4, and Tables 1 and 2.

Fig. 2. Adaptive selection results of the hidden layer’s node.

Fig. 3. Simulation results (case 1: additive noise).

Fig. 4. Simulation results (case 2: data distortion).

Table 1. Performance comparison of denoising algorithms (case 1).

Datasets	Signal strength according to ranges
Criterion	RMSE	PRD	\( SNR_{ip} \)
Conventional Proposed	0.5241 0.4494	32.1878 27.6001	9.8462 11.1818

 

Table 2. Performance comparison of denoising algorithms (case 2).

Datasets	Signal strength according to ranges
Criterion	RMSE	PRD	\( SNR_{ip} \)
Conventional Proposed	0.4317 0.3242	26.5142 19.9133	11.5304 14.0171

 

In simulations, the training data of the auto-coder is set to the received signal strength over distance and one of the several learning data was shown in the blue line (true) in Figs. 3 and 4. For comparison with the existing route loss model (3GPP, NLOS situation model), random noise and random bias are inserted in the test data, and as shown in Figs. 3 (random noise case) and 4 (random bias case), auto-coder-based model estimation techniques (green) are found to have more robust characteristics of noise compared to conventional model results (red dots). The additional noise levels used in this simulation are set to 10 dB and, in the case of signal distortion, random bias with 20 dB is added in two short periods of the input data.

Furthermore, even when the received signal is distorted, it can be seen that it is not affected by the distortion while reflecting the signal reception environment compared with the conventional technique as shown in Fig. 4. Although the parameter setting values of the existing models are not separately estimated due to the lack of additional information, there is a limit in estimating a precise path loss model only based on the received signal strength. On the other hand, in the case of using the proposed method, there is a signal attenuation error in the estimated path loss model, but the model estimation accuracy is higher than conventional methods. The performance comparison results are summarized in Tables 1 and 2 with evaluation parameters.

5. CONCLUSIONS

In this paper, a modeling method based on the auto-encoder network is proposed for the LTE path loss model. Due to its powerful nonlinear mapping capabilities, the proposed algorithm is used for noise reduction and distortion compensation of received signals. In addition, the adaptive network width is also designed in the proposed algorithm for increasing the efficiency of the algorithm. Simulation results show that the proposed method has better denoising performance compared with the conventional algorithm when there is noise and signal distortion in the received signal.