SAR Target Recognition Using the Multi-aspect-aware Bidirectional LSTM Recurrent Neural Networks

Fan Zhang, Chen Hu, Qiang Yin, Wei Li, Hengchao Li, Wen Hong

The outstanding pattern recognition performance of deep learning brings new vitality to the synthetic aperture radar (SAR) automatic target recognition (ATR). However, there is a limitation in current deep learning based ATR solution that each learning process only handle one SAR image, namely learning the static scattering information, while missing the space-varying information. It is obvious that multi-aspect joint recognition introduced space-varying scattering information should improve the classification accuracy and robustness. In this paper, a novel multi-aspect-aware method is proposed to achieve this idea through the bidirectional Long Short-Term Memory (LSTM) recurrent neural networks based space-varying scattering information learning. Specifically, we first select different aspect images to generate the multi-aspect space-varying image sequences. Then, the Gabor filter and three-patch local binary pattern (TPLBP) are progressively implemented to extract a comprehensive spatial features, followed by dimensionality reduction with the Multi-layer Perceptron (MLP) network. Finally, we design a bidirectional LSTM recurrent neural network to learn the multi-aspect features with further integrating the softmax classifier to achieve target recognition. Experimental results demonstrate that the proposed method can achieve 99.9% accuracy for 10-class recognition. Besides, its anti-noise and anti-confusion performance are also better than the conventional deep learning based methods.

Abstract—The outstanding pattern recognition performance of deep learning brings new vitality to the synthetic aperture radar (SAR) automatic target recognition (ATR). However, there is a limitation in current deep learning based ATR solution that each learning process only handle one SAR image, namely learning the static scattering information, while missing the space-varying information. It is obvious that multi-aspect joint recognition introduced space-varying scattering information should improve the classification accuracy and robustness. In this paper, a novel multi-aspect-aware method is proposed to achieve this idea through the bidirectional Long Short-Term Memory (LSTM) re- current neural networks based space-varying scattering informa- tion learning. Specifically, we first select different aspect images to generate the multi-aspect space-varying image sequences. Then, the Gabor filter and three-patch local binary pattern (TPLBP) are progressively implemented to extract a comprehensive spatial features, followed by dimensionality reduction with the Multi- layer Perceptron (MLP) network. Finally, we design a bidi- rectional LSTM recurrent neural network to learn the multi- aspect features with further integrating the softmax classifier to achieve target recognition. Experimental results demonstrate that the proposed method can achieve 99.9% accuracy for 10-class recognition. Besides, its anti-noise and anti-confusion performance are also better than the conventional deep learning based methods.

Index terms— Synthetic aperture radar (SAR), automatic target recognition (ATR), multi-aspect SAR, long short-term memory (LSTM).

Due to the imaging characteristics of day-and-night and weather-independent, synthetic aperture radar (SAR) has been widely used for Earth remote sensing for more than 30 years, and has come to play a significant role in geographical survey, climate change research, environment and Earth system

monitoring, multi-dimensional mapping and other applications [1], [2]. With the evolving of SAR technologies, massive SAR images with abundant characteristics (e.g., high resolution, multi-aspect, multi-dimension, multi-polarization) have been provided for further applications in the Earth Observation. Different from the corresponding optical counterparts in many aspects, such as speckle noise, backscattering oriented pixel intensity representation, geometric distortions, high sensitivity to the target position and so on [3], SAR image is relatively more difficult to interpret. To bridge the SAR systems and their applications, SAR image automatic interpretation, especially automatic target recognition (ATR), has become an important research topic in surveillance, military tasks, etc., and has been studied continuously for more than 20 years [4].

In current high resolution SAR imaging era, the target feature is more of significance such that the ATR research is more focused on the target type identification. The SAR target recognition mainly consists of three steps: pre-processing, feature extraction, and classification. The pre-processing step includes filtering [5] and segmentation [6], which is employed to provide pure target region in SAR image. The feature ex- traction aims to reduce redundant information of target image while keeps accurate target representation. On the basis of the former two steps, the classification step tends to get the exact category information by the classifier. The feature selection and design of classifier are the most important parts of SAR ATR algorithms. In terms of feature extraction, the widely used features can be described as the static features, which are selected from an independent image, like geometric feature, scattering feature, polarization feature, transform domain fea- ture and so on [7], [8], [9], [10]. As for the classification, there are mainly four kinds of methodologies, including template matching, model-based method, neural networks, and machine learning [7], [11], [12], [13], [14]. The template matching method is practical, but is more dependent on the construction of template library. If the SAR sensor changes, the accuracy will drop rapidly. To avoid the construction of template library, the model-based method employs the high-fidelity model to represent the target feature instead of image template, but its adaptability is still limited. The traditional neural network has the disadvantages of poor learning performance, big training data requirement. Compared with these methods, the machine learning based ATR proves to be stable, efficient, and accurate, for example, support vector machine (SVM) [11]. Although SVM seeks to separate classes by learning an optimal decision

hyperplane that best separates training samples in a kernel- induced high-dimensional feature space, there are some issues hindering its applicability, e.g., the selection of kernel function parameters, classification speed.

Despite numerous ATR research over the past thirty years, very few ATR algorithms have been applied into practical applications. One of the most important reasons is the poor false-alarm performance, which is related with the feature rep- resentation and classifier. Intrinsically, SAR image feature is a space-varying scattering feature, which changes dramatically with the variations of aspect and depression angle. Usually, the depression angle is known in advance, the aspect angle information is highly demanded for ATR application. Corre- spondingly, some aspect estimation algorithms are proposed to handle this issue [15], [16]. Even so, the classifiers trained with a specific aspect interval may not perform well if the test image falls out of this interval. To alleviate this problem, the complexity of classification has to be increased by multiple classifier methods, such as ensemble classifiers on different aspect angle, majority-voting strategy and so on [17]. So this kind of multi-aspect joint recognition seems to be a solid method to decrease the false-alarm rate, on the other hand, it may put forward higher requirements for the data acquisition. In fact, we can often obtain the multi-aspect SAR images in practice, i.e., multiple airborne/UAV SAR joint observation in different aspect angles, single SAR observations along a curvilinear or circular orbit. Correspondingly, the collected images can cover multiple aspect angles or even all-aspect angles, bring the comprehensive representation for target scat- tering signature, and provide one possibility to improve the recognition performance.

With the richness of multi-aspect signature, the classifica- tion technology is also further developed with the concept of deep learning. In recent years, the emerging deep learning methods demonstrate their excellent target recognition capabil- ity, and have been widely applied in signal processing, image processing, and remote sensing [18], [19], [20], [21], [22], [23], [24]. Also, the deep learning based SAR image process- ing has become a hot topic and outperformed the traditional approaches. For a deep neural network, a large amount of data sets are required to train millions of network weights. Compared with limited SAR ATR data, SAR image data for classification can fairly well meet the requirement and has been effectively classified by deep convolutional autoencoder (CAE) [25], deep belief network (DBN) [26], restricted Boltz- mann machine (RBM) [27], convolutional neural networks (CNN) [28] and so on. As for the SAR ATR application, shallow CAE and CNN are employed to conduct MSTAR and TerraSAR-X data recognition with sparsely connected con- volution architectures and fine-tuning strategies, and achieve high recognition accuracy over 98% in type identification [29], [30], [31]. However, for configuration recognition and confuser rejection, the recognition accuracy is reduced to 87%. Therefore, the deep learning based SAR ATR still be worthy of further studying.

The key perspective of deep learning is that these layers of features are not designed by human engineers: they are learned from data using a general-purpose learning procedure

[32]. Different from natural image in data and feature repre- sentation, SAR image brings two problems for deep learning based ATR solution. Firstly, SAR imagery is essentially a coherent image indicating the coherent backscatter. Secondly, the learned multi-level features are still limited to the static features, which are extracted from an independent image. Therefore, there is a most straightforward thought to decrease the false-alarm rate by using the space-varying scattering feature from the multi-aspect image sequence instead of the static feature from a single image. The multi-aspect even all- aspect images may offer such opportunity to construct the image sequence, which are employed to extract space-varying backscattering feature. This aspect-dependence property in SAR image is especially notable due to the physical com- position of the target. Therefore, the sequential information that extracted from the multi-aspect images at a single target, is capable of offering the potential to substantially improve identification performance [33]. In the multi-aspect informa- tion based ATR, the hidden Markov models (HMMs) based methods are the mainstream solutions to model such sequential data [33], [34], [35]. Due to the fact that it is difficult to find a relationship between the backscattering and the HMM state, its application has some limitations, e.g., the HMM modeling is basically derived from the practical experience and is not so solid for other scenarios.

In this work, a novel ATR framework based on multi-aspect- aware bidirectional Long Short-Term Memory (LSTM) recur- rent neural networks (MA-BLSTM) is proposed to improve the recognition performance by exploiting the sequential features of multi-aspect views on a single target. LSTM recurrent neu- ral networks were originally introduced for sequence learning [36]. After strengthened by bidirectional design, bidirectional LSTM has complete, sequential information about all points before and after it [37]. For SAR ATR, these networks includ- ing recurrently connected cells learn the dependent features among multiple aspect images, then transfer the probabilistic inference to the next and previous aspect image units. Recent works have indicated that LSTM outperforms hidden Markov models (HMMs) in modeling the stochastic sequences [38], [39], which was dominated by HMMs in the early 2000s. It is predictable that LSTM is suitable for multi-aspect feature based SAR target recognition. For each aspect image, the global Gabor features and the three-patch local binary pattern (TPLBP) features are combined in different orientations to extract more comprehensive spatial information, which is employed for multiple aspect images and further constructs the multi-aspect features. Furthermore, a fully-connected Multi- layer Perceptron (MLP) network for feature dimensionality reduction is integrated with the bidirectional LSTM recurrent neural networks to realize a highly efficient SAR ATR. Com- pared to the state-of-the-art deep learning based ATR methods [29], [30], [31], we make the following contributions.

• We present an idea of deep learning based ATR method, that is, considering multi-aspect image sequence based joint recognition instead of single aspect static feature based isolated recognition, which can be applied to multi- polarization, multi-static, multi-channel, multi-band and

so on.

  • We introduce the bidirectional LSTM recurrent neuralnetwork to memorize the context information for multi- aspect sequence data for learning the space-varying scat- tering feature. The static multi-orientation spatial infor- mation with gray-scale and rotation invariant charac- teristics are extracted from each image as the single aspect features, and further construct the pure multi- aspect features by concatenating all the static features in the sequence.
  • We propose a novel ATR framework including single as- pect features extraction by combining Gabor and TPLBP features, supervised feature dimensionality reduction with the MLP, multi-aspect features learning with the bidi- rectional LSTM, and softmax classifier, which tends to be discriminative and solid, and further improve the recognition accuracy as high as 99.9%.The rest of this paper is organized as follows. Section II specifically introduces the proposed MA-BLSTM ATR framework. Then, the experimental results and analysis are presented in Section III. Finally, conclusions are drawn in Section IV.II. THE PROPOSED ATR APPROACH

    In SAR image formation, the multi-aspect signatures are integrated to form a single image, thereby losing some of the explicit aspect dependence [34]. Correspondingly, most mainstream ATR approaches only perform target classification based on a single view of the target. Due to the received signals from different targets may be similar at certain as- pects, the space-variant backscattering feature bring troubles for reliable target recognition. The current ATR methods, e.g., the machine learning or deep learning, mainly focus on the independent backscattering feature rather than continuous space-variant backscattering features. On the other hand, to meet the practical demand for ATR in limited training data and confusing environment, the space-variant scattering features, namely the multi-aspect features, may be more essential and solid than separate backscattering feature. Therefore, the theo- retical and practical demands motivate us to exploit the multi- aspect sequence features to further improve the recognition performance and adaptability.

    The proposed target recognition framework is illustrated in Fig.1, including five processing steps: constructing the multi- aspect image sequence, extracting the separate backscattering feature using Gabor filter and TPLBP approach, reducing the feature dimensionality with a fully-connected MLP network, learning the multi-aspect features with the LSTM recurrent neural networks, and determining the category for each target sample via the softmax classifier. 

By SAR

Synthetic Aperture Radar (SAR) or SAR Journal is an industry trade journal which tracks the worldwide SAR industry. We offer news, education, and insights to the SAR industry. We are operated, moderated and maintained by members of the SAR community.This profile is run by multiple moderators who all represent the SyntheticApertureRadar.com If you would like to submit news or have questions about a post please email us here: SyntheticApertureRadarmag@gmail.com and someone will get back to you.

Leave a Reply

Your email address will not be published. Required fields are marked *