Abstract

Convolutional neural networks (CNNs) have made tremendous progress in solving many challenging problems. Good activation functions can improve the performance of CNNs. The existing activation functions exhibit inconsistent per-formance gains across different training settings, models, datasets and tasks. To solve this problem, we propose a general smoothed approximation for the maximum function maxðxi; axiÞ using the linear combination of the smoothed rectified linear unit and the identity function. And we use exponential moving average to training the negative slope in this smoothed approximation. To validate the effectiveness of our approach, we also present a smoothed approximation case named leaky power function linear unit (LPFLU) to compare with the current state-of-the-art activation functions. Experimental results demonstrate that our LPFLU outperforms the existing state-of-the-art activation functions in improved robustness across different training settings, models, datasets and tasks.

Recommended citation: Cheng Zha, Weidong Min*, Qing Han, Wei Li, Xin Xiong, Qi Wang, Meng Zhu. SAR ship localization method with denoising and feature refinement. Engineering Applications of Artificial Intelligence, 2023, 123: 1-13. DOI: 10.1016/j.engappai.2023.106444.
Download Paper | Download Slides