SRGAN Enhancement through Autoencoder-Pretrained U-Net with Residual Blocks for Improved Image Super-Resolution

Document Type : Research Paper

Authors

1 School of Electronics and Information, Northwestern Polytechnical University, Xi'an, China.

2 School of Electronics & Information, Northwestern Polytechnical University, Xi’an, Shaanxi, China.

3 School of Software Engineering, Northwest Polytechnic University, Xi'an, China.

4 School of Electronics & Information, Northwest Polytechnic University, Xi'an, China.

5 School of Aeronautics, Northwestern Polytechnical University, Xi'an, China.

6 School of Aeronautics, Northwestern Polytechnical University, Xi’an, Shaanxi, China.

Abstract

Super-resolution is a crucial task in image processing, enhancing the resolution of low-quality images for applications such as surveillance, remote sensing, and autonomous systems. Traditional methods often struggle to preserve fine details, leading to artifacts and reduced visual fidelity. This study introduces the Pretrained RU-SRGAN, an enhanced Super-Resolution Generative Adversarial Network (SRGAN) that incorporates U-Net architecture, residual learning, and autoencoder pretraining to improve both image quality and computational efficiency, particularly in resource-constrained environments like UAVs. The goal of this research is to evaluate how these architectural modifications can enhance super-resolution performance with limited data. Autoencoder pretraining enables the generator to leverage learned features from low-resolution images, accelerating convergence and improving high-resolution reconstructions. Experimental results show that Pretrained RU-SRGAN outperforms baseline models, achieving a PSNR of 25.7 dB and an SSIM of 0.83. These results highlight the model's ability to preserve fine details and structural integrity, making it particularly effective for real-time image enhancement in UAV applications. The Pretrained RU-SRGAN provides a robust solution for super-resolution tasks, balancing high-quality image reconstruction with computational efficiency, and is well-suited for practical deployment in dynamic, resource-limited environments.

Keywords

Main Subjects

References

[1]           Y. Wei, Y. Li, Z. Ding, Y. Wang, T. Zeng, and T. Long, “SAR Parametric Super-Resolution Image Reconstruction Methods Based on ADMM and Deep Neural Network,” IEEE Trans. Geosci. Remote Sensing, vol. 59, no. 12, pp. 10197–10212, Dec. 2021, doi: 10.1109/TGRS.2021.3052793.
[2]           D. Khaledyan, A. Amirany, K. Jafari, M. H. Moaiyeri, A. Z. Khuzani, and N. Mashhadi, “Low-Cost Implementation of Bilinear and Bicubic Image Interpolation for Real-Time Image Super-Resolution,” in 2020 IEEE Global Humanitarian Technology Conference (GHTC), Oct. 2020, pp. 1–5. doi: 10.1109/GHTC46280.2020.9342625.
[3]           A. M. John, K. Khanna, R. R. Prasad, and L. G. Pillai, “A Review on Application of Fourier Transform in Image Restoration,” in 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), Palladam, India: IEEE, Oct. 2020, pp. 389–397. doi: 10.1109/I-SMAC49090.2020.9243510.
[4]           Y. Yang, Z. Su, and L. Sun, “Medical image enhancement algorithm based on wavelet transform,” Electronics Letters, vol. 46, no. 2, pp. 120–121, Jan. 2010, doi: 10.1049/el.2010.2063.
[5]           W. Ren et al., “Wavelet Transform Based Network for Spectral Super-Resolution,” in IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, Pasadena, CA, USA: IEEE, Jul. 2023, pp. 7551–7554. doi: 10.1109/IGARSS52108.2023.10281411.
[6]           H. Chang, D.-Y. Yeung, and Y. Xiong, “Super-resolution through neighbor embedding,” in Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., Jun. 2004, p. I–I. doi: 10.1109/CVPR.2004.1315043.
[7]           J. Yang, J. Wright, T. S. Huang, and Y. Ma, “Image Super-Resolution Via Sparse Representation,” IEEE Transactions on Image Processing, vol. 19, no. 11, pp. 2861–2873, Nov. 2010, doi: 10.1109/TIP.2010.2050625.
[8]           S. Tang and N. Zhou, “Local Similarity Regularized Sparse Representation for Hyperspectral Image Super-Resolution,” in IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia: IEEE, Jul. 2018, pp. 5120–5123. doi: 10.1109/IGARSS.2018.8518168.
[9]           H. Li, K.-M. Lam, and M. Wang, “Image super-resolution via feature-augmented random forest,” Signal Processing: Image Communication, vol. 72, pp. 25–34, Mar. 2019, doi: 10.1016/j.image.2018.12.001.
[10]         C. Dong, C. C. Loy, K. He, and X. Tang, “Image Super-Resolution Using Deep Convolutional Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295–307, Feb. 2016, doi: 10.1109/TPAMI.2015.2439281.
[11]         J. Kim, J. K. Lee, and K. M. Lee, “Accurate Image Super-Resolution Using Very Deep Convolutional Networks,” presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1646–1654.
[12]         A. Rouhbakhshmeghrazi, B. Li, W. Iqbal, and G. Alizadeh, “Super-Resolution Reconstruction of UAV Images with GANs: Achievements and Challenges,” in 2024 International Conference on Cyber-Physical Social Intelligence (ICCSI), Doha, Qatar: IEEE, Nov. 2024, pp. 1–6. doi: 10.1109/ICCSI62669.2024.10799467.
[13]         C. Mollière, J. Gottfriedsen, M. Langer, P. Massaro, C. Soraruf, and M. Schubert, “Multi-Spectral Super-Resolution of Thermal Infrared Data Products for Urban Heat Applications,” in IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, Pasadena, CA, USA: IEEE, Jul. 2023, pp. 4919–4922. doi: 10.1109/IGARSS52108.2023.10283339.
[14]         W. Yang, Z. Ma, and Y. Shi, “SAR Image Super-Resolution based on Artificial Intelligence,” in IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia: IEEE, Jul. 2022, pp. 4643–4646. doi: 10.1109/IGARSS46834.2022.9884456.
[15]         J. Ferdousi, S. I. Lincoln, Md. K. Alom, and Md. Foysal, “A deep learning approach for white blood cells image generation and classification using SRGAN and VGG19,” Telematics and Informatics Reports, vol. 16, p. 100163, Dec. 2024, doi: 10.1016/j.teler.2024.100163.
[16]         M. Ullah, A. Hamza, I. Ahmad Taj, and M. Tahir, “Low Resolution Face Recognition using Enhanced SRGAN Generated Images,” in 2021 16th International Conference on Emerging Technologies (ICET), Islamabad, Pakistan: IEEE, Dec. 2021, pp. 1–6. doi: 10.1109/ICET54505.2021.9689885.
[17]         Q. Zhu, X. Fan, Y. Zhong, Q. Guan, L. Zhang, and D. Li, “Super Resolution Generative Adversarial Network Based Image Augmentation for Scene Classification of Remote Sensing Images,” in IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA: IEEE, Sep. 2020, pp. 573–576. doi: 10.1109/IGARSS39084.2020.9324043.
[18]         Y. Wang, H. Wu, L. Shuai, C. Peng, and Z. Yang, “Detection of plane in remote sensing images using super-resolution,” PLOS ONE.
[19]         D. Rathgamage Don, R. Aygun, and M. Karakaya, “A Multistage Framework for Detection of Very Small Objects,” in Proceedings of the 2023 6th International Conference on Machine Vision and Applications, Singapore Singapore: ACM, Mar. 2023, pp. 9–14. doi: 10.1145/3589572.3589574.
[20]         Sub-r-paChayanon, FanMing-Zhong, and ChenRung-Ching, “Super-resolution for traffic signs: a comparative analysis of SRGAN and ESRGAN performance,” IET Conference Proceedings, Dec. 2024, doi: 10.1049/icp.2024.4162.
[21]         P. Nandal, S. Pahal, A. Khanna, and P. Rogério Pinheiro, “Super-Resolution of Medical Images Using Real ESRGAN,” IEEE Access, vol. 12, pp. 176155–176170, 2024, doi: 10.1109/ACCESS.2024.3497002.
[22]         C. Ledig et al., “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” presented at the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4681–4690.
[23]         Y. Wang, Z. Xu, X. Wang, J. He, and X. Zhao, “An Improved SRGAN-Based Deblurring Model for Multiple Blurriness in Microscopy,” IEEE Transactions on Instrumentation and Measurement, vol. 73, pp. 1–13, 2024, doi: 10.1109/TIM.2024.3470059.
[24]         “SRGAN-LSTM-Based Celestial Spectral Velocimetry Compensation Method With Solar Activity Images | IEEE Journals & Magazine | IEEE Xplore.” Accessed: Nov. 15, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/10526305
[25]         S. Madhav, T. M. Nandhika, and M. K. Kavitha Devi, “Super Resolution of Medical Images Using SRGAN,” in 2024 Second International Conference on Emerging Trends in Information Technology and Engineering (ICETITE), Feb. 2024, pp. 1–6. doi: 10.1109/ic-ETITE58242.2024.10493588.
[26]         R. Duggal and A. Gupta, “P-TELU: Parametric Tan Hyperbolic Linear Unit Activation for Deep Neural Networks,” presented at the Proceedings of the IEEE International Conference on Computer Vision Workshops, 2017, pp. 974–978. Accessed: Nov. 15, 2024. [Online]. Available: https://openaccess.thecvf.com/content_ICCV_2017_workshops/w18/html/Duggal_P-TELU_Parametric_Tan_ICCV_2017_paper.html
[27]         S. Ji et al., “Super-resolution reconstruction of variable length infrared image sequences based on convolutional neural networks and pixel shuffling,” in 2024 International Conference on Optoelectronic Information and Optical Engineering (OIOE 2024), H. Bin Ahmad and M. Jiang, Eds., Kunming, China: SPIE, Jun. 2024, p. 10. doi: 10.1117/12.3030372.
[28]         K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” Apr. 10, 2015, arXiv: arXiv:1409.1556. doi: 10.48550/arXiv.1409.1556.
[29]         X. Ren and X. Li, “Research on image super-resolution based on attention mechanism and multi-scale,” J. Phys.: Conf. Ser., vol. 1792, no. 1, p. 012025, Feb. 2021, doi: 10.1088/1742-6596/1792/1/012025.
[30]         Y. Wang, “Single Image Super-Resolution with U-Net Generative Adversarial Networks,” in 2021 IEEE 4th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Chongqing, China: IEEE, Jun. 2021, pp. 1835–1840. doi: 10.1109/IMCEC51613.2021.9482317.
[31]         P. S. Hrishikesh, D. Puthussery, K. A. Akhil, and C. V. Jiji, “Relativistic GAN using Receptive Field Block for Single Image Super-Resolution with improved Perceptual Quality,” in 2023 11th International Symposium on Electronic Systems Devices and Computing (ESDC), May 2023, pp. 1–6. doi: 10.1109/ESDC56251.2023.10149876.
[32]         Z. Wu, “Research on Image Super-Resolution Using Attention Mechanisms based on Super-Resolution Generative Adversarial Network,” FCIS, vol. 5, no. 2, pp. 131–134, Sep. 2023, doi: 10.54097/fcis.v5i2.13142.
[33]         B. Sun, B. Chen, Y. Tian, and W. Chen, “TESRGAN: Transformer Enhanced Super-Resolution Generative Adversarial Networks,” in 2024 4th International Conference on Neural Networks, Information and Communication (NNICE), Guangzhou, China: IEEE, Jan. 2024, pp. 137–141. doi: 10.1109/NNICE61279.2024.10498773.
[34]         M. Hasan, M. Vijay, S. Sharanyaa, and V. S. D. Tejaswi, “ENSEMBLE MODEL WITH IMPROVED U-NET-BASED SEGMENTATION FOR LEUKEMIA DETECTION,” Biomed. Eng. Appl. Basis Commun., vol. 36, no. 03, p. 2450011, Jun. 2024, doi: 10.4015/S101623722450011X.
[35]         A. Rouhbakhshmeghrazi, B. Li, and W. Iqbal, “Color Image Segmentation of Dental Caries Using U-Net Enhanced with Residual Blocks and Attention Mechanisms,” in 2024 International Conference on Cyber-Physical Social Intelligence (ICCSI), Doha, Qatar: IEEE, Nov. 2024, pp. 1–6. doi: 10.1109/ICCSI62669.2024.10799395.
[36]         H.-H. Chang, S.-J. Yeh, M.-C. Chiang, and S.-T. Hsieh, “RU-Net: skull stripping in rat brain MR images after ischemic stroke with rat U-Net,” BMC Med Imaging, vol. 23, no. 1, p. 44, Mar. 2023, doi: 10.1186/s12880-023-00994-8.
[37]         S. Leclerc et al., “RU-Net: A refining segmentation network for 2D echocardiography,” in 2019 IEEE International Ultrasonics Symposium (IUS), Glasgow, United Kingdom: IEEE, Oct. 2019, pp. 1160–1163. doi: 10.1109/ULTSYM.2019.8926158.
[38]         J. Zhang et al., “Multispectral Drone Imagery and SRGAN for Rapid Phenotypic Mapping of Individual Chinese Cabbage Plants,” Plant Phenomics, vol. 2022, p. 0007, 2022, doi: 10.34133/plantphenomics.0007.
[39]         S. Nasrin, M. Z. Alom, R. Burada, T. M. Taha, and V. K. Asari, “Medical Image Denoising with Recurrent Residual U-Net (R2U-Net) base Auto-Encoder,” in 2019 IEEE National Aerospace and Electronics Conference (NAECON), Dayton, OH, USA: IEEE, Jul. 2019, pp. 345–350. doi: 10.1109/NAECON46414.2019.9057834.
[40]         A. K. Kakumani, L. P. Sree, C. S. Krishna, G. Uppalapati, G. S. S. Pavithra, and S. Harshini, “Semantic Segmentation of Cells in Microscopy Images via Pretrained Autoencoder and Attention U-Net,” in 2022 International Conference on Machine Learning, Computer Systems and Security (MLCSS), Bhubaneswar, India: IEEE, Aug. 2022, pp. 94–99. doi: 10.1109/MLCSS57186.2022.00025.
[41]         K. N. Alkhamaiseh, J. L. Grantner, S. Shebrain, and I. Abdel-Qader, “Towards reliable hepatocytic anatomy segmentation in laparoscopic cholecystectomy using U-Net with Auto-Encoder,” Surg Endosc, vol. 37, no. 9, pp. 7358–7369, Sep. 2023, doi: 10.1007/s00464-023-10306-4.
[42]         M. Z. Alom, M. Hasan, C. Yakopcic, T. M. Taha, and V. K. Asari, “Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation,” May 29, 2018, arXiv: arXiv:1802.06955. doi: 10.48550/arXiv.1802.06955.
[43]         J. Zhang, Z. Jiang, J. Dong, Y. Hou, and B. Liu, “Attention Gate ResU-Net for Automatic MRI Brain Tumor Segmentation,” IEEE Access, vol. 8, pp. 58533–58545, 2020, doi: 10.1109/ACCESS.2020.2983075.
 

Files

History

  • Receive Date: 24 January 2025
  • Revise Date: 03 March 2025
  • Accept Date: 16 April 2025

Share

How to cite

Statistics