一种改进CycleGAN的水下彩色图像增强方法

刘朝; 王红茹

doi:10.13433/j.cnki.1003-8728.20220162

一种改进CycleGAN的水下彩色图像增强方法

doi: 10.13433/j.cnki.1003-8728.20220162

刘朝^1,,
王红茹^2, ,

1.
江苏科技大学机械工程学院，江苏镇江　212003
2.
江苏科技大学江苏省船海机械先进制造及工艺重点实验室，江苏镇江　212003

基金项目: 国家重点研发计划项目（2018YFC0309100）

详细信息

作者简介:
刘朝（1994−），硕士研究生，研究方向为图像处理与机器视觉，374466828@qq.com

通讯作者:
王红茹，副教授，硕士生导师，wanghr@126.com

中图分类号: TP391.9
计量
- 文章访问数: 127
- HTML全文浏览量: 29
- PDF下载量: 23
- 被引次数: 0
出版历程
- 收稿日期: 2021-09-18
- 刊出日期: 2023-12-25

An Improved Underwater Color Image Enhancement Algorithm Based on CycleGAN

LIU Zhao^1
,,
WANG Hongru^{2
, ,}

1.
School of Mechanical Engineering, Jiangsu University of Science and Technology, Zhenjiang 212003, Jiangsu, China
2.
Key Laboratory of Advanced Manufacture and Process for Marine Mechanical Equipment Research Institute, Jiangsu University of Science and Technology, Zhenjiang 212003, Jiangsu, China

摘要

摘要: 针对基于深度学习方法的水下图像增强只考虑水下图像的RGB 颜色特征空间造成的增强效果不理想现象，本文在循环生成对抗网络的基础上改进了一种水下彩色图像增强算法。首先运用循环生成对抗网络在图像的RGB和HSV 颜色特征空间进行训练，将图像经过卷积网络下采样提取到的特征送入残差网络和扩展压缩模块，其中扩展压缩模块可以调整图像RGB 和HSV 通道的权重。预训练好的生成对抗网络作用在成对的水下降质图像与增强后的图像进行监督训练，采用特征融合网络将对抗生成网络输出的RGB 和HSV 六通道图像融合成RGB三通道图像。实验结果表明，该方法能够有效结合图像的RGB和HSV 空间的特征信息，提升水下图像的对比度和亮度，校正水下图像的颜色偏差。
- 水下彩色图像增强 /
- 循环对抗生成网络 /
- 卷积层压缩扩展 /
- 颜色空间融合
Abstract: The underwater image enhancement based on the deep learning method considers only the RGB feature space, therefore the image enhancement effect is unsatisfactory. To cope with this problem, this paper proposed an improved underwater color image enhancement algorithm based on the cyclic generative adversarial network (CycleGAN). Both RGB and HSV color spaces of an image are used to train the CycleGAN. The features down-sampled from the CycleGAN are input into the residual network and the expansion compression module to extract useful features. The weights of RGB and HSV spaces are adaptively adjusted in the expansion and compression module. The pre-trained CycleGAN acts on the paired water degraded image and the enhanced image for weakly supervised training. The feature fusion network is adopted to fuse the output of the CycleGAN into three channels of a new RGB image. The experimental results show that the algorithm can effectively combine the feature information on both RGB and HSV spaces, improves the contrast and brightness of the underwater image and corrects its color deviation.
- underwater color image enhancement /
- cyclic generative adversarial network /
- convolution layer expansion and compression /
- color space fusion

HTML全文

图 1 本文算法流程

Figure 1. Procedures of the Proposed Algorithm

下载: 全尺寸图片幻灯片

图 2 循环生成对抗网络结构

Figure 2. The Structure of the Cycle GAN

下载: 全尺寸图片幻灯片

图 3 生成网络结构

Figure 3. The Architeture of Generative Network

下载: 全尺寸图片幻灯片

图 4 残差和压缩扩展模块

Figure 4. Residual and Compression Expansion Module

下载: 全尺寸图片幻灯片

图 5 判别器网络结构

Figure 5. The Architeture of Discriminator Network

下载: 全尺寸图片幻灯片

图 6 特征融合网络

Figure 6. The Architeture of Feature Fuson Network

下载: 全尺寸图片幻灯片

图 7 本文算法和其他算法增强效果

Figure 7. Experimental Results of the Different Image Enhancement Algorithms

下载: 全尺寸图片幻灯片

表 1 增强图像的指标对比

Table 1. Performance parameters for enhanced images with different image enhancing algorithms

方法	MSE/10³	PSNR/dB	SSIM
Retinex-based	1.2924	17.0168	0.6071
Dense GAN	1.7298	15.7058	0.521
Water Cycle GAN	1.2152	17.2843	0.4426
Fusion-based	1.128	17.6077	0.7721
Ours	1.1681	18.4355	0.6808

下载: 导出CSV

参考文献(15)

[1]	WANG Y, ZHANG J, CAO Y, et al. A deep CNN method for underwater image enhancement[C]//2017 IEEE International Conference on Image Processing (ICIP). Beijing: IEEE, 2017: 1382-1386.
[2]	LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29: 4376-4389. doi: 10.1109/TIP.2019.2955241
[3]	LI J, SKINNER K A, EUSTICE R M, et al. WaterGAN: unsupervised generative network to enable real-time color correction of monocular underwater images[J]. IEEE Robotics and Automation Letters, 2018, 3(1): 387-394.
[4]	GUO Y C, LI H Y, ZHUANG P X. Underwater image enhancement using a multiscale dense generative adversarial network[J]. IEEE Journal of Oceanic Engineering, 2020, 45(3): 862-870. doi: 10.1109/JOE.2019.2911447
[5]	LI C Y, GUO J C, GUO C L. Emerging from water: underwater image color correction based on weakly supervised color transfer[J]. IEEE Signal Processing Letters, 2018, 25(3): 323-327. doi: 10.1109/LSP.2018.2792050
[6]	MEJJATI Y A, RICHARDT C, TOMPKIN J, et al. Unsupervised attention-guided image-to-image translation[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montréal: Curran Associates Inc. , 2018: 3697-3707.
[7]	李庆忠, 白文秀, 牛炯. 基于改进CycleGAN的水下图像颜色校正与增强[J]. 自动化学报, 2023, 49(4): 820-829. LI Q Z, BAI W X, NIU J. Underwater image color correction and enhancement based on improved cycle-consistent generative adversarial networks[J]. Acta Automatica Sinica, 2023, 49(4): 820-829. (in Chinese)
[8]	杨亚绒, 李恒, 赵磊, 等. 改进的同态滤波与多尺度融合的水下图像增强[J]. 机械科学与技术, 2022, 41(8): 1231-1239. doi: 10.13433/j.cnki.1003-8728.20200455 YANG Y R, LI H, ZHAO L, et al. Improved homomorphic filtering and multi-scale fusion method for underwater image enhancement[J]. Mechanical Science and Technology for Aerospace Engineering, 2022, 41(8): 1231-1239. (in Chinese) doi: 10.13433/j.cnki.1003-8728.20200455
[9]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal: MIT Press, 2014: 2672-2680.
[10]	ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2242-2251.
[11]	ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[J]. Machine Learning, 2017, 3: 1701-1732.
[12]	EDWARDS D A. On the Kantorovich-Rubinstein theorem[J]. Expositiones Mathematicae, 2011, 29(4): 387-398. doi: 10.1016/j.exmath.2011.06.005
[13]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large scale image recognition[J]. Computer Science, 2014, 1(6): 1-14.
[14]	FU X Y, ZHUANG P X, HUANG Y, et al. A retinex-based enhancing approach for single underwater image[C]//2014 IEEE International Conference on Image Processing (ICIP). Paris: IEEE, 2014: 4572-4576.
[15]	ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 81-88.