|
|
论文:2021,Vol:39,Issue(3):641-649 |
|
|
引用本文： |
|
|
宋佰霖, 许华, 蒋磊, 饶宁. 一种基于深度强化学习的通信抗干扰智能决策方法[J]. 西北工业大学学报 |
|
|
SONG Bailin, XU Hua, JIANG Lei, RAO Ning. An intelligent decision-making method for anti-jamming communication based on deep reinforcement learning[J]. Northwestern polytechnical university |
|
|
|
|
|
|
|
| 一种基于深度强化学习的通信抗干扰智能决策方法 |
|
| 宋佰霖, 许华, 蒋磊, 饶宁 |
|
| 空军工程大学 信息与导航学院, 陕西 西安 710077 |
| 摘要: |
| 为解决战场通信智能抗干扰决策问题，设计了一种基于深度强化学习的通信抗干扰决策方法。该方法在DQN算法架构下引入经验回放和基于爬山策略（PHC）的动态ε机制，提出动态ε-DQN智能决策算法，该算法能够根据决策网络状态更优地选择ε值，提高收敛速度和决策成功率。在决策过程中，对所有通信频率是否存在干扰信号进行检测，将结果作为干扰判别信息输入决策算法，使算法可在无先验干扰信息条件下智能决策通信频率，在尽量保证通信不中断的前提下，有效躲避干扰。实验结果表明，所提方法适应多种通信模型，决策速度较快，算法收敛后的平均成功率可达95%以上，较已有方法具有较大优势。 |
| 关键词:
通信抗干扰
智能决策
深度强化学习
|
|
| An intelligent decision-making method for anti-jamming communication based on deep reinforcement learning |
|
| SONG Bailin, XU Hua, JIANG Lei, RAO Ning |
|
| Information and Navigation College, Air Force Engineering University, Xi'an 710077, China |
| Abstract: |
| In order to solve the problem of intelligent anti-jamming decision-making in battlefield communication, this paper designs an intelligent decision-making method for communication anti-jamming based on deep reinforcement learning. Introducing experience replay and dynamic epsilon mechanism based on PHC under the framework of DQN algorithm, a dynamic epsilon-DQN intelligent decision-making method is proposed. The algorithm can better select the value of epsilon according to the state of the decision network and improve the convergence speed and decision success rate. During the decision-making process, the jamming signals of all communication frequencies are detected, and the results are input into the decision-making algorithm as jamming discriminant information, so that we can effectively avoid being jammed under the condition of no prior jamming information. The experimental results show that the proposed method adapts to various communication models, has a fast decision-making speed, and the average success rate of the convergent algorithm can reach more than 95%, which has a great advantage over the existing decision-making methods. |
| Key words:
anti-jamming communication
intelligent decision-making
deep reinforcement learning
|
|
| 收稿日期: 2020-08-27
修回日期:
|
| DOI: 10.1051/jnwpu/20213930641 |
| 通讯作者:
Email： |
| 作者简介: 宋佰霖(1997-),空军工程大学硕士研究生,主要从事通信对抗、通信抗干扰研究。
|
|
| 相关功能 |
|
|
|
|
| 作者相关文章 |
|
| 宋佰霖 在本刊中的所有文章 |
| 许华 在本刊中的所有文章 |
| 蒋磊 在本刊中的所有文章 |
| 饶宁 在本刊中的所有文章 |
|
|
|
|
|
|
|
|
参考文献: |
|
|
[1] VARDHAN S. Information jamming in electronic warfare:operational requirements and techniques[C]//International Conference on Electronics, 2015:49-54
[2] XIAO L, CHEN T, LIU J, et al. Anti-jamming transmission stackelberg game with observation errors[J]. IEEE Communications Letters, 2015, 19(6):949-952
[3] XIAO L, LIU J, LI Q, et al. User-centric view of jamming games in cognitive radio networks[J]. IEEE Trans on Information Forensics & Security, 2015,10(12):2578-2590
[4] XIA T, PINY R, QINGH D, et al. Securing wireless transmission against reactive jamming:a stackelberg game framework[C]//IEEE Global Communications Conference, 2016:1-6
[5] JIA L, XU Y, SUN, et al. A game-theoretic learning approach for anti-jamming dynamic spectrum access in dense wireless networks[J]. IEEE Trans on Vehicular Technology, 2018,68(2):1646-1656
[6] YANG D, XUE G, ZHANG J, et al. Coping with a smart jammer in wireless networks:a stackelberg game approach[J]. IEEE Trans on Wireless Communications, 2013, 12(8):4038-4047
[7] JIA L, YAO F, SUN, et al. A hierarchical learning solution for anti-jamming stackelberg game with discrete power strategies[J]. IEEE Wireless Communications Letters, 2017, 6(6):818-821
[8] NASIR Y S, GUO D. Multi-agent deep reinforcement learning for dynamic power allocation in wireless networks[J]. IEEE Journal on Selected Areas in Communications, 2019, 37(10):2239-2250
[9] LIU X, XU Y, JIA L, et al. Anti-jamming communications using spectrum waterfall:a deep reinforcement learning approach[J]. IEEE Communications Letters, 2017, 22(5):998-1001
[10] LIU S, XU Y, CHEN X, et al. Pattern-aware intelligent anti-jamming communication:a sequential deep reinforcement learning approach[J]. IEEE Access, 2019, 7:169204-169216
[11] YAO F, JIA L. A collaborative multi-agent reinforcement learning anti-jamming algorithm in wireless networks[J]. IEEE Wireless Communication Letters, 2019, 8(4):1024-1027
[12] MACHUZAK S, JAYAWEERA S K. Reinforcement learning based anti-jamming with wideband autonomous cognitive radios[C]//IEEE/CIC International Conference on Communications, 2016:1-5
[13] AREF M A, JAYAWEERA S K, MACHUZAK S. Multi-agent reinforcement learning based cognitive anti-jamming[C]//2017 IEEE Wireless Communications and Networking Conference(WCNC), 2017:1-6
[14] LOPEZ-BENITEZ M, CASADEVALL F. Improved energy detection spectrum sensing for cognitive radio[J]. IET Communications, 2012, 6(8):785-796
[15] ZHANG Lin, TAN Junjie, LIANG Yingchang, et al. Deep reinforcement learning-based modulation and coding scheme selection in cognitive heterogeneous networks[J]. IEEE Trans on Wireless Communications, 2019, 18(6):3281-3294
[16] HE Ying, ZHANG Zheng, YU Richard, et al. Deep-reinforcement-learning-based optimization for cache-enabled opportunistic interference alignment wireless networks[J]. IEEE Trans on Vehicular Technology, 2017, 66(11):10433-10445
[17] YANG Bo, LEI, et al. A novel multi-agent decentralized win or learn fast policy hill-climbing with eligibility trace algorithm for smart generation control of interconnected complex power grids[J]. Elsevier Energy Conversion & Management, 2015, 103:82-93
[18] MNIH V, KAVUKCUOGLU K, SILVER D. et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518:529-533
[19] CHI Jin, ZEYUAN Allenzhu, SEBASTIEN Bubeck, et al. Is Q-learning Provably Efficient?[C]//32nd International Conference on Neural Information Processing Systems, 2018:4868-4878 |
|
|
|
|
|
|
|
