[1]高东祥,张 洪,修伟杰,等.基于深度强化学习改进的Smith预估器温度控制[J].工业仪表与自动化装置,2024,(03):54-59+99.[doi:DOI:10.19950/j.cnki.CN61-1121/TH.2024.03.010]
　GAO Dongxiang,ZHANG Hong,XIU Weijie,et al.Improved smith predictor temperature control based on deep reinforcement learning[J].Industrial Instrumentation & Automation,2024,(03):54-59+99.[doi:DOI:10.19950/j.cnki.CN61-1121/TH.2024.03.010]
点击复制

基于深度强化学习改进的Smith预估器温度控制()

《工业仪表与自动化装置》[ISSN:1000-0682/CN:61-1121/TH]

卷:
期数:
2024年03期
页码:
54-59+99
栏目:
出版日期:
2024-06-15

文章信息/Info

Title:
Improved smith predictor temperature control based on deep reinforcement learning
文章编号:
1000-0682（2024）03-0054-06
作者:
高东祥1; 张 洪1; 2; 修伟杰1; 张 林3
（1.江南大学机械工程学院，江苏 无锡 214122；2.江苏省食品先进制造装备技术重点实验室，江苏 无锡 214122；3.江苏惠霖环保科技有限公司，江苏 无锡 214122）
Author(s):
GAO Dongxiang1; ZHANG Hong1; 2; XIU Weijie1; ZHANG Lin3
(1. School of Mechanical Engineering, Jiangnan University, Jiangsu Wuxi 214122, China; 2. Jiangsu Provincial Key Laboratory of Advanced Food Manufacturing Equipment Technology, Jiangsu Wuxi 214122, China;3. Jiangsu Huilin Environmental Protection Technology Co., Ltd., Jiangsu Wuxi 214122, China）
关键词:
温度控制; Smith预估器; 强化学习; 神经网络; 时滞系统
Keywords:
temperature control;  Smith estimator;  reinforcement learning;  neural network;  time lag system
分类号:
TP23
DOI:
DOI:10.19950/j.cnki.CN61-1121/TH.2024.03.010
文献标志码:
A
摘要:
针对牛粪发酵过程具有惯性大、时滞性、参数变化非线性的特点，提出了一种基于深度确定性策略梯度（DDPG）改进Smith模糊PID控制器的温度控制方法。首先，针对传统模糊PID不能对时滞系统有效控制的问题，建立Smith预估模糊PID控制器。其次，使用DDPG算法改进温度控制器，对设计的智能体进行离线训练。最后，通过仿真对所设计控制器进行实验验证。实验结果表明：DDPG改进的Smith模糊PID控制器能有效消除时滞对温度控制的影响，减少超调量和误差，且能避免被控对象参数随时间变化产生动态偏离时造成的系统不稳定。
Abstract:
Aiming at the characteristics of large inertia, time lag and nonlinear parameter change in the fermentation process of cow manure, a temperature control method based on deep deterministic strategy gradient to improve Smith fuzzy PID is proposed. Firstly, to address the issue that traditional fuzzy PID cannot effectively control time-delay systems, a Smith predictive fuzzy PID controller is established. Secondly, use the DDPG algorithm to improve the temperature controller and conduct offline training on the designed intelligent agent. Finally, the designed controller is experimentally validated through simulation. The results show that the Smith PID controller improved by DDPG can eliminate the influence of time delay on temperature control, reduce overshoot and errors, and avoid system instability caused by dynamic deviation of controlled object parameters over time.

参考文献/References:

[1] 谢光辉，包维卿，刘继军,等. 中国畜禽粪便资源研究现状述评[J].中国农业大学学报, 2018,23: 75-87.

[2] MICHEL F, O’Neill T, RYNK R, et al. Chapter 7 - Contained and in-vessel composting methods and methods summary[C]//R. RYNK. The Composting Handbook.Academic Press,2022:271-305.
[3] 苏佳佳, 李凤鸣, 李伟,等. 畜禽粪污堆肥技术装备发展现状与趋势[J].农业工程, 2022,12: 12-18.
[4] 刘泽龙, 王选, 曹玉博,等. 立式筒仓反应器堆肥技术工艺优化研究[J].中国生态农业学报(中英文), 2020,28: 1979-1989.
[5] AJMAL M, SHI A, AWAIS M, et al. Ultra-high temperature aerobic fermentation pretreatment composting: Parameters optimization, mechanisms and compost quality assessment [J]. Journal of Environmental Chemical Engineering, 2021,9: 105453.
[6] MU D, QU F, ZHU Z, et al. Effect of Maillard reaction on the formation of humic acid during thermophilic phase of aerobic fermentation [J]. Bioresource Technology, 2022,357: 127362.
[7] 刘浩,沈星,曲浩丽,等.粒子群优化的神经网络PID沼气干发酵温度控制[J].计算机工程与设计,2017,38(03):784-788.
[8] 贺自名, 牛江川, 张静. 基于Smith变论域模糊自适应PID蒸发源温度控制[J]. 控制工程, 2021,28: 1308-1314.
[9] 张皓, 高瑜翔. 前馈反馈Smith预估模糊PID组合温度控制算法[J]. 中国测试, 2020,46: 132-138+168.
[10] 郑仰东. 采用Smith预估器模型的时滞系统自适应控制[J]. 控制理论与应用, 2021,38: 416-424.
[11] 周志勇,莫非,赵凯,等.基于PPO的自适应PID控制算法研究[J/OL].系统仿真学报,1-8 https://doi.org/10.16182/j.issn1004731x.joss.23-0137.
[12] LAWRENCE N P, FORBES M G, LOEWEN P D, et al. Deep reinforcement learning with shallow controllers: An experimental application to PID tuning [J]. Control Engineering Practice, 2022,121: 105046.
[13] 吴敏, 王晓璐, 姜玉东,等. 深度确定性策略梯度与模糊PID的协同温度控制[J]. 控制理论与应用, 2022,39: 2358-2365.
[14] BU Q, CAI J, LIU Y, et al. The effect of fuzzy PID temperature control on thermal behavior analysis and kinetics study of biomass microwave pyrolysis[J]. Journal of Analytical and Applied Pyrolysis, 2021,158: 105176.
[15] 孙军, 张典, 黄青山,等. 基于串级模糊自适应PID的蒸发器温度控制系统设计[J]. 过程工程学报, 2023,23(09):1290-1299.
[16] 荆中亚,陈 为.基于STM32的密炼机自整定PID温控系统设计[J].电子设计工程,2024,32(5):51-55.
[17] 杨智显,胡安杰,刘 东.基于STM32的冷暖两联供控制系统设计[J].电子设计工程,2023,31(21):1-6.
[18] Al-Dhaifallah M. Fuzzy fractional-order PID control for heat exchanger[J]. Alexandria Engineering Journal, 2023,63: 11-16.
[19] OLIVEIRA F S S, SOUZA F O, PALHARES R M. PID Tuning for Time-Varying Delay Systems Based on Modified Smith Predictor 11This work has been supported by the Brazilian agencies CAPES, CNPq, and FAPEMIG[J]. IFAC-PapersOnLine, 2017,50: 1269-1274.
[20] LEE D, LEE M, SUNG S, et al. Robust PID tuning for Smith predictor in the presence of model uncertainty[J]. Journal of Process Control, 1999,9: 79-85.[21] SHI S, LIU Q. Deep Deterministic Policy Gradient With Classified Experience Replay[J]. Acta Automatica Sinica, 2022,48: 1816-1823.
[22] SIRASKAR R. Reinforcement learning for control of valves[J]. Machine Learning with Applications, 2021,4: 100030.
[23] XU J, ZHANG H, QIU J. A deep deterministic policy gradient algorithm based on averaged state-action estimation [J]. Computers and Electrical Engineering, 2022,101: 108015.
[24] KANG J-L, MIRZARI S, ZHOU J-A. Robust control and training risk reduction for boiler level control using two-stage training deep deterministic policy gradient[J]. Journal of the Taiwan Institute of Chemical Engineers, 2022,130.

相似文献/References:

[1]王志俊,张吉月.重力式芯片处理机温度控制技术研究[J].工业仪表与自动化装置,2015,(02):36.
　WANG Zhijun,?ZHANG Jiyue.The study of gravity chip handler’s temperature control technology[J].Industrial Instrumentation & Automation,2015,(03):36.
[2]王晓侃,上官建林,朱振伟.基于PIC16F877单片机的加热炉模糊控制系统设计与研究[J].工业仪表与自动化装置,2015,(03):59.
　WANG Xiaokan,SHUANGGUAN Jianlin,ZHU Zhenwei.Design and research of heating furnace fuzzy control system based on PIC16F877 MCU[J].Industrial Instrumentation & Automation,2015,(03):59.
[3]杨 智,段鹏斌.一种基于模糊控制的温度控制器设计[J].工业仪表与自动化装置,2015,(03):90.
　YANG Zhi,DUAN Pengbin.A temperature controller design based on fuzzy control[J].Industrial Instrumentation & Automation,2015,(03):90.
[4]张英坤,刘会忠.基于模糊PID的石墨化炉温度控制系统[J].工业仪表与自动化装置,2015,(06):26.
　ZHANG Yingkun,LIU Huizhong.The temperature control system of graphitization furnace based on fuzzy PID[J].Industrial Instrumentation & Automation,2015,(03):26.
[5]陈宏希.基于Jess的HVAC温度控制仿真系统[J].工业仪表与自动化装置,2015,(06):122.
　CHEN Hongxi.Simulation system of HVAC temperature control based on Jess[J].Industrial Instrumentation & Automation,2015,(03):122.
[6]金 鹏,李 晶.基于智能算法的双平板导热系数测试仪[J].工业仪表与自动化装置,2015,(06):29.
　JIN Peng,LI Jing.A Double–guarded hot plate thermal conduction coefficient measuring instrument based on intelligent algorithms[J].Industrial Instrumentation & Automation,2015,(03):29.
[7]汝翰霖,李长录,孙铭阳,等.矿用膜分离制氮机温度控制的实现[J].工业仪表与自动化装置,2016,(01):106.
　RU Hanlin,LI Changlu,SUN Mingyang,et al.The realization of temperature control for coal mine film separation and preparation[J].Industrial Instrumentation & Automation,2016,(03):106.
[8]郭建松,包建东,朱建晓,等.低压注塑机注射装置智能化温度控制研究[J].工业仪表与自动化装置,2016,(06):27.
　GUO Jiansong,BAO Jiandong,ZHU Jianxiao,et al.Intelligent temperature control method in Injection system of low pressure injection molding machine[J].Industrial Instrumentation & Automation,2016,(03):27.
[9]赵 静,李建勇.一种新型智能控制算法的仿真研究[J].工业仪表与自动化装置,2018,(02):113.[doi:1000-0682（2018）02-0000-00]
　ZHAO Jing,LI Jianyong.A new intelligent control algorithm simulation research[J].Industrial Instrumentation & Automation,2018,(03):113.[doi:1000-0682（2018）02-0000-00]
[10]范蟠果,刘经纬,王超然,等.基于PLC的模糊PID冷却液温度控制系统的设计[J].工业仪表与自动化装置,2020,(01):69.
　FAN Panguo,LIU Jingwei,WANG Chaoran,et al.Design of fuzzy-PID coolant temperature control system based on PLC[J].Industrial Instrumentation & Automation,2020,(03):69.

备注/Memo

备注/Memo:
收稿日期：2024-01-06第一作者：高东祥(1999— )，硕士，主要研究方向为机电控制，自动化。E-mail：2397536176@qq.com 通信作者：张洪（1966— )，副教授，博士，主要研究方向为机电测试与控制技术、机器人技术。E-mail：1105399774@qq.com
更新日期/Last Update: 1900-01-01