Computer vision based early fire-detection and firefighting mobile robots oriented for onsite construction
Abstract
Fires are one of the most dangerous hazards and the leading cause of death in construction sites. This paper proposes a video-based firefighting mobile robot (FFMR), which is designed to patrol the desired territory and will constantly observe for fire-related events to make sure the camera without any occlusions. Once a fire is detected, the early warning system will send sound and light signals instantly and the FFMR moves to the right place to fight the fire source using the extinguisher. To improve the accuracy and speed of fire detection, an improved YOLOv3-Tiny (namely as YOLOv3-Tiny-S) model is proposed by optimizing its network structure, introducing a Spatial Pyramid Pooling (SPP) module, and refining the multi-scale anchor mechanism. The experiments show the proposed YOLOv3-Tiny-S model based FFMR can detect a small fire target with relatively higher accuracy and faster speed under the occlusions by outdoor environment. The proposed FFMR can be helpful to disaster management systems, avoiding huge ecological and economic losses, as well as saving a lot of human lives.
Keyword : convolutional neural network, firefighting, fire accidents prevention, mobile robot, improved YOLOv3-Tiny model, construction sites
How to Cite
Kong, L., Li, J., Guo, S., Zhou, X., & Wu, D. (2024). Computer vision based early fire-detection and firefighting mobile robots oriented for onsite construction. Journal of Civil Engineering and Management, 30(8), 720–737. https://doi.org/10.3846/jcem.2024.21360
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Ahn, Y., Choi, H., & Kim, B. S. (2023). Development of early fire detection model for buildings using computer vision-based CCTV. Journal of Building Engineering, 65, Article 105647. https://doi.org/10.1016/j.jobe.2022.105647
Ando, H., Ambe, Y., Ishii, A., Konyo, M., Tadakuma, K., Maruyama, S., & Tadokoro, S. (2018). Aerial hose type robot by water jet for fire fighting. IEEE Robotics and Automation Letters, 3(2), 1128–1135. https://doi.org/10.1109/LRA.2018.2792701
Bosheng, S. (2022, November 25). Heavy construction fire accidents reported in 2020 (in Chinese). https://www.163.com/dy/article/GU0RDHF00552HS5R.html
Campbell, R. (2020). Fires in structures under construction or renovation. National Fire Protection Association.
CE Safety. (2022). Fires in buildings under construction caused by hot work. https://cesafety.co.uk/news/how-to-reduce-the-risk-of-fires-caused-by-hot-works/
Chang, C. K., Siagian, C., & Itti, L. (2010). Mobile robot vision navigation & localization using gist and saliency. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 4147–4154). IEEE.
Chaoxia, C., Shang, W., & Zhang, F. (2020). Information-guided flame detection based on faster r-cnn. IEEE Access, 8, 58923–58932. https://doi.org/10.1109/ACCESS.2020.2982994
Chen, C., Guorun, Y., Chenyu, W., Sotirios, G., & Shaohua, W. (2022). Enhancing the robustness of object detection via 6G vehicular edge computing. Digital Communications and Networks, 8(6), 923–931. https://doi.org/10.1016/j.dcan.2022.10.013
Chen, X., Xue, Y. P., Hou, Q. S., Fu, Y. & Zhu, Y. L. (2023). RepVGG-YOLOv7: A Modified YOLOv7 for fire smoke detection. Fire, 6(10), Article 383. https://doi.org/10.3390/fire6100383
Di Paola, D., Milella, A., Cicirelli, G., & Distante, A. (2010). An autonomous mobile robotic system for surveillance of indoor environments. International Journal of Advanced Robotic Systems, 7(1), 19–26. https://doi.org/10.5772/7254
Edirisinghe, R. (2019). Digital skin of the construction site: Smart sensor technologies towards the future smart construction site. Engineering, Construction and Architectural Management, 26(2), 184–223. https://doi.org/10.1108/ECAM-04-2017-0066
Fire Safety Matters. (2020). One-fifth of construction industry-related fires in England caused by hot work. https://www.fsmatters.com/Causes-of-construction-sector-fires-revealed
Geng, Y., Lai, M., Tian, X. C., Xu, X. L., Jiang, Y., & Zhang, Y. K. (2023). A novel seam extraction and path planning method for robotic welding of medium-thickness plate structural parts based on 3D vision. Robotics and Computer-Integrated Manufacturing, 79, Article 102433. https://doi.org/10.1016/j.rcim.2022.102433
Girshick, R. (2015). Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1440–1448). IEEE. https://doi.org/10.1109/ICCV.2015.169
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 580–587). IEEE. https://doi.org/10.1109/CVPR.2014.81
Han, X. F., Jin, J. S., Wang, M. J., Jiang, W., Gao, L., & Xiao, L. P. (2017). Video fire detection based on Gaussian Mixture Model and multi-color features. Signal, Image and Video Processing, 11(8), 1419–1425. https://doi.org/10.1007/s11760-017-1102-y
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824
He, W., Huang, Z., Wei, Z., Li, C., & Guo, B. (2019). TF-YOLO: An improved incremental network for real-time object detection. Applied Sciences, 9(16), Article 3225. https://doi.org/10.3390/app9163225
Hefeeda, M., & Bagheri, M. (2007, October). Wireless sensor networks for early detection of forest fires. In 2007 IEEE International Conference on Mobile Adhoc and Sensor Systems. IEEE. https://doi.org/10.1109/MOBHOC.2007.4428702
Hong, X., Wang, W., & Liu, Q. (2019, June). Design and realization of fire detection using computer vision technology. In 2019 Chinese Control and Decision Conference (CCDC) (pp. 5645–5649). IEEE. https://doi.org/10.1109/CCDC.2019.8832897
Innocente, M. S., & Grasso, P. (2019). Self-organising swarms of firefighting drones: Harnessing the power of collective intelligence in decentralised multi-robot systems. Journal of Computational Science, 34, 80–101. https://doi.org/10.1016/j.jocs.2019.04.009
Jiao, Z., Zhang, Y., Xin, J., Mu, L., Yi, Y., Liu, H., & Liu, D. (2019, July). A deep learning based forest fire detection approach using UAV and YOLOv3. In 2019 1st International Conference on Industrial Artificial Intelligence (IAI). IEEE. https://doi.org/10.1109/ICIAI.2019.8850815
Kim, J. H., Jo, S., & Lattimer, B. Y. (2016). Feature selection for intelligent firefighting robot classification of fire, smoke, and thermal reflections using thermal infrared images. Journal of Sensors, 2016, Article 8410731. https://doi.org/10.1155/2016/8410731
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. https://doi.org/10.1145/3065386
Li, H., Lu, M., Hsu, S. C., Gray, M., & Huang, T. (2015). Proactive behavior-based safety management for construction safety improvement. Safety Science, 75, 107–117. https://doi.org/10.1016/j.ssci.2015.01.013
Li, S., Feng, C., Niu, Y., Shi, L., Wu, Z., & Song, H. (2019). A fire reconnaissance robot based on SLAM position, thermal imaging technologies, and AR display. Sensors, 19(22), Article 5036. https://doi.org/10.3390/s19225036
Li, Z., Mihaylova, L., & Yang, L. (2021). A deep learning framework for autonomous flame detection. Neurocomputing, 448, 205–216. https://doi.org/10.1016/j.neucom.2021.03.019
Li, S., Wang, Y. H., Feng, C. Y., Zhang, D., Li, H. Z., Huang, W. & Shi, L. (2022). A thermal imaging flame-detection model for firefighting robot based on YOLOv4-F model. Fire, 5(5), Article 172. https://doi.org/10.3390/fire5050172
Liu, S., Tu, D., & Zhang, Y. (2009, November). Multiparameter fire detection based on wireless sensor network. In 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems (Vol. 3, pp. 203–206). IEEE. https://doi.org/10.1109/ICICISYS.2009.5358197
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Lecture notes in computer science: Vol. 9905. Computer vision – ECCV 2016 (pp. 21–37). Springer, Cham. https://doi.org/10.1007/978-3-319-46448-0_2
Madhevan, B., Sakkaravarthi, R., Singh, G. M., Diya, R., & Jha, D. K. (2017). Modelling, simulation and mechatronics design of a wireless automatic fire fighting surveillance robot. Defence Science Journal, 67(5), Article 572. https://doi.org/10.14429/dsj.67.10237
Management & Emergency. (2020). Nationalwide fire and fire rescue reported in 2020 (in Chinese). https://www.119.gov.cn/article/3xBeEJjR54K
McNeil, J. G., & Lattimer, B. Y. (2017). Robotic fire suppression through autonomous feedback control. Fire Technology, 53(3), 1171–1199. https://doi.org/10.1007/s10694-016-0623-1
Mir-Nasiri, N., Siswoyo J. H., & Ali, M. H. (2018). Portable autonomous window cleaning robot. Procedia Computer Science, 133, 197–204. https://doi.org/10.1016/j.procs.2018.07.024
Muhammad, K., Ahmad, J., & Baik, S. W. (2018). Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing, 288, 30–42. https://doi.org/10.1016/j.neucom.2017.04.083
Nguyen, A. Q., Nguyen, H. T., Tran, V. C., Pham, H. X. & Pestana, J. (2021). A visual real-time fire detection using single shot MultiBox detector for UAV-based fire surveillance. In 8th IEEE International Conference on Communications and Electronics (IEEE ICCE), Vietnam. https://doi.org/10.1109/ICCE48956.2021.9352080
Palmer, A. (2012). Hotel construction site catches fire. Top Stories, Midland Reporter-Telegram. Texas.
Park, J., Cho, Y. K., & Martinez, D. (2016). A BIM and UWB integrated mobile robot navigation system for indoor position tracking applications. Journal of Construction Engineering and Project Management, 6(2), 30–39. https://doi.org/10.6106/JCEPM.2016.6.2.030
Pincott, J., Tien, P. W., Wei, S., & Calautit, J. K. (2022). Development and evaluation of a vision-based transfer learning approach for indoor fire and smoke detection. Building Services Engineering Research and Technology, 43, 319–332. https://doi.org/10.1177/01436244221089445
Prema, C. E., Vinsley, S., & Suresh, S. (2018). Efficient flame detection based on static and dynamic texture analysis in forest fire detection. Fire Technology, 54(1), 255–288. https://doi.org/10.1007/s10694-017-0683-x
Qiu, X., Xi, T., Sun, D., Zhang, E., Li, C., Peng, Y., Wei, J., & Wang, G. (2018). Fire detection algorithm combined with image processing and flame emission spectroscopy. Fire Technology, 54(5), 1249–1263. https://doi.org/10.1007/s10694-018-0727-x
Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7263–7271). IEEE. https://doi.org/10.1109/CVPR.2017.690
Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv. https://doi.org/10.48550/arXiv.1804.02767
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 779–788). IEEE. https://doi.org/10.1109/CVPR.2016.91
Ren, S., He, K., Girshick, R., & Sun, J. (2016). Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Roberto, G. F., Branco, K. C., Machado, J. M., & Pinto, A. R. (2013, April). Local data fusion algorithm for fire detection through mobile robot. In 2013 14th Latin American Test Workshop-LATW. IEEE. https://doi.org/10.1109/LATW.2013.6562667
Shen, D., Chen, X., Nguyen, M., & Yan, W. Q. (2018, April). Flame detection using deep learning. In 2018 4th International conference on control, automation and robotics (ICCAR) (pp. 416–420). IEEE. https://doi.org/10.1109/ICCAR.2018.8384711
So, A. T., Lo, T. Y., & Chan, W. L. (1996). An autonomous robotic cladding inspector for high-rise buildings in Hong Kong. HKIE Transactions, 3(2), 37–45. https://doi.org/10.1080/1023697X.1996.10667701
Sridhar, P., & Sathiya, R. (2021). Computer vision based early electrical fire-detection in video surveillance oriented for building environment. Journal of Physics: Conference Series, 1916, Article 012024. https://doi.org/10.1088/1742-6596/1916/1/012024
Su, Y., Mao, C., Jiang, R., Liu, G., & Wang, J. (2021). Data-driven fire safety management at building construction sites: Leveraging CNN. Journal of Management in Engineering, 37(2), Article 04020108. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000877
Tesema, F. B., Lin, J., Ou, J., Wu, H., & Zhu, W. (2018). Feature fusing of feature pyramid network for multi-scale pedestrian detection. In 2018 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) (pp. 10–13). IEEE. https://doi.org/10.1109/ICCWAMTIP.2018.8632614
Tso, S. K., & Feng, T. O. N. G. (2003, September). Robot assisted wall inspection for improved maintenance of high-rise buildings. In 20th International Symposium on Automation and Robotics in Construction (pp. 449–455). https://doi.org/10.22260/ISARC2003/0071
Victores, J. G., Martínez, S., Jardón, A., & Balaguer, C. (2011). Robot-aided tunnel inspection and maintenance system by vision and proximity sensor integration. Automation in Construction, 20(5), 629–636. https://doi.org/10.1016/j.autcon.2010.12.005
Vishaal, R., Raghavan, P., Rajesh, R., Michael, S., & Elara, M. R. (2018). Design of dual purpose cleaning robot. Procedia Computer Science, 133, 518–525. https://doi.org/10.1016/j.procs.2018.07.065
Wang, Y., Xing, J. P., Guo, H., & Wang, L. J. (2017). Key technologies of tunnel firefighting robots. IETE Technical Review, 34(1), 3–10. https://doi.org/10.1080/02564602.2016.1139475
Wang, Z., Li, H., & Zhang, X. (2019). Construction waste recycling robot for nails and screws: Computer vision technology and neural network approach. Automation in Construction, 97, 220–228. https://doi.org/10.1016/j.autcon.2018.11.009
Wang, Z., Li, H., & Yang, X. (2020). Vision-based robotic system for on-site construction and demolition waste sorting and recycling. Journal of Building Engineering, 32, Article 101769. https://doi.org/10.1016/j.jobe.2020.101769
Wu, H., Wu, D., & Zhao, J. (2019). An intelligent fire detection approach through cameras based on computer vision methods. Process Safety and Environmental Protection, 127, 245–256. https://doi.org/10.1016/j.psep.2019.05.016
Wu, Z., Xue, R., & Li, H. (2022). Real-time video fire detection via modified YOLOv5 network mode. Fire Technology, 58(4), 2377–2403. https://doi.org/10.1007/s10694-022-01260-z
Xiao, D., Shan, F., Li, Z., Le, B.T., Liu, X., & Li, X. (2019). A target detection model based on improved Tiny-Yolov3 under the environment of mining truck. IEEE Access, 7, 123757–123764. https://doi.org/10.1109/ACCESS.2019.2928603
Xie, Y., Zhu, J., Cao, Y., Zhang, Y., Feng, D., Zhang, Y., & Chen, M. (2020). Efficient video fire detection exploiting motion-flicker-based dynamic features and deep static features. IEEE Access, 8, 81904–81917. https://doi.org/10.1109/ACCESS.2020.2991338
Xu, Z., Guo, Y., & Saleh, J. H. (2020). Tackling small data challenges in visual fire detection: A deep convolutional generative adversarial network approach. IEEE Access, 9, 3936–3946. https://doi.org/10.1109/ACCESS.2020.3047764
Yang, H., Jang, H., Kim, T., & Lee, B. (2019). Non-temporal lightweight fire detection network for intelligent surveillance systems. IEEE Access, 7, 169257–169266. https://doi.org/10.1109/ACCESS.2019.2953558
Yar, H., Khan, Z. A., Ullah, F. U. M., Ullah, W. & Baik, S. W. (2023). A modified YOLOv5 architecture for efficient fire detection in smart cities. Expert Systems with Applications, 231, Article 120465. https://doi.org/10.1016/j.eswa.2023.120465
Yi, Z., Yongliang, S., & Jun, Z. (2019). An improved TINY-YOLOV3 pedestrian detection algorithm. Optik, 183, 17–23. https://doi.org/10.1016/j.ijleo.2019.02.038
Zhan, H. W., Pei, X. Y., Zhang, T. H. & Zhang, L. Q. (2023). Research on flame detection method based on improved SSD algorithm. Journal of Intelligent & Fuzzy Systems, 45, 6501–6512. https://doi.org/10.3233/JIFS-232645
Zhang, J., Jin, Z., & Feng, H. (2018). Type synthesis of a 3-mixed-DOF protectable leg mechanism of a firefighting multi-legged robot based on GF set theory. Mechanism and Machine Theory, 130, 567–584. https://doi.org/10.1016/j.mechmachtheory.2018.08.026
Zhang, H., Qin, L., Li, J., Guo, Y., Zhou, Y., Zhang, J., & Xu, Z. (2020). Real-time detection method for small traffic signs based on Yolov3. IEEE Access, 8, 64145–64156. https://doi.org/10.1109/ACCESS.2020.2984554
Zhang, H., Wang, Z., Chen, M., Peng, Y., Gao, Y., & Zhou, J. (2021). An improved YOLOv3 algorithm combined with attention mechanism for flame and smoke detection. In X. Sun, X. Zhang, Z. Xia, & E. Bertino (Eds.), Lecture notes in computer science: Vol. 12736. Artificial intelligence and security. ICAIS 2021 (pp. 226–238). Springer, Cham. https://doi.org/10.1007/978-3-030-78609-0_20
Zhang, T., Wang, Z., Zeng, Y., Wu, X., Huang, X., & Xiao, F. (2022). Building artificial-intelligence digital fire (AID-Fire) system: A real-scale demonstration. Journal of Building Engineering, 62, Article 105363. https://doi.org/10.1016/j.jobe.2022.105363
Zhu, J., Li, W., Lin, D., Cheng, H., & Zhao, G. (2020a). Intelligent fire monitor for fire robot based on infrared image feedback control. Fire Technology, 56(5), 2089–2109. https://doi.org/10.1007/s10694-020-00964-4
Zhu, J., Pan, L., & Zhao, G. (2020b). An improved near-field computer vision for jet trajectory falling position prediction of intelligent fire robot. Sensors, 20(24), Article 7029. https://doi.org/10.3390/s20247029
Ando, H., Ambe, Y., Ishii, A., Konyo, M., Tadakuma, K., Maruyama, S., & Tadokoro, S. (2018). Aerial hose type robot by water jet for fire fighting. IEEE Robotics and Automation Letters, 3(2), 1128–1135. https://doi.org/10.1109/LRA.2018.2792701
Bosheng, S. (2022, November 25). Heavy construction fire accidents reported in 2020 (in Chinese). https://www.163.com/dy/article/GU0RDHF00552HS5R.html
Campbell, R. (2020). Fires in structures under construction or renovation. National Fire Protection Association.
CE Safety. (2022). Fires in buildings under construction caused by hot work. https://cesafety.co.uk/news/how-to-reduce-the-risk-of-fires-caused-by-hot-works/
Chang, C. K., Siagian, C., & Itti, L. (2010). Mobile robot vision navigation & localization using gist and saliency. In 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 4147–4154). IEEE.
Chaoxia, C., Shang, W., & Zhang, F. (2020). Information-guided flame detection based on faster r-cnn. IEEE Access, 8, 58923–58932. https://doi.org/10.1109/ACCESS.2020.2982994
Chen, C., Guorun, Y., Chenyu, W., Sotirios, G., & Shaohua, W. (2022). Enhancing the robustness of object detection via 6G vehicular edge computing. Digital Communications and Networks, 8(6), 923–931. https://doi.org/10.1016/j.dcan.2022.10.013
Chen, X., Xue, Y. P., Hou, Q. S., Fu, Y. & Zhu, Y. L. (2023). RepVGG-YOLOv7: A Modified YOLOv7 for fire smoke detection. Fire, 6(10), Article 383. https://doi.org/10.3390/fire6100383
Di Paola, D., Milella, A., Cicirelli, G., & Distante, A. (2010). An autonomous mobile robotic system for surveillance of indoor environments. International Journal of Advanced Robotic Systems, 7(1), 19–26. https://doi.org/10.5772/7254
Edirisinghe, R. (2019). Digital skin of the construction site: Smart sensor technologies towards the future smart construction site. Engineering, Construction and Architectural Management, 26(2), 184–223. https://doi.org/10.1108/ECAM-04-2017-0066
Fire Safety Matters. (2020). One-fifth of construction industry-related fires in England caused by hot work. https://www.fsmatters.com/Causes-of-construction-sector-fires-revealed
Geng, Y., Lai, M., Tian, X. C., Xu, X. L., Jiang, Y., & Zhang, Y. K. (2023). A novel seam extraction and path planning method for robotic welding of medium-thickness plate structural parts based on 3D vision. Robotics and Computer-Integrated Manufacturing, 79, Article 102433. https://doi.org/10.1016/j.rcim.2022.102433
Girshick, R. (2015). Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1440–1448). IEEE. https://doi.org/10.1109/ICCV.2015.169
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014). Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 580–587). IEEE. https://doi.org/10.1109/CVPR.2014.81
Han, X. F., Jin, J. S., Wang, M. J., Jiang, W., Gao, L., & Xiao, L. P. (2017). Video fire detection based on Gaussian Mixture Model and multi-color features. Signal, Image and Video Processing, 11(8), 1419–1425. https://doi.org/10.1007/s11760-017-1102-y
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824
He, W., Huang, Z., Wei, Z., Li, C., & Guo, B. (2019). TF-YOLO: An improved incremental network for real-time object detection. Applied Sciences, 9(16), Article 3225. https://doi.org/10.3390/app9163225
Hefeeda, M., & Bagheri, M. (2007, October). Wireless sensor networks for early detection of forest fires. In 2007 IEEE International Conference on Mobile Adhoc and Sensor Systems. IEEE. https://doi.org/10.1109/MOBHOC.2007.4428702
Hong, X., Wang, W., & Liu, Q. (2019, June). Design and realization of fire detection using computer vision technology. In 2019 Chinese Control and Decision Conference (CCDC) (pp. 5645–5649). IEEE. https://doi.org/10.1109/CCDC.2019.8832897
Innocente, M. S., & Grasso, P. (2019). Self-organising swarms of firefighting drones: Harnessing the power of collective intelligence in decentralised multi-robot systems. Journal of Computational Science, 34, 80–101. https://doi.org/10.1016/j.jocs.2019.04.009
Jiao, Z., Zhang, Y., Xin, J., Mu, L., Yi, Y., Liu, H., & Liu, D. (2019, July). A deep learning based forest fire detection approach using UAV and YOLOv3. In 2019 1st International Conference on Industrial Artificial Intelligence (IAI). IEEE. https://doi.org/10.1109/ICIAI.2019.8850815
Kim, J. H., Jo, S., & Lattimer, B. Y. (2016). Feature selection for intelligent firefighting robot classification of fire, smoke, and thermal reflections using thermal infrared images. Journal of Sensors, 2016, Article 8410731. https://doi.org/10.1155/2016/8410731
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. https://doi.org/10.1145/3065386
Li, H., Lu, M., Hsu, S. C., Gray, M., & Huang, T. (2015). Proactive behavior-based safety management for construction safety improvement. Safety Science, 75, 107–117. https://doi.org/10.1016/j.ssci.2015.01.013
Li, S., Feng, C., Niu, Y., Shi, L., Wu, Z., & Song, H. (2019). A fire reconnaissance robot based on SLAM position, thermal imaging technologies, and AR display. Sensors, 19(22), Article 5036. https://doi.org/10.3390/s19225036
Li, Z., Mihaylova, L., & Yang, L. (2021). A deep learning framework for autonomous flame detection. Neurocomputing, 448, 205–216. https://doi.org/10.1016/j.neucom.2021.03.019
Li, S., Wang, Y. H., Feng, C. Y., Zhang, D., Li, H. Z., Huang, W. & Shi, L. (2022). A thermal imaging flame-detection model for firefighting robot based on YOLOv4-F model. Fire, 5(5), Article 172. https://doi.org/10.3390/fire5050172
Liu, S., Tu, D., & Zhang, Y. (2009, November). Multiparameter fire detection based on wireless sensor network. In 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems (Vol. 3, pp. 203–206). IEEE. https://doi.org/10.1109/ICICISYS.2009.5358197
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C. Y., & Berg, A. C. (2016). SSD: Single shot multibox detector. In B. Leibe, J. Matas, N. Sebe, & M. Welling (Eds.), Lecture notes in computer science: Vol. 9905. Computer vision – ECCV 2016 (pp. 21–37). Springer, Cham. https://doi.org/10.1007/978-3-319-46448-0_2
Madhevan, B., Sakkaravarthi, R., Singh, G. M., Diya, R., & Jha, D. K. (2017). Modelling, simulation and mechatronics design of a wireless automatic fire fighting surveillance robot. Defence Science Journal, 67(5), Article 572. https://doi.org/10.14429/dsj.67.10237
Management & Emergency. (2020). Nationalwide fire and fire rescue reported in 2020 (in Chinese). https://www.119.gov.cn/article/3xBeEJjR54K
McNeil, J. G., & Lattimer, B. Y. (2017). Robotic fire suppression through autonomous feedback control. Fire Technology, 53(3), 1171–1199. https://doi.org/10.1007/s10694-016-0623-1
Mir-Nasiri, N., Siswoyo J. H., & Ali, M. H. (2018). Portable autonomous window cleaning robot. Procedia Computer Science, 133, 197–204. https://doi.org/10.1016/j.procs.2018.07.024
Muhammad, K., Ahmad, J., & Baik, S. W. (2018). Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing, 288, 30–42. https://doi.org/10.1016/j.neucom.2017.04.083
Nguyen, A. Q., Nguyen, H. T., Tran, V. C., Pham, H. X. & Pestana, J. (2021). A visual real-time fire detection using single shot MultiBox detector for UAV-based fire surveillance. In 8th IEEE International Conference on Communications and Electronics (IEEE ICCE), Vietnam. https://doi.org/10.1109/ICCE48956.2021.9352080
Palmer, A. (2012). Hotel construction site catches fire. Top Stories, Midland Reporter-Telegram. Texas.
Park, J., Cho, Y. K., & Martinez, D. (2016). A BIM and UWB integrated mobile robot navigation system for indoor position tracking applications. Journal of Construction Engineering and Project Management, 6(2), 30–39. https://doi.org/10.6106/JCEPM.2016.6.2.030
Pincott, J., Tien, P. W., Wei, S., & Calautit, J. K. (2022). Development and evaluation of a vision-based transfer learning approach for indoor fire and smoke detection. Building Services Engineering Research and Technology, 43, 319–332. https://doi.org/10.1177/01436244221089445
Prema, C. E., Vinsley, S., & Suresh, S. (2018). Efficient flame detection based on static and dynamic texture analysis in forest fire detection. Fire Technology, 54(1), 255–288. https://doi.org/10.1007/s10694-017-0683-x
Qiu, X., Xi, T., Sun, D., Zhang, E., Li, C., Peng, Y., Wei, J., & Wang, G. (2018). Fire detection algorithm combined with image processing and flame emission spectroscopy. Fire Technology, 54(5), 1249–1263. https://doi.org/10.1007/s10694-018-0727-x
Redmon, J., & Farhadi, A. (2017). YOLO9000: better, faster, stronger. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7263–7271). IEEE. https://doi.org/10.1109/CVPR.2017.690
Redmon, J., & Farhadi, A. (2018). Yolov3: An incremental improvement. arXiv. https://doi.org/10.48550/arXiv.1804.02767
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016). You only look once: Unified, real-time object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 779–788). IEEE. https://doi.org/10.1109/CVPR.2016.91
Ren, S., He, K., Girshick, R., & Sun, J. (2016). Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
Roberto, G. F., Branco, K. C., Machado, J. M., & Pinto, A. R. (2013, April). Local data fusion algorithm for fire detection through mobile robot. In 2013 14th Latin American Test Workshop-LATW. IEEE. https://doi.org/10.1109/LATW.2013.6562667
Shen, D., Chen, X., Nguyen, M., & Yan, W. Q. (2018, April). Flame detection using deep learning. In 2018 4th International conference on control, automation and robotics (ICCAR) (pp. 416–420). IEEE. https://doi.org/10.1109/ICCAR.2018.8384711
So, A. T., Lo, T. Y., & Chan, W. L. (1996). An autonomous robotic cladding inspector for high-rise buildings in Hong Kong. HKIE Transactions, 3(2), 37–45. https://doi.org/10.1080/1023697X.1996.10667701
Sridhar, P., & Sathiya, R. (2021). Computer vision based early electrical fire-detection in video surveillance oriented for building environment. Journal of Physics: Conference Series, 1916, Article 012024. https://doi.org/10.1088/1742-6596/1916/1/012024
Su, Y., Mao, C., Jiang, R., Liu, G., & Wang, J. (2021). Data-driven fire safety management at building construction sites: Leveraging CNN. Journal of Management in Engineering, 37(2), Article 04020108. https://doi.org/10.1061/(ASCE)ME.1943-5479.0000877
Tesema, F. B., Lin, J., Ou, J., Wu, H., & Zhu, W. (2018). Feature fusing of feature pyramid network for multi-scale pedestrian detection. In 2018 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP) (pp. 10–13). IEEE. https://doi.org/10.1109/ICCWAMTIP.2018.8632614
Tso, S. K., & Feng, T. O. N. G. (2003, September). Robot assisted wall inspection for improved maintenance of high-rise buildings. In 20th International Symposium on Automation and Robotics in Construction (pp. 449–455). https://doi.org/10.22260/ISARC2003/0071
Victores, J. G., Martínez, S., Jardón, A., & Balaguer, C. (2011). Robot-aided tunnel inspection and maintenance system by vision and proximity sensor integration. Automation in Construction, 20(5), 629–636. https://doi.org/10.1016/j.autcon.2010.12.005
Vishaal, R., Raghavan, P., Rajesh, R., Michael, S., & Elara, M. R. (2018). Design of dual purpose cleaning robot. Procedia Computer Science, 133, 518–525. https://doi.org/10.1016/j.procs.2018.07.065
Wang, Y., Xing, J. P., Guo, H., & Wang, L. J. (2017). Key technologies of tunnel firefighting robots. IETE Technical Review, 34(1), 3–10. https://doi.org/10.1080/02564602.2016.1139475
Wang, Z., Li, H., & Zhang, X. (2019). Construction waste recycling robot for nails and screws: Computer vision technology and neural network approach. Automation in Construction, 97, 220–228. https://doi.org/10.1016/j.autcon.2018.11.009
Wang, Z., Li, H., & Yang, X. (2020). Vision-based robotic system for on-site construction and demolition waste sorting and recycling. Journal of Building Engineering, 32, Article 101769. https://doi.org/10.1016/j.jobe.2020.101769
Wu, H., Wu, D., & Zhao, J. (2019). An intelligent fire detection approach through cameras based on computer vision methods. Process Safety and Environmental Protection, 127, 245–256. https://doi.org/10.1016/j.psep.2019.05.016
Wu, Z., Xue, R., & Li, H. (2022). Real-time video fire detection via modified YOLOv5 network mode. Fire Technology, 58(4), 2377–2403. https://doi.org/10.1007/s10694-022-01260-z
Xiao, D., Shan, F., Li, Z., Le, B.T., Liu, X., & Li, X. (2019). A target detection model based on improved Tiny-Yolov3 under the environment of mining truck. IEEE Access, 7, 123757–123764. https://doi.org/10.1109/ACCESS.2019.2928603
Xie, Y., Zhu, J., Cao, Y., Zhang, Y., Feng, D., Zhang, Y., & Chen, M. (2020). Efficient video fire detection exploiting motion-flicker-based dynamic features and deep static features. IEEE Access, 8, 81904–81917. https://doi.org/10.1109/ACCESS.2020.2991338
Xu, Z., Guo, Y., & Saleh, J. H. (2020). Tackling small data challenges in visual fire detection: A deep convolutional generative adversarial network approach. IEEE Access, 9, 3936–3946. https://doi.org/10.1109/ACCESS.2020.3047764
Yang, H., Jang, H., Kim, T., & Lee, B. (2019). Non-temporal lightweight fire detection network for intelligent surveillance systems. IEEE Access, 7, 169257–169266. https://doi.org/10.1109/ACCESS.2019.2953558
Yar, H., Khan, Z. A., Ullah, F. U. M., Ullah, W. & Baik, S. W. (2023). A modified YOLOv5 architecture for efficient fire detection in smart cities. Expert Systems with Applications, 231, Article 120465. https://doi.org/10.1016/j.eswa.2023.120465
Yi, Z., Yongliang, S., & Jun, Z. (2019). An improved TINY-YOLOV3 pedestrian detection algorithm. Optik, 183, 17–23. https://doi.org/10.1016/j.ijleo.2019.02.038
Zhan, H. W., Pei, X. Y., Zhang, T. H. & Zhang, L. Q. (2023). Research on flame detection method based on improved SSD algorithm. Journal of Intelligent & Fuzzy Systems, 45, 6501–6512. https://doi.org/10.3233/JIFS-232645
Zhang, J., Jin, Z., & Feng, H. (2018). Type synthesis of a 3-mixed-DOF protectable leg mechanism of a firefighting multi-legged robot based on GF set theory. Mechanism and Machine Theory, 130, 567–584. https://doi.org/10.1016/j.mechmachtheory.2018.08.026
Zhang, H., Qin, L., Li, J., Guo, Y., Zhou, Y., Zhang, J., & Xu, Z. (2020). Real-time detection method for small traffic signs based on Yolov3. IEEE Access, 8, 64145–64156. https://doi.org/10.1109/ACCESS.2020.2984554
Zhang, H., Wang, Z., Chen, M., Peng, Y., Gao, Y., & Zhou, J. (2021). An improved YOLOv3 algorithm combined with attention mechanism for flame and smoke detection. In X. Sun, X. Zhang, Z. Xia, & E. Bertino (Eds.), Lecture notes in computer science: Vol. 12736. Artificial intelligence and security. ICAIS 2021 (pp. 226–238). Springer, Cham. https://doi.org/10.1007/978-3-030-78609-0_20
Zhang, T., Wang, Z., Zeng, Y., Wu, X., Huang, X., & Xiao, F. (2022). Building artificial-intelligence digital fire (AID-Fire) system: A real-scale demonstration. Journal of Building Engineering, 62, Article 105363. https://doi.org/10.1016/j.jobe.2022.105363
Zhu, J., Li, W., Lin, D., Cheng, H., & Zhao, G. (2020a). Intelligent fire monitor for fire robot based on infrared image feedback control. Fire Technology, 56(5), 2089–2109. https://doi.org/10.1007/s10694-020-00964-4
Zhu, J., Pan, L., & Zhao, G. (2020b). An improved near-field computer vision for jet trajectory falling position prediction of intelligent fire robot. Sensors, 20(24), Article 7029. https://doi.org/10.3390/s20247029