Abstract
Ensuring safe human-robot interactions is a crucial concern in robotics research and development. However, controlling the social distance between quadruped robots and humans poses a significant challenge due to the large motion range of robots and the lack of specific social distance rules for human safety. This study proposes an innovative framework that integrates proxemic theory and ISO safety strategy to provide a new social distance control mechanism for human-quadruped robot interactions. The proposed framework uses a gated spike filter neural network (GSN) that fuses LiDAR and RGB-D data to estimate human-robot distance and ensures quantified human-robot social distance with continuous velocity limitations, which current approaches do not offer. The experimental results demonstrate that the GSN algorithm offers high accuracy and is well-suited for large-range human-robot distance control of quadruped robots. It has an average prediction error that is 31.4% lower than traditional Kalman filter methods and offers a more efficient and precise solution for real-time social distance control. Additionally, this study includes a detailed use case that demonstrates how to apply the proposed framework to human-quadruped robot interactions. Overall, this study’s findings have implications for improving the safety and efficiency of human-robot interactions in various applications.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and material
All data generated or analyzed during this study are included in this published article.
Code Availability
The code for the neural network architecture of GSN and a portion of the data are available online at https://gitee.com/shuaizhang_zju/gated-spike-filter-neural-network-framework.git. The entire code for the GSN framework will be released later.
References
Maslow AH (1943) A theory of human motivation. Psychol Rev 50(4):370
Coronado E, Kiyokawa T, Garcia Ricardez GA, Ramirez-Alpizar IG, Venture G, Yamanobe N (2022) Evaluating quality in human-robot interaction: A systematic search and classification of performance and human-centered factors, measures and metrics towards an industry 5.0. J Manuf Syst 63:392–410
Scassellati B, Vázquez M (2020) The potential of socially assistive robots during infectious disease outbreaks. Science Robotics 5(44):eabc9014
Saunderson S, Nejat G (2019) How robots influence humans: A survey of nonverbal communication in social human-robot interaction. International Journal of Social Robotics 11:575–608
Chemweno P, Pintelon L, Decre W (2020) Orienting safety assurance with outcomes of hazard analysis and risk assessment: A review of the iso 15066 standard for collaborative robot systems. Saf Sci 129:104832
Salvini P, Paez-Granados D, Billard A (2021) On the safety of mobile robots serving in public spaces: Identifying gaps in en iso 13482: 2014 and calling for a new standard. ACM Transactions on Human-Robot Interaction (THRI) 10(3):1–27
Scalera L, Giusti A, Vidoni R, Di Cosmo V, Matt D, Riedl M (2020) Application of dynamically scaled safety zones based on the iso/ts 15066: 2016 for collaborative robotics. International Journal of Mechanics and Control 21(1):41–49
Byner C, Matthias B, Ding H (2019) Dynamic speed and separation monitoring for collaborative robot applications-concepts and performance. Robot Comput Integr Manuf 58:239–252
Zhang S, Li S, Li X, Xiong Y, Xie Z (2022) A human-robot dynamic fusion safety algorithm for collaborative operations of cobots. J Intell Robot Syst 104:1–14
Franklin CS, Dominguez EG, Fryman JD, Lewandowski ML (2020) Collaborative robotics: New era of human-robot cooperation in the workplace. J Saf Res 74:153–160
Tanaka Y, Shirai Y, Lin X, Schperberg A, Kato H, Swerdlow A, Kumagai N, Hong D (2022) Scaler: A tough versatile quadruped free-climber robot. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 5632–5639. IEEE
Iqbal A, Gao Y, Gu Y (2020) Provably stabilizing controllers for quadrupedal robot locomotion on dynamic rigid platforms. IEEE/ASME Transactions on Mechatronics 25(4):2035–2044
Hooks J, Ahn MS, Yu J, Zhang X, Zhu T, Chae H, Hong D (2020) Alphred: A multi-modal operations quadruped robot for package delivery applications. IEEE Robotics and Automation Letters 5(4):5409–5416
Cao Z, Hidalgo G, Simon T, Wei S, Sheikh Y (2019) Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp 1–6
Pathi SK, Kiselev A, Kristoffersson A, Repsilber D, Loutfi A (2019) A novel method for estimating distances from a robot to humans using egocentric rgb camera. Sensors (Basel, Switzerland) 19(14):1–13
Edwards A, Edwards C, Westerman D, Spence PR (2019) Initial expectations, interactions, and beyond with social robots. Comput Hum Behav 90:308–314
Patompak P, Jeong S, Nilkhamhang I, Chong NY (2020) Learning proxemics for personalized human-robot social interaction. International Journal of Social Robotics 12:267–280
Millán-Arias C, Fernandes B, Cruz F (2022) Proxemic behavior in navigation tasks using reinforcement learning. Neural Comput & Applic, pp 1–16
Leichtmann B, Nitsch V (2020) How much distance do humans keep toward robots? literature review, meta-analysis, and theoretical considerations on personal space in human-robot interaction. J Environ Psychol 68:101386
Hall ET (1969) The Hidden Dimension, vol 4 of 10. Anchor Books, New York, 3 edn, 7. An optional note
Gonzalez-Trejo JA, Mercado-Ravell DA, Jaramillo-Avila U (2022) Monitoring social-distance in wide areas during pandemics: a density map and segmentation approach. Appl Intell 52(12):13824–13838
Neggers MME, Cuijpers RH, Ruijten PAM, IJsselsteijn WA (2022) Determining shape and size of personal space of a human when passed by a robot. International Journal of Social Robotics, pp 1–12
Camara F, Fox C (2021) Space invaders: Pedestrian proxemic utility functions and trust zones for autonomous vehicle interactions. International Journal of Social Robotics 13(8):1929–1949
Sprunk C, Lau B, Pfaff P, Burgard W (2017) An accurate and efficient navigation system for omnidirectional robots in industrial environments. Auton Robot 41(2):473–493
Karagiannis P, Kousi N, Michalos G, Dimoulas K, Mparis K, Dimosthenopoulos D, Tokçalar Ö, Guasch T, Gerio GP, Makris S (2022) Adaptive speed and separation monitoring based on switching of safety zones for effective human robot collaboration. Robot Comput Integr Manuf 77:102361
Zhang S, Li S, Wang H, Li X (2022) An intelligent manufacturing cell based on human-robot collaboration of frequent task learning for flexible manufacturing. The International Journal of Advanced Manufacturing Technology 120(9–10):5725-5740
Ferraguti F, Landi CT, Costi S, Bonfè M, Farsoni S, Secchi C, Fantuzzi C (2020) Safety barrier functions and multi-camera tracking for human-robot shared environment. Robot Auton Syst 124:103388
Kolar P, Benavidez P, Mo J (2020) Survey of datafusion techniques for laser and vision based sensor integration for autonomous navigation. Sensors 20(8):2180
Graf F, Odabaşi Ç, Jacobs T, Graf B, Födisch T (2019) Mobika-low-cost mobile robot for human-robot interaction. In 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp 1–6. IEEE
Lee W, Won C, Lim J (2020) Unified calibration for multi-camera multi-lidar systems using a single checkerboard. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 9033–9039
Kummerle J, Kuhner T (2020) Unified intrinsic and extrinsic camera and lidar calibration under uncertainties. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 6028–6034
Park K, Kim S, Sohn K (2020) High-precision depth estimation using uncalibrated lidar and stereo fusion. IEEE Trans Intell Transp Syst 21(1):321–335
Li S, Zhang S, Fu Y, Wang H, Han K (2020) Task-based obstacle avoidance for uncertain targets based on semantic object matrix. Control Eng Pract 105:104649
Cheng X, Zhong Y, Dai Y, Ji P, Li H (2019) Noise-aware unsupervised deep lidar-stereo fusion. In 2019 IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 6332–6341
Narkhede P, Walambe R, Poddar S, Kotecha K (2021) Incremental learning of lstm framework for sensor fusion in attitude estimation. PeerJ Computer Science 7:e662
Shao Z, Cheng G, Ma J, Wang Z, Wang J, Li D (2021) Real-time and accurate uav pedestrian detection for social distancing monitoring in covid-19 pandemic. IEEE transactions on multimedia 24:2069–2083
Pan J, Chitta S, Manocha D (2012) Fcl: A general purpose library for collision and proximity queries. In 2012 IEEE International Conference on Robotics and Automation, pp 3859–3866. IEEE
Munaro M, Basso F, Menegatti E (2012) Tracking people within groups with rgb-d data. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2101–2107
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 1, pp 886–893
Munaro M, Menegatti E (2014) Fast rgb-d people tracking for service robots. Auton Robot 37(3):227–242
Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9–13. IEEE
Morgan Q, Ken C, Brian G, Josh F, Tully F, Jeremy L, Rob W, Andrew N (2009) Ros: an open-source robot operating system. In ICRA workshop on open source software, vol 3, p 5. Kobe, Japan
Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn 47(6):2280–2292
Stampe DM (1993) Heuristic filtering and reliable calibration methods for video-based pupil-tracking systems. Behavior Research Methods, Instruments, & Computers 25(2):137–142
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32:8026–8037
Fujii K (2013) Extended kalman filter. Refernce Manual, pp 14–22
Hornyak V, Vanswearingen JM, Brach JS (2012) Measurement of gait speed. Topics in geriatric rehabilitation 28(1):27–32
Acknowledgements
This research work received support from the National Natural Science Foundation of China (NO.32200890) and China Postdoctoral Science Foundation (NO.2022M712791).
Author information
Authors and Affiliations
Contributions
Shuai Zhang and Zhiguo Wang conceived and designed the study. Yongkai Li designed the neural network. Shuai Zhang and Zehao Huang performed the experiments. Shuai Zhang wrote the paper. Shuai Zhang, Zhiguo Wang and Rong Wang reviewed and edited the manuscript. All authors read and approved the manuscript.
Corresponding authors
Ethics declarations
Ethics approval
The study was conducted in accordance with the Declaration of Helsinki, with the research protocol approved by the Institutional Review Board at the Center for Psychological Sciences, Zhejiang University (protocol code 2022-008, approved on 13 December 2022).
Consent to participate
Informed consent was obtained from all human subjects to participate in the study.
Consent for publication
Informed consent was obtained from the family to publish the data in an anonymized manner.
Conflict of interest statement
We declare that we have no financial or personal relationships with other people or organizations that influenced our work. There is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, S., Li, Y., Huang, Z. et al. Social distance control for quadruped robots in a gated spike filter neural network framework. Appl Intell 53, 24089–24105 (2023). https://doi.org/10.1007/s10489-023-04832-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-04832-w