Automatic Counting Passenger System using Online Visual Appearance Multi-Object Tracking

Volume 7, Issue 5, Page No 113-128, 2022

Author’s Name: Javier Calle^a), Itziar Sagastiberri, Mikel Aramburu, Santiago Cerezo, Jorge García

View Affiliations

Fundacion Vicomtech, Basque Research and Technology Alliance (BRTA), Donostia-San Sebastian, 20009, Spain

^a)whom correspondence should be addressed. E-mail: jcalle@vicomtech.org

Adv. Sci. Technol. Eng. Syst. J. 7(5), 113-128 (2022); DOI: 10.25046/aj070514

Keywords: People-counting, Multi-object tracking, Fisheye camera, Public transport, Visual appearance modelling

Download Now!

205 Downloads

Export Citations

Abstract

In recent years, people-counting problems have increased in popularity, especially in crowded indoor spaces, e.g., public transport. In peak hours, trains move significant numbers of passengers, producing delays and inconveniences for their users. Therefore, analysing how people use public transport is essential to solving this problem. The current analysis estimates how many people are inside a train station by using the number of people entering and leaving the ticket gates or estimating the train occupancy based on conventional CCTV cameras. However, this information is insufficient for knowing the train occupancy. The required data includes vehicle usage: how many people enter or leave a vehicle or which door is the most used. This paper presents a solution to the stated problem based on a multi-object tracker with a sequential visual appearance predictor and a line-based counting strategy to analyse each passenger’s trajectory using an overhead fisheye camera. The camera selection inside the train was made after profoundly studying the railway environment. The method proposes a module to compute the total train occupancy. The solution is robust against occlusions thanks to the selected tracker and the fisheye camera field of view. This work shows a proof of concept dataset containing pseudo-real case scenarios of people’s affluence in train doors recorded by fisheye cameras. Its purpose is to prove the system’s functionality in these scenarios. The proposed approach achieved an overall accuracy of counting people getting on and off of 90.78% in the pseudo-real dataset, proving that this approach is valid.

Received: 31 August 2022, Accepted: 08 October 2022, Published Online: 25 October 2022

Full Text

References (45)

I. Sagastiberri, N. v. d. Gevel, J. Garc´ıa, O. Otaegui, “Learning Sequential Visual Appearance Transformation for Online Multi-Object Tracking,” in 2021 17th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 1–7, 2021, doi:10.1109/AVSS52988.2021.9663809.
Y. Zhang, C. Wang, X. Wang, W. Zeng, W. Liu, “FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking,” arXiv preprint arXiv:2004.01888, 2020.
E. Khoumeri, H. Fraoucene, E. Khoumeri, C. Hamouda, R. Cheggou, People Counter with Area Occupancy Control for Covid-19, 405–415, 2021, doi: 10.1007/978-3-030-63846-7 38.
A. Naser, A. Lotfi, J. Zhong, “Adaptive Thermal Sensor Array Placement for Human Segmentation and Occupancy Estimation,” IEEE Sensors Journal, 21(2), 1993–2002, 2021, doi:10.1109/JSEN.2020.3020401.
Z.-Q. Cheng, J.-X. Li, Q. Dai, X. Wu, A. Hauptmann, “Learning Spa- tial Awareness to Improve Crowd Counting,” in 2019 IEEE/CVF Inter- national Conference on Computer Vision (ICCV), 6151–6160, 2019, doi: 10.1109/ICCV.2019.00625.
Y. Li, X. Zhang, D. Chen, “CSRNet: Dilated Convolutional Neural Net- works for Understanding the Highly Congested Scenes,” 1091–1100, 2018, doi:10.1109/CVPR.2018.00120.
Y. Miao, Z. Lin, G. Ding, J. Han, “Shallow Feature Based Dense Attention Network for Crowd Counting,” in The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020, 11765–11772, AAAI Press, 2020.
Q. Song, C. Wang, Z. Jiang, Y. Wang, Y. Tai, C. Wang, J. Li, F. Huang, Y. Wu, “Rethinking Counting and Localization in Crowds:A Purely Point-Based Frame- work,” 2021.
S. A. Velastin, R. Ferna´ndez, J. E. Espinosa, A. Bay, “Detecting, Tracking and Counting People Getting On/Off a Metropolitan Train Using a Standard Video
Camera,” Sensors, 20(21), 2020, doi:10.3390/s20216251.
D. Kuplyakov, Y. Geraskin, T. Mamedov, A. Konushin, “A Distributed Track- ing Algorithm for Counting People in Video by Head Detection,” paper26–1, 2020, doi:10.51130/graphicon-2020-2-3-26.
J.-W. Kim, K.-S. Park, B.-D. Park, S.-J. Ko, “Real-time vision-based people counting system for the security door,” in Proceedings of the IEEK Conference, 1416–1419, The Institute of Electronics and Information Engineers, 2002.
J. Barandiaran, B. Murguia, F. Boto, “Real-time people counting using mul- tiple lines,” in 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services, 159–162, IEEE, 2008.
J. Ahmad, H. Larijani, R. Emmanuel, M. Mannion, A. Javed, “An intelligent real-time occupancy monitoring system using single overhead camera,” in Proceedings of SAI Intelligent Systems Conference, 957–969, Springer, 2018.
S. Yu, X. Chen, W. Sun, D. Xie, “A robust method for detecting and count- ing people,” in 2008 International conference on audio, language and image processing, 1545–1549, IEEE, 2008.
P. Chato, D. J. M. Chipantasi, N. Velasco, S. Rea, V. Hallo, P. Constante, “Im- age processing and artificial neural network for counting people inside public transport,” in 2018 IEEE Third Ecuador Technical Chapters Meeting (ETCM), 1–5, IEEE, 2018.
M. Bouazizi, C. Ye, T. Ohtsuki, “Low-Resolution Infrared Array Sensor for Counting and Localizing People Indoors: When Low End Technology Meets Cutting Edge Deep Learning Techniques,” Information, 13(3), 2022, doi:10.3390/info13030132.
R. L. dos Santos, H. C. de Oliveira, M. C. de Almeida, D. F. Vieira, E. P. L. Junior, T. Ji, “A Low-Cost Bidirectional People Counter Device for Assisting Social Distancing Monitoring for COVID-19,” Journal of Con- trol, Automation and Electrical Systems, 33(4), 1148, 1160, 2022, doi: 10.1007/s40313-022-00916-z.
P. Bergmann, T. Meinhardt, L. Leal-Taixe, “Tracking Without Bells and Whis- tles,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019.
W. Tian, M. Lauer, L. Chen, “Online Multi-Object Tracking Using Joint Do- main Information in Traffic Scenarios,” IEEE Transactions on Intelligent Trans- portation Systems, 21(1), 374–384, 2020, doi:10.1109/TITS.2019.2892413.
W. V. Ranst, F. De Smedt, J. Berte, T. Goedeme´, “Fast Simultaneous People Detection and Re-identification in a Single Shot Network,” in 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2018, doi:10.1109/AVSS.2018.8639489.
L. Ren, J. Lu, Z. Wang, Q. Tian, J. Zhou, “Collaborative Deep Reinforcement Learning for Multi-object Tracking,” in V. Ferrari, M. Hebert, C. Sminchisescu,
Y. Weiss, editors, Computer Vision – ECCV 2018, 2018.
X. Dong, J. Shen, “Triplet Loss in Siamese Network for Object Tracking,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018.
J. Son, M. Baek, M. Cho, B. Han, “Multi-object Tracking with Quadruplet Convolutional Neural Networks,” in 2017 IEEE Conference on Computer Vi- sion and Pattern Recognition (CVPR), 3786–3795, 2017, doi:10.1109/CVPR. 2017.403.
J. Yin, W. Wang, Q. Meng, R. Yang, J. Shen, “A Unified Object Motion and Affinity Model for Online Multi-Object Tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
N. Wojke, A. Bewley, D. Paulus, “Simple Online and Realtime Tracking with a Deep Association Metric,” 2017.
P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. D. Reid, S. Roth,
K. Schindler, L. Leal-Taixe´, “MOT20: A benchmark for multi object tracking in crowded scenes,” CoRR, abs/2003.09003, 2020.
R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” Transactions of the ASME–Journal of Basic Engineering, 82(Series D), 35–45,
1960.
P. C. Mahalanobis, “On the Generalised Distance in Statistics,” Proceedings of the National Institute of Sciences of India, 2(1), 49—55, 1936.
R. Jonker, A. Volgenant, “A shortest augmenting path algorithm for dense and sparse linear assignment problems,” Computing, 38(4), 325–340, 1987.
S. Li, M. Tezcan, P. Ishwar, J. Konrad, “Supervised People Counting Using An Overhead Fisheye Camera,” in 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 1–8, 2019, doi: 10.1109/AVSS.2019.8909877.
X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W. kin Wong, W. chun Woo, “Con- volutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting,” 2015.
S. A. Rahman, D. A. Adjeroh, “Deep Learning using Convolutional LSTM estimates Biological Age from Physical Activity,” Scientific Reports, 2019, doi:10.1038/s41598-019-46850-0.
G. Vass, “Applying and removing lens distortion in post production,” 2003.
A. Rosebrock, “OpenCV People Counter,” https://pyimagesearch. com/2018/08/13/opencv-people-counter/, 2018, [Online; ac- cessed 12- Aug- 2022].
Z. Duan, M. O. Tezcan, H. Nakamura, P. Ishwar, J. Konrad, “RAPiD: Rotation- Aware People Detection in Overhead Fisheye Images,” 2020.
T. Scheck, R. Seidel, G. Hirtz, “Learning from THEODORE: A Synthetic Omnidirectional Top-View Indoor Dataset for Deep Transfer Learning,” 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), 2020,
doi:10.1109/wacv45572.2020.9093563.
B. E. Demiroz, I. Ari, O. Eroglu, A. A. Salah, L. Akarun, “Feature-based tracking on a multi-omnidirectional camera dataset,” in 2012 5th International Symposium on Communications, Control and Signal Processing, 1–5, 2012, doi:10.1109/ISCCSP.2012.6217867.
R. B. Knapp, N. F. Polys, J.-B. Huang, A. Ibrahim, N. Ma, C. Hurt, Y. xiao, “MW-18Mar Dataset,” .
U. P. d. M. G.-U. Grupo de Tratamiento de Ima´genes, “PIROPO Database: People in Indoor ROoms with Perspective and Omnidirectional cameras,” .
S. Manen, M. Gygli, D. Dai, L. V. Gool, “PathTrack: Fast Trajectory Annota- tion with Path Supervision,” 2017.
R. Stiefelhagen, K. Bernardin, R. Bowers, J. Garofolo, D. Mostefa, P. Soundararajan, “The CLEAR 2006 Evaluation,” in R. Stiefelhagen, J. Garo- folo, editors, Multimodal Technologies for Perception of Humans, 1–44, Springer Berlin Heidelberg, Berlin, Heidelberg, 2007.
H. S. P. Brauer, Camera based Human Localization and Recognition in Smart Environments, Ph.D. thesis, University of the West of Scotland, 2014.
B. Demiroz, A. Salali, L. Akarun, “Multiple person tracking using omnidirec- tional cameras,” 1231–1234, 2014, doi:10.1109/SIU.2014.6830458.
G. Gemignani, BTLD+:A Bayesian Approach to Tracking Learning Detection by Parts, Ph.D. thesis, UNIVERSIT‘A DEGLI STUDI DI MILANO, 2013.
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, Courville, Y. Bengio, “Generative adversarial nets,” in Advances in neural
information processing systems, 2672–2680, 2014.

Automatic Counting Passenger System using Online Visual Appearance Multi-Object Tracking