An Intelligent Occupancy Detection Framework Using Multimodal Sensory Data in Smart Buildings

Authors

  • Giorgia Petrocelli Department of Computer Science, Binghamton University, Binghamton, NY, USA. Author
  • Rozalia Utrata Department of Computer Science, University of Central Florida, Orlando, FL, USA. Author

Keywords:

occupancy detection; multimodal sensing; data fusion; probabilistic inference; uncertainty calibration; smart buildings

Abstract

Occupancy detection in smart buildings is a decision problem under uncertainty: single sensors fail under occlusion, drift, environmental lag, or changing schedules, while building control requires timely and calibrated presence estimates. This manuscript develops an intelligent occupancy detection framework that fuses heterogeneous sensory streams—passive infrared events, carbon dioxide, temperature, humidity, illuminance, acoustics, and plug or circuit power—through modality-specific encoders, reliability-weighted attention, temporal inference, and cost-sensitive decision making. The central contribution is a coherent probabilistic formulation that treats each sensor stream as an imperfect observation of a latent occupancy state and learns both discriminative representations and modality reliability. The framework separates representation learning, cross-modal fusion, and temporal smoothing so that missing or degraded modalities can be discounted rather than forcing a brittle fixed fusion rule. A hidden-state transition model and a conditional temporal objective enforce duration and switching regularities; calibration terms and an uncertainty decomposition support abstention, sensor maintenance, and risk-aware actuation. The paper specifies model assumptions, objective functions, inference steps, boundary conditions, and evaluation logic, and it analyzes failure modes such as carbon dioxide lag, passive infrared silence during stillness, acoustic contamination, and prior shift across zones. Because the framework is grounded in explicit probabilistic and decision-theoretic relationships, it can be instantiated with different neural encoders and label sources without altering the occupancy inference logic. The resulting design offers a technically defensible route from multimodal sensing to calibrated occupancy evidence for lighting, ventilation, and heating control.

References

1. Candanedo, L. M., & Feldheim, V. (2016). Accurate occupancy detection of an office room from light, temperature, humidity and CO2 measurements using statistical learning models. Energy and Buildings, 112, 28–39. https://doi.org/10.1016/j.enbuild.2015.11.071

2. Dodier, R. H., Henze, G. P., Tiller, D. K., & Guo, X. (2006). Building occupancy detection through sensor belief networks. Energy and Buildings, 38(9), 1033–1043. https://doi.org/10.1016/j.enbuild.2005.12.001

3. Yang, J., Santamouris, M., & Lee, S. E. (2016). Review of occupancy sensing systems and occupancy modeling methodologies for the application in institutional buildings. Energy and Buildings, 121, 344–355. https://doi.org/10.1016/j.enbuild.2015.12.018

4. Nguyen, T. A., & Aiello, M. (2013). Energy intelligent buildings based on user activity: A survey. Energy and Buildings, 56, 244–257. https://doi.org/10.1016/j.enbuild.2012.09.005

5. Agarwal, Y., Balaji, B., Gupta, R., Lyles, J., Wei, M., & Weng, T. (2010). Occupancy-driven energy management for smart building automation. In Proceedings of the 2nd ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Buildings (pp. 1–6). ACM. https://doi.org/10.1145/1878431.1878433

6. Balaji, B., Xu, J., Nwokafor, A., Gupta, R., & Agarwal, Y. (2013). Sentinel: Occupancy based HVAC actuation using existing WiFi infrastructure within commercial buildings. In Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems (pp. 1–14). ACM. https://doi.org/10.1145/2517351.2517370

7. Chen, Z., Jiang, C., & Xie, L. (2018). Building occupancy estimation and detection: A review. Energy and Buildings, 169, 157–170. https://doi.org/10.1016/j.enbuild.2018.03.031

8. Ryu, S. H., & Moon, H. J. (2016). Development of an occupancy prediction model using indoor environmental data based on machine learning techniques. Building and Environment, 107, 1–9. https://doi.org/10.1016/j.buildenv.2016.06.039

9. Saha, H., Florita, A. R., Henze, G. P., Sarkar, S., & Vaze, J. (2019). Occupancy sensing in buildings: A review of data analytics approaches. Energy and Buildings, 188–189, 263–278. https://doi.org/10.1016/j.enbuild.2019.02.020

10. Labeodan, T., Zeiler, W., Boxem, G., & Zhao, Y. (2015). Occupancy measurement in commercial office buildings for demand-driven control applications—A survey and detection system evaluation. Energy and Buildings, 93, 303–314. https://doi.org/10.1016/j.enbuild.2015.02.028

11. Liu, J., Luo, D., Fu, X., Lu, Q., & Kang, K. Y. (2022). Design strategy of multimodal perception system for smart environment. In Internet of Things for Smart Environments (pp. 93-115). Cham: Springer International Publishing.

12. Gravina, R., Alinia, P., Ghasemzadeh, H., & Fortino, G. (2017). Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges. Information Fusion, 35, 68–80. https://doi.org/10.1016/j.inffus.2016.09.005

13. Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010). Multimodal fusion for multimedia analysis: A survey. Multimedia Systems, 16(6), 345–379. https://doi.org/10.1007/s00530-010-0182-0

14. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

15. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30, pp. 5998–6008).

16. Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257–286. https://doi.org/10.1109/5.18626

17. Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (pp. 282–289). Morgan Kaufmann.

18. Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (pp. 1050–1059). PMLR.

19. Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations.

20. Calì, D., Matthes, P., Huchtemann, K., Streblow, R., & Müller, D. (2015). CO2 based occupancy detection algorithm: Experimental analysis and validation for office and residential buildings. Building and Environment, 86, 39–49. https://doi.org/10.1016/j.buildenv.2014.12.011

Downloads

Published

2026-10-11