Bayesian Deep Learning for Uncertainty Estimation in Urban Scene Perception Models
Keywords:
Bayesian deep learning; uncertainty calibration; urban scene perception; semantic segmentation; distribution shift; risk-aware decision makingAbstract
Urban scene perception models now support autonomous driving, municipal mapping, accessibility auditing, and infrastructure inspection, yet deterministic segmentation and detection networks often produce overconfident outputs in occluded, rare, or distribution-shifted conditions. This manuscript develops a Bayesian deep learning framework for uncertainty estimation in dense urban scene perception, with emphasis on separating epistemic model uncertainty from aleatoric observation and annotation ambiguity. The framework treats semantic labels and object-level predictions as outcomes of a posterior predictive distribution, uses variational inference and ensemble-style posterior approximation for scalable training, and defines spatial and class-aware likelihood weighting so that small but safety-critical classes are not overwhelmed during optimisation. It further specifies a calibration layer for dense predictions, an uncertainty decomposition based on predictive entropy and mutual information, and a risk-aware decision rule that permits abstention when epistemic uncertainty is too high. The analysis clarifies why pixel-independent likelihood assumptions understate spatially correlated uncertainty, how class imbalance distorts calibration, and why post hoc temperature scaling alone cannot correct posterior inadequacy under dataset shift. The contribution is methodological and analytical rather than an empirical benchmark: it provides a coherent inference and evaluation design, identifies failure mechanisms specific to urban scenes, and links uncertainty estimates to operational decisions such as relocalisation, human review, and conservative perception fallback.
References
1. Kendall, A., & Gal, Y. (2017). What uncertainties do we need in Bayesian deep learning for computer vision? In Advances in Neural Information Processing Systems (Vol. 30, pp. 5574–5584).
2. Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (PMLR Vol. 48, pp. 1050–1059).
3. Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2017). Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (Vol. 30, pp. 6402–6413).
4. Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., Abbass, H. A., & Nahavandi, S. (2021). A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76, 243–297.
5. Blundell, C., Cornebise, J., Kavukcuoglu, K., & Wierstra, D. (2015). Weight uncertainty in neural networks. In Proceedings of the 32nd International Conference on Machine Learning (PMLR Vol. 37, pp. 1613–1622).
6. Graves, A. (2011). Practical variational inference for neural networks. In Advances in Neural Information Processing Systems (Vol. 24, pp. 2348–2356).
7. Gal, Y., Hron, J., & Kendall, A. (2017). Concrete dropout. In Advances in Neural Information Processing Systems (Vol. 30, pp. 3581–3590).
8. Maddox, W. J., Izmailov, P., Garipov, T., Vetrov, D. P., & Wilson, A. G. (2019). A simple baseline for Bayesian uncertainty in deep learning. In Advances in Neural Information Processing Systems (Vol. 32).
9. Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (PMLR Vol. 70, pp. 1321–1330).
10. Aman, J., Matisziw, T. C., Kim, J. B., & Luo, D. (2022, April). Sensing the city: Leveraging geotagged social media posts and street view imagery to model urban streetscapes using deep neural networks. In Proceedings of the International Conference on Computer-Aided Architectural Design Research in Asia (pp. 595-604). Association for Computer-Aided Architectural Design Research in Asia.
11. Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016). The Cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3213–3223).
12. Geiger, A., Lenz, P., & Urtasun, R. (2012). Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3354–3361).
13. Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., & Beijbom, O. (2020). nuScenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11621–11631).
14. Kendall, A., Badrinarayanan, V., & Cipolla, R. (2017). Bayesian SegNet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding. In Proceedings of the British Machine Vision Conference.
15. Hendrycks, D., & Gimpel, K. (2017). A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations.
16. Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., & Snoek, J. (2019). Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems (Vol. 32).
17. Jospin, L. V., Laga, H., Boussaid, F., Buntine, W., & Bennamoun, M. (2022). Hands-on Bayesian neural networks—A tutorial for deep learning users. IEEE Computational Intelligence Magazine, 17(2), 29–48.
18. Sensoy, M., Kaplan, L., & Kandemir, M. (2018). Evidential deep learning to quantify classification uncertainty. In Advances in Neural Information Processing Systems (Vol. 31, pp. 3179–3189).
19. Wilson, A. G., & Izmailov, P. (2020). Bayesian deep learning and a probabilistic perspective of generalization. In Advances in Neural Information Processing Systems (Vol. 33).
20. Izmailov, P., Vikram, S., Hoffman, M. D., & Wilson, A. G. (2021). What are Bayesian neural network posteriors really like? In Proceedings of the 38th International Conference on Machine Learning (PMLR Vol. 139, pp. 4629–4640).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Computational Intelligence Systems

This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.