Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation

Kipngetich Gideon; Victor Muthama Musau; Margaret Wambui Kinyua

doi:doi:10.11648/j.ijdsa.20261202.11

Research Article |

| Peer-Reviewed

Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation

Kipngetich Gideon^*

, Victor Muthama Musau, Margaret Wambui Kinyua

Published in International Journal of Data Science and Analysis (Volume 12, Issue 2)

Received: 18 April 2026 Accepted: 3 May 2026 Published: 10 June 2026

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

Malaria has been one of the major public health issues that has not been extensively addressed. Controlling the spread of infectious diseases in space and time requires robust adaptive policies that significantly account for heterogeneity, uncertainty, and optimal sequential decision-making. This study presents an innovative framework that integrates Bayesian spatiotemporal modeling with reinforcement learning (RL) with the 5D3 algorithm. The disease risk at location i and time t is modeled using a logistic regression with spatial random effects and Bayesian inference performed using the non-reversible Metropolis-Hastings algorithm, and the parameter estimates are used to calibrate a stochastic reinforcement learning environment via episodic parameter sampling. The study identified significant drivers of malaria risk: rainfall, temperature, secondary and tertiary levels of education, higher wealth index, female gender, treated nets, and spray repellents, while quantifying uncertainty via credible intervals. The spatial random effect captured unmeasured local heterogeneity, and the temporal effect accounted for seasonality, which is essential for reliable parameter estimation. Therefore, a reinforcement learning agent can learn optimal, spatially adaptive intervention policies under uncertainty, making the model suitable for public health decision-making where spatial heterogeneity and uncertainty are prominent. The proposed calibrated model within a policy-learning environment using posterior samples can be replicated to simulate realistic transmission scenarios for malaria and evaluate dynamic control strategies.

Published in	International Journal of Data Science and Analysis (Volume 12, Issue 2)
DOI	10.11648/j.ijdsa.20261202.11
Page(s)	17-24
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2026. Published by Science Publishing Group

Keywords

Geospatial Calibration, Bayesian, Reinforcement Learning, Malaria, Transmission, Parameter Estimation

References

[1]	P. Gholizadeh, M. Sanogo, A. Oumarou, M. N. Mohamed, Y. Cissoko, M. S. Sow, P. Pagliano, P. Akouda, S. Soufiane, A. A. Iknane, et al., “Fighting covid-19 in the west africa after experiencing the ebola epidemic,” Health Promotion Perspectives, vol. 11, no. 1, p. 5, 2021.
[2]	W. Avis, “Malaria, hiv and tb in tanzania: Epidemiology, disease control challenges and interventions,” 2022.
[3]	L. Palombi and S. Moramarco, “Health in sub-saharan africa: Hiv, tb and malaria epidemiology,” in Multidisciplinary Teleconsultation in Developing Countries, pp. 3–16, Springer, 2018.
[4]	A. Yu, “Practical compartmental models for infectious disease dynamics in closed populations,” Intelligence Planet Journal of Mathematics and Its Applications, vol. 2, no. 4, 2025.
[5]	J. A. L. Marques, F. N. B. Gois, J. Xavier-Neto, and S. J. Fong, “Epidemiology compartmental models sir, seir, and seir with intervention,” in Predictive Models for Decision Support in the COVID-19 Crisis, pp. 15–39, Springer, 2020.
[6]	R. Van de Schoot, S. Depaoli, R. King, B. Kramer, K. Martens, M. G. Tadesse, M. Vannucci, A. Gelman, D. Veen, J. Willemsen, et al., “Bayesian statistics and modelling,” Nature Reviews Methods Primers, vol. 1, no. 1, p. 1, 2021.
[7]	B. J. Reich and S. K. Ghosh, Bayesian statistical methods. Chapman and Hall/CRC, 2019.
[8]	C. Szepesvari, Algorithms for reinforcement learning. Springer nature, 2022.
[9]	M. Morris, K. Wheeler-Martin, D. Simpson, S. J. Mooney, A. Gelman, and C. DiMaggio, “Bayesian hierarchical spatial models: Implementing the besag york mollie´ model in stan,” Spatial and spatio-temporal epidemiology, vol. 31, p. 100301, 2019.
[10]	B. Schrodle and L. Held, “Spatio-temporal disease mapping using inla,” Environmetrics, vol. 22, no. 6, pp. 725–734, 2011.
[11]	M. L. Ozbilen, E. Eugriboz, R. Halepmollasi, I. Bilgen, and M. Haklidir, “Deep reinforcement learning for simulation-based determination of covid-19 pandemic mitigation policies,” Artificial Intelligence Theory and Applications, vol. 1, no. 2, pp. 29–38, 2021.
[12]	S. N. Khatami and C. Gopalappa, “Deep reinforcement learning framework for controlling infectious disease outbreaks in the context of multi-jurisdictions,” medRxiv, pp. 2022–10, 2022.
[13]	P. J. K. Libin, A. Moonens, T. Verstraeten, F. Perez-Sanjines, N. Hens, P. Lemey, and A. Nowe, “Deep reinforcement learning for large-scale epidemic control,” in Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 155–170, Springer, 2020.
[14]	S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in International conference on machine learning, pp. 1587–1596, PMLR, 2018.
[15]	Q. Lan, Y. Pan, A. Fyshe, and M. White, “Maxmin q-learning: Controlling the estimation bias of q-learning,” arXiv preprint arXiv: 2002.06487, 2020.
[16]	M. Vialaret and F. Maire, “On the convergence time of some non-reversible markov chain monte carlo methods,” Methodology and Computing in Applied Probability, vol. 22, no. 3, pp. 1349–1387, 2020.
[17]	Y. Song, P. N. Suganthan, W. Pedrycz, J. Ou, Y. He, Y. Chen, and Y. Wu, “Ensemble reinforcement learning: A survey,” Applied Soft Computing, vol. 149, p. 110975, 2023.
[18]	A. Katsevich, “Improved dimension dependence in the bernstein–von mises theorem via a new laplace approximation bound,” Information and Inference: A Journal of the IMA, vol. 14, no. 3, p. iaaf020, 2025.
[19]	E. A. Mordecai, K. P. Paaijmans, L. R. Johnson, C. Balzer, T. Ben-Horin, E. De Moor, A. McNally, S. Pawar, S. J. Ryan, T. C. Smith, et al., “Optimal temperature for malaria transmission is dramatically lower than previously predicted,” Ecology letters, vol. 16, no. 1, pp. 22–30, 2013.
[20]	P. E. Parham and E. Michael, “Modeling the effects of weather and climate change on malaria transmission,” Environmental health perspectives, vol. 118, no. 5, p. 620, 2009.
[21]	L. S. Tusting, B. Willey, H. Lucas, J. Thompson, H. T. Kafy, R. Smith, and S. W. Lindsay, “Socioeconomic development as an intervention against malaria: a systematic review and meta-analysis,” The Lancet, vol. 382, no. 9896, pp. 963–972, 2013.
[22]	M. Ngutu, D. O. Omia, T. O. Ngage, C. A. Oduor, N. O. Ouko, B. Oingo, I. Oluoch, S. Kariuki, J. Chikovore, W. Onyango-Ouma, et al., “Gender-related factors affecting community malaria-related perceptions and practices in migori county, kenya,” Malaria journal, vol. 24, no. 1, p. 196, 2025.
[23]	G. M. Diiro, H. D. Affognon, B. W. Muriithi, S. K. Wanja, C. Mbogo, and C. Mutero, “The role of gender on malaria preventive behaviour among rural households in kenya,” Malaria Journal, vol. 15, no. 1, p. 14, 2016.

Cite This Article

Plain Text BibTeX RIS

APA Style

Gideon, K., Musau, V. M., Kinyua, M. W. (2026). Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation. International Journal of Data Science and Analysis, 12(2), 17-24. https://doi.org/10.11648/j.ijdsa.20261202.11

Copy | Download

ACS Style

Gideon, K.; Musau, V. M.; Kinyua, M. W. Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation. Int. J. Data Sci. Anal. 2026, 12(2), 17-24. doi: 10.11648/j.ijdsa.20261202.11

Copy | Download

AMA Style

Gideon K, Musau VM, Kinyua MW. Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation. Int J Data Sci Anal. 2026;12(2):17-24. doi: 10.11648/j.ijdsa.20261202.11

Copy | Download

@article{10.11648/j.ijdsa.20261202.11,
  author = {Kipngetich Gideon and Victor Muthama Musau and Margaret Wambui Kinyua},
  title = {Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation
},
  journal = {International Journal of Data Science and Analysis},
  volume = {12},
  number = {2},
  pages = {17-24},
  doi = {10.11648/j.ijdsa.20261202.11},
  url = {https://doi.org/10.11648/j.ijdsa.20261202.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ijdsa.20261202.11},
  abstract = {Malaria has been one of the major public health issues that has not been extensively addressed. Controlling the spread of infectious diseases in space and time requires robust adaptive policies that significantly account for heterogeneity, uncertainty, and optimal sequential decision-making. This study presents an innovative framework that integrates Bayesian spatiotemporal modeling with reinforcement learning (RL) with the 5D3 algorithm. The disease risk at location i and time t is modeled using a logistic regression with spatial random effects and Bayesian inference performed using the non-reversible Metropolis-Hastings algorithm, and the parameter estimates are used to calibrate a stochastic reinforcement learning environment via episodic parameter sampling. The study identified significant drivers of malaria risk: rainfall, temperature, secondary and tertiary levels of education, higher wealth index, female gender, treated nets, and spray repellents, while quantifying uncertainty via credible intervals. The spatial random effect captured unmeasured local heterogeneity, and the temporal effect accounted for seasonality, which is essential for reliable parameter estimation. Therefore, a reinforcement learning agent can learn optimal, spatially adaptive intervention policies under uncertainty, making the model suitable for public health decision-making where spatial heterogeneity and uncertainty are prominent. The proposed calibrated model within a policy-learning environment using posterior samples can be replicated to simulate realistic transmission scenarios for malaria and evaluate dynamic control strategies.
},
 year = {2026}
}

Copy | Download

TY  - JOUR
T1  - Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation

AU  - Kipngetich Gideon
AU  - Victor Muthama Musau
AU  - Margaret Wambui Kinyua
Y1  - 2026/06/10
PY  - 2026
N1  - https://doi.org/10.11648/j.ijdsa.20261202.11
DO  - 10.11648/j.ijdsa.20261202.11
T2  - International Journal of Data Science and Analysis
JF  - International Journal of Data Science and Analysis
JO  - International Journal of Data Science and Analysis
SP  - 17
EP  - 24
PB  - Science Publishing Group
SN  - 2575-1891
UR  - https://doi.org/10.11648/j.ijdsa.20261202.11
AB  - Malaria has been one of the major public health issues that has not been extensively addressed. Controlling the spread of infectious diseases in space and time requires robust adaptive policies that significantly account for heterogeneity, uncertainty, and optimal sequential decision-making. This study presents an innovative framework that integrates Bayesian spatiotemporal modeling with reinforcement learning (RL) with the 5D3 algorithm. The disease risk at location i and time t is modeled using a logistic regression with spatial random effects and Bayesian inference performed using the non-reversible Metropolis-Hastings algorithm, and the parameter estimates are used to calibrate a stochastic reinforcement learning environment via episodic parameter sampling. The study identified significant drivers of malaria risk: rainfall, temperature, secondary and tertiary levels of education, higher wealth index, female gender, treated nets, and spray repellents, while quantifying uncertainty via credible intervals. The spatial random effect captured unmeasured local heterogeneity, and the temporal effect accounted for seasonality, which is essential for reliable parameter estimation. Therefore, a reinforcement learning agent can learn optimal, spatially adaptive intervention policies under uncertainty, making the model suitable for public health decision-making where spatial heterogeneity and uncertainty are prominent. The proposed calibrated model within a policy-learning environment using posterior samples can be replicated to simulate realistic transmission scenarios for malaria and evaluate dynamic control strategies.

VL  - 12
IS  - 2
ER  -

Copy | Download

Author Information

Kipngetich Gideon

Pure and Applied Sciences, Kirinyaga University, Kerugoya, Kenya

Contact Email

http://orcid.org/0000-0003-2874-4266
Victor Muthama Musau

Pure and Applied Sciences, Kirinyaga University, Kerugoya, Kenya

Contact Email
Margaret Wambui Kinyua

Mathematics, Statistics and Actuarial Sciences, Karatina University, Karatina, Kenya

Contact Email

Download PDF

Submit an Article

Sections

Plain Text BibTeX RIS

APA Style

Gideon, K., Musau, V. M., Kinyua, M. W. (2026). Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation. International Journal of Data Science and Analysis, 12(2), 17-24. https://doi.org/10.11648/j.ijdsa.20261202.11

Copy | Download

ACS Style

Gideon, K.; Musau, V. M.; Kinyua, M. W. Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation. Int. J. Data Sci. Anal. 2026, 12(2), 17-24. doi: 10.11648/j.ijdsa.20261202.11

Copy | Download

AMA Style

Gideon K, Musau VM, Kinyua MW. Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation. Int J Data Sci Anal. 2026;12(2):17-24. doi: 10.11648/j.ijdsa.20261202.11

Copy | Download

@article{10.11648/j.ijdsa.20261202.11,
  author = {Kipngetich Gideon and Victor Muthama Musau and Margaret Wambui Kinyua},
  title = {Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation
},
  journal = {International Journal of Data Science and Analysis},
  volume = {12},
  number = {2},
  pages = {17-24},
  doi = {10.11648/j.ijdsa.20261202.11},
  url = {https://doi.org/10.11648/j.ijdsa.20261202.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.ijdsa.20261202.11},
  abstract = {Malaria has been one of the major public health issues that has not been extensively addressed. Controlling the spread of infectious diseases in space and time requires robust adaptive policies that significantly account for heterogeneity, uncertainty, and optimal sequential decision-making. This study presents an innovative framework that integrates Bayesian spatiotemporal modeling with reinforcement learning (RL) with the 5D3 algorithm. The disease risk at location i and time t is modeled using a logistic regression with spatial random effects and Bayesian inference performed using the non-reversible Metropolis-Hastings algorithm, and the parameter estimates are used to calibrate a stochastic reinforcement learning environment via episodic parameter sampling. The study identified significant drivers of malaria risk: rainfall, temperature, secondary and tertiary levels of education, higher wealth index, female gender, treated nets, and spray repellents, while quantifying uncertainty via credible intervals. The spatial random effect captured unmeasured local heterogeneity, and the temporal effect accounted for seasonality, which is essential for reliable parameter estimation. Therefore, a reinforcement learning agent can learn optimal, spatially adaptive intervention policies under uncertainty, making the model suitable for public health decision-making where spatial heterogeneity and uncertainty are prominent. The proposed calibrated model within a policy-learning environment using posterior samples can be replicated to simulate realistic transmission scenarios for malaria and evaluate dynamic control strategies.
},
 year = {2026}
}

Copy | Download

TY  - JOUR
T1  - Bayesian Geospatial Calibration of Reinforcement Learning for Malaria Transmission Control: Parameter Estimation

AU  - Kipngetich Gideon
AU  - Victor Muthama Musau
AU  - Margaret Wambui Kinyua
Y1  - 2026/06/10
PY  - 2026
N1  - https://doi.org/10.11648/j.ijdsa.20261202.11
DO  - 10.11648/j.ijdsa.20261202.11
T2  - International Journal of Data Science and Analysis
JF  - International Journal of Data Science and Analysis
JO  - International Journal of Data Science and Analysis
SP  - 17
EP  - 24
PB  - Science Publishing Group
SN  - 2575-1891
UR  - https://doi.org/10.11648/j.ijdsa.20261202.11
AB  - Malaria has been one of the major public health issues that has not been extensively addressed. Controlling the spread of infectious diseases in space and time requires robust adaptive policies that significantly account for heterogeneity, uncertainty, and optimal sequential decision-making. This study presents an innovative framework that integrates Bayesian spatiotemporal modeling with reinforcement learning (RL) with the 5D3 algorithm. The disease risk at location i and time t is modeled using a logistic regression with spatial random effects and Bayesian inference performed using the non-reversible Metropolis-Hastings algorithm, and the parameter estimates are used to calibrate a stochastic reinforcement learning environment via episodic parameter sampling. The study identified significant drivers of malaria risk: rainfall, temperature, secondary and tertiary levels of education, higher wealth index, female gender, treated nets, and spray repellents, while quantifying uncertainty via credible intervals. The spatial random effect captured unmeasured local heterogeneity, and the temporal effect accounted for seasonality, which is essential for reliable parameter estimation. Therefore, a reinforcement learning agent can learn optimal, spatially adaptive intervention policies under uncertainty, making the model suitable for public health decision-making where spatial heterogeneity and uncertainty are prominent. The proposed calibrated model within a policy-learning environment using posterior samples can be replicated to simulate realistic transmission scenarios for malaria and evaluate dynamic control strategies.

VL  - 12
IS  - 2
ER  -

Copy | Download