A reinforcement learning approach for individualizing erythropoietin dosages in hemodialysis patients

José D. Martín-Guerrero, Faustino Gomez, Emilio Soria-Olivas, Jürgen Schmidhuber, Mónica Climente-Martí, N. Víctor Jiménez-Torres

Research output: Contribution to journalArticlepeer-review

46 Scopus citations


This paper presents a reinforcement learning (RL) approach for anemia management in patients undergoing chronic renal failure. Erythropoietin (EPO) is the treatment of choice for this kind of anemia but it is an expensive drug and with some dangerous side-effects that should be considered especially for patients who do not respond to the treatment. Therefore, an individualized treatment appears to be necessary. RL is a suitable approach to tackle this problem. Moreover, resulting policies are similar to medical protocols, and hence, they can easily be transferred to daily practice. A cohort of 64 patients are included in the study. An implementation of the Q-learning algorithm based on a state-aggregation table and another implementation using the multi-layer perceptron as a function approximator (Q-MLP) are compared with the protocols followed in the Nephrology Unit. The policy obtained by the Q-MLP approach outperforms the hospital policy in terms of the ratio of patients that are within the targeted range of hemoglobin (11.5-12.5 g/dl) at the end of the analyzed period, since an increase of 25% is observed. It ensures an improvement in patients' quality-of-life and considerable economic savings for the health care system due to both the expensiveness of EPO treatment and the costs incurred by the health care system in order to alleviate problems related to EPO over-dosing. It should be pointed out that the approach presented here is completely general, and therefore, it can be applied to any problem of drug dosage optimization. © 2009 Elsevier Ltd. All rights reserved.
Original languageEnglish (US)
Pages (from-to)9737-9742
Number of pages6
JournalExpert Systems with Applications
Issue number6
StatePublished - Aug 1 2009
Externally publishedYes

Bibliographical note

Generated from Scopus record by KAUST IRTS on 2022-09-14

ASJC Scopus subject areas

  • Artificial Intelligence
  • General Engineering
  • Computer Science Applications


Dive into the research topics of 'A reinforcement learning approach for individualizing erythropoietin dosages in hemodialysis patients'. Together they form a unique fingerprint.

Cite this