Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics

Matteo Togninalli, Xu Wang, Tim Kucera, Sandesh Shrestha, Philomin Juliana, Suchismita Mondal, Francisco Pinto, Velu Govindan, Leonardo Crespo-Herrera, Julio Huerta-Espino, Ravi P Singh, Karsten Borgwardt, Jesse Poland

Research output: Contribution to journalArticlepeer-review


Motivation: Developing new crop varieties with superior performance is highly important to ensure robust and sustainable global food security. The speed of variety development is limited by long field cycles and advanced generation selections in plant breeding programs. While methods to predict yield from genotype or phenotype data have been proposed, improved performance and integrated models are needed. Results: We propose a machine learning model that leverages both genotype and phenotype measurements by fusing genetic variants with multiple data sources collected by unmanned aerial systems. We use a deep multiple instance learning framework with an attention mechanism that sheds light on the importance given to each input during prediction, enhancing interpretability. Our model reaches 0.754 ± 0.024 Pearson correlation coefficient when predicting yield in similar environmental conditions; a 34.8% improvement over the genotype-only linear baseline (0.559 ± 0.050). We further predict yield on new lines in an unseen environment using only genotypes, obtaining a prediction accuracy of 0.386 ± 0.010, a 13.5% improvement over the linear baseline. Our multi-modal deep learning architecture efficiently accounts for plant health and environment, distilling the genetic contribution and providing excellent predictions. Yield prediction algorithms leveraging phenotypic observations during training therefore promise to improve breeding programs, ultimately speeding up delivery of improved varieties.
Original languageEnglish (US)
StatePublished - May 23 2023

Bibliographical note

KAUST Repository Item: Exported on 2023-05-26
Acknowledgements: We sincerely appreciate the support of CIMMYT field staff with assistance in management of the field trials. Byron Evers and Mark Lucas made valuable contributions to data management and organization and Shuangye Wu for genotyping support. This material is based upon work supported by the National Science Foundation under Grant No. (1238187), the Feed the Future Innovation Lab for Applied Wheat Genomics through the U.S. Agency for International Development (Contract No AID-OAA-A-13- 00051) and the U.S. NIFA International Wheat Yield Partnership (grant no. 2017-67007-25933/project accession no. 1011391).

ASJC Scopus subject areas

  • Biochemistry
  • Computational Theory and Mathematics
  • Computational Mathematics
  • Molecular Biology
  • Statistics and Probability
  • Computer Science Applications


Dive into the research topics of 'Multi-modal deep learning improves grain yield prediction in wheat breeding by fusing genomics and phenomics'. Together they form a unique fingerprint.

Cite this