On June 29 2021, the World Health Organization (WHO) reported around 45,951 confirmed cases and 817 deaths of COVID-19 in India, and 64,903 confirmed cases and 1,839 deaths in Brazil. This virus has been determined as a global pandemic by WHO. Accurate forecast of COVID-19 cases has become a crucial task in the decision-making of hospital managers to optimally manage the available resources and staff. In this study, the Gaussian process regression (GPR) model tuned by Bayesian optimization (BO) was used to forecast the recovered and confirmed COVID-19 cases in two highly impacted countries, India and Brazil. Specifically, the BO algorithm is employed to find the optimal hyperparameters of the GPR model to improve the forecasting quality. We compared the performance of the Optimized GPR with 14 models, including Support vector regression with different kernels, GPR with different kernels, Boosted trees, and Bagged trees. We also applied the BO to the other investigated predictors to maximize their forecasting accuracy. Three statistical criteria are used for the comparison. The daily records of confirmed and recovered cases from Brazil and India are adopted in this study. Results reveal the high performance of the GPR models compared to the other models.