In this paper, we study the numerical approximation of a system of PDEs which arises from an optimal control problem for the time-fractional Fokker-Planck equation with time-dependent drift. The system is composed of a backward time-fractional Hamilton-Jacobi-Bellman equation and a forward time-fractional Fokker-Planck equation. We approximate Caputo derivatives in the system by means of L1 schemes and the Hamiltonian by finite differences. The scheme for the Fokker-Planck equation is constructed in such a way that the duality structure of the PDE system is preserved on the discrete level. We prove the well-posedness of the scheme and the convergence to the solution of the continuous problem.