Dynamic Caching in Space over Heterogeneous Mega-Constellations: A DRL-Guided Approach

Caching content on servers closer to end-users is a crucial solution for Internet content providers (ICPs) to reduce the burden on backhaul traffic. The emergence of mega-constellations of satellites has created avenues to enhance global connectivity. By incorporating network caching technique into the architecture of satellite communication systems, significant advancements can be made in terms of increasing network scalability and bridging the digital divide. In this work, we consider a heterogeneous satellite constellation composed of low earth orbit (LEO) satellites and medium earth orbit (MEO) satellites that can be utilized by ICPs to deliver content to terrestrial end-users. Due to the time-varying topology resulting from the constant movement of satellites, how to optimally adapt content caching decisions at satellite servers remains an open challenge. To tackle this challenge, we propose a multi-layer optimization framework aimed at minimizing long-term operating costs, encompassing both satellite server storage expenses and network traffic expenses. Specifically, our hierarchical framework comprises a deep reinforcement learning (DRL) agent that progressively learns by accumulating experience over time to determine the optimal content caching strategy, as well as a linear programming (LP) solver for traffic flow allocation. Extensive simulations conducted using realistic satellite constellations, including OneWeb, Starlink, and O3b, validate the effectiveness of our proposed algorithm.