NetworkGym: Democratizing Network AI via Sim-aaS.#

NetworkGym is an innovative Simulation-as-a-Service (Sim-aaS) framework to democratize Network AI research and development, based on the streamlined concept of “agent-environment loop” as seen in reinforcement learning.

_images/network_gym_intro.png

Objectives#

The NetworkGym Sim-aaS framework consists of four key components: Agent, Client, Server, and Environment, and is designed to achieve the following objectives:

  • AI Development with No Knowledge of Network Simulation: (AI algorithm) Agent is fully customizable and controlled by developer. (network simulation) Environment is hosted in the cloud. By utilizing open-source based NetworkGym Client and APIs, an Agent can interact with the Environment to collect measurement data and inject action, and perform AI training for the configured use-case without any network simulation expertise.

  • Flexibility in Language and Platform: The separation of Agent and Environment provides the freedom to employ different programming languages for AI and network simulation respectively. For instance, a Python-based Agent can smoothly interact with a C++(ns-3) based simulation Environment.

  • Controlled Access: The access to Environment is controlled through NetworkGym APIs to hide the details of how a function or feature is implemented in the Environment from developers. This controlled access maintains security and oversight.

  • Independent Deployment: Separating Agent and Environment allows them to be deployed on different machines or platforms, optimized for specific workload. They can also be developed and maintained by different entities.

Agent#

NetworkGym offers four types of network agents, each serving a specific purpose when interacting with NetworkGym Environments:

  • System Default Agent: By sending an “empty” action to the environment, this agent enables the collection of data for offline training purposes.

  • Custom Algorithm Agent: A flexible option that allows users to define their own specialized agents.

  • Stable-Baselines3 Agent: A RL agent includes the State-of-the-Art (SOTA) Reinforcement Learning (RL) algorithms sourced from Stable-Baselines3.

  • CleanRL Agent: A CleanRL agent is available for custom algorithms.

We have included additional information on how to train the agent in the Training Agents tutorial. This tutorial provides detailed guidance and instructions for effectively training your agent using various network agents offered by NetworkGym.

Gymnasium API#

The interaction between the network agent and the environment is facilitated through the standard Gymnasium API, including action (↓), observation (↑), and reward (↑).

NetworkGym API#

NetworkGym consists of three essential components and two interfaces, each playing a crucial role in the framework:

  • NetworkGym Client: connects Agent to the Server, configures the Environment, and enables data conversion between Gym and NetworkGym format.

    • Northbound Interface: interface between Client and Server, including network configuration (↓), action (↓), and network stats (↑).

  • NetworkGym Server: manages connections between Clients and Environments.

    • Southbound Interface: interface between Server and Environment, including network configuration (↓), action (↓), and network stats (↑).

  • NetworkGym Environment: simulates or implements network environments tailored for various network AI use cases. E.g., open-source network simulation tools such as ns-3.

For more comprehensive information about the key components of NetworkGym, please refer to the Overview page.

NetworkGym Scope and Limitations#

✔️ In-Scope:

  • NetworkGym focuses on the development of open and reference AI models/algorithms tailored for networking research (Layer 2 and above).

  • It also involves the development of abstract models, such as channel, compute, power, etc., simulating aspects of the physical world and systems.

  • NetworkGym endeavors to create full-stack models that can effectively simulate end-to-end (Access/Edge/Cloud) network operations with high fidelity.

❌ Out-of-Scope:

  • NetworkGym does not involve the development of proprietary AI algorithms or models.

  • It is not centered around creating AI applications or software, like xApp/rApp, based on specific controller architectures (e.g., O-RAN RIC).

  • NetworkGym is not designed to create a Digital Twin to simulate the physical world with high fidelity.