|
|
--- |
|
|
license: apache-2.0 |
|
|
pipeline_tag: reinforcement-learning |
|
|
tags: |
|
|
- rlgym |
|
|
- rocket-league |
|
|
- RLBot |
|
|
- PPO |
|
|
--- |
|
|
|
|
|
# CanoPy |
|
|
|
|
|
CanoPy is a self-playing reinforcement learning Rocket League agent designed for the `RLBot Championship 2025`. |
|
|
It uses PPO (Proximal Policy Optimization) to learn 2v2 gameplay through self-play. The agent is trained to play effectively on both blue and orange teams and can generalize to various team compositions. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Framework:** RLGym + RLBot v5 |
|
|
- **Algorithm:** PPO (via `rlgym-ppo`) |
|
|
- **Team size:** 2v2 |
|
|
- **Action repeat:** 8 |
|
|
- **Observations:** `DefaultObs` with normalized positions, angles, velocities, and boost |
|
|
- **Action space:** Lookup table actions with repeat frames |
|
|
- **Reward shaping:** Combined reward including: |
|
|
- Speed toward ball |
|
|
- In-air bonus |
|
|
- Ball velocity toward goal |
|
|
- Goal scoring reward |
|
|
|
|
|
 |
|
|
 |
|
|
|
|
|
## Training Configuration (from `config.json`) |
|
|
|
|
|
- **Number of processes:** 4 |
|
|
- **Minimum inference ratio:** 80% |
|
|
- **Steps per checkpoint:** 1,000,000 |
|
|
- **PPO batch size:** 100,000 |
|
|
- **PPO minibatch size:** 50,000 |
|
|
- **PPO epochs per update:** 2 |
|
|
- **Experience buffer size:** 300,000 |
|
|
- **Policy network layers:** [256, 128] |
|
|
- **Critic network layers:** [256, 128] |
|
|
- **Policy learning rate:** 0.0001 |
|
|
- **Critic learning rate:** 0.0001 |
|
|
- **PPO entropy coefficient:** 0.01 |
|
|
- **Standardize returns:** true |
|
|
- **Standardize observations:** false |
|
|
- **Total training steps:** 1,000,000,000 |
|
|
- **Checkpoint directory:** ./checkpoints |
|
|
|
|
|
## Intended Use |
|
|
|
|
|
CanoPy is intended for research, competition, and experimentation within the RLBot framework. It is designed to compete in the ML bot bracket of the RLBot Championship 2025. |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- Performance is dependent on training; untrained or partially trained models may perform poorly. |
|
|
- The bot has been trained for standard Rocket League 2v2 matches; it may not generalize to unusual map sizes, mutators, or game modes. |
|
|
- Does not include human-like strategy beyond what PPO has learned from self-play. |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
CanoPy can be evaluated using the `evaluate()` function in the training script. Expected evaluation includes average episode returns and gameplay against copies of itself. |
|
|
- **Note:** To meet RLBot Championship submission requirements, further testing against Psyonix Pro bots may be necessary. |
|
|
|
|
|
## Contact / Author |
|
|
|
|
|
- **Author:** FlameF0X /// Discord handler `@flame_f0x` |
|
|
- **Competition:** RLBot Championship 2025 |
|
|
|
