ThreeJs and Reinforcement Learning in 3D

I built a simple 3D game in November 2024 mostly for fun. The objective was to minimize the time taken for a player to destroy all the pillars. Then I got a brilliant (in hindsight not so brilliant) idea 💡. Why not add a Reinforcement Learning (RL) agent to play the game I made? I’d always wanted to do some hands-on RL. Sounded like a great idea with the best of intentions, what could possibly have gone wrong.

If you wanted to view the current version of the game find it here

Project Overview

This project featured a robot and an ‘alien’ character navigating through a custom-designed 3D environment.

• Gameplay Objective: The primary goal was to minimize the time it took for the RL agent to destroy all the pillars within the environment. Each pillar, when destroyed, scored 100 points, adding a competitive edge to the gameplay.
• Environment: A 3D space where the robot and ‘alien’ character interacted with pillars. Here’s a visual representation of the game:

Reinforcement Learning Agent

• Initial Setup: The RL agent started with an epsilon value of around 0.4. This meant:
• 40% of the actions taken by the agent were random (exploratory), allowing it to discover new strategies.
• 60% of the actions were based on exploitation, maximizing reward based on previously learned information.
• Learning Process: As the game progressed, the epsilon value decreased, transitioning the agent from exploration to exploitation, refining its strategy over time. To decrease epsilon I divided it by a constant factor I had hardcoded.
• Current Status: The agent encountered an issue where it got stuck after destroying approximately 4 pillars. The robot character also had jerky motion that just didn’t look good especially at the start. This was where development paused, but hopefully there was potential for further refinement.

Here’s an earlier screen recording I made below

Features

• Destruction Mechanics: The game featured dynamic explosions when the character impacted a pillar, enhancing the visual and interactive experience.
• Scoring System: Each pillar destroyed awarded the player 100 points, encouraging efficiency and strategy in gameplay.

Technologies Used

• Three.js: For rendering the 3D environment and characters.
• ReactJS: For managing the game’s UI and state, if you were interested in contributing or experimenting with the RL agent.

Getting Started

If you were interested in continuing the development or experimenting with the RL agent, here’s what you needed:

1. Basic Knowledge: Familiarity with ReactJS and 3D modeling would have been beneficial.
2. Environment Setup: Ensure you had Node.js installed for ReactJS development. Or you could have used an online IDE like stackblitz (honestly that was the same environment I used to code this)
3. The link to the repository was here

Follow the Development

For updates and progress on this project, follow me on X (formerly Twitter).

Acknowledgements

The initial template for the project was made with bolt.new.

Feel free to fork the repository, contribute, or reach out with suggestions or questions. Let’s bring this game to its full potential!