Add class/bot registering functionality

* Add class/bot registering functionality. * Refactor .md files. * Run black linter.
giogix2 · Oct 16, 2024 · e5fab0d · e5fab0d
1 parent 9db8227
commit e5fab0d
Show file tree

Hide file tree

Showing 11 changed files with 318 additions and 89 deletions.
diff --git a/README.md b/README.md
@@ -1,10 +1,18 @@
 # Pygame_spiel: Pygame-based UI to play board Reinforcement Learning games from OpenSpiel
 PygameSpiel is a [Pygame](https://www.pygame.org)-based library to play board games from the library [OpenSpiel](https://github.com/google-deepmind/open_spiel) against AI algorithms.
 
-## Install
-```bash
-pip install pygame_spiel
-```
+
+Pygame_spiel provides a graphical UI interface to play games from the library [OpenSpiel](https://github.com/google-deepmind/open_spiel). While by default Pygame_spiel allows to play against a few existing algorithms from OpenSpiel, it also enables users who are working with OpenSpiel to quickly test new algorithms. With this in mind, Pygame_spiel offers the possibility to plugin new BOTs in runtime for quick testing.
+
+## Index
+* [Installation](docs/installation.md)
+* [Get started](docs/get_started.md)
+
+## Overview
+
+Use your mouse to select the cell (tic tac toe) or select pawn and destination cell (breakthrough).
+
+![breakthrough_tic_tac_toe](https://github.com/giogix2/pygame_spiel/assets/5859539/dd5f8709-f383-497e-8317-a113ca50d1e7)
 
 ## Version 1.0.0
 Games currently available: 
@@ -15,15 +23,3 @@ AI algorithms available:
 * mcts, DQN (currently only for breakthrough)
 
 **more to come...**
-
-## Overview
-Run Pygame_spiel with:
-
-```bash
-pygame_spiel
-```
-![menu](https://github.com/giogix2/pygame_spiel/assets/5859539/aa0f41c8-a619-489d-bbef-592555fb8afa)
-
-Use your mouse to select the cell (tic tac toe) or select pawn and destination cell (breakthrough).
-
-![breakthrough_tic_tac_toe](https://github.com/giogix2/pygame_spiel/assets/5859539/dd5f8709-f383-497e-8317-a113ca50d1e7)
diff --git a/docs/get_started.md b/docs/get_started.md
@@ -0,0 +1,46 @@
+# Get started
+
+## Play
+If the user just wants to play with the available Bots and games, the procedure is simple. After launching PygameSpiel, the main menu appears showing two dropdown menus which include the available games and Bots. Simply select the game and BOT and click play.
+
+## Register a new bot
+To dynamically add new algorithms, Pygame_spiel uses the Bot class from pyspiel as interface (pyspiel.Bot). This class is present in the OpenSpiel library, and it's currently used as template for other available algorithms in OpenSpiel (e.g., [MCTSBot](https://github.com/google-deepmind/open_spiel/blob/master/open_spiel/python/algorithms/mcts.py), [RandomUniform](https://github.com/google-deepmind/open_spiel/blob/master/open_spiel/python/bots/uniform_random.py), [Human](https://github.com/google-deepmind/open_spiel/blob/master/open_spiel/python/bots/human.py), ...). This class offers a useful format to create a generic interface between new algorithms and Pygame_spiel. The user can create a new class, which includes the logic of their new algorithm.
+
+### Create a Bot class
+In the snippet below we show an example of a simple Bot which always select the first legal action:
+
+```python
+import pyspiel
+
+class Dummy(pyspiel.Bot):
+
+  def __init__(self, game, player_id):
+    pyspiel.Bot.__init__(self)
+    self._player_id = player_id
+
+  def step(self, state):
+    legal_actions = state.legal_actions(self._player_id)
+    if not legal_actions:
+      return [], pyspiel.INVALID_ACTION
+    action = legal_actions[0] # Always choose 1st action
+    return action
+```
+This class definition can be saved in a .py file in any folder.
+**NOTE:** There are two requirements when creating a new Bot:
+1) The class needs to be inherited from pyspiel.Bot
+2) The class needs to have the two mandatory arguments **game** and **player_id**. While these might not be used by the Bot, Pygame_spiel expects these parameters.
+
+### Register the  class
+After launching Pygame_spiel, the user can provide the path the the Python file containing the Bot definition, which will be loaded and visible in the list of Bots (called "Opponent" in the main menu). In the *Module* dropdown menu navigate through the filesystem and select the Python file
+
+![alt text](images/module_dropdown_1.png)
+
+After selecting the file, the file name will be visualized below the dropdown menu as in the next image. Also, if the Python file contains classes inherited from pyspiel.Bot, the class names will be printed next to the file name. This information can be used to know if the Bots have been correctly registered or not. If no names are printed after the text "new Bots:", the classes have not been correctly registered.
+
+![alt text](images/module_dropdown_2.png)
+
+If the new classes are correctly registered, the class names will appear in the *Opponent* dropdown menu, after the Bots which are already available in Pygame_spiel by default (see the new class "Dummy" in the image below).
+
+![alt text](images/module_dropdown_3.png)
+
+After this, just click on *Play*!.
diff --git a/docs/images/module_dropdown_1.png b/docs/images/module_dropdown_1.png
diff --git a/docs/images/module_dropdown_2.png b/docs/images/module_dropdown_2.png
diff --git a/docs/images/module_dropdown_3.png b/docs/images/module_dropdown_3.png
diff --git a/docs/installation.md b/docs/installation.md
@@ -0,0 +1,36 @@
+# Installation
+
+## Just play
+If the user just wants to play with the algorithms and games available by default, without changing or adding extra functionalities to Pygame_spiel or OpenSpiel, they can just install the PyPI package and run the game. To install Pygame_spiel run:
+```bash
+pip install pygame_spiel[spiel]
+```
+Adding the parameter **[spiel]** within square brackets forces pip to install OpenSpiel as a dependency. OpenSpiel is not installed by default, because the main use case in PyGame_spiel is to support users who are working on a local clone of OpenSpiel, and by specifying it as dependency, the PyPI installation of OpenSpiel would override the local one.
+
+NOTE: if you're using a zsh terminal, specify pygame_spiel[spiel] within single quotes:
+```bash
+pip install 'pygame_spiel[spiel]'
+```
+
+To launch Pygame_spiel run:
+```bash
+pygame_spiel
+```
+
+## With the local version of OpenSpiel
+If you are working with a OpenSpiel source code locally, you can install PyGame_spiel as an extra library for local development. There are two ways to install PyGame_spiel: 1) clone the repo and install it, 2) via pip install.
+
+### Install via pip
+If the user only needs to experiment with new algorithms, and no additional games or changes in the GUI are required, the best way to install Pygame_spiel is via pip install. Unlike with the installation in the previous section, when working with an existing version of OpenSpiel installed locally, run the following:
+```bash
+pip install 'pygame_spiel'
+```
+Here we don't specify the parameter **[spiel]**, since we don't want to override the local version of OpenSpiel with the library from pip. Here we assume that OpenSpiel is already installed locally via pip (as explained in secion 4 in [OpenSpiel Installation tutorial](https://github.com/google-deepmind/open_spiel/blob/master/docs/install.md)).
+
+### Clone the repo
+If you want to customize PyGame_spiel while also working on a local version of OpenSpiel, the simplest way is to clone the repository and install it.
+1) Clone pygame_spiel
+2) run pip install .
+3) run pygame_spiel from terminal as usual to launch it
+
+By cloning the repo the user full access to the code, and the possibility to modify any part of the code (including the graphical UI and elements).
diff --git a/pygame_spiel/bots/dummy.py b/pygame_spiel/bots/dummy.py
@@ -0,0 +1,14 @@
+import pyspiel
+
+
+class Dummy(pyspiel.Bot):
+    def __init__(self, game, player_id):
+        pyspiel.Bot.__init__(self)
+        self._player_id = player_id
+
+    def step(self, state):
+        legal_actions = state.legal_actions(self._player_id)
+        if not legal_actions:
+            return [], pyspiel.INVALID_ACTION
+        action = legal_actions[0]  # Always choose 1st action
+        return action
diff --git a/pygame_spiel/games/base.py b/pygame_spiel/games/base.py
@@ -5,9 +5,14 @@
 import site
 from pathlib import Path
 import os
+import numpy as np
 
 from pygame_spiel.games.settings import SCREEN_SIZE, BREAKPOINTS_DRIVE_IDS
-from pygame_spiel.utils import init_bot, download_weights
+from pygame_spiel.utils import download_weights
+from pygame_spiel.bots import dqn
+
+from open_spiel.python.bots import uniform_random, human
+from open_spiel.python.algorithms import mcts
 
 
 class Game(metaclass=abc.ABCMeta):
@@ -25,20 +30,81 @@ def __init__(self, name, current_player):
         pygame.display.set_caption(name)
 
         self._package_path = site.getsitepackages()[0]
+        self._registered_bots = []
 
     @abc.abstractmethod
     def play(
         self, mouse_pos: t.Tuple[int, int], mouse_pressed: t.Tuple[bool, bool, bool]
     ) -> None:
         """
-        Abstact interface of the function play(). At each iteration, it requires the mouse position
+        Abstract interface of the function play(). At each iteration, it requires the mouse position
         and state (which button was pressed, if any).
 
         Parameters:
             mouse_pos (tuple): Position of the mouse (X,Y coordinates)
             mouse_pressed (tuple): 1 if the i-th button is pressed
         """
 
+    def _init_bot(
+        self,
+        bot_type: str,
+        game: pyspiel.Game,
+        player_id: int,
+        breakpoint_dir: str = None,
+    ) -> None:
+        """
+        Returns a bot of type bot_type for the player specified by player_id.
+
+        Parameters:
+            bot_type (str): Bot type (mcts, random or dqn)
+            game (pyspiel.Game): open_spiel game
+            player_id (int): id of the player that the bot will be driving
+            breakpoint_dir (str): Path to the DQN weigths (optional)
+
+        Returns:
+            None
+        """
+        if bot_type not in list(self._registered_bots.keys()) + [
+            "mcts",
+            "random",
+            "dqn",
+            "human",
+        ]:
+            ValueError("Invalid bot type: %s" % bot_type)
+        rng = np.random.RandomState(42)
+        if bot_type == "mcts":
+            utc = 2  # UCT's exploration constant
+            max_simulations = 1000
+            rollout_count = 1
+            evaluator = mcts.RandomRolloutEvaluator(rollout_count, rng)
+            solve = True  # Whether to use MCTS-Solver.
+            verbose = False
+            bot = mcts.MCTSBot(
+                game,
+                utc,
+                max_simulations,
+                evaluator,
+                random_state=rng,
+                solve=solve,
+                verbose=verbose,
+            )
+            return bot
+        if bot_type == "random":
+            bot = uniform_random.UniformRandomBot(1, rng)
+            return bot
+        if bot_type == "dqn":
+            # We need to load bots for both players, because the models have been trained
+            # using the script breakthrough_dqn.py, causing the issue reported in
+            # https://github.com/deepmind/open_spiel/issues/1104.
+            # Only the Bot related to the specified player is returned.
+            bot0 = dqn.DQNBot(game, player_id=0, checkpoint_dir=breakpoint_dir)
+            bot1 = dqn.DQNBot(game, player_id=1, checkpoint_dir=breakpoint_dir)
+            return bot0 if player_id == 0 else bot1
+        if bot_type == "human":
+            return human.HumanBot()
+
+        return self._registered_bots[bot_type](game=game, player_id=player_id)
+
     def set_bots(
         self, bot1_type: str, bot1_params: str, bot2_type: str, bot2_params: str
     ) -> None:
@@ -52,14 +118,11 @@ def set_bots(
             bot2_type (str): Bot type of player 1
             bot2_params (str): Bot's parameters (e.g., neural network breakpoints)
         """
-        # TODO self._bot_params is not used. Remove
-
-        self._bot_params = [bot1_params, bot2_params]
         self._bots = []
 
         for i, bot_type in enumerate([bot1_type, bot2_type]):
             bot_breakpoint_dir = None
-            if bot_type in ["dqn"]:
+            if bot_type in ["dqn"]:  # TODO move next code inside DQN bot definition
                 breakpoint_dest_dir = Path(
                     self._package_path,
                     "pygame_spiel/data/breakpoints",
@@ -76,7 +139,18 @@ def set_bots(
                         file_id=file_id, dest_folder=str(breakpoint_dest_dir)
                     )
                 bot_breakpoint_dir = Path(breakpoint_dest_dir, "weights_default")
-            bot = init_bot(
+            bot = self._init_bot(
                 bot_type, self._game, player_id=i, breakpoint_dir=bot_breakpoint_dir
             )
             self._bots.append(bot)
+
+    def register_bots(self, registered_bots: dict[str, type]):
+        """
+        Register a new Bot class definition.
+        This is a class which is loaded at runtime from the pygame_spiel menu.
+
+        Parameters:
+            registered_bots (dict[str, type]): dictionary with class names
+            and definitions
+        """
+        self._registered_bots = registered_bots
diff --git a/pygame_spiel/main.py b/pygame_spiel/main.py
@@ -1,8 +1,6 @@
 #!/usr/bin/env python
 
 import pygame
-import pygame_menu
-from pygame_menu import themes
 
 from pygame_spiel.games.settings import GAMES_BOTS
 from pygame_spiel.games.factory import GameFactory
@@ -18,15 +16,20 @@ def pygame_spiel():
     menu.display()
     game_name = menu.get_selected_game()
     bot_type = menu.get_selected_opponent()
+    registered_bots = menu.get_registered_bots()
 
     player_id = 0
 
+    list_available_bots = list(GAMES_BOTS[game_name].keys()) + list(
+        registered_bots.keys()
+    )
     assert (
-        bot_type in GAMES_BOTS[game_name].keys()
+        bot_type in list_available_bots
     ), f"""Bot type {bot_type} not available for game {game_name}. List of 
-        available bots: {list(GAMES_BOTS[game_name].keys())}"""
+        available bots: {list_available_bots}"""
 
     game = GameFactory.get_game(game_name, current_player=player_id)
+    game.register_bots(registered_bots)
     game.set_bots(
         bot1_type="human",
         bot1_params=None,