The goal of this project was to detect chess pieces in photos of live chess games. We constructed our own hand-labeled dataset of 500 photos.

Our training set was relatively small with a lot of noise (angle of photos, background, etc.). Using Hough Transforms, we were able to find the corners of the boards and warp every image into a perfect square, which significantly reduced the noise. With these simpler images, we were able to use a relatively straightforward CNN to classify the piece on each of the 64 spaces in the board images with almost perfect accuracy.

Our paper can be found here.

• Samuel Ryan (samryan@seas)
• Mukund Venkateswaran (mukundv@seas)
• Michael Deng (michdeng@wharton)
• Kurt Convey (kconvey@seas)

1. Follow instructions below to preprocess labeled images
2. Upload the most up to real dataset notebook date notebook from clean_notebooks/ into Google Colab and update path to the preprocessed dataset.

Only necessary for a new dataset. There is a preprocessed dataset available here which is referenced in the current notebook.

$ git clone https://github.com/samryan18/chess-ray-vision.git
$ cd chess-ray-vision
$ pip install -e .
$ preprocess [OPTIONS] # or run with no options to be prompted for inputs

# Example run with inputs
$ preprocess --verbose --glob_path="path_to_images/*.jpeg" --dest_path="./preprocessed"

• Include --glob_path="path_to_directory/*.jpeg" to specify where the images to preprocess are stored and what the file format is
• Prompted for if not included

• Include --dest_path="path_to_destination_directory" to specify where to put preprocessed images
• Prompted for if not included

• Flag for whether to print info about the run
• Include --verbose to run in this mode

• Run $ preprocess --help for help.

We are nearing 100% validation accuracy.

Queen's Gambit :)

As a proof of concept, we ran our models on a much easier dataset of online chessboard imgaes.

1. Upload the most up to date easy dataset notebook from clean_notebooks/ into Google Colab and run all cells.

Preprocessing code adapted from here. (Thanks to @Elucidation)

1. Convert image to binary bitmap
2. Blur the image
3. Sobol Filter
4. Canny Edge Detectors
5. Finding Contours with cv2
6. "Prune the contours"—Ramer–Douglas–Peucker Algorithm to reconcile "almost lines"
7. Find line intersections
8. Sanity checks
   • convex hull
   • correct num points
   • check angles between lines
9. Warp images
10. Write to new files

• PyTorch (for training models)
• opencv-python (for image preprocessing)