This repository contains a Keras model for Google's quickdraw dataset to classify sketches, an Android app for demonstration and describes the whole workflow to integrate a custom model to an Android app applying Tensorflow Lite.
- "Android" folder: Includes the Android Studio Project of the SketchRecognition App
- "SketchRecognition" folder: Directory for the complete recognition task
- "ModelTransform" folder: Scripts for model conversion (Keras -> Tensorflow -> TFLite)
I used a subset from Google's quickdraw dataset (see Quick, draw!) which contains more than 50 million drawings across 345 categories. For this project I used the simplified drawings stored in Numpy bitmaps. These bitmaps contain 28x28 grayscale images.
I used a CNN architecture presented by Tensorflow's MNIST tutorial with a few modifications.
| # classes | # training data | # test data | top-1 accuracy | top-3 accuracy |
|---|---|---|---|---|
| 20 | 8000 | 2000 | 89.76% | 95.34% |
| 345 | 4000 | 1000 | 65.57% | 82.71% |
The much higher top-3 accuracy using 345 classes is justified by the fact that there are some quite similar classes inside the dataset.
The folder "preparation_helper" contains Python and Shell scripts for preprocessing steps like dataset download, dataset instances reduction and dataset split in train and test set. "recognition" is the python project for the classification task. Besides it contains a "models" directory for all generated results (training process, images, frozen model, list of labels).
-
Download the full dataset (ca. 40GB) via script or download a subset manually (Link) and put the files into a "dataset" directory inside "SketchRecognition".
-
Adapt the parameters in
split_train_test.pyto setup the directories, to reduce the number of instances per class you want to keep and split the dataset in train and test data. Then run the script.
The classification.py script contains model training, visualisation and evaluation. Before run, adapt the dataset path.
The main function describes the complete workflow from preprocessing over the model training to the evaluation. So feel free to make changes. Again, you can decide how many instances you want to use for training and test in total.
Ubuntu 18.04
Python 3.6.5 with Anaconda
Keras version 2.2.0
Tensorflow version 1.8.0
- Convert generated Keras model (*.h5) to tensorflow frozed graph (*.pb).
Run the nice script
keras_to_tensorflow.pywritten by Amir H. Abdi (see Repo), but before specify parameters like input and output file names. - Take the generated tensorflow model (*.pb) and convert it to tensorflow lite using toco. You can run the
toco.shscript. Important note: Use the correct input and output array names and also the correct input shape size. To inspect and verify these settings you can modify and runpb_view.pywhich loads the frozen tensorflow graph (*.pb).
Take the generated Tensorflow Lite model (.tflite) and the related "labels.csv" which were generated in the training task and move it to androids "assets" directory.
macOS High Sierra v. 10.13.6
Android Studio 3.1.3
Python 3.6.3 with Anaconda
Keras 2.1.6
Tensorflow 1.7.0 (For model conversion Tensorflow 1.7 is required!)