Quantization aware training and CNN model converter for FPGA based Intuitus hardware accelerator.

Quantization aware training is based on https://github.com/SpursLipu/YOLOv3v4-ModelCompression-MultidatasetTraining-Multibackbone.git.

• Training and testing multiple Yolov3 or Yolov4 implementations
• Quantization-aware training
• Testing expected FPGA behaviour using Torch model with GPU supported computations
• Converting Torch model into Keras
• Post-training quantization using tensorflow quantizer
• Translation of quantized Torch or Keras models into FPGA interpretable commands

• conv2d (kernel sizes: [1x1,3x3,5x5]; strides: [1,2])
• inplace maxpool2d (stride 2 only)
• maxpool2d
• upsample
• concat
• split
• inplace yolo layer
• fully connected
• inverse bottleneck
• residual

cd Intuitus-converter
pip install -e .

1. Select a pretrained model and download pretrained weights
2. Use train_torch_yolo.py for quantization aware training
3. Fuse batchnormalization layer and retrain till convergence if neccessary (set quantize 0, FPGA = False)
4. Change activation to ReLU by editing cfg file (neccessary for FPGA) and retrain till convergence (keep quantize = 0)
5. Set quantize to 1 and retrain till convergence
6. Use torch_convert_postscale.py to fuse quantization scaling to a single scale (shift)
7. Set quantize to 2 and test if mAP has changed to to scale fusing (should not have changed)
8. Set FPGA = True to test which mAP can be achieved using the Intuitus hardware accelerator
9. Generate commands for the Intuitus hardware accelerator using generate_intuitus_from_torch.py
10. Test the results on hardware following the steps in the related projects

Lukas Baischer lukas_baischer@gmx.at