This is a demo project showing a proof of concept use for Machine Learning at my fifth semester. The project is not supposed to be very accurate, but only to showcase the use of ML in a real world application, using real world training data.

(back to top)

A local event agency contacted our firm to get a machine learning model of the use of their location and parking place. They would like an AI to monitor their area for 6 months to show how many people visit them, and how many cars park there. The purpose of this task is to:

1. Take an image from a security camera and process it
2. Locate the wanted objects on the given image (people + vehicles)
3. Summarize the amount of recognized objects on the image
   • I ran into several opstacles using ML.Net, and instead of switching to Python (the course was primarily about ML.Net) I decided to go with a different approach
4. Locate a known object on the image and return it as a string, if no object found return "Not found" and save the image in a learning-folder to use it later for a captcha-style learning process.
5. Create logic for further training. Save unknown images in ./unknown and save new known images in the respective folder.
   • After n new files in the known folder, re-run the training, including the new images.
   • Have a captcha-style "what is this" presented for users, to help give the model more accuracy.
     • Make a fail-safe logic for images that eg. is marked "person" when the confidence of the current model is <20 to avoid trolls
6. Output
   • Return the image via the WebAPI to the sender with a mark around the identified objects (If I can make it work on macOS, and if I have time for it)
   • Return a string/JSON from the WebAPI to the sender with a summarization of amount of identified objects the identified object, or Not found if no known object was located.
   • Return a string in a console application from the image with a summarization (Failsafe model if the API doesn't get completed in time)

(back to top)

ML Project
│   README.md
│   LICENSE.txt    
└───API
│   │───Program.cs
│   └───data
│   │   │───test        - Test dataset
│   │   │───train       - Train dataset
│   │   │───val         - Validation dataset
│   │   └───unknown     - Unknown (captcha)
│   │
│   └───Interfaces
│   │   └───IService.cs - Interface for the services used in the API
│   │
│   └───service
│   │   └───Service.cs  - Image classification services
│   │
│   └───trainer
│       └───Tainer.cs   - Model train engine (generation incrementer)
│   
│───Frontend
│   │───Program.cs
│   │───Constants
│   │   └───AsciiArt.cs - ASCII art for Nicolai aka Merlin
│   │
│   └───Pages
│       │───Admin       - Administrative area (retraining)
│       │───FAQ         - FAQ (link to this readme)
│       │───Index       - The Imagerecogniz-o-matic 2000 main page
│       └───Trainer     - A "Captcha" light, where you can help train the AI
│
└───Razor
│   │───Program.cs

(back to top)

The project is a two-layer solution consisting of an API backend and a Blazor WASM frontend.

The API serves as a pipeline from a given frontend to the ML.Net backend, receiving and dealing images and guesstimates back and forth. The frontend is simply just a Blazor WASM that requests images from the API or serves an image to the API to receive a guess.

(back to top)

(back to top)

(back to top)

• Gather enough data (and sanitize it)
• Generate the machine model and verify the accuracy of it (30% confidence is my minimum as PoC)
• Create the logic behind the object recognition
• Test the logic, and see if it is accurate/satisfying enough
  • Test the identification logic (is it a person, a car, a bike, a tree or other?)
  • Test the re-train logic to see if we can make it more accurate over time
  • Test the fail-safe logic, avoiding trolls (an image of a flower marked as a car)
• Create frontend
  • Console all-in-one (smoke test)
  • Blazor + WebAPI returning a Tuple/JSON
  • Razor + WebAPI returning an image with the identified objects (marked on the original image)
  • Razor + WebAPI returning an array of images with each of the identified objects cropped out of the original image
  • Bonus - Razor frontend (I decided to go that way because the API could be left unexposed to the WWW)

(back to top)

(back to top)

• API: GPLv3
• Frontend: GPLv3
• Razor: GPLv3

(back to top)

Jan Andreasen - jan@tved.it

Project Link: https://github.com/Thoroughbreed/H5.ML_Project

(back to top)