Self-driving AI system for GTA:SA using OpenCV and CNN

Gta_AutoPilot is a computer vision project representing AI system for an autonomous driving in GTA:San Andreas. In the project, we read frames directly from the desktop using OpenCV, instead of working with the game's code. This approach enables to use the system basically on any other game. The system can be devided into 3 parts: collecting data, learning using CNN and exploiting.

Collecting data consists in reading and processing frames using OpenCV. Parallely, we read keyboard keys, which correspond to the prediction classes. To provide derterminism in the system's predictions, we use simple driving function to enable constant car speed if possible.

Note: If you want to test already trained system on GTA:SA jump to “Example” section.

• to-do

• Run the game in windowed mode, 640x480 resolution, at the top left of your screen.
• For best perfomance, use first person view, if possible.
• Run scanner.py
• Press following keys to save frames with corresponding moves: 'O' - sraight, 'K' - left, 'L' - right.
  Interface keys: 'P' - pause/resume data collection, 'Q' - quit.

Note: As for now, for simplicity, pressing AWSD keys is not recognizable by the program. Collecting AWSD keys brings a lot of bias and complicates next steps.

• Run model.py

• Run the game in windowed mode, 640x480 resolution, at the top left of your screen.
• For best perfomance, use first person view, if possible.
• Run predictor.py Interface keys: 'P' - pause/resume autopilot, 'Q' - quit.

• Run GTA:SA in windowed mode, 640x480 resolution, at the top left of your screen.
• For best perfomance:
  • Use first person view (Use 'V' key)
  • Type SLOWITDOWN to slow the time
  • Type VROCKPOKEY to spawn the vechicle used while training
• Run predictor.py Interface keys: 'P' - pause/resume autopilot, 'Q' - quit.

• Use Alex net.
• Improve constant car speed algorithm + speed adjusting.
• Make interface more friendly and intuitive.
• Test autopilot in other game.
• Archive repo.