While playing my clarinet, I realized that it would be fun to hear other instruments playing alongside me. Instruments like guitar, piano or even a choir. It would also be nice if the melodies could be transcribed on paper. All existing solutions to these problems require a bulky computer or a cell phone. I realized that creating this compact device would combine my interest for music with my passion for engineering and math.

This project creates a small, affordable and accurate device that listens to a musical instrument and recognizes the notes played. It displays the music as a a clef or piano roll. These notes can then be sent to a synthesizer using MIDI. This allows musician to hear other instruments playing alongside with them, and allows them store their compositions.

A short video of an early prototype

Clone the repository and its submodules to a local directory. The --recursive flag automatically initializes and updates the submodules in the repository.

git clone --recursive https://github.com/jvonk/Arduino_Pitch-Detector.git

or using ssh

git clone --recursive git@github.com:jvonk/Arduino_Pitch-Detector.git

This project uses input from amplified microphone and outputs to a TFT display and USB-MIDI connection. To use the optional USB-MIDI output, you need to replace the firmware on the ATmega16U2 companion chip.

Power

Logic

To upload the sketch, the USB-MIDI switch needs to be open. The replay button was used during development to replay stored MIDI notes.

Remember to connect the 3.3 Volt output from the Arduino to the AREF input on the Arduino. If you forget this, no notes will be displayed.

For the microphone, I use the Adafruit microphone breakout, because it has a 1.25 Volt DC bias and includes an automatic gain control. The "max gain" is set to 40 dB by connecting the GAIN to 5V. Other microphones will work for as long as they have a DC biased output, and the output signal is strong enough.

The popular Arduino UNO R3 forms the heart of the system. Note that the REPLAY signal was only used during debugging. This board is now getting hard to find, but you can also use the Arduino METRO 328 or possibly other boards.

If you’re going to reprogramming the Atmega16u2, you need access the companion chip header (ICSP1) as marked in the illustration below.

The implementation changes the sound wave to an electrical signal; uses autocorrelation to find frequency played; display frequency as note on screen while sending to a MIDI synthesizer.¹

• Install the libraries.
• If switch SW1 is closed, open it and power-cycle the device.
• Using the Arduino IDE or Visual Code
  • Specify the board, COM port and baud rate.
  • Build and upload the sketch .

By default the USB port is used for USB-MIDI, so monitoring the serial port will not show anything useful.

The device is now ready to show the pitch of notes played!

If you want to connect it using USB-MIDI, continue reading.

Instructions are for Windows, but should equally apply to Linix and MacOS.

The USB on the Arduino talks USB-SERIAL with the host computer. That is how the Serial.print() statements end up on the serial monitor. A small companion chip (ATmega16U2) on the Arduino acts a UART/USB-serial bridge.

We reprogram this companion chip so the Arduino UNO R3 so that, when SW1 is closed and you power-cycle the device it appears as a MIDI peripheral (HID), and otherwise as a the usual USB-SERIAL port.

This is important that the Arduino can still behave as USB-SERIAL, because this is the usual method of uploading new sketches.

I patched dualMocoLUFA, so that it matches what is reported by Windows Device Manager's property details.

USB\VID_2341&PID_0043&REV_0001

If your device shows different details, read dualMocoLUFA/PATCHES.md first.

To install the modified dualMocoLUFA firmware on the ATmega16U2m.

• Install the Atmel FLexible In-system Programmer (FLIP).

• Put the Arduino Uno R3 in DFU mode by briefly connecting the RESET2* to GND on the atmega16u2 ICSP1 header. These are the leftmost pins, near the USB connector, as shown above.

• In Windows Device Manager, it should show up as “Atmel USB Devices > ATmega16U2“. If you encounter “unknown device“, install the driver from U:\Program Files (x86)\Atmel\Flip 3.4.7\usb\atmel_usb_dfu.inf.

• Start Atmel FLIP

  • Device » Select » ATmega16U2
  • Settings » Communication » USB
  • File » Load HEX File » dualMocoLUFA/LUFA-100807-patched/Projects/dualMoco.hex

At this point you should be able to connect the Arduino to a computer and see MIDI events using e.g. MidiView

If you have a real synthesizer to connect to, you can try that. Otherwise, I suggest installing a software synthesizer such as

• CoolSoft MIDIMapper and VirtualMIDISynth with sound fonts FluidR3 GM and/or Timbres of Heaven.

There are many Software MIDI editors and sequencers to choose from. A few that I tried are:

• MidiEditor, free. Can record. (Press the "Connect Midi In/Out" button.)
• MuseScore, free.
• Anvil Studio, freemium. Can record.
• Cakewalk
• ScoreCloud, freemium; requires sign-in.
• Presonus Notation, commercial

A video recording the MIDI with SoundCloud.

More information on the design and implementation can be found at here.