Plantasia

Custom Musical Instrument

I created a capacitive touch based instrument that can connect to any water/metal based items to control a C major scale and animation via serial communication on a computer

Time: FALL 2021 // Creative Technology

Role: Engineer, Creative Coder

Tools: Arduino, Capacitive Touch MPR121, Processing, Succulent, Banana, Lemon, Rosarita Vegetarian Refried Beans



BREIF

Create a system that allows you to play music (and possibly video) from your computer using a physical custom interface of your own design and creation. Will you make it for your own personal use or more general, usable by a larger group of people? Is it mean for solo preformance or multiple people? Will it be specific to a genre of music? How do you select the parameters that you can manipulate

LEARNING THE BASICS

This was my first time working with Arduino/ESP32 so I needed to start with the basics. I created a few simple tests to learn how to use the Arduino IDE and connect/control elements on a breadboard



The next hurdle, was learning how to have interface elements communicate with processing via serial communication. I started by tasking myself to control a processing sketch via serial communication

I am pleased with the sketch I made so here is a screengrab of the sketch to give it a moment of glory

The last puzzle piece was to learn how to have multiple elements communicate with processing via serial communication

Now that the basics were solidified, I could move on to the actual build!

VERSION 1

My initial idea was to use piezo touch sensors as a way to drive the sound. My first test was to simply get the piezo working and hooked up to my plant to see how the interaction felt

Before connecting it to processing, I hooked up a simple piezo speaker to test out the system. I started to become hesitant about using piezo touch sensors for this instrument as they wouldn't trigger the note reliably which is an important aspect of music creation

Nonetheless, I finished the build to give it a solid chance

Some initial musical tests with the piezo setup:

VERSION 2

While researching this project, I stumbled across a capacitive touch breakout board built for arduino that seemed like it may be a more reliable alternative and, if it worked correctly, would include the plant into the system more directly rather than being more of an illusion gimmick.

First, I needed to test if my plant would function well as a capacitive sensor:

With the succesful first test, I decided to move forward with this build. Due to the fact that the breakout board allowed for multiple connections via one data stream, It was relatively straightforward to connect more inputs to communicate with processing. I built out a basic synth in processing that would randomly cycle through various octaves per note trigger to create an ambient but musical interaction

Instead of having the octaves jump around randomly, I felt it would be more playable to put the octave control onto a slider as well as give the user attack and release controls on a knob. Enter the multiple serial communication basics!

Control surface + prototype casing:

Lastly, I created a visual component to supplement the note choices that the user would input

Final Performance:

REFLECTIONS

This was a very exciting and useful project for myself. Getting to bring my background in music into this instrument was a very fun and rewarding way to apply some basic yet important hardware principles all together in one unified project. I would like to expand on the idea by adding more notes to make up a full 12 note scale, restricting to specific keys, and synthesizing a more unique sound rather than a simple sine wave (Which I will admit does work well with the plant theme as is). I look forward to utilizing these tools in future projects!

Instructor: Maxim Safioulline