What is the 'Synesthesia Lens'?
The Synesthesia Lens is a code-generated visualization of how Alexander Scriabin, a famous composer and pianist, perceived his synesthesia. As you enter the Synesthesia Lens, the audio from your microphone is translated into lines of color depending on each instance's frequency, sporadically appearing on the page.
What is synesthesia?
Synesthesia is an involuntary and automatic perception of a bodily sense when another bodily sense is stimulated. For example, some synesthetic people associate specific colors with specific words. For Scriabin, he saw notes as specific colors, such as C being red, A being light green, and B being light blue.
How does this work?
This project uses P5.js and ML5.js libraries to create the synesthetic visualization and HTML, CSS, and JS to create the website.
Live audio data is taken in from the microphone and their frequencies are identified in real time via ML5.js. This frequency is categorized into one of the 12 distinct notes found in Western music, then mapped to the color Scriabin saw for that note. I used P5.js to display each instance of the audio as a colorful line that appears for an instant than fades slowly.
Below is the mapping of notes to colors Scriabin had that I used:
| C | Red | F#/Gb | Blue |
| C#/Db | Purple | G | Orange |
| D | Yellow | G#/Ab | Fuchsia |
| D#/Eb | Magenta | A | Light Green |
| E | Pale Sea Foam | A#/Bb | Burgundy |
| F | Dark Red | B | Light Blue |
Who am I?
My name is Jacqueline Li. As of April 2023, I am a 5th year student at Northeastern University studying computer science and design graduating in May. This project is my senior capstone—a culmination of the knowledge and skills I’ve acquired throughout my time in undergrad poured into one stand alone project.
What other work have I produced?
Check out my other project on my website or my Instagram.
How can you reach me?
My Instagram is @didotfanclub, my TikTok is @influenczar, and my email is whoisjacquelineli@gmail.com