# Visual Interps of Audio Input **URL:** \ **Category:** Factual Questions\ **Created:** [August 8, 2025, 3:49am UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843 "2025-08-08T03:49:29Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![AHunter3](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ahunter3/32/368_2.png) [@AHunter3](https://boards.straightdope.com/u/AHunter3)\ **Post date:** [August 8, 2025, 3:49am UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/1 "2025-08-08T03:49:29Z")
Way back when MTV made its debut and I was first hearing about this phenomenon of visual accompaniment to the soundtrack of an audio recording, I somehow imagined that what they’d do is really represent the sounds visually. Of course that’s not what MTV, or subsequent music videos, were about. That doesn’t make it a bad idea. I’m not fascinated by soundwave representation; I deal with it all the time when editing sound files, but it’s just not evocative. What I’d rather see is… • Color-code the blasts of color on screen according to what KEY they’re playing in. e.g., C major is blue, D major is red, E major is green, etc… for all 13 major keys, with variants for their associated minors. And additional variants for 7th major and 7th minor and 9th and so on commonly recurring combo chords. • When the chord being played doesn’t reconcile to a single major or minor or 7th or 9th or whateverish chord, superimpose the colors. • Melody-line tracks in similar colors as a dot superimposed upon the colorswatch, based on what note is the predominant melody note. Or something like that. Back to Factual Questions: Does there exist software that can represent audio in this fashion, loosely speaking? ---
**Author:** ![DaveUnknown](https://avatars.discourse-cdn.com/v4/letter/d/94ad74/32.png) [@DaveUnknown](https://boards.straightdope.com/u/DaveUnknown)\ **Post date:** [August 8, 2025, 4:49am UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/2 "2025-08-08T04:49:11Z")
Deleted ---
**Author:** ![DaveUnknown](https://avatars.discourse-cdn.com/v4/letter/d/94ad74/32.png) [@DaveUnknown](https://boards.straightdope.com/u/DaveUnknown)\ **Post date:** [August 8, 2025, 4:50am UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/3 "2025-08-08T04:50:52Z")
Double deleted ---
**Author:** ![pjd](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pjd/32/3009_2.png) [@pjd](https://boards.straightdope.com/u/pjd)\ **Post date:** [August 8, 2025, 11:31am UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/4 "2025-08-08T11:31:32Z")
VLC media player has Audio Visualizations. You can load new ones - see [here.](https://www.vlchelp.com/how-to-install-new-audio-visualizations-in-vlc/) (I don’t know if any will achieve what you want, but maybe worth a look.) ---
**Author:** ![Eonwe](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/eonwe/32/240_2.png) [@Eonwe](https://boards.straightdope.com/u/Eonwe)\ **Post date:** [August 8, 2025, 12:24pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/5 "2025-08-08T12:24:00Z")
> [@AHunter3](#): > > Color-code the blasts of color on screen according to what KEY they’re playing in. e.g., C major is blue, D major is red, E major is green, etc… for all 13 major keys, with variants for their associated minors. And additional variants for 7th major and 7th minor and 9th and so on commonly recurring combo chords. In true SDMB fashion, I’m going to be slightly pedantic about this. There is a difference between the key of a song or section of a song and the chord that can be defined (in simple terms) by the sounds happening moment to moment. It sounds like you don’t want the key but rather dynamic chord/harmonic analysis. ---
**Author:** ![AHunter3](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ahunter3/32/368_2.png) [@AHunter3](https://boards.straightdope.com/u/AHunter3)\ **Post date:** [August 8, 2025, 1:41pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/6 "2025-08-08T13:41:22Z")
> [@pjd](#): > > VLC media player has Audio Visualizations. > You can load new ones - see [here.](https://www.vlchelp.com/how-to-install-new-audio-visualizations-in-vlc/) It does?? (Opens VLC media player, fishes around in menus) … can’t find where to invoke such things. All the “video” items I’ve examined seem to pertain to playback of the video portion of movie files. ---
**Author:** ![pjd](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pjd/32/3009_2.png) [@pjd](https://boards.straightdope.com/u/pjd)\ **Post date:** [August 8, 2025, 1:45pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/7 "2025-08-08T13:45:25Z")
It’s in the Audio menu → visualizations ---
**Author:** ![AHunter3](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ahunter3/32/368_2.png) [@AHunter3](https://boards.straightdope.com/u/AHunter3)\ **Post date:** [August 8, 2025, 1:49pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/8 "2025-08-08T13:49:34Z")
Right under my nose. Hmm, not bad! Thanks! ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [August 8, 2025, 2:09pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/9 "2025-08-08T14:09:14Z")
Coloring by key has several problems. First, you can’t define what key a single sound is: Any given note shows up in 7 out of 12 major keys and 7 out of 12 minor keys. When a song starts off with a single prolonged note, what key do you assign it to? Do you just wait until there are enough other notes to tell, and color it retroactively? Second, it’s fairly typical for an entire song to be in the same key, in which case you’d… just have the screen showing that same color for the entire song? There’s also the issue that an image has more than just color; it also has two dimensions of position, where sound has zero positional dimensions. But on the other hand, human vision has lousy spectral resolution, with only three channels, while sound has, effectively, a full continuum of spectral resolution (or, for the sounds generally classed as “music”, at least as many channels as there are notes). There are ways to map the sort of information that sound has to the sort of information in an image, as you allude to with wave representations in sound editing, but it doesn’t “look the same” as the music, in the way that you’re trying for. ---
**Author:** ![blondebear](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/blondebear/32/1022_2.png) [@blondebear](https://boards.straightdope.com/u/blondebear)\ **Post date:** [August 8, 2025, 2:39pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/10 "2025-08-08T14:39:21Z")
What comes to mind is a “color organ” like the ones sold in the 70s. There were several versions, some that hooked up to your speaker terminals, others had a microphone. I think they were limited to 3 channels. There are kits available today and I suppose someone with a little tech savvy could boost that capability. ---
**Author:** ![DPRK](https://avatars.discourse-cdn.com/v4/letter/d/4491bb/32.png) [@DPRK](https://boards.straightdope.com/u/DPRK)\ **Post date:** [August 8, 2025, 3:22pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/11 "2025-08-08T15:22:32Z")
MilkDrop-style visualizations are just supposed to look cool, and run in real time, so it does not matter _exactly_ what it is doing. Incidentally, an arbitrary piece of pop music may not even be in a well-defined key, strictly music-theoretically speaking (also, at some level that has to do with how the song sounds to you and how you interpret the harmony; it’s not all purely mathematical). Now, I am not sure if the OP is interested in a psychedelic music visualizer or something in a DAW that would enhance the usual amplitude-envelope visualization (graph of the amplitude as a function of time) with some kind of polyphonic pitch detection. There is absolutely published code that grabs a FFT and tracks what “key” best matches the spectrum, or what chord is playing at any given moment [look at some of the MIREX stuff], but I am not sure if any of them are already integrated into a DAW plugin with the desired color coding. ---
**Author:** ![Francis\_Vaughan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/francis_vaughan/32/3093_2.png) [@Francis\_Vaughan](https://boards.straightdope.com/u/Francis_Vaughan)\ **Post date:** [August 8, 2025, 3:27pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/12 "2025-08-08T15:27:13Z")
For what I find a remarkable and useful visual representation of the music, this is my favourite. Considering the work visualised, it is quite something. This isn’t going to happen automatically. > **[Stravinsky, Rite of Spring (complete), animated graphical score](https://www.youtube.com/watch?v=5IXMpUhuBMs%5C)** > > Stravinsky's The Rite of Spring, accompanied by a graphical score.FAQQ: What do the shapes indicate?A: Each shape corresponds to a family of instruments: el... ---
**Author:** ![AHunter3](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ahunter3/32/368_2.png) [@AHunter3](https://boards.straightdope.com/u/AHunter3)\ **Post date:** [August 8, 2025, 3:59pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/13 "2025-08-08T15:59:22Z")
I’m realizing I did not have a specific algorithm in mind. At any given moment the sound at that point could be considered a “chord” which could be analyzed and recognized, but there’s a lot of arbitrary decision involved when someone does that, the more so as you increase the number of “incidental” tones that are considered to have been “added to” the fundamental “chord”. ---
**Author:** ![AHunter3](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/ahunter3/32/368_2.png) [@AHunter3](https://boards.straightdope.com/u/AHunter3)\ **Post date:** [August 8, 2025, 4:01pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/14 "2025-08-08T16:01:01Z")
Oh, that Stravinsky representation is very nice!! ---
**Author:** ![Reply](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/reply/32/15952_2.png) [@Reply](https://boards.straightdope.com/u/Reply)\ **Post date:** [August 8, 2025, 5:18pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/15 "2025-08-08T17:18:09Z")
> [@Francis\_Vaughan](#): > > For what I find a remarkable and useful visual representation of the music, this is my favourite. > [@AHunter3](#): > > Oh, that Stravinsky representation is very nice!! The creator of that visualization has a detailed write-up about it at [Harmonic Coloring](https://www.musanim.com/HarmonicColoring/) [![](https://www.musanim.com/HarmonicColoring/bop131m1234_Overview.png) ](https://www.musanim.com/HarmonicColoring/bop131m1234_Overview.png) And academic papers at [https://ccrma.stanford.edu/~craig/papers/01/icmc01-tonal.pdf](https://ccrma.stanford.edu/~craig/papers/01/icmc01-tonal.pdf) and [https://ismir2007.ismir.net/proceedings/ISMIR2007\_p189\_mardirossian.pdf](https://ismir2007.ismir.net/proceedings/ISMIR2007_p189_mardirossian.pdf) There’s also a playlist of additional visualizations/songs at [https://www.youtube.com/playlist?list=PLtj\_HurkS7ZymxGee5I16LB1rcSQ8oNmf](https://www.youtube.com/playlist?list=PLtj_HurkS7ZymxGee5I16LB1rcSQ8oNmf) ---
**Author:** ![Marvin\_the\_Martian](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/marvin_the_martian/32/2898_2.png) [@Marvin\_the\_Martian](https://boards.straightdope.com/u/Marvin_the_Martian)\ **Post date:** [August 8, 2025, 5:52pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/16 "2025-08-08T17:52:05Z")
The actual visualizations in the harmonic coloring videos appear to be taking the color organ idea to the extreme, with channels a half-tone wide and also capturing attack/sustain/decay. I don’t believe there is any change in the colorization of a particular note based on key, the magic is in the assignment of colors to each note so that the overall color palette of a section of music changes with key. The point of the visualization is to assist in the identification of key changes, but not to actually identify the key. I think that generating such a visualization in real time is certainly doable, although strongly inharmonic instruments could cause problems. ---
**Author:** ![Hoops](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/hoops/32/6408_2.png) [@Hoops](https://boards.straightdope.com/u/Hoops)\ **Post date:** [August 8, 2025, 6:36pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/17 "2025-08-08T18:36:44Z")
Here’s a visualization of The Great Curve, from Talking Heads’ Remain In Light. Although it is a deep cut by the Heads’ standard, aficionados of the band seem to rate it very highly. [![](https://img.youtube.com/vi/LqTyfMDnK-k/maxresdefault.jpg "The Anatomy of: Talking Heads - The Great Curve") ](https://www.youtube.com/watch?v=LqTyfMDnK-k) It obviously takes as its starting point the pedestrian audio track visual model, but it does do a great job of explicating the complexity of what is basically just a one chord jam with layered vocals (and wild guitar solos courtesy of Adrian Belew). To radically shift gears, I can think of two classical (well, 20th Century) composers. Scriabin and Messiaen, who worked towards a theory of color visualization, though neither codified it into anything like a unified theory. > **[Alexander Scriabin | Influence of colour](https://en.wikipedia.org/wiki/Alexander_Scriabin#Influence_of_colour)** > > Though Scriabin's late works are often considered to be influenced by synesthesia, an involuntary condition wherein one experiences sensation in one sense in response to stimulus in another, it is doubted that Scriabin actually experienced this. His colour system, unlike most synesthetic experience, accords with the circle of fifths, which tends to prove it was mostly a conceptual system based on Sir Isaac Newton's Opticks.\[citation needed\] > Scriabin did not, for his theory, recognize a differenc... > **[Olivier Messiaen | Colour](https://en.wikipedia.org/wiki/Olivier_Messiaen#Colour)** > > Colour lies at the heart of Messiaen's music. He believed that terms such as "tonal", "modal" and "serial" are misleading analytical conveniences. For him there were no modal, tonal or serial compositions, only music with or without colour. He said that Monteverdi, Mozart, Chopin, Wagner, Mussorgsky, and Stravinsky all wrote strongly coloured music. > In some of Messiaen's scores, he notated the colours in the music (notably in Couleurs de la cité céleste and Des canyons aux étoiles...)—the purpos... ---
**Author:** ![xtenkfarpl](https://avatars.discourse-cdn.com/v4/letter/x/c2a13f/32.png) [@xtenkfarpl](https://boards.straightdope.com/u/xtenkfarpl)\ **Post date:** [August 9, 2025, 6:36pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/18 "2025-08-09T18:36:09Z")
> [@AHunter3](#): > > Does there exist software that can represent audio in this fashion, loosely speaking? Not automatically, I think. But have you come across smalin on YouTube? Some very interesting visual interpretations! ---
**Author:** ![rsThump](https://avatars.discourse-cdn.com/v4/letter/r/a88e4f/32.png) [@rsThump](https://boards.straightdope.com/u/rsThump)\ **Post date:** [August 10, 2025, 5:25pm UTC](https://boards.straightdope.com/t/visual-interps-of-audio-input/1021843/19 "2025-08-10T17:25:37Z")
There’s nothing off the shelf I’m aware of - Melodyne’s DNA editing is closest, but that GUI is based on the standard dynamic waveform display, just using multiple instances spread across a piano roll to indicate the pitch in polyphonic program material. It’s an interesting idea that happens to dovetail with a musical/sonic feature extraction project I’m working on. So, I played around with the concepts a bit yesterday and came up with a method and rough prototype to extract labeled chords (confidence varies) and beats, so the chord “blobs” could (not are, _could_) be animated along with the audio track. Next would be to figure out color-coding for the diatonic behavior, which is where things can get tricky, due to evolving nature of Western Common Practice. After that, melodic extraction and analysis…but, those features are beyond the scope of what I’m playing with. Factual Answer: not currently in existence, but could be done.