Is there an app, where I can point my phone’s camera at a fragment of musical notation and the melody would play on my phone?
if not, why not?
It looks like there are apps that can read and play sheet music. I don’t have any knowledge or experience of them, but here are some links to information I found:
I’ve seen musicians (e.g. all four members of the Dover quartet) use tablets to display the music. But they have to advance the page with a foot pedal and I have wondered if it would be possible for the software to follow the music and advance the page automatically. Of course, there could be a problem with repeats, but I imagine that could be programmed beforehand.
T_B…thanks for the links! PlayScore does exactly what I was looking for: you snap a picture of the score, and it plays it back instantly (using piano tones). The free version only allows two staves, but that’s fine for my purposes.
Polyphonic music recognition is not trivial, but it is possible - I have implemented code (not mine) that recognises chords in an audio stream, and the same basic principle (a Constant-Q Fourier Transform) can be extended to the entire range of musical notes. I’m actually not even sure I’d do that, though.
A chromagram is a 12-point graph that plots notes from C to B by summing the octaves. You can calculate the approximate chromagram at any point from the score, and compare it to the chromagram of the audio, and make a pretty good attempt at tracking the progress. Add in some beat-matching to compute the tempo, and I suspect it would work pretty well.
I’ll have to play …
I wrote the first app for this in 1989 with Musitek