Aside from the odd numbing effect, it is kinda bitter, a little sweet, and it feels cold on your tongue, kinda like certain mints. Depending on what it is cut with, it may have other tastes to it.
What's funny about how the cops in TV shows take a taste is that if they aren't regular users of cocaine with a developed tolerance and it is fairly decent stuff, they should get pretty high from doing that. A lot of cocaine users like to do 'numbys' where they take a pinch and rub it between their jaw and their lip. Usually they do it when there's not enough left for a line on the mirror, plate, picture frame, whatever, and even an experienced user will feel some effect from it, other than the numbing of the mouth. If a non-user took a tiny pinch of fairly pure cocaine from a bag and put it in their mouth, they would probably feel like they just did 10 shots of espresso and a couple of novocaine.