I am a big fan of NPR’s Piano Jazz with Marian McPartland. She will say to her guest, “why don’t we play this tune?” They may mention a key, maybe not. There is no rustling of charts. And out comes this two part, magnificent music. Which appears to be off the cuff. Can these people really do this? Or do they edit the tape? Discuss what they are going to play before the show? It appears that they just meet and start playing. Which I think is incredible, but does it really happen that way. Thank you for clearing this up for me.
Well, with a nod to the OP’s name, John Popper from Blues Traveler can spontaneously riff on his harmonica. He did a segment on the Carolla podcast where he improvised over Maneater and Abracadabra that was amazing.
Some of them, sure. The ones that can’t tend not to go on shows where it’s going to be demanded of them. 
Seriously, if you ever get to sit in on a “jam”, do it. I don’t play anything, and I’m a mediocre singer at best, but it’s wonderfully amazing to sit with skilled musicians who can do this - play unrehearsed, sometimes improvised music together. And plenty of people who aren’t even professional musicians can do a great job with it; it’s just another skill.
Got a local Irish themed watering hole? They often have “Open Sessions”, sometimes on a Sunday afternoon. It’s something of a hybrid of concert, open mike and jam. You generally have a person or three leading events, and they’ll play some, and then invite people to play some, and then the whole things winds up with everyone who wants to and can playing together - often traditional Irish reels with a lot of room for improvisation. Go, you won’t regret it.
Short answer, yes they can
Longer answer, it comes from experience and practice even if the musician does not know the song they will likely know the form, a simple example is 12 bar blues
Blues songs are typically I-IV-V in the key of E this would be E-A-B so if you know this you can lay back and play the chords until you get the feel of a song you don’t know, cop the melody and start inserting those licks you have spent years developing voila instant awesome. The real key is being able to listen
Jazz is more complicated because the progressions tend to be more complicated but the same principles apply, also Jazz musicians tend to know standards or tunes just about everyone knows atleast the form and melody thereof. Jazz songs are often built for improvisation.
I am being overly simplistic and left out most of the real music stuff but basically it is listening, practice and experience.
Capt
I don’t listen to that show, but do they tend to stick to jazz standards? If they do, odds are they’ve played all those songs dozens of times.
I once read advice from a jazz musician, saying to practice as many “standards” as you can, as often as you can, in different keys.
I’m not a jazz musician, but I know a lot of jazz musicians.
That program and Marian McPartland’s skill absolutely freaking boggle my mind.
Rock/blues guitarist here. Yeah, what they said - there are some classic chord progressions that, once identified, are easy to latch onto. And for jazz musicians, fuggedaboutit.
ETA: once the sequence is understood, what’s fun is to sell the next chord change: varying the dynamics and pace until the switch. Leads to good communication and audience rapport.
I play a weekly jazz gig and this happens all the time when someone sits in. most of the time, it’s a classic standard or one of more common jazz tunes in the repertoire.
You also get much better at covering mistakes, cueing off others and recognizing common motions and movements.
The biggest factor is just practice. I’ve been a pro sideman/studio bassist for.years and after a while, you just know what to.expect and also can react to things much quicker.
there are only so many motifs, structures and forms. I often compare a new tune to one I’ve been playing for years, and just note where it differs. so for a simple example…Blue Monk is a standard blues, but the four chord in bar 5 resolves up a half step to the mi7b5 before the extended turnaround. the melody walks chromatically up to the 5 each iteration. that’s enough for me to cover that perfectly well even if I’ve never heard the tune.
The only thing somewhat unusual is that they don’t always name a key. But, then again, if they can see each other, they can probably figure it out. I’ve done that a lot.
A large part of playing by ear is being able to do this sort of thing. As mentioned, there are patterns, so it’s not as hard as it seems. But you also learn tricks to make any mistakes not sound like they were mistakes. Honestly, I think it’s a bit easier with jazz, since a discordant note can be played off as part of a jazz chord.
Good answer, Picker.
To add; Studio musicians (Sometimes paid by the hour) Have to be able to sight read. No rehearsals, (Cost Money) Just a quick tune-up with the rest of the group, sit down and honk away. If you’re not proficient enough, there’s 15 other people waiting outside to take your place. I sat in on a rehearsal for West Side Story and just couldn’t cut it. (Reeds). Came close though! 
As a kid my guitar teacher was an ex-session musician, the man was a monster sight reader. Any style, any tempo and he could knock out all but the most extremely complex pieces in a passable fashion in one go. It was pretty much a job requirement for him back in the day.
I think most of us underestimate just how many talented musicians there are out there. As Jake points out above for every musician with a paying gig there’s probably at least 15 to 20 others who are pretty much as good. Hell I’m willing to bet that any decent-sized high school would have 2 or 3 frightningly good players amongst its students.
Another sideman here - good answers/commentary from all. I was recently discussing this same topic with a guitar player who wants to double on bass for casual gigs. My advise was pretty standard:
Know your scales and be able to play them without constant reference - you’re going to be concentrating on listening and watching the other players for musical and visual clues & cues.
Learn the standards of your chosen genre(s).
Be understated, stay “in the pocket”, swing when it feels right - the drummer and soloists will appreciate it.
Oh yeah - listen and listen some more.
Yeah, for bass the four rules are:
know your feels/styles,
play less/groove more,
develop your ear and pitch training
and
be able to read anything.
For bass its the rhythms that are brutal, the lines aren’t that difficult but the timing and syncopation is very precise.
McPartland may well tip any guest off beforehand of any song they’ll be expected to play so they’re both prepared for it.
Here’s an excellent example, IMO. Dave Brubeck, who I think was already in his 70s by the time of this Russian tour, was surprised when a young Russian violinist stood up and joined in on a song. You can clearly see Brubeck’s surprise, and there is a little hesitancy, but in very short order they are playing off each other’s riff. Pretty good for an old guy.
Yes, absolutely. I had a working jazz quartet for many years. Most seasoned jazz musicians know the changes of the standard jazz tunes from memory and can transpose them fairly easily. If you were to sit down with any professional jazz musician and ask them to play a tune that is “standard” or fairly well known they’d most likely be able to do it and acquit themselves very well.
I have a book of the 100 or so songs that were part of my repertoire. Some of them are very “standard” but I have a unique arrangement. If I am working with a musician who is unfamiliar with my book, I have to remind them that I have a special arrangement and not the standard changes. If I don’t, we wouldn’t all be on the same page because the new guy would be playing the changes from memory instead of looking at my book. Minor train wreck ensues while they find their place in the chart and start actually reading.
If I’m sitting in with another band or attending a jazz jam I make sure to call tunes that I think everyone will know and tell them what key I want to sing them in. Never had a problem with musicians knowing jazz standards and being able to play them in my key. OK, that’s a little broad, I’ve certainly sat in with people who couldn’t play very well but that’s a whole 'nother story.
Miles Davis’s “Kind of Blue” was kind of like this. Cite:
For experienced musicians used to playing in an improvisational format, no, this is not an unusual skill. To do it very well is certainly impressive, but it’s not terribly rare. “Jamming” and sitting in with other musicians to play unrehearsed music is a big part of being a good musician, at least in popular music genres like rock, jazz, folk, blues, etc. That’s what makes it, to me, so fun and social. You sit down with a bunch of other musicians, pick a song or call out some chord changes, and go at it and see what happens. It’s like the musical equivalent of a conversation. Lots of fun, and every once in awhile, true magic happens.
Very nice link! And it is LIVE! Incredible!