Young lady, cute, had short hair, played the ukelele? My google-fu skills have atrophied.
Nellie McKay?
Maybe Amanda Plummer?
Kate Micucci? (of Garfunkel & Oates and Raising Hope fame)?
Don’t forget The Big Bang Theory. 
Sigh No, none of those. She did a funny song about…science, I think, recently? Maybe I have the wrong show?
Or Amanda Palmer?
::facepalms self::
Yes, Amanda Fucking Palmer, as she apparently sometimes refers to herself.
Hmmm, no luck so far. Perhaps it was Jake Shimabukuro, uke gunslinger extraordinaire, in a pixie-blonde wig? 
OP, find the original broadcast date and look up Craig’s guests. I Googled McKay - who is the most talented, fun and eccentric (yes, even more than AFP) of those named, but didn’t find any recent broadcast dates, although she has been a guest…
Complete shot in the dark since I don’t see any evidence of her having been on an American late night show:
But she plays ukulele, sings about science, is funny, and has short hair.
who else was on?
Things I learned from this thread: there are a lot more short-haired, young, famous, female ukulele players running around then I would have guessed.
Hey, I have Craig’s shows on my DVR - give me a date, and I’ll look it up for you!
Well, she didn’t play a ukelele, but Ariel Tweto was on recently - she’s just about the cutest thing EVER!
He also had Esther Povitsky on recently, but she also didn’t play the ukulele.
He also had Megan Markle on; you guessed it, she did not play the ukulele.
Diane Kruger - no ukelele.
ETA: I should say, I assumed - I didn’t look.
Minnie Driver - again, I assume no ukulele.
Are you sure you saw this, Love Rhombus? Maybe you were tripping on acid. 
Georgia King - no ukulele confirmed. 
She is, however, in Cockneys Vs. Zombies, which I really want to see.