Oh, I was gonna chime in with the piano solo for “Aladdin Sane,” but you don’t mean musically unconventional so much as instrumentally unconventional (though sax & harmonica are pretty common, IMO)
Well, the classics would be:
The ocarina solo in “Wild Thing.” (The Troggs)
The bagpipes in “Long Way to the Top if You Wanna Rock and Roll” (AC/DC)
The spoon solo in “Spoonman” (Soundgard)
Anything with Jethro Tull’s flute
There’s gotta be a sitar one out there, but all the sitar songs that immediately come to mind don’t really have an improvised solo section. I’m sure I must be missing something obvious.
Phish has this weird thing where the drummer occasionally plays a “vacuum” solo.
Here’s a bizarre number where (starting at 1:12) Daniele Vidal harmonizes her voice over an electronic machine, to make it sound weirdly like a voice synthesizer.
Queen, again: “Night at the Opera” has “Seaside Rendezvous,” which has a verse-long solo by Freddie and Roger (playing kazoos?). Roger sings so high it makes my nether regions ache.
I can only thnk of two times I’ve heard a duet of harmonizing saxophones, one of thme the classic bridge in Van Morrison’s "Into the Mystic’ . The other, deliciously, in Ibrahim Ferrer’s “Aquellos Ojos Verdes” (at 2:15)