If something is "dropped" from ISS altitude, do we need to worry about it burning up on re-entry

Way to avoid answering the question. Yes, the terminus of a space elevator must necessarily be above (not at) geosynchronous height. So don’t drop it from the terminus. Ride the elevator up only partway, to ISS height (which would not be in the ISS’s orbit, because you’re going way too slow), and drop from there.

It’s sort of true that you would be limited by terminal velocity, but that’s not the whole picture. Terminal velocity depends on the thickness of the atmosphere, so very high up where the atmosphere is only just barely there, the terminal velocity is much higher than it is at ground level. I think (though I haven’t done the calculations) that a dropped object would fall through the layers of atmosphere more quickly than it would decelerate due to air resistance, so that, for most points on its fall, it would be moving at faster than the terminal velocity for that point.

In any event, the simple answer is that you can get, at most, the gravitational potential energy of the object, and that’s not going to be very much. A kilogram dropped from 200 km up is going to have approximately 470 kcal of energy. If that kilogram is all water, it’ll take 540 kcal to evaporate all of it (plus more to raise the temperature first, plus more to melt it if it started out as ice). And of course, water is about the most volatile material you could be doing this with: Metal or rock would just shrug it off.