It’s partly to due with that - if you’re in cold lake water, it’s similar to really cold air and hypothermia… after a while you don’t feel cold any more; that’s when you start to get into trouble. Although with hot bath water, part of your getting used to it comes from the water simply cooling down. Within 5 or 6 minutes, the water gets noticably cooler no matter how hot it was to start.
Set up three glasses, one filled with ice water, one with water at room temp and the final one filled with hot water (not too hot, scalding in the name of science is to be avoided).
Put a finger from one hand into the ice cold glass and one from your other hand into the hot one. Wait a while until they feel comfortable, then plunge them both into the room temp glass. You should find that the one from the cold glass registers being much warmer than the other.
Conclusion: your skin is better at measuring temperature differences than absolute temperature.
Based on this I would guess that the OP phenomena is caused by the skin gradually warming/cooling to be closer to the ambient and so registering less of a difference.