Our fridge has a filter for the water dispenser, and to me, the water coming from it tastes a little funky for about a week or so after changing the filter. It’s like something has to wash out, or plug up for the water to taste “right”, and still better than the tap.
Or, if the filter is particularly old, there could be something growing within the filter, I suppose.
And last, I suppose the possibility technically exists, that something is getting filtered out that masks the taste in tap water of something else that isn’t getting filtered out, so that when you drink the filtered water, the masking flavor isn’t there.