Published in the late 70’s in Fantasy and Science Fiction magazine.
A man is living the life of a middle aged loser in a city somewhere. Job he hates, single, crappy apartment, out of shape. One day he hears the voice of an alien in his head. At first he understandably freaks out, but eventually he comes to accept it and not be scared. The alien agrees to set up private rooms and a common area in the man’s mind.
The alien explains his mission. He comes from a Universe fantastically different from our own. The alien analogizes it to a gigantic incredibly complex mathematical equation, where every part is necessary to the logical meaning of the whole. It’s a world of icy abstract beauty, cold and logical. Now one piece of the universe has detached itself and come here, to our universe. Without that piece, the alien’s universe doesn’t make sense any more. It is difficult for the alien to convey the magnitude of the crisis, but he asks the man to imagine “Time running backwards, the sun turning into a cold ball of iron, glaxies flapping their arms like demented starfish.” The Alien has been sent to our universe to track the fugitive and return him, thus saving his world.
The alien begins to make changes in the man’s life. He gets the man in shape, gets him a better job, begins investing in a market he can easily predict. The alien is overwhelmed by sensation, having never experienced it before. Even the man’s poor bachelor diet fascinates him. Then the alien discovers sex, which he won’t shut up about for three days. Eventually the man becomes wealthy. He and the alien start traveling the globe, ostensibly searching for the fugitive, but the man notices they always wind up in places of art, beauty and pleasure. Soon he is leading the life of a decadent cultured playboy.
One day the man is looking at paintings at a gallery in the Greek Islands. As he is about to enter a room the alien starts yelling in his mind. “Get out! Get Out!” The man leaves the gallery. The alien apologizes and claims he was simply bored in the gallery. It’s never staed explicitly but it’s obvious what happened: The alien found his quarry, but has lost the desire to return to his universe.
At the end the man accepts his life. It’s a pretty good life after all, the life of a wealthy aesthete. But sometimes he can’t stop thinking about time running backwards or the sun becoming a cold ball of iron.
That story has stayed with me but I can’t remember what it was called or who wrote it.
ETA: if anyone knows this let me know, not just the OP.