The question can’t be answered without considering the proper motion of stars and the motions of galaxies in our local group. Their gravitational attraction overwhelms expansion, so one day our local group will be all we can see, with everything else receded so far away it will be outside our ‘light bubble’ and permanently invisible to us - along with most evidence of the evolution of the universe. But that’s a LONG way into the future.
Before that happens, but still a Long way in the future (like 10 times the current age of the universe), all 54 galaxies in our local group will merge into one gigantic elliptical galaxy. That should result in a lot more stars visible in the sky (but maybe not - it will be REALLY big, and it’s not clear if the density of stars near enough to us to be seen with the naked eye will be larger or smaller). Of course, our sun will be long gone by then.
In the shorter time frame (‘shorter’ being 4 billion years or so), the Andromeda galaxy will collide with ours. So for a long time before that happens, Andromeda will get brighter and bigger in the sky. It’s already visible with the naked eye in a really dark sky and much larger than the moon. One day it will be spectacular and fill the sky with billions of stars we can’t resolve today.
Somewhat before that happens, the gravitational influence of the other galaxy will start to disturb gas clouds in the Milky Way, leading to bright new star forming regions. So there may be an increase in stars visible in our galaxy say a billion years before the actual collision.
Then there is the proper motion of other stars near us. At large scales, the galaxy looks like a fairly ordered spiral with everything moving together. But if you zoom in the spiral arm we are in, you will see a lot of motion in all directions between the stars. Some are moving towards us, some moving away. This will eventually change the number of stars that you can see with the naked eye, since most naked eye stars are fairly local to us.
For example, Gliese 710 is 64 light years away right now, but in about a million and a half years will pass so close to us it will go through the Oort cloud. This could have interesting effects, like dislodging a rain of comets on the inner solar system. It will also be very bright.
In general, I would say that small changes in visible star count will happen over periods of thousands of years, but it won’t really change the perception of the number of stars in the sky and will be of interest mainly to astronomers. The big changes won’t happen for billions of years.