The short answer is, never give up on your dreams.
The long answer is, you still shouldn’t give up on your dreams, but you do have to have some pragmatism mixed in there. Take something lofty, like wanting to be an astronaut or a movie star or the president. If you reach a point in your life, as most people who aspire to those types of careers do, that it’s not realistically possible to achieve those anymore, the question isn’t to ask whether or not you should give up on your dreams. Instead, the real question to ask is what about those careers draws you to them?
Without going through my life story, music is my passion, and had things gone a little differently, maybe I’d be a professional musician, but it didn’t work out that way. That I’m not making a living doing that doesn’t make me feel like a failure. In fact, to a certain extent, having learned what I have about that industry, I may very well be better off pursuing my passion for music as an amateur, so I’m not beholden to my success to be able to provide for myself. As a result, I have less time for my passion, but I also know that when I DO get to engage in my passion, that it is always purely for the love of it.
So, what is it about astrophysics that interests you? Sure, if you want to be the next Einstein or Hawking or whatever, that probably won’t happen. But there’s nothing about astrophysics that says you have to pursue it in that way. Maybe you can become a teacher or professor. Maybe you can become a writer or editor. Maybe a consultant or contractor related to various projects.
As a similar example, I’m a computer scientist, and as you might expect, a ton of the people I ran into in undergrad and going into it are interested mostly because of their passion for games. So many get in there, realize how utterly inglorious and tedious some of it can be, particularly when it comes to game programming, and they quickly give up. As it turned out, they were never passionate about computer science, they were interested in creative design aspects. They wanted to be the next Shigeru Miyamoto, which is like going into Hollywood wanting to be the next Steven Spielberg.
Frankly, a big part of my original interest in computer science was video games too, but as I studied it, I found out that I was even more drawn to the logic and problem solving. In fact, that’s the same thing that drew me to the types of games I liked, complex puzzles and creative solutions to them. It wasn’t long before I realized that I’d never want to work in the video game industry, and I had found an interest I could pursue for my career, at least for the time being.
So, I guess my point is, there’s more of an underlying aspect to your dream, and if you can get to the substance of that, then you absolutely can still achieve that dream. Hell, you might even find that you were pursuing a dream with a misunderstanding of what that actually involved, and if you knew that, you never would have gotten started. A lot of life in that regard is a little bit of trial and error.
So, pursue your education as much as you can and as much as you still desire to. If you find your interest is waning, make adjustments to your path to compensate. And in the meantime, find a job that meets what you want to do as closely as is reasonably possible. Maybe you can’t get one directly related to astrophysics, but maybe something else in the sciences or something that sparks your curiosity and problem solving. But never ever just settle. Even if you HAVE to for a period of time to pay bills, keep your eyes out for the next step and move on when that time comes.