Getting that out of the way, my fundamental assumption is that the amount of “stuff” in the Universe remains constant. By stuff, I mean energy (including mass, which is a form of energy).
Okay, when we push something forward in time, universal energy remains constant. If we are talking about the twin paradox (one is sent off to space and travels near light speed, one stays home) then the traveling twin will appear to slow down, shrink, and gain mass in relation to the homebody twin. In other words, the traveling twin won’t be able to get a date.
Now, note that the traveling twin does not blink out of existence. I believe this is because the amount of energy in the universe must remain constant. Instead, the various “properties” of the twin are (relatively) changed to compensate for the (relative) time slowing.
For reverse time travel, I would think the same must hold true. You can’t just blink out of existence, totally removing the mass/energy from one frame of reference and add that mass/energy to another. Each frame of reference must have a consistent amount of energy. So, although a reverse time traveler would look really weird (with effect preceding cause), the reverse time traveler must still “exist” at every “moment” for every frame of reference.
Therefore, assuming every frame of reference must have a constant energy, reversing time travel must be impossible. Otherwise, you could keep on increasing the energy in one frame of reference and decreasing the energy in another.
Think about it. Since we already said you couldn’t blink in and out of existence, you must be present in the universe at every “moment.” So you wake up one day and see someone (who looks exactly like you) going in reverse, with effect preceding cause. As you are watching that, you come up to you and say “that must have been us while we were going in reverse. I switched directions a couple of minutes ago (or “from now,” depending on your reference). So, in the “normal” reference point, there are 3 of you. But, according to my premise, you can’t change mass/energy in a frame of reference. Therefore such a scenario is impossible.
Of course, the same prohibition holds true regardless of how far in the past you go. You will always be adding mass/energy into the universe
So, my conclusion is that once you are going in one direction, you cannot reverse directions.
I should mention that I see one loophole with my theory. Once again, IANAP, so I’m not sure if the loophole is relevant or not. My premise is that you cannot reverse directions. However, what if you find a whole bunch of matter that is already going in the backwards direction? Anything you did in your future to the reverse stream would effect your present reverse stream. Once you start sending yourself messages, you can run into the grandfather paradox all over again.