How long have you been working out? It’s fairly common that when you’re starting out you make faster progress than when you’ve been doing it for a while. Most people then plateau somewhat, and after that it’s harder to make additional gains.
But I agree with you on the length of workouts. I think the returns diminish beyond a point; spending 2 hours at the gym isn’t twice as good as spending one hour.