No, you’re not misunderstanding anything. We never want to admit that the problems encountered by a poor person are by-and-large their own fault. Instead, we have a tendency to seek out external reasons to absolve them of personal responsibility.
I grew up in Dayton, OH. I know Dayton well. Put me dead-center in the poorest section of Dayton, and I can guarantee you I can find fresh fruits and vegetables within a 10-minute walk.
However much we want to believe it’s true, the reason poor people do not eat a healthy diet is not due to “limited access” of healthy food. Healthy food can be easily acquired; the problem is that they don’t want it.