That particular one is used even in Tennessee for any place further eastward, which is mostly North Carolina, South Carolina and Virginia. Other references are Up North, Out West and Down South. When I was living in Alabama anywhere north of Birmingham was Up North. From Tennessee, Georgia is more South even if it’s not all that Down, and Florida is definitely Down: I cannot recall anybody ever saying they were going (from here) Back to Florida; always Down. Anything north of Virginia, whether east or not is Up North, which would include Kentucky and the Midwest.
The Midwest has always been a mysterious region (directionwise) for me. In some usages it starts at Ohio and goes all the way to Minnesota and includes Missouri and Kansas and maybe even Nebraska. Iowa I’m not so sure of but probably more Midwest that West. I guess the Rockies establish West even though Texas and Oklahoma are West to me.
The Northwest may start in Idaho but stops at California going southward.
I get the impression (from various sources) that to New Englanders and New Yorkers (maybe even Pennsylvanians) that Down South starts in Maryland.
Out West seems to begin at the Mississippi unless the speaker is himself/herself further west that that.
The one I’ve always had a fascination for is Mainers’ Down East. After visiting the Maine coast not so long ago it did make some sense in that from other parts of the state the coast is both Down (South) and East. I’m just not sure if people in Portland and other places in the southernmost sections of the state also refer to Bar Harbor as Down East. That would be quaint.
Some other threads that delve into this issue include:
Up To or Down To The Capital
Down/Up/Over/Out/Back/etc. in Giving Directions