I’ve tried 10 or 20 variants, but can’t find it at dictionary.com or via google searches.
It’s not spelled:
Squahoggy
Squayhoggy
Squawhoagy
Squahaugy
Skwahoggy
Skwahaugy
Skwayhaugy
etc. etc. etc.
It’s a slang term meaning “more than a bit cattywumpus.”
And *cattywumpus * means…?
Sorry, cattywumpus means “out of square”, as in “the foundation has settled, and the rooms of my old house are cattywumpus.”
Well I guess I learn something new everyday on the SD !
cattywumpus
*Skewed or sideways to, out of alignment.*
Bubba, you got that manifold gasket in there all cattywumpus.
cattywumpus
*to grab another's head and brain suck*
He yelled "Cattywumpus" and I crushed his skull with my palm.
I’ve spent three bleedin’ hours trying to help you. I give up. Here’s what I can tell you:
It’s a Native American word (but I knew that when I first read your question). 
And so is catawampus (which I vaguely remember - and no, I do not agree with the spelling enshrined in the Urban Dictionary. I’m older than anyone who posted, I bet - maybe as old as them altogether). 
Wherever you (or your family) originally got the word is your best clue to what tribe it came from. It kinda sorta sounds like NY/New England to me.
I’ve Googled till I’m tired of it. There are zillions of NA websites. Once you know which tribe/nation you’re dealing with, you can probably find some kind of online contact to ask.
There useta be some Native maillists with large circulation that were open to anybody, and you could get answers there (I know; I useta belong to some of 'em, >10 years ago). But things like chats and eGroups, etc., have apparently killed 'em off. I was able to find some “maillists” that purported to be what you will want, but they listed such pitiful enrollments that they’d be useless.
And then there are the newsgroups, which I flatly refuse to check (sorry). Try investigating alt.native and soc.culture.native , and good luck!
If you can give me your (or your family’s) geographic origin, but have been unable to find anyone who would/could answer your questions, I’ll try once more to help. But I think you should do some work on it yourself. Unless some other Doper can help you off the top of his/her head.
Are you sure it’s a Native American word? If the OP has encountered the word only orally, there would be a wide uncertainty in the actual spelling, and perhaps even the pronunciation has drifted over time. The initial “squa” portion of the word is similar to “skew,” which would jibe with the definition. I submit the British slang skew-whiff as something similar.
Note: I am not a linguist, nor Native American, nor British, nor have I ever knowingly uttered “squahoggy” or any of its variants.
Well, thanks for trying. I’ve spent quite a bit of time looking myself, and was surprised at how obscure the word is.
The location is southern Wisconsin, so the tribes would be Sauk or Fox. Then again, I vaguely remember someone telling me that the word came from Maine. That might make it a variant on the quahog, or quahaug clam?
Interesting. I’ve spent my entire life in southern Wisconsin and I’ve never heard this term used. Cattywumpus on the other hand is quite common (as, of course, you know). Sorry I can’t help.
Don’t be so sure of a Native American etymology of catawampus. Web3 says it’s probably a modification of catercorner via folk etymology. RHUD2, a more recent reference, says it’s from cata- “diagonal” plus wampish a Scots word meaning “to wave something about”.
Okay, Dan, I’ll yield on catawampus.
I did more thinking after I went offline, and I think you’re right. Probably is Scots.
However, I still maintain that squahoggy (or however it’s properly spelled) is NDN.
No one here seems to know about squahoggy so perhaps Squink should ask the question on Dave Wilton’s WordOrigins message board. I think you now have to register to post there, but it’s painless.