Kobayashi Maru?
Bituminous infant?
Oooooo, Star Trek reference. I like it. 
Repercussions.
Pandora’s Box
Morass
Labyrinth
Web
and one that I never heard of…
Boscage - A mass of trees or shrubs; a thicket.
When you are at the bottom of a hole you have been digging and it is too deep, stop digging.
“holding a wolf by the ears”
Caught on a slippery slope?
I like this one also. Like “riding a tiger,” I think. Hang on!
Instead of “punched the tar baby” I now say “put your dick in the crazy”.
Hey, this question hit close to home for me. A few years ago, a black Pomeranian puppy wandered up to our house. We took him in (cause he’s awesome) and innocently named him Tarbaby- because he’s small, black, and crap sticks to him constantly. Didnt think of the racial thing at all. We soon figured out that it’s kind of uncomfortable to go outside and yell “Tarbaby!..Here, Tarbaby!” in a mostly black neighborhood. We just call him Tar these days. I realize this isn’t helpful, so I’ll just go away now.
Grabbing a tiger by the tail.
Hey, I think that’s it!
“Quagmire” seems the closest, but even that doesn’t really convey the idea that it is your aggression which is getting you into deeper and deeper trouble.
I think there is no perfect metaphorical substitute for “tarbaby.”