Painful Contractions (and I'm not even with child)

“Wouldn’t it be nice to own a Malibu beach house, just like Barbi?” “Wouldn’t” is a contraction, meaning, “would not”.

Is the correct answer (assuming “yes”), “Yes, it would not be nice.”

Why does it sound so wrong and, most certainly, not like the intent of the answer?