Hmm… Well I’m not too certain on the correct grammar, but it sounds okay that way. But I think it should be more like, “My room smells bad because of cats.”
Then again, I don’t quite understand how a room can smell bad because “of cats.” Maybe a litter box, but not cats. They don’t smell. Dogs do, but not cats.
That’s different ("…badly of cats"). Now you are again modifying a verb, a complex verb “to smell of cats”. Your room is doing it badly. But grammatically.