If the ancient Greeks believed their gods lived atop Mt. Olympus, then what did the ancient Romans believe? Did they have an equivalent mountain?
No. There is no one overarching Roman religion or mythology; it was a mishmash of their original cults, those of the peoples they conquered, and most of all the Greek.
The Roman versions of the Greek gods (Jupiter, Juno, et al) still lived in Olympus. The rest, nowhere in particular.