Ummm ok, so what is oscillating exactly in an electromagnetic wave?

RE: What’s oscillating.

So to simplify, is it safe to say that electromagnetism—being a fundamental force—isn’t oscillating in any medium, but propagates under its own fundamental laws within spacetime and matter?

That’s probably about as accurate as you can get in a non-mathematical explanation.