There’s a difference between being able to stay on a horse in motion, and riding.
Balance and body awareness are key, flexibility and strength (whole body, not arm!) factor in too. I spent over 20 years teaching people of all ages to ride, and some picked it up really easily, some had to really work at it, some were hooked, and some just didn’t care about it much. The ones who had an easy time of it were not necessarily the ones that loved it, and those categories covered all ages. I’ve taught 3 yo beginners and 83 yo beginners.
One of the hardest things is your own balance. The stirrups on the saddle and the reins in your hands are NOT to keep you on the horse. The saddle is for comfort - you and the horse. The stirrups can be a help, but they are mostly for the rider’s comfort. They don’t provide a lot of stability though, they’re essentially a strap hanging from a single pivot point. Western saddles swing less than English, but it’s still not like feet-on-the-floor stability.
Communication with the horse is via the reins in your hands, your legs, and your body weight + position (your “seat”). The reins will not keep you on the horse, and mishandled reins can cause the horse pain. A horse in pain is not likely to be cooperative for long. At a very basic level the reins are used to signal turns and stopping. In more finessed riding, the reins are only a small part of the conversation, and the speed and direction are mostly influenced by the rider’s seat and legs.
I mentioned body awareness above. One of the real needs in riding is to be able to use one part of your body without it affecting another part. In can get very, very nuanced, but in broad terms you need to think of splitting yourself in 4 + chunks:
Upper (waist up, sometimes including your head but not always)
Lower (waist down)
Left (shoulder to heel)
right (shoulder to heel )
Head & eyes
As an example, if you need to make a tight right turn, maybe threading between trees, you’d need to use your head/eyes to look at the path you want to take, use a bit of right rein to cue the direction, softly close your left leg on the horse’s side to stop any drift that direction.
In a more beginner-centric example, you need to be able to keep your upper body from influencing your leg position. When humans get tense on a horse, we curl up and ‘go fetal’. That motion tends to make the legs swing up and back towards the horse’s butt, and that is when it’s easiest to fall off. To be solid in the saddle you need an upright position, with legs long and softly straight underneath you. Sometimes you do need to lean forward, but it has to be done still keeping your legs long.
Rein grip is a another hard thing, starting out. Humans control their world with hands, and when things get a bit scary we want to grab, clutch, pull. You can’t do that with reins, it won’t keep you on (and it will hurt/piss off the horse). They are attached to the horse’s mouth, which is at the end of a long and muscular neck - ie: no stability there.
The staying on part of riding needs soft strength, the real riding part of riding needs the ability to control your own body enough to influence theirs. If you can balance well on one leg, (and equally well on the left or the right), can sit on a balance ball or the edge of a rocking chair with your feet off the floor, you should have a relatively easy time learning to sit a horse, and then you can learn to ride.
Be careful, it’s addicting!
(I’m mostly an English rider, Western is a little different at the basic level, but not a lot. As you go up the levels and branch out into different disciplines within horse sports things get a good deal more varied!)