That’s it in a nutshell. Circular orbits are possible for any sort of gravity at all, but in most cases, they’re not stable. In our normal inverse-square gravity, if you slightly perturb an object in a circular orbit (say, slow it down slightly), it’ll move closer to the star for half an orbit, then come back out for the other half, ending up back where it started after tracing out a complete ellipse. But in inverse-cube (or higher powers) gravity, if you nudge an object in a circular orbit, it’ll either spiral into the star, or out off into infinity, and never return to its original distance.
This is not an insurmountable obstacle for higher-dimensional theories. First of all, this only applies when the central object is a point source or the equivalent (a hypersphere that fills all of the available dimensions is equivalent to a point source). If, instead, your central object is a very long line extending through one of the dimensions, then its gravity will fall off as the inverse square of the distance from that line, and objects restricted to the space perpendicular to that line will still follow familiar Keplerian orbits.