Simply not true. Hardwoods (or broadleaf trees) do not grow much slower than softwoods (or conifers). They usually grow faster. In my shop, white ash, elm, birch, maple and red oak lumber with up to 3/8" growth rings is common as dirt. With conifers around here, 3/8" growth rings is rare to non-existent. The average ring width in the locally-ubiquitous Scots pine is 1/16", too thin to be found on any of the hardwoods.
Most hardwoods are expensive relative to most softwoods because they are rarely cultivated in man-made monoculture forests where every tree is of uniform size and shape and of the right species; many commercial softwoods naturally and invariably grow into a shape that’s conducive to processing (ie. a single, straight bole with evenly distributed small-diameter branches), unlike many hardwoods with short or multi-stemmed boles and a wide, irregular crown with difficult-to process heavy branch wood. So, filling a truck with pine is much quicker and easier than filling a truck with maple, and many more truckloads of pine can be taken per area. Hardwoods, being on average much denser than softwoods, are also harder on the woodcutting and woodmilling machinery, more time-consuming to season properly (in expensively-run kilns) and much heavier in seasoned form, adding hardware, maintenance, utility and transportation costs relative to softwoods.
There’s also a premium put on by lumberyards on the hardwoods used on more prestigious projects than the lowly knotty pine, douglas fir etc. No-one builds scaffolding from rock elm, but it sure makes a pretty table top. This is reflected in the price.