Yes, but that’s true everywhere. It takes time to build infrastructure and it will be done incrementally.
When we first started shipping stuff across oceans it was incredibly expensive. The first oil wells were very expensive. But today we can pump oil out of the ground on the other side of the world, get it to a port, ship it across the world, refine it, then ship,it across a continent then sell it at a profit for a couple of bucks per gallon - cheaper than bottled water. Humans are ingenious when it comes to shaving costs.
One advantage the moon holds over Earth is that the environment there is incredibly static. A machine built for harvesting regolith doesn’t have to worry about wind, rain, mud, randomly variable temperature, and all the rest of the complexity of mining on Earth. The moon being in near vacuum enables a lot pf processes that are expensive on Earth, and enables large scale automated processes.
Imagine setting some autonomous robot miners free on a huge tract of regolith. These things scoop,up regolith and bring it to large solar furnaces which melt the regolith, releasing oxygen, iron, titanium, and all sorts of useful stuff. Absent weather, the solid materials can just be put in huge piles with no protection from the elements required. We could even fill craters with the stuff, using them as pre-made silos.
Now we build a large reactor, either uranium or thorium, and we use that power to drive coils to electrmagnetically launch payloads. We also use the plant to power industrial processes on the Moon and provide life support to workers. We can sinter regolith with solar ovens to make building materials for the moon.
The launcher could launch stuff like ores that can handle high g’s without any rocket stage at all, or for more sensitive stuff it could act as a first stage at much lower g forces. Once in space, the payload can use lunar-sourced aluminum fuel to take it to LEO.
How cheap could this process get? Who knows? Maybe we’ll be able to ship stuff from the moon to LEO for a buck a kilo some day.