Well, it wouldn’t be same-sex forever (unless you’re sorting out only XX sperm) and it would probably be a good idea to have some older males as behavioral models for the next generation of boys.
A skewed sex ratio, say 10:1 women to men, might work better.
I assume you’re talking about a skewed sex ratio in addition to the frozen sperm? Because just having a small proportion of men would not give the best genetic diversity. But having the sperm bank + a few adult men in the starting population might be the best way to maximize genetic diversity, while still retaining some behavioral norms.
Assuming that the Enclave wants those behavioral norms, that is.
I really don’t think the gene pool is a crucial issue. Little comes of inbreeding except recessive genetic diseases pair up homozygously more often. And things won’t get worse with time. The tenth generation will have as many individuals with rare genetic diseases as the second (and not too many either case). And it could have a lot less, if carriers aren’t allowed to breed.
It’s when you leave your cave to repopulate the earth that you’ll wish for a bigger gene pool.