Your argument is found in Guns, Germs, and Steel, by Jared Diamond. Although his book explains why agriculture and civilization began where they began, he ignores the fact that once these begin they have different population pressures than a hunter gatherer existence.
Agriculture requires the ability to plan for the future, defer gratification, and endure long periods of tedium. Civilization selects for superior intelligence because men of superior intelligence become successful merchants, government officials, artisans, artists, and so on, and live better lives than manual laborers. Consequently, they are likely to have more children who survive and reproduce.
The longer a racial group has practiced agriculture and city life the lower its crime rate is likely to be, and the higher its average intelligence. This can readily be seen in the charts in the following websites.
An excellent supplement and partial rebuttal of Guns, Germs and Steel can be found in The 10,000 year Explosion: How Civilization Accelerated Human Evolution, by Gregory Cochran and Henry Harpending.
The 10,000 Year Explosion explains the evolution of racial differences that are documented in The Bell Curve, by Richard Herrnstein and Charles Murray.