The difference between chalk and cheese
The cite to Brewer’s [BTW thanks for that bibliophage, saves me fossicking through my hard copy :)] makes the important distinction that the phrase relates to things that have some superficial similarity yet are totally different in character.
Look at some of the excellent English aged cheddar cheeses e.g. Cheshire which is chalk white. I hazard a WAG that the original derivation might date back to a time when an unscrupulous traders might have substituted one for the other.
IIRC, the phrases “Buying a pig in a poke” and “letting the cat out of the bag” relate to a smilar agricultural con trick i.e. a cat for a suckling pig.