"Yellow and yellow-bellied haven’t always had a negative connotation. An earlier and completely different use of “yellow-bellies” apparently first arose in England as a humorous reference to residents of Lincolnshire, without the connotation of cowardly, as far as I can tell. The regimental flag had a yellow background, and frogs found in the fens were in fact yellow-bellied, so we find the term in print around 1796. It was a word like redcoats, meant as descriptive and humorous, slightly ridiculing. In the 1890s yellow-bellied was used in a literal sense to indicate fish, such as sole and flounder, that have light-colored undersides. "
Thaks for this!! I was born in Lincolnshire and had always wondered why my dad called me a yellow belly! I didn’t think he was calling me a coward, but its nice to know its not an insult! 
)