I think insects may often have been avoided because some of them are unpleasant-tasting and/or toxic, discouraging experimental eating (once you’ve munched and spat out three or four different sorts of nasty caterpillar, you may be less inclined to try additional kinds, even though those might happen to tast like chicken).
They may also have been overlooked or avoided because they are often associated with decay.