There are two broad categories of food poisoning (although they overlap a bit too); infectious and toxic.
Infectious food poisoning occurs when you ingest a sufficient population of pathogens (such as salmonella) to overwhelm your body’s defences; they reproduce out of control and cause damage to tissues by a number of means, but mainly by secretion of toxic waste products. Eventually, your immune system catches up and wipes them out. Well, either that, or you die.
Toxic food poisoning occurs when you ingest food contaminated with sufficient levels of toxins to cause chemical poisoning of your body. These toxins may have been produced by pathogenic bacteria living in the food (and they may persist even if cooking kills or attenuates the live pathogens), or they could be chemical toxins from other sources such as cleaning materials (at least they lumped this in with toxic food poisoning on my food hygiene courses).
You won’t get ill from ingesting a single pathogenic bacterium; it takes a population of them to get past your defences (although I expect this is a statistcial curve where the likelihood of infection increases continuously with the size of the sample ingested).
No food is really aseptic either; there are probably low levels of food poisoning pathogens in just about everything - they are abundant in soil, dust, animal faeces, human hair and skin, etc, but again, they’re only a problem when you encounter them en masse.
Food poisoning normally occurs when food is improperly handled or stored; the pathogens (both the infectious ones and those that produce residual toxins) require food, moisture, time and warmth to do the damage.
The food they require consists usually of protein-rich items such as meat, dairy, fish, eggs.
So, a typical story would be:
Juices from a defrosting chicken are allowed to drip into a bowl of fresh cream on the shelf below (this is cross-contamination through improper storage); the pathogens present in the chicken will not be a problem, because that will be cooked before consumption, but the cream is whipped and plonked onto some cakes, which are then left unrefrigerated in a room for a couple of hours; by the time they are eaten, the pathogens have multiplied dramatically in the cream and are now present at levels sufficient to cause illness.