simplicity. they just want to fit the engine into the space between the tractor wheels, and since 3 cyclinder diesel is smooth enough, they use it… Diesel cylinders can be any size, since all the diesel is ignited at the same time, by the compression.
Petrol gas spark ignition… the fire starts at the spark plug, and then takes time to travel down.
So for gas/petrol, with the spark plug ignition, ~500 cc cylinder size is the sweet spot, and its terrible to go larger limits the RPM too much… the flame front has to travel from the spark plug down through the cylinder… the larger cylinder the longer it takes…
Motorbikes are so efficient, they can choose to be a bit less efficient by using smaller cylinders…to get higher RPM …shorter burn time for the flame front to set the whole cylinder’s worth alight.