If you assume that the aliens in question have a somewhat similar biology to us, then the lower limit comes from the brain. Intelligence is basically limited by the number of neurons, and (again, assuming an Earth-like biology, which may or may not be a valid assumption) you can only make neurons so small.
Birds, needing to minimize weight in order to fly, have a much higher neuron density than humans. The obvious conclusion from this is that humans are nowhere near the lower size limit. But you are probably talking about something the size of a large bird for the lower limit.
For something the size of a blue whale, you’d definitely need a planet with a significantly lower gravity. The massive Saturn V rocket had a payload of 120 tons and that payload includes the entire habitable part of the spacecraft, including life support systems, food and water, power, computers, etc. The blue whale weighs about 200 tons which is almost double the payload capacity of the Saturn V and that’s before you’ve even started building a spaceship around Mr. Whale.