Again, oversimplifying: we are made of water, because we came from water. What made it possible for us to leave the water was that we developed a way (oversimplifying) to carry a self-contained “ocean” around with us: our circulatory system. Warmblooded animals developed a pretty sophisticated self-regulating internal pumping system to keep oxygen and nutrients supplied to each cell in our bodies, while many the smallest of the oceanliving animals have enough of their cells within direct proximity to the actual ocean (as opposed to our “artificial” one) that they can collect what they need directly through osmosis.
Now, the ocean is extremely stable, certainly as compared to a single land dwelling animal’s body; it can take decades to vary by a fraction of a degree, or minute changes in chemistry. So animals who “stayed behind” didn’t necessarily need to evolve the kind of self-regulating systems that we did; the system they live in doesn’t really require regulation because it’s so very very stable.
This is a complicated (yet still vastly oversimplified) explanation to explain the different degrees to which animals are dependent upon the stability of their environment for survival. Fish are separated from the larger circulatory system of the body of water they live in only by an osmotic membrane, so fish that live in extremely stable systems can be drastically affected by minute changes in the water, chemically or temperature-wise. So, with many exceptions of course, many species of fish tend to live in a relatively narrow window of tolerance. So fish that have evolved in warm water tend not to adapt favorably in cold, and vice versa. Fish that evolved in ecosystems that change rapidly–mountain streams; tide pools; rainy-season waterholes–tend, of course, to have higher tolerances for sudden changes.