'The Limit of the Heat'?

AKA Hysteresis

As long as there is still sufficient heat input past the peak of heat input (which there is, because peak input is beyond the input level necessary for balance), the system will continue to get warmer.

So is the longest day (solstice) also the point in the year when the Northern Hemisphere receives the most direct solar radiation (i.e. tilted more toward the sun)?

All other variables being equal, yes, because the daylength is longest (longest period of input) and the angle of inclination is highest (so the radiation intensity is highest).

To express the hysteresis thing a different way: Assuming the days before(-1) and after(+1) the longest day are equal, and there was enough solar radiation on day -1 to make get warmer, then it should also get warmer on day +1