Sunny and hot last year – 80-100 degrees typically in the summer; overcast in the mornings, since I’m relatively near the ocean, which burns off well before noon. Very low humidity. No rain at all most summers, everything needs to be irrigated or hand-watered (I did the latter). Also, I have had excellent luck with tomatoes in this climate; not as good as where I grew up (Illinois), but really good. Except two years ago when we got tomato blight (I forget the real name – black spots on the leaves from some fungus); we changed all the dirt in the raised beds last year, and got small fruits, despite liberal cowflop in the soil.
You might not have been giving them enough water, and you might want to go for less fertilizer, and a more balanced one (like 13-13-13) when you do give it to them.
And, living in Gilroy, shouldn’t you be growing garlic?