It depends on the temperature, but I usually start wearing my summer clothes about mid-October.
Stupid upside down land, confusing everyone.
Well, I only live a hunded or so miles north of you, so I am quite familiar with the climate! My point is that I have enough clothes to cope with summer and winter, but I have one small wardrobe and a small chest of drawers. It all fits in there quiet nicely without any need to shift things around from season to season. Simples!
Ah, I see what you’re saying - I thought by wardrobes you meant “collections of clothing”. Yes, same as me, all my clothing is in one wardrobe even though it varies quite a lot.
Should have noticed you’re in the same country as me. :o
Nobody in our house has that huge a wardrobe, and we have enough clothing storage space that we don’t switch summer and winter clothes around. However, it has become a tradition that I wash the woolens and the winter coats on May 1st, which is a holiday around these parts, before they disappear into the appropriate closets. That way I’m sure they’re all clean before the summer.
The first warm week, when it’s at least 60 degrees. I always keep a couple of hoodies and sweaters out year round just in case it gets chilly. The winter sweaters get washed and put in boxes till next year, with the ‘winter’ slacks and jeans. The tank tops, t-shirts, skirts, and sundresses come out and hung up in the winter clothes’ place. I try to pick through and get rid of clothes at this time, too.
Based on past experience, about 48 hours prior to a late patch of really crappy weather. 
I pretty much wear shorts, a t-shirt and sandals all year, even when it’s like 4ºC outside. At night, in winter, it can go as low as -7ºC or so, and I’ll still be in shorts, a t-shirt and sandals.