While this seems superficially plausible, if you drill down it’s not an explanation for the Canada/Australia imbalance. In fact, you make my point for me.
As you say, Canada has a number of very popular, “every street corner has a game” winter sports like hockey, skating and skiing, which are winter olympic events and so Canada should win winter olympic events in those sports. And it does.
The sports which Australians love, upon which there national focus, are cricket and three codes of football (League, Union and Australian Rules). These are the comparable popular, streetcorner games. None of these are even Olympic sports. So by your reasoning, Australia should win nothing.
But Australia does win olympic medals. It wins them in sports that are not the national focus. In sports that are small minority, like swimming and athletics and perhaps cycling, and indeed in sports so tiny that even describing them as “minority” gives an exaggerated impression (equestrian, shooting, boxing, rowing, canoeing, field hockey, weights, sailing, judo).
In short Australia wins medals in precisely the type of minority, non-focus sports that you excuse Canada from not winning.