Not really, although I’ve heard similar claims myself. I’d say that Swiss German just has different features to High German, some of which just happen to correspond with Yiddish.
There are also different shadings of Yiddish, both written and spoken, some of which come much closer to/are mixed more with Standard German (esp. Yiddish which was spoken in German-speaking and not Slavic-speaking areas).
For that matter, I’ve even heard someone speak Standard German (with a strong American accent and a couple of Yiddishisms!) on Kol Israel’s Yiddish radio service. But of course Yiddish is a separate language. Average Germans have a very hard time understanding Yiddish spoken normally (often a watered-down version is used in Klezmer music, etc. for German audiences). It would seem that it’s generally easier for Yiddish speakers to understand German than vice-versa.