I just assumed that it filtered the background by identifying pixels in frame that don’t change over time (without requiring an initial image). You can use image analysis methods to build an invariant background around the edges of the less invariant face. It’s similar to digital image stabilization. If there isn’t enough lighting/resolution to create contrast so the camera catches detectable variation of the foreground face, you can get the background bleeding into the face.
That’s how I’d approach it, anyhow (maybe with some face recognition to set focus areas).
Well, I joined a corporate Zoom meeting today - the meeting ID has more digits (11 from 9), as does the participant ID (6 or 8 maybe, from 3). This increases the attack space to identify meetings. My corporate account also mandates passwords and waiting rooms now. It’s a bit of a pain to remember to allow customers into the Zoom meetings for support calls.
It’s more sophisticated than that. I can sit absolutely still, and yet the system doesn’t paint the background over me. My wife can walk by behind me, and the system is smart enough to continue painting the background over her.
On the one hand, it’s pretty amazing, on the other hand it wouldn’t pass muster in an Ed Wood production.
It’s identifying your face, which is pretty typical for technology these days, then uses contrast and pattern recognition to retain your hair and body from the camera feed. It works, and we’re fine with it messing up our hair or shoulders from time to time because it’s a Zoom meeting, not a Hollywood Blockbuster.
Some observations from using Microsoft Teams (a different software that uses a similar feature):
[ul]
[li]it is reasonably good with heads with hair on top, but when confronted with male pattern baldness it tends to take a bite out of a bald pate[/li][li]it can be fooled by draping a blanket over one’s shoulders (a colleague called it a Harry Potter effect)[/li][li]when my wife walks behind me she is part of the background, but when she looks over my shoulder she suddenly is part of the foreground.[/li][/ul]
It does a terrible job on my system. I have a Lenovo Thinkpad running Windows 10 Pro. In a reasonably well-lit space, using the integrated camera on either Zoom or WebEx, I get a decent image of my face and whatever is behind me. But when I try to use the virtual background on Zoom, things behind me still appear, while pieces of me disappear. I found a small patch of bare white wall to stand in front of, and that almost worked, but the background showed up in the whites of my eyes! I don’t have a good way to hang a blanket or similar to use as a green screen.