Is it possible we will soon have some version of police-style facial recognition in our pockets?
I mean a pocket device you can point at someone and identify them. Can a civilian subscribe to the fancy databases that the police now use? Can all the data required of such a service be squeezed down into a consumer product?
I suppose there are technical, legal and moral issues to this of course.
You could scrape the names and matching faces of everyone who has those public facing, through sites such as Facebook and LinkedIn. Once you have that, you just need a location to house the data, a webservice to front the data, and an app, mobile or otherwise, to access it.
The fact that scraped data on the internet is considered scrapable and reusable isn’t as well tested in court as I’d like for it to be, so you’ll likely get sued by someone, but many companies have made a business out of collecting scraped data and reselling it in many forms.
This is what Clearview AI has done. Further, we can presume every intelligence and police agency in the world is doing the same thing. This is horrible. This is terrible.
When can I get a service like that for my own personal use?
Right now, the only way to build one that I’m aware of is to build it yourself. The hardest part will be the scraping of data. Not because it is difficult, but because it is fiddly and time-consuming. You have to put natural-appearing pauses in your scrapes and occasionally rotate which IP address you are coming from, or most major sites will simply block you in an automated fashion. You can also use a proxy service that rotates your request through thousands of IP addresses, but that costs money in addition to the time. The fiddly part is that once you are well on your way to scrape those 3 billion (assuming you want to match Clearview) image/identifying data combinations, one of your sources will change their layout and you’ll have to modify your scraping code.
When you have your database, everything else is a piece of cake.
It would be useful, such as when you meet someone at a cocktail party whom you should recognize from the last social gathering but can’t remember their name. Perhaps the system could whisper their name into your earpiece, along with some basic biographical information, so you can sound like you haven’t forgotten their kid just started college.
Sure, you could also use it to get the phone number or maybe even address of that attractive person that is coming towards you on the sidewalk. Not that that would be creepy or anything. 
A few years ago (A century in technology time) some tech-savvy geek was identifying pretty girls on the Moscow metro. This was what first alerted me that this technology was of age.
Further, I suspect the really good stuff is quite advanced from the systems we know about. I bet the SCS has some really neat stuff.
This would be all kinds of illegal in Europe. Even if you post a picture on a public-facing site that is viewable by anyone, it’s not lawful for a private company to scrape and save it for themselves. (There are exceptions for law enforcement.)
“Really good stuff” in this context would apply almost entirely to the number of records and features of each person that you have in your database. The actual facial recognition piece is pretty well mastered and available to the public in many libraries. There are two parts to the usage. First up is facial detection, which is the piece that identifies the bounding box of a face in the current frame. No need to do a lookup of that stop-sign that they are standing under. Then you have to do the recognition piece, which tries to find a matching face in your data to the face in the detected bounding box.
And in fact one of the early articles in The New York Times on Clearview AI describes how investors, clients and friends of the company used the software in social settings.
I’m not surprised by that, but from past experience (not faces, but other images), it is not fully answered in the US. Our particular general counsel came down on the side of “it’s legal to use images scraped from the internet, but it can be a hassle to deal with the issues surrounding doing so, such as lawsuits.”
One thing to remember is that in large part this is already done on social media and cell phones, or at least the facial recognition part is. They just make you label that person, rather than looking it up in a database somewhere.
Another thing to consider is that based on the way the machine learning works for this kind of thing, each AI doing the recognizing does it differently and likely stores that pattern differently. So when Google identifies someone, that pattern can’t be used by Facebook’s recognition engines, because they’re two different systems. There’s not AFAIK, a standardized facial recognition template for this sort of thing.
I would imagine that the best way to do it overall would be some kind of opt-in system, where you’d grant consent to the facial matching company to store and process your social media images, contact info, etc… along with a sort of “friends, friends & family, public, etc…” type level of consent for who could see that.
I’d even go so far as to suggest that it allow for public/private identities, so that if your “day” job is as Mistress Dominatrixa the Terrible, you could set up a public identity/face-matching profille that would identify you as that to the public, but identify you as Betty Smith of Sheboygan, WI to friends/family/financial institutions/etc… And on both, you could limit what you choose to expose- maybe just your name, maybe your name & email, etc…
That way, I imagine if you were to wander around with some sort of smart monocle or something, it would identify some people but not all, and of those it identifies, you’d only see what they wanted you to see, which might be commercial- you might be in the grocery store, and your digital monocle might recognize a guy, and it would say “Joe Smith - Joe’s Roofing (222) 333-4444 for all your roofing needs!” without really letting you know anything about Joe. Or people might be stupid and list everything on there for anyone to see; their choice.
Can you tell me more about this?
For a quick starting point, I’d suggest this tutorial, which is built in Python using available libraries and includes a sample dataset to work with. In practice, which libraries you use could be dependent on the type of capture you are using to get the faces to be matched. Are you surreptitiously capturing them? You need something that is better at handling faces at odd angles. Are they voluntarily posing with their face looking straight ahead? Most libraries work fine.
While I’ve used image recognition plenty in my career and done web scraping projects many times over, facial recognition is just something that I play around with. I’ve been working on a Raspberry Pi connected camera setup as a morning greeter for the office if we ever return back there. Still a work in progress or I’d simply share my hardware config and final code with you.
Since we’re now getting into the nitty gritty anyway, if you really want to move forward, I’d probably just use Python (pick a 3.x version, no matter what the grumpy old folks might tell you) with BeautifulSoup for the web scraping of the billions of images you are going to want to start collecting. I wouldn’t normally recommend Python for tasks of this volume, but when it comes to scraping, the speed of the programming language isn’t an issue. If you try to grab thousands of pages per second on Facebook from a single IP, I’m guessing you wouldn’t last more than a minute or two before you’re blocked.
I’m not thinking it makes sense to build your own database. I’m waiting for Google or Facebook or somebody to make their whole database available as an API. The app on my phone takes a quick pic, sends it to them, and the dossier comes back in a couple seconds.
That would enable exactly the uses that @bump describes in #12 above.
I’m not sure which is worse: the government having this capability while almost none of us really know or think about it, or everyone having the capability which means we all know and think about being on both the sending and the receiving end of it.
What’s to stop family/friends/financial institutions from seeing what “the public” sees?
You could, practically, give a small, select group access to more information. You can’t, practically, give a small, select group access to less information.
I don’t want to scare you any worse, but I know hundreds of things about you. When I say you, I don’t mean the “royal you”, I mean “you, specifically” (assuming you are 18 or older and reside in the US). Good news: I don’t know which “you, specifically” happens to use the name @LSLGuy on this site. Bad news: I’m not part of the government and there are thousands of people with access to similar data.
Oh, I (personally I) fully understand. Part of why I left here in 2017 was getting a little too concerned about how wide a trail I leave here pointing right at my real, not figurative, forehead.
As discussed in many other threads, the US public is hell bent on handing corporations, criminals, and hostile governments the keys to their own individual kingdoms. It will end in tears for many.
When it comes to accuracy, there may be problems. From csis.org:
" In ideal conditions, facial recognition systems can have near-perfect accuracy. Verification algorithms used to match subjects to clear reference images (like a passport photo or mugshot) can achieve accuracy scores as high as 99.97% on standard assessments like NIST’s Facial Recognition Vendor Test (FRVT). This is comparable to the best results of iris scanners. This kind of face verification has become so reliable that even banks feel comfortable relying on it to log users into their accounts.
However, this degree of accuracy is only possible in ideal conditions where there is consistency in lighting and positioning, and where the facial features of the subjects are clear and unobscured. In real world deployments, accuracy rates tend to be far lower. For example, the FRVT found that the error rate for one leading algorithm climbed when matching against high-quality mugshots to 9.3% when matching instead to pictures of individuals captured “in the wild,” where the subject may not be looking directly at the camera or may be obscured by objects or shadows. Ageing is another factor that can severely impact error rates, as changes in subjects’ faces over time can make it difficult to match pictures taken many years apart. NIST’s FRVT found that many middle-tier algorithms showed error rates increasing by almost a factor of 10 when attempting to match to photos taken 18 years prior."
MIT Technology Review Are Face Recognition Systems Accurate? Depends on Your Race. | MIT Technology Review
states " Law enforcement agencies haven’t provided many details on how they use facial recognition systems, but in June the Government Accountability Office issued a report saying that the FBI has not properly tested the accuracy of its face matching system, nor that of the massive network of state-level face matching databases it can access.
And while state-of-the-art face matching systems can be nearly 95 percent accurate on mugshot databases, those photos are taken under controlled conditions with generally coöperative subjects. Images taken under less-than-ideal circumstances, like bad lighting, or that capture unusual poses and facial expressions, can lead to errors."