How Does Facial Recognition Work?

How you look might start to matter so check that mirror before you step outside: Facial recognition technology is on the rise and it’s here to stay. Private companies use it to make our lives easier than ever, from unlocking phones, to finding a mate on dating sites, to automatically tagging friends in group Facebook photos. And from high-level federal surveillance work down to local traffic regulations, the government uses it too. Local law enforcement agencies have relied on the technology for decades to identify investigative leads in traffic cases as well as misdemeanor and felony larcenies. In fact, only a handful of states – including Washington – have banned its use.

Lately, mainstream media has released a salvo of concerns over the technology’s use and abuse, its effectiveness as a tool and its questionable legality in light of privacy concerns. And while it’s uncertain how these concerns will pan out legally (a smattering of local governments have banned its use in various capacities, including using footage from traffic cameras to personally identify violators), what we do know is that the practice of using facial recognition software is so deeply entrenched in both the private and public sector that the data, at least, is not going anywhere – and that it’s virtually impossible to be incognito. 

So, whether you’re dodging authorities for some nefarious reason or simply hoping no one saw you blow through that red light, in many cases, you’re out of luck.

What is Facial Recognition?

The internet contains a massive set of data: images of faces garnered from sites like Facebook and Instagram, for instance, among others. The data is used to train artificial neural networks to first detect and recognize faces. What we refer to as “facial recognition” is, essentially, an automated enforcement technology that runs images of faces through these massive databases to produce matches.

Take, for example, Identix, ® a Minnesota-based company and among the major developers of facial recognition technology. The company licenses software called Facelt, ® a tool that can select a face from a crowd, compare it to a database of stored images, and present potential matches.

In returning matches, the Facelt algorithm employs two key steps: detection and identification. Each face contains about 80 distinguishable characteristics, “nodal points,” which the software algorithm uses to detect and identify human faces. The points include features like the distance between eyes, the depth of eye sockets, the width of noses, the shape of cheekbones, and the length of jawlines. As to the detection step, the algorithm distinguishes faces from the rest of the image background – i.e., this is a face, not a street sign or a tree. And, it’s a human face, not an animal one. Then, the software runs the image through a database – images of other faces gleaned from a dizzying mass of data points – to identify a match.

The very best technologies are increasingly advanced. A recent test by the U.S. National Institute of Standards and Technology (NIST) found that between 2014 and 2018, facial recognition systems improved substantially (i.e., they reportedly got about twenty times better) in finding a match in a database of twelve million portrait photos, and the failure rate fell from four percent to .02 percent throughout the testing period. According to NIST, this technology is driving a new “industrial revolution” in facial recognition.

What’s Old Is New Again

While many believe that facial recognition technology is new terrain, like most things, it’s just a reincarnation of an older tool. The first known use of facial recognition hearkens to the mid-1960s when scientists used machines to recognize human faces. The Department of Defense eventually leveraged it to spot fugitives who crossed state borders, and recognizing the technology’s value in locating criminals, private companies started financing research on its use.

Fast forward to the 2001 Tampa Superbowl, where facial recognition technology was used to spot sought-after criminals in the crowd. This set off a firestorm of using facial recognition to sniff out felonious activities and now the technology is virtually everywhere.

Facial Recognition Technology in the Private Sector

The improvements in the technology’s accuracy have led to commercial opportunities for private companies. Airlines are using facial recognition to expedite the cumbersome boarding process, apartment complexes leverage it to allow residents to enter without keys, and even the hospitality industry has found a way to use it improve customer service.

The technology has also allured players like Amazon, as well as Facebook and other social media applications. You’re likely already aware of this if you’ve ever been mistagged in someone else’s photos. If you keep up with the constant iPhone iterations, then you’ve likely opened your phone by simply glancing at it. And popular dating sites use facial recognition to pair you with compatible candidates.

Facial Recognition Technology in the Public Sector

Where the rubber meets the road legally, though, is facial recognition as a tool in the hands of government players. Law enforcement agencies in major cities like New York, Los Angeles, and Chicago use the technology, as does the FBI and other federal agencies. And while the datasets in use are by no means comprehensive, a 2016 study found that roughly half of American adults are in a law enforcement facial recognition database – giving the government a substantial amount of data against which to run photos and images of suspects. 

At the local level, law enforcement officers have been using facial recognition technology for years to identify investigative leads. Departments commonly use still images from crime scenes and run them against databases of booking photos.

It works something like this:

In some states, like Florida, the practice has been a part of law enforcement work for decades. In fact, in part of Tampa Bay, officers have access to more than thirty million images, including drivers’ licenses, mug shots, and juvenile booking photographs.

The use is spreading across large cities down to small municipalities across the country as more and more departments are purchasing facial recognition software. According to market research firm Grand View Research, this “facial biometrics” market is expected to hit $375 million by 2025. This is largely because the software is relatively easy and cheap to install as opposed to costly DNA evidence. As such, departments can use facial recognition technologies to quickly identify suspects and solve routine crimes.

Law enforcement officers in many regions use private criminal databases, not at-large data like DMV records or social media profiles, allegedly assuaging concerns about privacy intrusions. But as debates over its legality continue to unfold, police are simply increasing their use of the technology. What started in Pinellas County, Florida (which received a series of grants to fund continued research on the technology), has since spread to mainstream use in Los Angeles, San Diego, Chicago, and New York, as well as hundreds of other state and local agencies. In Colorado, local police have stopped credit-card fraudsters in their tracks, Indiana police have caught a serial robber, Pennsylvania agents foiled a serial rapist, and Oregon police have caught numerous shoplifters red-handed. This barely scratches the surface – there are few limits, it seems to what law enforcement agencies can do with this technology.

Local and State Bans on Facial Recognition Use

Nonetheless, governmental use of the technology has sparked controversy. While law enforcement agencies and officers deem it indispensable to securing public safety, various state and local governments have banned its use due to fears about privacy violations and false matches. The Washington State Legislature is currently entertaining proposals to ban government use of facial recognition technology. Massachusetts has a similar proposal pending. And Oregon is ahead of the curve, having already implemented laws prohibiting the use of facial recognition technology in tandem with body cameras.

Local governments are joining the movement: San Francisco and Oakland have banned its use. And although there are currently no federal laws governing the use of facial recognition, momentum is building just enough at the local level to create a pressure cooker of opinions – so we can likely expect to see some movement at the federal level before long. Change is already afoot: A Representative from Massachusetts is co-sponsoring federal legislation to prohibit facial recognition in federally-funded public housing. The ACLU recently filed a federal lawsuit in Massachusetts claiming that the FBI, DOJ, and DEA have failed to heed a request for surveillance records. And a bipartisan group of lawmakers sent a request to FBI Director Christopher Wray and former Secretary for the Department of Homeland Security Kevin McAleenan requesting information about how the agencies use facial recognition technology (the agencies have yet to respond).

And recently, civil liberties advocates have joined the chorus of voices decrying the practice for fear that in the wrong hands, it can become a tool for authoritarian control as in China, where it is used for racial profiling.

How Facial Recognition Affects Citizens in the Traffic Context: Local Bans Mushroom into Broader Due Process Concerns

Unless you are sought by the federal government or you are a high-profile celebrity ducking public scrutiny at every turn, the use of your image and likeness is unlikely to phase you. But there is one context in which it undeniably affects us all: the otherwise innocuous and undetected traffic violation.

If you have human DNA, then you’re familiar with this scenario: You blow through a red light. Maybe you knew you were doing it (you were already so, so late) or maybe it was an honest-to-goodness accident. Either way, you sweat a little. You look right and you look left. You glance nervously into your rearview. Whew. No flashing lights. You think you got off scot-free. But did you? A week later, you receive that telltale packet in the mail with a photo of your very vehicle, trucking right through the intersection. The violation you were so sure that no one saw.

Speeding, red-light, and school bus cameras are a common type of automated enforcement technology used to deter traffic violations in Washington. If you regularly traverse the Washington State roads, here’s what you need to know: Yes, traffic cameras are legal in our State – but with requirements. Nine, to be exact.

For example, the statute says that traffic cameras must be paired with a sign or notice informing motorists of their presence. The cameras are only allowed in certain places, for instance, railroad crossings, school speed zones, and “intersections of two arterials with traffic control signals that have yellow change interval durations[.]” But the one that really, truly matters is this:

The camera must not capture your face. Only your vehicle or your license plate.

In other words, in Washington, surveillance cameras cannot be used to personally identify a driver. Like many other states and municipalities that have decided this is a violation of privacy, Washington has enacted legislation that specifically precludes the use of facial recognition as an investigative lead in criminal cases, including simple traffic violations.

Nonetheless, caution is in order: Not all states follow suit. While it is illegal in Washington, it is legal in other states. As of last year, about a dozen states passed restrictions on the use of facial recognition technology to identify potential criminal violations. (Here is a map showing the breakdown). In the others, however, it is still fair game – and law enforcement will likely continue to use it until they can’t.

Against Due Process?

But traffic issues are small potatoes compared to some of the broader implications: namely, misidentifying suspects and the disproportionate error rate when it comes to the identification of minorities. The concerns intensify when it comes to government use of the technology. Unlike private companies, government agencies are not naturally checked by market competition. As such, their errors tend to more deeply impact people’s fundamental rights and freedoms. After all, it’s not a big deal if Facebook misidentifies you in a photo, but it is very much a big deal if you’re misidentified as a suspect in a crime.

The technology is still maturing. It can be glitchy, and although it requires the highest standards of accuracy, the algorithm is only as good as the underlying data. Incomplete or lopsided data sets are ripe for misidentification errors. Imagine a database that contains mostly images of Caucasian males. A photograph of a black female, for instance, would yield a higher rate of misidentification when run against the system as compared to an image of yet another Caucasian male. This, precisely, exposes the system’s vulnerability to inevitable human biases.

Take for instance Amazon’s Rekognition software, a face-scanning technology used by law enforcement officers in Oregon and Orlando, which has been criticized for disproportionately misidentifying minorities. Recently, the ACLU conducted a study of the software and reported that it incorrectly matched twenty-eight members of Congress and that these matches were, disproportionately, people of color. The Congressional Black Caucus wrote to Amazon’s CEO, Jeff Bezos, expressing this concern that such skewed data could negatively affect communities of minorities. The issue was exacerbated by a published study conducted by MIT researchers showing that systems from Microsoft and IBM were better at identifying the gender of white male faces than dark-skinned female faces.

So far, only one court is known to have ruled on the constitutionality of law enforcement use of facial recognition software. In 2015, sheriff’s deputies in Jacksonville, Florida, photographed a man selling cocaine. Unable to identify him, they ran his image through a Face Analysis Comparison System (FACES), drawing from a dataset of more than 30 million drivers’ licenses and law enforcement photos. The software returned images of defendant Willie Allen Lynch and four other suspects. Lynch was ultimately arrested and sentenced to eight years in prison.

At trial, Lynch argued that he was misidentified, but the court denied his request to see the photos of the other four suspects. On appeal to the Florida appellate court, Lynch claimed that he was wrongfully denied access to potentially exculpatory evidence. The court affirmed his conviction. Technological evidence is subject to analysis through separate proceedings, but generally, the results of facial recognition have not been considered sufficiently reliable to withstand judicial scrutiny. This is likely because there is no uniform standard for how systems like FACES are used as investigative tools. When someone’s liberty is at stake, should a computer match be the sole basis for conviction?

Recall that this is an iterative process and depends not only upon the quality of the images, but also the quality, quantity, and diversity of the datasets to which it’s compared. Its accuracy depends on the testing. And while image quality has improved, in surveillance situations, law enforcement has not been able to reach a level of reliability that will assuage misidentification concerns.

What This Means for You

So, how does this impact you on a day-to-day level? And how will it play out?

The bottom line: No one really knows. The future of facial recognition by local, state, and federal governments is uncertain but for now, you should know that in many (most, actually) states it is fair game. So if you’re going to blow through that red light, at least flash a smile on your way through. Unless you are in Washington, in which case you are safe. For now.

I Got a Traffic Camera Ticket in Washington State. Do I Need To Hire A Lawyer?

Typically, no. In most cases, a camera ticket will not hit your driving record and should not affect your insurance rates, so long as you pay it. Essentially, it is treated like a parking ticket.

In Washington, you can fight your ticket by requesting a contested hearing – but make sure you check out the video associated with your infraction. Tickets received via mail include a list of instructions that break down how you should proceed if you choose to contest your ticket.

You may choose to request a mitigation hearing, saying that yes, you violated the traffic laws, but for good reason. A Washington judge has the discretion to reduce your fine. This is different from a contested hearing, however, in which you claim you did not violate the traffic laws. Granted, this would be difficult to prove if your violation is captured on camera. Not to mention, if your violation was in a school zone or you passed a stopped school bus, don’t bother: A judge will virtually never mitigate your charges in these cases.

Under Washington law, a traffic violation detected via a traffic camera is presumed to have been committed by the vehicle’s registered owner. If you weren’t driving, you can fight this presumption if you state, under oath, that someone else was driving at the time of the incident. you can do so by mailing in a declaration of non-responsibility.

In most cases, camera tickets are straightforward, but nonetheless, this does not mean that you won’t have questions, or that some situations might not warrant legal assistance. Our application makes this as easy as a few screen taps. Our SmartMatch algorithm will connect you with a lawyer in your region who can help you handle your ticket.

While we don’t recommend fighting a camera ticket, if you have gotten a more traditional ticket, we at Off The Record have made the process of hiring a traffic ticket attorney as easy as possible through our app or website: Just snap a photo of your ticket, answer a few questions, and let your lawyer take it from there. Get started with our website or iOS app and we will connect you with an attorney in a matter of minutes.

Image credit: Mike MacKenzie CC BY 2.0