Introduction To Face Detection
What is face detection?
Face detection is a version of object detection that, as the name implies, detects faces that are displayed in a visual medium, often having to pick them out from other parts of the image such as landscapes, other body parts, or other people. While face detection itself has few direct uses, it is the first step in many key applications used in fields such as biometrics, law enforcement, and even entertainment.
Face detection algorithms are generally divided into 4 categories, distinguished by what the algorithm is looking for. These categories are known as “knowledge-based”, which detects faces based on a set of rules that all faces must follow, “feature-based”, which detects faces based on extracting structural features of a face, “template-matching” which uses pre-defined templates to compare against potential faces, and “appearance-based”, which uses machine learning to find characteristics of a face. Appearance-based algorithms are further broken down into 9 subcategories based on the specific algorithm being used.
When the algorithm is requested to detect a face often the first thing that is done to the image is to convert it to grayscale. This is done because identifying characteristics are easier to perceive in grayscale. Next, as it is the most easily recognizable feature of a human face, most face detection algorithms start by identifying the eyes. From there it may attempt to identify further identifying features such as a nose, mouth, eyebrows, etc, but once a region is concluded to have a high probability of being a face, additional tests are performed to get confirmation and the detection data is sent back to the application that requested it.
Facial detection vs facial recognition
While these terms are often used interchangeably, they are not the same thing. Facial detection, as described above, refers to the detection of faces in a visual medium. Facial recognition on the other hand is a further application of face detection, and goes much further than simply detecting when and if a face is present. Once a face is detected the biometric data of the individual’s face is mapped and saved as a face print. This face print is then compared to a database of other face prints to find potential matches.
What is face detection used for?
If you’ve ever used a camera app on your phone, you’ve most likely already seen face detection in use. If you’ve taken a picture and seen a colored square appear around the photo target’s head, that’s face detection. When applied in cameras, this software is most often used to auto-focus cameras and identify if people have entered the frame, helping photographers take clearer and cleaner shots. Furthermore, face detection can tell a computer what part of an image or video to blur to maintain that person’s privacy.
Another common usage that you may have seen, is facial recognition. Facial recognition is used for everything from an iOS phone’s face ID to law enforcement. Many phones come with an option to unlock the device with a face ID in place of a passcode or pin, which uses facial recognition to identify if the person attempting to unlock the phone is the owner. While this is the use case the average person is most likely to encounter, it is certainly not the only one. In September of 2020, the LA Times reported that the LAPD had used facial recognition software nearly 30,000 times since 2009, comparing photos of suspects against their mugshot database.
While it is still in the early stages of development, one potential use of face detection is automated lip-reading. Automated lip-reading would use the visual data obtained through face recognition, and potentially combine it with the audio data from speech, to convert auditory speech to visual text. This would be particularly useful for people who are deaf or hard of hearing, especially in the time we are in now where most social interactions take place through a screen. Automated lip-reading even has applications in security, such as a computer detecting who is saying what on a screen, or picking apart overlapped conversations.
In a similar vein, emotional inference software is based on facial detection systems. Emotional inference software uses facial recognition to identify human emotions. One particular use of this software would be to help people on the autism spectrum identify and understand the feelings of people around them.
An application that takes facial detection a step further than even facial recognition is facial motion capture. Where facial recognition typically tries to simply gain a clear print of the face to compare against a database, facial motion capture creates a database of several face prints to track the position and shape of different facial features over time. These faceprints can then be used for CGI in a variety of mediums, being used to create far more realistic graphics in movies, video games, and the like.
Advantages of face detection
- Privacy
Unlike facial recognition, face detection does not store any identifying data. Face detection will typically only identify the presence of a face, and at most will use facial analysis to store general, unspecific data about the face detected such as age or gender. With cameras already nearly everywhere in our day-to-day lives, an addition of facial recognition software could serve as a nail in the coffin to whatever privacy we still have. While this may help law enforcement to detain criminals, it can still be used to keep tabs on the average citizen. Imagine a database containing hits from facial recognition software taken from every camera you pass by in your day, for every person in the area. It could, if storage allowed, feasibly have an accurate accounting for your whereabouts at every point of every day for an extended period of time. This isn’t even an unheard-of concept, as some governments are already implementing similar practices with phone records and other collectible data. - Improved security and safety
Even using only face detection, having an automated process to determine a person is moving towards a secure area could be a large boon to security workers. Adding in the benefits posed by facial recognition, law enforcement would have a much easier time identifying and tracking down criminals. The TSA is even piloting a biometrics program that makes check-ins at airports less time-consuming and physically invasive by utilizing facial recognition to cut down on time spent.
This technology is not necessarily limited to crime prevention either, as New Delhi found over 3000 missing children over the course of 4 days using facial recognition systems. This is just a drop in the bucket compared to the number of missing persons in India alone, but further refinements to the software and the ability to search for the missing persons earlier could reduce the number of missing persons significantly.
Disadvantages of face detection
- Data Storage
Face detection and all of its applications use machine learning to identify a face, and the storage requirements for such a process aren’t insignificant and would likely be unavailable to the average person. When taken to the further applications that require databases to match against or store additional data, the requirements can escalate quickly. - Vulnerability
This is a particular concern with facial recognition applications in fields involving security, such as biometric locks or law enforcement. While these algorithms tend to be more accurate than manual checks, false positives and false negatives aren’t uncommon. While a false negative might just mean you need to scan again, a false positive can allow unauthorized users into areas that should remain secure, whether that’s your phone or something bigger. In addition, facial recognition software is particularly vulnerable to larger changes in appearance that a manual check would otherwise catch. - Algorithmic Bias
These algorithms in their current state are far from perfect, but with boasts of classification accuracy exceeding 90% in some cases, you might think otherwise. The real problem comes from the disproportionately low accuracy when identifying minorities. The “Gender Shades” project in 2018 found that in particular, subjects who are female, black, and in the 18–30-year-old age group consistently have the poorest identification accuracy, with averages reaching as low as 20%.
Like every emerging technology, it is exciting to imaging the possibilities the new tech can be used for, but it is always important to remain informed of the risks and disadvantages instead of getting caught up in that excitement. This isn’t to say that new tech can’t be used for good, just that regulations need to be put in place to ensure that the potentially harmful usage of technology is minimized or prevented where possible. In the words of Thad Starner, “Any tech can be used for good or bad; it’s about encouraging good behavior”
Bernstein, C. (2020, February 03). What is face detection and How does it work? Retrieved May 10, 2021, from https://searchenterpriseai.techtarget.com/definition/face-detection
Dwivedi, D. (2019, March 27). Face detection for beginners. Retrieved May 10, 2021, from https://towardsdatascience.com/face-detection-for-beginners-e58e8f21aad9
Face detection vs. face recognition: What’s the difference? (2019, May 09). Retrieved May 13, 2021, from https://www.facefirst.com/blog/face-detection-vs-face-recognition/
Face recognition. (2021, February 15). Retrieved May 13, 2021, from https://www.eff.org/pages/face-recognition#:~:text=Face%20recognition%20systems%20use%20computer,in%20a%20face%20recognition%20database.
RECTOR, K., & WINTON, R. (2020, September 21). Despite past DENIALS, LAPD has used facial recognition software 30,000 times in the last decade, records show. Retrieved May 13, 2021, from https://www.latimes.com/california/story/2020-09-21/lapd-controversial-facial-recognition-software
Marchildon, J. (n.d.). Facial recognition TECHNOLOGY Identifies 3,000 missing children. Retrieved May 14, 2021, from https://www.globalcitizen.org/fr/content/missing-children-found-india/#:~:text=Hundreds%20of%20thousands%20of%20children%20go%20missing%20in%20India%20every%20year.&text=Close%20to%203%2C000%20missing%20children,software%20program%2C%20according%20to%20NDTV.
Puckett, J. (n.d.). Can I opt out of facial recognition at the airport? Retrieved May 14, 2021, from https://www.cntraveler.com/story/how-to-opt-out-of-facial-recognition-at-the-airport
Najibi, A. (2020, October 26). Racial discrimination in face recognition technology. Retrieved May 14, 2021, from https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/