Course 2 - 206 - Facial Recognition for Retail Flashcards
What is facial recognition?
Facial recognition is a way of identifying or confirming an individual’s identity using biometric software to map an individual’s facial features.
What is machine learning?
the use and development of computer systems that are able to learn and adapt without following explicit instructions by using algorithms and statistical models to analyze and draw inferences from patterns in data.
What is deep learning?
A subset of machine learning, deep learning is an artificial intelligence function that imitates the workings of the human brain in processing data and creating patterns to extract higher-level features. Deep learning uses a multi-layered network to repeatedly perform calculations and make predictions, using the available information to get better and better at identifying and classifying data on its own, progressively “learning” and gradually improving the accuracy of the outcome over time.
What is computer processing power?
The fundamental rate at which a computer performs its most basic operations, computer processing power is a rough indication of how many calculations a processor can make each second – the higher the number, the more calculations that can be made.
What are some of the more common ways that facial recognition is being used in the retail environment?
Loss Prevention: The most common use case for biometric surveillance is to assist in the reduction and/or prevention of retail shrink. Similar to traditional video surveillance methods, data is typically collected from cameras mounted near entrances, at customer service desks, in warehouses, or other locations where the technology can be successfully applied.
In the artificial intelligence (AI) industry, biometric surveillance is commonly referred to as “in the wild” facial recognition. The goal is to instantly know the moment a known criminal suspect, such as a banned shoplifter, enters a retail location. The loss prevention team is immediately alerted so they can proactively deter criminal behavior, potentially pursue an apprehension if criminal activity occurs, or contact law enforcement if necessary.
Violence Prevention: Facial recognition can also be used to identify potentially violent offenders when they enter stores. By identifying when known or suspected violent offenders enter the stores, retail employees can deter, prevent, or de-escalate potential incidents of violence, protect customers and employees from potential threats, and involve law enforcement if and when necessary.
Investigations: Facial recognition is commonly used to aid, enhance, and expedite investigations. Rather than manually reviewing hours of surveillance footage, a facial recognition image search can reveal specific store locations, zones within those locations, and times where suspected individuals have appeared. This can also help unearth evidence of additional crimes and locations where suspects haven’t been previously identified.
Access Control: Facial recognition can be used to establish a virtual perimeter around sensitive locations. Imagine an invisible gate requiring a person’s face to be used as the key to enter.
When facial recognition is used to ensure that only approved individuals can gain access to a location, this process is commonly referred to as “whitelisting.”
Facial recognition can also be used to ensure that specific individuals do not gain access to a location, a process commonly referred to as “blacklisting.”
Access control deployments using both whitelisted and blacklisted individuals is commonly referred to as “greylisting.”
Identity Verification: Facial recognition can also be used to validate identity or age during checkout, such as during alcohol purchases or at self-checkout. The technology can be used as a primary form of identification, or can be used as a second factor of identification. The primary goal for the retailer is to reduce fraud, ensure legal compliance in sales of restricted merchandise, and improve operational efficiencies.
Customer Experience: Facial recognition can be used as a triggering mechanism to enhance in-store engagement, improve customer assistance, or even in advertising. For example, facial recognition enables retailers to instantly recognize when individuals enter the store, enabling the retailer to send a personalized coupon, promotion, or offer of service. This process allows retail customers to opt-in to a facial recognition program, the same way they might opt-in to receive coupons via email.
Visitor Analytics: Facial detection, which is related to facial recognition, can be used to estimate in-store traffic volume, demographics, and even customer sentiment. Rather than using biometrics to identify individuals, the technology is used to generate measurable data reports that will be useful for store operations, marketing, and other non-security uses.
What is ground truth data?
Facial recognition is a form of artificial intelligence (AI) that uses machine learning techniques to train algorithms—computer calculations and rules, that automatically measure thousands of points, landmarks, and other defined features across a human face—to identify and differentiate individuals. Long before systems are deployed to retail locations, these algorithms are trained using a neural network, a system modeled after the human brain to make authentications, and tens of millions of facial images, typically referred to as “ground truth data.” This ground truth data is typically extremely diverse and helps to eliminate any bias and significantly increases solution accuracy.
Over time, the system is trained to recognize a wide variety of human faces across every demographic. A system trained with demographically diverse videos and images will identify individuals with a high degree of accuracy despite changes in appearance such as facial hair, hats, or other factors.
What is match probability and how is it used as part of a facial recognition program?
Match alerting is an extremely important component for any facial recognition system used for retail security. Alerts are typically sent to in-store loss prevention teams through mobile app messaging, text message, email, or through interactions between multiple software applications (known as application programming interface, or API) with other systems.
Match probability is also an important component of most facial recognition systems. Match probability is an estimation of the likelihood that a person is really a match with an existing file image in the gallery. Match probability settings help determine how many alerts are sent. For example, a loss prevention manager may wish to set the system to alert only if a match probability is at or above 95%. Conversely, an alerting threshold of 80% would typically result in more alerts. Setting proper accuracy thresholds is critical to accuracy and ensuring that retailers minimize false positives.
What are the primary privacy goals to consider when using a facial recognition system?
Restrict Data Access. Enforce a role-based hierarchical system for biometric data access, ensuring that only individuals that truly need to view, edit, or otherwise administer data can do so.
Collect Only Required Data. Collect only the data needed to meet project objectives. For example, when using a facial recognition solution to identify individuals who have been banned from a retail location, consider turning demographic estimation reporting such as gender and age off. This avoids the collection and storage of unnecessary data.
Minimize Data Storage Duration. Retailers should avoid storing biometric data for longer than absolutely necessary. Actionable steps to avoid unneeded data storage include:
Configure systems to automatically purge anonymous data at regular intervals, including mobile devices that may incorporate such data.
Set expiration dates for enrolled system data.
Schedule data audits to determine which personally identifiable data should remain in the system, and for how long.
What is an anonymous visitor? What are some simple steps that can be taken to ensure anonymity?
Ensure Anonymity Whenever Possible. As part of a security-focused biometric surveillance situation, a facial recognition platform attempts to recognize every person within camera view. The facial recognition software converts the images of all such persons into biometric templates. It then uses those templates to match against images of individuals in a gallery. Visitors who enter a store, but are not on a watchlist or enrolled in an opt-in loyalty program, should be treated as “anonymous visitors.” To respect the privacy of these individuals, retailers should:
Take no additional action to attempt to identify these individuals through additional research third-party data source integration.
Use a facial redaction or image blurring feature to visually mask anonymous faces from system administrators
What are some of the steps that can be taken to ensure that data is not stored longer than absolutely necessary?
Be Transparent. Organizations using facial recognition for security purposes should post public notice informing individuals that their data is being collected and used, and for what purpose. Some states require public notice, although the exact notice or language is typically not defined. An appropriately worded sign may be sufficient. In addition, some retailers will post notices to their website privacy policy, which is recommended.
Obtain Consent for Marketing or Commercial Purposes. Using biometric data for marketing or commercial purposes falls outside the typical jurisdiction of most loss prevention professionals. In addition, most states and municipalities do not require consent to use facial recognition for security purposes, such as to identify suspected shoplifters. Nevertheless, loss prevention professionals should be aware that some states require specific consent to use biometric information for marketing purposes. Retailers are strongly urged to obtain consent in such situations, as not doing so may be considered a breach of privacy and consumer trust.
Why is it important to establish clear gallery enrollment guidelines when managing a facial recognition program?
When used in a security context, facial recognition galleries should be populated by facial images or video clips falling into distinct categories. While each retailer must determine which types of data they are most comfortable with, establishing clear guidelines is essential to the successful management of the program. Establishing and following clear guidelines greatly enhances success rates and operational efficiency while maximizing consumer privacy. Such guidelines may include but are not limited to:
Barring Notices
Should individuals signing barring notices be automatically enrolled?
Should the enrollment expiration date correspond with the duration of the barring period?
Should loss prevention personnel attempt to gain consent for inclusion in the gallery?
Video Evidence
Should video evidence of individuals leaving the store with stolen merchandise be used for enrollment?
If so, under what circumstances and how long should the data be stored?
Police Assistance
When law enforcement asks for retailer cooperation in finding a wanted individual, should those individuals be enrolled in the gallery?
If so, under what circumstances and how long should the data be stored?
Missing Persons
Should missing children or other individuals be enrolled upon request from families, nonprofit groups or police?
If so, under what circumstances and for how long should the data be stored?
Threats Against the Company or Employees
If the retailer has evidence that an individual has threatened physical violence against a retail location, employee or customers, should that individual be enrolled in the gallery?
If so, under what circumstances and for how long should the data be stored?
Demographic Data During Enrollment Generally Considered Unnecessary
As a general rule, when facial recognition is being used for retail security purposes, there is no need to document an individual’s race or gender in enrollment notes. The image itself is enough to drive the actions.
What are some general steps that can be taken to limit access to biometric data?
Limiting system access is vital for any software system, and this certainly applies to the management of the facial recognition process. Successful management of the program should include:
Enforcing Strong Credentials. Access to any part of a facial recognition solution should always be administered with strong passwords, two-factor authentication and other industry-standard safeguards.
Enforcing Access Hierarchies. Ensure that individuals at various seniority levels have access only to the data they need, as follows:
Purpose Limitation: Individuals may only have access to data collected for expressed purposes. For example, authorized retail loss prevention personnel may only have access to data collected for purposes of crime prevention or case investigation, while authorized guest services personnel may have access to data collected for purposes of customer engagement.
Role Limitation: Only individuals who are authorized and trained to use a system should be granted access to data collected within that software system.
Role-Based Access Control (RBAC) should offer the ability to enable and disable features per user. Access to audit logs should also be configured on a per-user basis.
Keep Data Logs. Ensure that all administrative access is logged and regularly audited.
Why is biometric data encryption important?
Encrypted facial recognition templates are very hard to spoof or gain access to. But since some records may be attached to personally identifiable information, it is important to make sure that this data is adequately protected at every stage. Without proper encryption, data might be vulnerable to attacks.
All data should be encrypted at every stage of storage and transmission.
Logical access to a live customer environment should only be established via a secure encrypted session and is restricted to authorized personnel only.
Why is human verification considered necessary when using a facial recognition system?
Artificial Intelligence is far better than human beings at recognizing individuals. Yet while research has indicated that facial recognition often exceeds 99 percent accuracy, it should still not be considered flawless. “False positive” is the industry term for false matches. False positive rates have dropped dramatically across the industry, but they still exist, and retailers using a low match threshold may get more false positives than what might be expected. Therefore, it is essential that all matches are verified by human eyes before any action is taken.
Retailers should ensure that human verification of any positive match is an important step in the crime prevention process. Following a match alert, if there is any doubt regarding a particular individual’s identity, loss prevention professionals should revert to simply offering that person great customer service.
Why is it important to align match alert language with authorized loss prevention actions?
Alerts typically include historical data about crimes that an enrollee is suspected of committing. It is also important to document what types of items they are suspected of having stolen and include relevant information in alerting metadata.
Most importantly, when considering the prescribed action to be taken as the result of the alert, prescribed actions should be completely consistent with established loss prevention procedures regarding approaching individuals within stores. Failing to adhere to established protocols could result in the unauthorized invasion of customer privacy, avoidable violence, or other unwanted or unexpected consequences.
Furthermore, these prescribed actions should be made in consultation with all relevant stakeholders on company leadership and legal teams. While every retailer will have their own specific policies, practices, and terminology, some examples might include:
Offer Customer Service. Approaching an individual who has been banned from the retail location by offering customer service will often deter or prevent potential criminal activity or other unwelcomed outcomes.
Monitor Behavior. Maintain observation and monitor behavior while taking no action unless there is explicit cause to do so.
Pursue Apprehension. If and when appropriate, loss prevention professionals should always follow all established company policies, practices, and protocols regarding the apprehension process.
Alert Police. If and when appropriate, loss prevention professionals should always follow all established company policies, practices, and protocols concerning law enforcement in these situations.
