By Robert Moss, an independent writer who focuses on the value gained through computer vision, AI, machine learning, IoT and other technologies

Company executives around the world are under pressure to overcome a variety of challenges – some directly related to the coronavirus pandemic, and some exacerbated by it.

For example, the healthcare sector, already under increased strain on resources, must ensure the protection of staff and patients by isolating suspected COVID-19 cases from other patients. Grocery stores and restaurants face stricter hygiene requirements, such as limiting customer numbers and keeping surfaces clean to prevent the spread of COVID-19. While retail businesses are concerned that an increase in shoplifting, which has risen since the pandemic, will depress already thin margins.

For other sectors, there have been opportunities. Organizations that experienced a boost in visitors, such as pharmacies, grocery stores and banks, see highly-targeted out-of-home advertising as a means to better communicate with customers.

To rise above these challenges and exploit opportunities, more companies are turning to computer vision, artificial intelligence (AI) and machine learning solutions.

These technologies enable organizations to get real-time data on what’s happening in their premises. For example, a retail business can compare images of what’s occurring in their stores with images in a database. This enables them to automatically determine situations such as:

Hygiene status

• Whether a surface or object needs cleaning?
• Has a cleaning crew properly scrubbed a surface or object, readying it for the next user?
  

Customer behavior

• Is a consumer interested in learning about a product or purchasing it?
• Might a shoplifter be intending to steal a product?
  

Social distancing

• Has the number of people who’ve entered reached its allotment?
• Are people properly spaced out within a location?
  

According to Thor Turrecha, meldCX EVP of SaaS, AI can help organizations solve these and other challenges. “Each vision AI solution requires a database consisting of thousands if not millions of images. They’re needed to train an application to recognize people, products and objects as well as activities and behaviors with a high degree of confidence.”

Organizations need to collect and analyze images in real-time in their locations. This is performed by edge devices mounted on site. When an image corresponds to a relevant image in the database, the appropriate notification can be sent. But how can data collection comply with privacy laws and be achieved at low cost?

Anonymized synthetic annotated images lower barriers to entry

To reduce the cost and meet regulations, developers are working with anonymized synthetic annotated images. This enables them to produce the millions of images needed for machine learning training at a relatively low cost. Likewise, capturing and anonymizing images in places such as supermarkets conforms to consumer expectations of privacy as well as confidentiality laws. 

Anonymized data removes bias from both the training and operational stages of a computer vision AI solution. The technology can then focus on movement and behaviors, which forms the relationships between people and objects or products. 

“Removing individually identifiable elements helps ensure the AI can be designed to avoid conscious and unconscious bias, and thus meet standards of ethics and accuracy,” Turrecha explains.

Image annotation plays a role in the process by enabling machines to learn how to assign metadata, such as captions or keywords, to digital images. The technique is used to organize and locate specific images in a database, allowing AI to complete a range of tasks.

How many anonymized synthetic images are needed to be accurate?

The number of images required depends on the particular task to be performed by the AI. The greater the level of specificity, the more anonymized synthetic images are needed.

For example, people counting needs relatively few images. The AI simply detects when an individual has entered or exited a space. It isn’t analyzing whether they’re interested in a product or are performing a specific action. A mall, shop, clinic or other business may use this functionality to ensure that visitors have enough room to maintain social distance. 

Other tasks demand a significant amount of sample images. Using computer vision to decipher handwritten mailing labels on parcels is such an instance. Many millions of images are needed to train the solution to accurately read a wide range of handwriting styles. In addition to handwriting, the database must also contain every possible mailing address permutation. Using synthetic images and data makes a solution of this scale cost effective.

Using AI to track multiple models and behaviors 

Tasks such as shoplifting prevention require training the machine learning solution to do more than simply spot an individual concealing a product. It must learn to recognize complex suspicious behaviors such as:

• looking less at merchandise and more for the presence of staff and shoppers
• handing off products to accomplices
• loitering
• wearing a big raincoat on a sunny day 
• performing normal shopping activities, such as examining products, only when another person approaches them
  

By tracking these and other behaviors, the solution can focus on suspicious individuals and ignore those who appear to be legitimate shoppers.

To meet privacy regulations, security cameras mounted in public areas use masking algorithms to ensure no identifiable data is captured. Initial detection of characteristics is used to create a numeric token to represent each individual who enters the store. 

From that point on, organizations can use the numeric token to track a person from aisle to aisle and camera to camera, and focus on those whose behavior suggests they are there to shoplift. 

Object recognition can also be gathered and analyzed. For example, the brand of shoes worn by an individual can be used to help understand customer preferences – as well as to help make a positive identification if an individual is involved in theft.

Image augmentation brings clarity and detail

Cameras mounted in public locations use wide angle lenses. While necessary to capture an expansive view, they can make it difficult to distinguish between similar looking products. 

For example, imagine an individual picks up a shampoo and conditioner from the same brand. They read the labels before placing one back on the shelf and slipping the other into their pocket. The resolution of the raw image may not be high enough to determine which bottle was concealed. An image augmentation algorithm can enhance the data, enabling the positive identification required to prove that shoplifting had occurred.

Tracking attention and brand engagement

Beyond theft prevention, on-site cameras combined with data processing can measure which products or promotional messages capture the most attention. This can be used to perform tests, enabling marketers to improve their messages.

In addition to tracking attention, organizations can recognize the brands that customers wear or drive. This can be used to tailor programmatic out-of-home advertising to display relevant brands. 

Turrecha notes that luxury brands, which have exhausted online marketing, are now looking to target specific segments outside the home. While high-end venues such as concert halls are closed, consumers still visit grocery stores and fill their vehicles up with gas. Using computer vision and AI to identify luxury consumers and deliver targeted ads can fill this void. 

Making AI and computer vision affordable

If built from scratch, a developer must also write code that enables multiple custom machine learning models to work together. This includes those that detect and track people and behaviors and others that recognize and track objects or products. This increases cost and development time.

Instead of developing each solution from the ground up, meldCX uses a digital building block approach to create machine learning applications for its clients. This technique cuts time and lowers the cost of producing computer vision AI solutions that work with anonymized annotated images and data.

This method also enables businesses to select multiple pre-built AI models and ensure that they will all blend and work together in an edge device without additional development. This allows organizations to solve a variety of challenges relevant to their needs, while meeting both privacy laws and budget constraints.

To learn how your business can benefit by using AI and anonymized synthetic annotated images, schedule a live demo with one of our product experts.