In-Store Traffic Analytics: Retail Sensing with Intelligent Object Detection

1. What is Store Traffic Analytics?

In-store traffic analytics allows data-driven retailers to collect meaningful insights about customer’s behavioral data.

The retail industry receives millions of visitors every year. Along with fulfilling the primary objective of a store, it is can also extract valuable insights from this constant stream of traffic.

The footfall data, or the count of people in a store, creates an alternate source of value for retailers. One can collect traffic data and analyze key metrics to understand what drives the sales of their product, customer behavior, preferences, and related information.

2. How does it help store potential?

Customer Purchase Experience

Store Traffic Analytics helps provide insights and in-depth knowledge of customer shopping and purchasing habits, their in-store journey, etc., by capturing key data points such as the footfall at different periods, the preferred product categories identifying traffic intensity across departments, among others. Retailers can leverage such analytics to strategize and target their customers such that it enhances customer experience and drive sales.

Customer Dwell Time Analysis

Dwell time is the length of time a person spends looking at the display or remains in a specific area. It grants an understanding of what in a store holds customer attention and helps in optimizing store layouts and product placements for higher sales.

Demographics Analysis

Demographic analysis separates store visitors into categories based on their age and gender, aiding in optimizing product listing. For instance, a footwear store footfall analysis shows that the prevalent customers are young men between the age group of 18-25. The information helps the store manager list products that appeal to this demographic group, ensuring better conversion rates.

Human Resource Scheduling

With the help of store traffic data, workforce productivity can also be enhanced by effective management of staff schedules according to peak shopping times to meet demands and provide a better customer experience, directly impacting operational costs.

3. Customer Footfall Data

The first step for Store Traffic Analytics is to have a mechanism to capture customer footfall data. Methods to count people entering the store (People Counting) have been evolving rapidly. Some of them are as follows –

• Manual tracking
• Mechanical counters
• Pressure mats
• Infrared beams
• Thermal counters
• Wi-Fi counting
• Video counters.

This article will take a closer look into the components of an AI-based object detection and tracking framework for Video counters using Python, Deep Learning and OpenCV, by leveraging CCTV footage of a store.

4. People Counting (Video Counters)

Following are the key components involved in building a framework for people counting in CCTV footage:

1. Object Detection – Detecting objects (persons) in each frame or after a set of a fixed number of frames in the video.
2. Object Tracking – Assigning unique IDs to the persons detected and tracking their movement in the video stream.
3. Identifying the entry/exit area – Based on the angle of the CCTV footage, identifying the entry/exit area tracks the people entering and exiting a store.

Since object detection algorithms are computationally expensive, we can use a hybrid approach where objects are detected once every N frames (and not in each frame). And when not in the detecting phase, the objects are tracked as they move around the video frames. Tracking continues until the Nth frame, and then the object detector is re-run. We then repeat the entire process. The benefit of such an approach helps to apply highly accurate object detection algorithms without much computational burden.

4.1 Object Detection

Object detection is a computer vision technique that allows us to determine where the object is in an image/frame. Some object detection algorithms include Faster R-CNN, Single Shot Detectors (SSD), You Only Look Once (YOLO), etc.

Illustration of YOLO Object Detector Pipeline (Source)

YOLO being significantly faster and accurate, can be used for video/real-time object detection. YOLOv3 model is pre-trained on the COCO dataset to classify 80 different classes, including people, cars, etc. Using the machine learning concept of transfer learning (where knowledge gained from solving a problem helps solve similar problems), for people counting, the pre-trained model weights, developed by the darknet team, can be leveraged to detect persons in the frames of the video stream.

Following are the steps involved for object detection in each frame using the YOLO model and OpenCV –

1. Load the pre-trained YOLOv3 model using OpenCV’s DNN function-

2. Determine the output layer names/classes from YOLO model and construct blob from the frame –

3. For each object detected in ‘layerOutputs,’ filter objects labeled ‘Person’ to identify all the persons present in the video frame.

4.2 Object Tracking

The persons detected using object detection algorithms are tracked with the help of an object tracking algorithm that accepts the input coordinates (x,y) of where the person is in the frame and then assigns a unique ID to that particular person. The tracked person moves around a video stream (in different frames) by predicting the new object location in the next frame based on various factors of the frame such as gradient, speed, etc. Few object tracking algorithms are Centroid Tracking, Kalman Filter tracking, etc.

Since the position of a person in the next frame is determined to a great extent by their velocity and position in the current frame, the Kalman Filter tracking algorithm tracks old or new persons detected. Kalman Filter allows model tracking based on velocity and position, predicting likely possible positions. It does so by using Gaussians. When it receives a new reading, it uses probability to assign measurements to its prediction and update itself. Accordingly, the object assigns to existing or unique IDs. This blog explains the maths behind Kalman Filter.

4.3 Identifying the entry/exit area

To keep track of people entering/exiting a particular area of the store, based on the CCTV angle, the entry/exit area in the video stream is specified to accurately collect data of the customer journey and traffic in the store.

In the image below (the checkout counter), the yellow boundary specifies the entry/exit area, and the status of store traffic is updated accordingly.

A reference frame from the video (Source of the original video)

5. Key Challenges –

Some key challenges observed during object detection and tracking framework for footfall data capturing are listed below –

• Speed for real-time detection – The object detection and prediction time needs to be incredibly fast to accurately capture the traffic in the store with frequently visiting customers.
• The angle of CCTV Cameras – The camera angle should accurately capture the footfall traffic.
• Video Clarity – For object detection and tracking algorithms to accurately capture the people in a video, the quality of the video plays an important role. It should not be too blurry, have proper lighting, etc.

6. Conclusion

The need for Store Traffic Analytics has become apparent with the growing complexity of the industry. Retail businesses face fierce competition that pressures them to guarantee that the right products and services are available to their customers.

Collecting and analyzing data that accurately reveals customer behavior has therefore become a crucial part of the process.

7. References

• https://www.pyimagesearch.com/2018/11/12/yolo-object-detection-with-opencv/
• https://www.pyimagesearch.com/2018/08/13/opencv-people-counter/
• https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/
• https://arxiv.org/pdf/1506.02640.pdf
• https://towardsdatascience.com/r-cnn-fast-r-cnn-faster-r-cnn-yolo-object-detection-algorithms-36d53571365e
• https://blog.compliantia.com/2019/12/17/the-evolution-of-foot-traffic-analysis-and-how-to-use-it-in-2020/
• https://www.retailsensing.com/people-counting/retail-store-traffic-analytics/