A new initiative has been launched at the Aruk Mitajiri supermarket in Hofu, Japan. The supermarket has begun analyzing the behaviour of its customers while they shop and using this data to power a generative AI. This AI creates a virtual avatar that offers personalized suggestions for potential purchases.
Equipped with advanced technology from Fujitsu, the system employs video cameras to observe shoppers who engage in various behaviours, such as spending time at displays, comparing products, reaching for items on shelves, picking up products, and interacting with in-store content.
These behavioural insights are then utilized to fine-tune prompts for a generative AI, a customized “Avatar concierge.” This digital assistant generates tailored sales pitches and content based on individual shopping patterns and behaviours.
The hope is that this innovative approach will encourage customers to return to the shelves with a clear sense of what they wish to buy, leading to more confident purchases and a satisfying shopping experience, all while making payments in Yen.
The Japanese technology giant collaborated with a research team led by Naoto Onzo, the director of the Institute of Marketing and Communication at Japan’s Waseda University. This group advised Fujitsu that both “warmth and competence,” indicators of a customer service provider’s personality and knowledge, and the “design and functionality of the product” impact consumer purchasing decisions.
Navigating Supermarkets with AI-Powered Product Identification: Convenience, Privacy, and Accuracy Considerations
Using this insight, they developed a model that can “predict the probability of behaviour transitions for presenting customer service and promotional content.” This prediction is based on individual customers’ behaviour and attribute data, analyzed using human sensing technology.
Fujitsu believes this concept is valuable because modern customers increasingly desire an “engaging and consumer-friendly experience.” However, crafting the content to deliver such an experience requires significant time, money, and human resources. AI can accomplish this task more efficiently and cost-effectively.
For countless individuals who assume the role of the family shopper, navigating the aisles in search of specific products can be daunting. The challenge of identifying products based on vague or elusive descriptions can lead to moments of bewilderment. Fujitsu’s newly developed technology seeks to alleviate this frustration by utilizing advanced algorithms to identify and locate products more efficiently.
The premise is simple yet transformative: as a shopper walks through the supermarket, the technology processes visual and contextual cues to understand their preferences and needs. A man searching for products traditionally associated with women, or vice versa, could find the shopping experience more seamless, with the system intuitively guiding them to the desired items.
However, as this technology promises convenience, it raises important questions about privacy and accuracy. Can the system accurately discern the intent behind a shopper’s product selection? Will it mistakenly assume preferences based on external appearances, potentially leading to awkward or uncomfortable situations?
Striking the Balance Between Enhanced Shopping and Privacy Concerns
The ongoing trial, scheduled until October 15, will provide valuable insights into the system’s capabilities and limitations. The supermarket’s collaboration with Fujitsu aims to fine-tune the technology, ensuring it enhances the shopping experience without compromising individual privacy or perpetuating stereotypes.
Fujitsu has outlined ambitious plans to launch a comprehensive solution based on the trial’s outcomes. The anticipated rollout, projected to take place during Fujitsu’s 2023 financial year, which concludes on March 31, 2024, underscores the company’s commitment to shaping the future of retail.
As the trial period unfolds, consumers, experts, and privacy advocates will closely watch to gauge the technology’s effectiveness and potential pitfalls. Balancing the allure of an enhanced shopping experience with concerns over privacy and accuracy will be pivotal in determining the technology’s long-term success.
