The recent meteoric rise of artificial intelligence (AI) models is revolutionising how we interact with computers and access information online. But has AI made a significant difference in ratings and reviews, and is it what shoppers want?
Amazon has rolled out generative AI to create succinct product-level summaries of reviews for many of its products to support shoppers' decision-making process. Several clients have asked us to explore how this has impacted a shopper’s experience when using Amazon and understand how these features are perceived compared to ‘traditional’ ratings and reviews.
To do this, we created a study to research how regular Amazon shoppers use the site. The study simulates the shopping process for a selection of grocery category products.
We received over 2500 responses, and our key findings challenge the notion that AI automatically benefits users.
The 4 key takeaways are:
- Individual Ratings and reviews are as important as ever
- Shoppers don't hang around when shopping for groceries on Amazon
- Amazon's AI features are hardly used or even noticed
- There is little evidence of AI improving the shopping experience, but the generations split opinion
The Research set-up
To do the research, we created a new study to research our Amazon shoppers' site use by simulating the shopping process with a series of selected products. Members chose the product category they were most interested in and viewed the product information page (PIP) on Amazon as if they wanted to purchase.
Shoppers were asked to choose one of the chocolate, dog treats, hand cream, toilet roll or whisky categories they were interested in shopping from and then taken to the Amazon website to "shop as normal." When they were done, they returned to the survey to answer questions about their experience.
Below are some highlighted areas that shoppers were questioned about (not mentioned in advance).
Amazon Product Information Page (PIP) Content and Layout
For those less familiar with Amazon PIPs, this one for Amazon Fire TV Stick is over 15 screens long and covers a vast array of product information, including descriptions, ratings, technical information, marketing data, alternatives and customer product reviews.
Our study clearly shows that the information at the top and bottom are the most used for groceries.
1. Individual Ratings and reviews are as important as ever
When rating the utility of all the content available on a PIP on Amazon, we found that shoppers paid the most attention to the top-of-page product information - the name, brief description, and imagery of the product, as you may expect. From here, shoppers bypassed the middle section of the page, such as manufacturer content and Amazon’s recommendations, and moved straight to rating and reviews as the next most significant feature. 
Within the reviews information, it is evident from this chart that shoppers prioritised the ratings and review text rather than finding much benefit from the AI-generated ‘Customers say’ summary of the review content. With so many respondents being experienced in online shopping, shoppers know the information they trust to make product decisions. This is supported by our knowledge that not all reviews are equal, based on our influence score of how influential shoppers find a review to be.
We found that the longer shoppers looked at the page, the higher scores they gave for all of these features. This suggests that the summaries can be useful in addition to the rest of the review information, informing customers of the product's key features. However, it may be rare for products to be afforded this time on an e-commerce platform as populated as Amazon when looking for groceries.
2. Shoppers don't hang around when shopping for groceries on Amazon
87% of respondents said they viewed the page for 2 minutes or less, with 42% claiming they took less than a minute to decide on the product. This highlights shopper’s impatience when they have easy access to the large range of options that Amazon provides and the small window of time your product has to make an impression.
We found a similar pattern when asking respondents about the average rating or number of reviews for their product, with just 74% and 76% being able to recall these figures correctly. We did notice differences here based on which category was chosen, with whisky and hand cream shoppers recalling information the best and dog-treat shoppers the worst.
Rather than recalling specifics, shoppers seem to be looking for a minimum threshold. We know from previous studies that if a product's rating is above 4.0 Stars, and there are more than 30 reviews (thousands in this case), shoppers do not use the exact score to decide between products. This means that on Amazon, with similar average ratings and often so many reviews making differentiation between products difficult, the contents of the top 3 reviews can be pivotal in influencing shoppers' decision-making.
This perhaps sheds light on Amazon’s introduction of AI-generated review summaries and product attributes, which attempt to save time and simplify the shopping experience. But have they had that impact?
3. Amazon's AI features are hardly used or even noticed
When shoppers spend such little time viewing a product yet still value ratings and reviews, an AI summary of reviews is the ideal solution. However, we found that just 22% of respondents used these two features. These numbers increased to 30% of those who could correctly recall the number of ratings or reviews, suggesting that those who studied the page more closely found use in the summaries but potentially missing the purpose Amazon had intended.
And yet, 30% of respondents said they had noticed these features previously, meaning some were aware of them and chose not to use them. As seen below, this awareness increased the more frequently shoppers ordered from Amazon, but even with those making weekly orders, just 41% of respondents had noticed the AI features before.
Whilst this may be partially due to poor visibility resulting from the small amounts of space taken up by the summaries on pages so dense with information, are shoppers looking for these AI features to be more prevalent?
4. Little evidence of AI improving the shopping experience, but opinion is split by generations
When asking respondents how they felt these features had impacted their shopping experience, 45% gave a damning ‘not at all’. Compared to a measly 6% for whom the AI-generated summaries ‘significantly’ improved the experience, this would suggest that shoppers are not convinced of the usefulness of AI when using Amazon, particularly when it comes to the user-generated content of ratings and reviews.
However, attitudes differed significantly based on age when assessing respondents' interest in including more AI-driven features when shopping online in the future, particularly from supermarkets.
Whilst just 20% of those over 54 answered ‘probably’ or ‘definitely’ to wanting more of this in the future, this more than doubled to 44% for those aged 34 and under.
It is obvious that AI will have a role to play in ratings and reviews, and the demand is growing, but displacing authentic consumer opinions is proving a difficult change for shoppers with online retailers such as Amazon.
Conclusions
The research shows ratings and reviews are important when shopping from e-commerce platforms like Amazon. They act as a focal point that is trusted and useful for shoppers to make decisions.
A vast amount of choice, coupled with easy access, means shoppers can move between products on Amazon and other retail sites faster than ever, so they will only pay attention to the most valuable information. Therefore, maximising this time through clear product information at the top of the page and a strong rating and Top 3 reviews at the bottom is essential in helping your product stand out.
Whilst AI-generated summaries should supplement this, it is clear that they are not a substitute for genuine customer reviews (they are just a summary, after all) and that, in Amazon’s case, they are struggling to have a significant impact. To improve public perception, these features must be more visible, detailed, and aligned with areas in which customers want improvement.
However, younger generations are significantly more open to using AI as part of their shopping experience, reflected by supermarkets looking to incorporate AI on their sites to improve the shopping experience, which is likely to increase. The continuously improving capabilities of AI may be more powerful when used alongside user-generated content rather than being used for the same purpose, which is to provide streamlined insights that improve the e-commerce experience for shoppers.
If you want to know more about Amazon shoppers or how we can help with ratings and reviews in any retailer, then Contact us.
fans 15,785
followers 4,312