Key Findings From “Do Gatekeepers Develop Worse Products? Evidence from Online Review Platforms”
- 对于平均业务评级,Yelp has a more uniform distribution between one and five stars, whereas Google, Facebook, and HomeAdvisor have skewed rating distributions that disproportionately favor higher star ratings.
- Reviews that contain 501 to 1,000 characters made up about 28% of Yelp reviews, compared to only 6% of Google reviews. In fact, 50% of Google’s reviews contain less than 100 characters — 32% of these so-called “reviews” contain no text at all.
- For suspicious businesses with a large number of consumer complaints and a low letter grade from the Better Business Bureau (BBB), businesses on Google had a significantly higher and favorable average star rating (3.5 stars) compared to Yelp (2.4 stars).
This week, trial began in a case brought by the Justice Department, joined by dozens of states, against Google over its anticompetitive conduct in the online search industry. One of Google’s primary arguments to defend its dominant position is that its features are better for consumers, but when it comes to local business information like consumer reviews this is plainly not the case. Illustrating this, U.S. Federal Trade Commission (FTC) Economist Devesh Raval, recentlyreleased a studythat supportsargumentsagainst Google’s anticompetitive self-preferencing conduct. Raval examined if “gatekeeper platforms,” such as Google and Facebook, “develop worse products” for local business reviews compared to other platforms like Yelp, HomeAdvisor, and the Better Business Bureau (BBB). The study states, “A prominent concern in antitrust is that digital platforms may stifle innovation by steering consumers to their own products.” Google’s behavior harms consumers by prioritizing their own lower-quality features over competitor content with higher quality.
Through Raval’s analyses, he found that Google, Facebook, and HomeAdvisor exhibit signs of rating inflation, even for lower-quality businesses that have received a poor letter grade from the BBB or had numerous consumer complaints filed against them. The study also finds that Yelp’s content policies, and trust and safety practices help mitigate the impact of lower quality or potentially fake reviews.
Average business ratings on Yelp have a more uniform distribution than other platforms
By evaluating a sample of nearly 135,000 businesses across all five platforms, the study found that average business ratings on Yelp are “much more uniform across the rating distribution” — about 32% of businesses are above four stars, 58% are between two and four stars, and 10% are below two stars.
Google, Facebook, and HomeAdvisor have rating distributions that heavily skews toward higher star ratings. The percentage of businesses with at least four star ratings on each platform were 59% on Google, 79% on Facebook, and 96% on HomeAdvisor. Only 4% of businesses on Google and 2% of businesses on Facebook had an average rating below two stars. The study likens this to theLake Wobegon effect, or a tendency to believe that the businesses listed on the respective platforms are all above average.
Low-quality businesses have higher ratings on Google than Yelp
The study categorizes the quality of a business based on the number of complaints filed with various consumer protection related organizations (such as the BBB or the FTC) and theletter gradea business receives from the BBB.
- High-quality tier:businesses with almost no complaints and almost all receive A+ grades from the BBB (27% of businesses in the sample)
- Medium-quality tier:businesses with complaints and grades in between A+ and F from the BBB (63% of businesses)
- Low-quality tier:businesses with a large number of complaints and more likely to receive a F grade from the BBB (10% of businesses)
A low quality business on Google has about the same average rating as a medium quality business on Yelp or a high quality business on the BBB’s platform.
Raval found that both low- and high-quality businesses have higher ratings on Google, Facebook, and HomeAdvisor compared to the BBB and Yelp. The average business ratings on Google, Facebook, and HomeAdvisor only see a slight decrease as business quality falls, whereas Yelp and the BBB see significant decreases. Google and Facebook both fell by 0.9 stars from high-quality to low-quality businesses, and HomeAdvisor fell by 0.5 stars.
此外,确认如果轻微下降average business ratings between high-quality and low-quality businesses may be due to the volume of Google reviews with less than 100 characters, Raval removed these reviews from his sample and still found a similar pattern between high-quality and low-quality businesses (both average ratings decreased). Raval concludes that, “A platform that spends less effort on reducing fake reviews is likely to have inflated reviews for low quality businesses.”
Yelp takes extensive measures to mitigate against misinformation on our platform. Over the years,industry experts,the media, andregulatorshave shown that Yelp is one of the most aggressive and successful at identifying and weeding out unreliable reviews.
Google tends to have many lower quality reviews
One differentiator between Yelp and other review platforms is that Yelp has always required review text to accompany ratings. Google allows users to leave a star rating with no review text, which the study found made up 32% of Google’s reviews and saw an overwhelmingly positive average of 4.3 stars. The study also revealed that 50% of Google reviews were 100 characters or less, compared to only 2% of Yelp reviews. Relatedly, aSOCi research reportfound that Google reviews with text have fallen 28% between January 2015 and July 2022, as bare ratings lacking context made up more of their local business content.
Given the ease of leaving a star rating with no text on Google, bad actors have been able to weaponize Google ratings in anextortion schemetargeting numerous restaurants in major cities in 2022. In a Yelp-commissionedsurvey conducted by Material, 85% of respondents who read reviews say they’re more likely to trust a written review over just a star rating with no review text.*
艾丽萨的研究还发现,28% of Yelp reviews were between 501 and 1,000 characters, compared to only 6% of Google reviews. While Yelp and Google reviews with more than 1,000 characters both had an average rating of 2.5 stars, these longer reviews made up 18% of Yelp’s sample and only 2% of Google’s sample. The study highlights the value of requiring review text, stating, “Requiring a reviewer to write text imposes greater costs on reviewers, which might reduce the quantity of reviews but increase their quality.”
The Material survey also found that77%of respondents said they are reading more online reviews now than they ever have before, and72%said it’s rare for them to visit an unfamiliar business without checking online reviews first.* This is why the quality and reliability of reviews has long been a top priority for Yelp. The trust consumers have in the ratings and reviews on Yelp is why we invest in significant measures to protect the integrity of content on our platform, including ourautomated recommendation software, reporting by Yelp’s community of business owners and users,human moderation, and ourConsumer Alertsprogram.
With average ratings of businesses on Google heavily skewed towards four or five stars, people may unwittingly be driven to businesses with numerous consumer complaints and F ratings from the BBB. As Raval puts it, “Google and Facebook are dominant platforms in search and social media markets; reviews are a small part of their business.” “Gatekeeper platforms” like Google are able toleverage their monopoly distribution to steer consumerslooking for a business on Google Search or Google Maps to their own business reviews, creating less incentive for them to take serious steps in reigning in the growing issue of fake reviews.
Learn more in FTC Economist Devesh Raval’s full study:Do Gatekeepers Develop Worse Products? Evidence from Online Review Platforms
* Material Survey 2022. This survey was fielded by Material among 2,000 people aged 18+ in the United States. The survey was conducted online during the period of August and September 2022 and has a margin of error of +/- 2%. Results of any sample are subject to sampling variation. The magnitude of the variation is measurable and is affected by the number of interviews and the level of the percentages expressing the results.
