Update on the New Peer Review Process

A few weeks back, we annouced changes to the Peer Review process. I am making good on my promise to deliver our findings on what we have seen since we introduced these changes. These findings reflect activity in the month of November:

  • In total, 231,000 reviews were completed.
  • The average time a product was in Peer Review was 6.5 hours, with most items passing in less than 1 hour.
  • The Customer Service Peer Review (CSPR) queue was zeroed out, with new products in this queue being reviewed within 24 hours — most within a few hours.
  • The average review accuracy of guest accounts is 87%, and the average review accuracy of non-guest accounts is 94%, and we expect the guest users to continue to get better as time passes (if not, consistently inaccurate reviewers run the risk of being banned from participation).
  • Most days, we give out no more than 1 million promo credits. This is a small amount compared to other ways users can earn promo credits, and we haven’t seen any major effect on the economy. In fact, credit spending is significantly up, which may be due to items getting through Peer Review faster.
  • Customer Care & Education has already taken action on hundreds of accounts owned by customers submitting UFI content.  We have also removed the ability for well over 100 people to vote in Peer Review due to consistently low accuracy rates.   

Our findings show that contrary to popular concern, guest accounts are actually taking their participation in this process very seriously (likely because they don’t want to be banned from the opportunity to earn promo credits).

We are going to continue to monitor this process, but so far, we find these results promising.

I know that many of you are still concerned that an increased number of UFI and AP items rated GA are getting into the catalog.  While we can’t confirm this to be the case, I can tell you that the resources freed up from reviewing products in the CSPR queue are being directed at flagged products. What’s more, we are continuing to work on the project that will improve the flagging and re-rating process, allowing us to respond to inaccurately rated products more quickly.

I hope this helps to ease some of the continued concern I’ve seen on this topic. I’ll continue to keep you posted on new developments.

Thank you,
Jami
Sr. Community Manager

Share on Facebook0Share on Google+0Tweet about this on TwitterPin on Pinterest0Share on Tumblr0
13 Comments