New Peer Review Process

Since introducing the Peer Review (PR) process, we’ve heard the following persistent concerns:

  • Products take much too long to get through Peer Review.
  • Not enough people are reviewing products to keep up with daily product submissions.
  • Review delays lead to unmanageable Customer Support queues.
  • Reviewers routinely pass/fail products incorrectly to increase their Tier scores, to publish unsuitable for IMVU (UFI) content, or to simply disrupt the process.
  • Most Creators are operating within the Virtual Goods Policy (VGP), but are still being penalized by the actions of the few who aren’t.

Not only do we agree that these are legitimate concerns, but we are ready to take action. Rather than simply make small changes to deal with each individual concern, we’ve decided instead to completely overhaul the current process. Below is an overview of what is changing.

Goals
Before describing the changes, we first want to make our goals with this overhaul extremely clear. Our overarching goal is to ensure that products submitted to Peer Review are accurately reviewed and visible in the catalog within 48 hours. In order to do this, we are focusing on three main areas:

1)      We are broadening the eligibility of who can review products.

2)      We are introducing a clearer, more intuitive review process.

3)      We will monitor the accuracy of reviews more closely. Users who consistently vote incorrectly will not be allowed to participate in this process. Additionally, Creators who routinely submit mis-rated/UFI products will have their account(s) suspended or permanently disabled.

Changes to who can participate
To get more users engaged in Peer Review, we are opening the process to all users with the following rewards and restrictions:

  • Only users who are 18 and over, have been on IMVU for at least 30 days, and use 3D Chat (since reviewers are required to view products in 3D in Peer Review) may participate in Peer Review.
  • Reviewers will earn 5 promo credits for each product they review, with a limit of 500 reviews per day.
  • Existing Creators who are 18 and over and have not yet been on IMVU for 30 days will still be able to participate in Peer Review.
  • The first time a user visits Peer Review, they will see a pop-up window with an explanation of what Peer Review is. They will learn the rules for participating, and they’ll be required to familiarize themselves with the Virtual Goods Policy (VGP).
  • Voters who consistently vote incorrectly will be prohibited from participating in Peer Review and earning promo credits.

By opening the review process to non-Creators, we are also developing a new channel for these users to get an exclusive sneak peek at new products. If someone sees something they like while reviewing, they will now be able to add the item to their Wishlist so they can remember to buy it when it passes Peer Review and is available in the catalog.

Changes to the review process
In order to address some of the concerns raised, we’ve made some fundamental changes to the review process itself:

  • Reviewers are now required to view the product in 3D for a minimum period of time, since it is unlikely that a product will be rated accurately if it’s never viewed in 3D from all angles.
  • We are increasing the number of reviews required to pass or fail a product.
  • Products that have not passed Peer Review will not appear in the catalog.
  • We are requiring voters to confirm a failing vote to to ensure the reasons they’ve selected accurately reflect the product being reviewed.

Changes to Customer Support’s involvement in this process
Customer Support will closely monitor individual Peer Review performance and behavior. They will ban users from this process who routinely pass or fail products inaccurately. Additionally, Customer Support will track the product re-rating history of all Creators to look for deliberate attempts to pass UFI content through to the catalog.  Creators who routinely submit mis-rated/UFI products will have their account(s) suspended or permanently disabled.

Changes to when items appear in the Product Catalog
We will no longer display products that are less than one week old in the catalog to customers who have been on IMVU for less than 30 days. Thus, new users will have the best possible experience while minimizing the exposure of mis-rated items.

What’s the impact to the economy?
Based on the number of products submitted to Peer Review and typical number of votes per product, we expect to issue between 750,000 and 1 million promo credits daily (with a maximum of 2,500 per user per day). This represents a very small portion of overall credit economy, so we do not expect a material impact to creators or resellers.  Nevertheless, as with all credit activity, we will be closely monitoring the effects of these changes over the next few months and will adjust the reward if necessary.

We hope you are as excited as we are about this overhaul to Peer Review which should be visible to all users toward the end of this week. As mentioned, we will be actively monitoring the effects of these changes and will continue to make changes as necessary. We encourage you to share your feedback with us by commenting below.

Thank you for all of the feedback and suggestions you have shared with us on this process!

Best,
Jami
Sr. Community Manager

**********************************************

Frequently Asked Questions (FAQs)
Q. Who can participate in Peer Review?

A. Any user who is 18 and over, has been on IMVU for at least 30 days, and uses 3D Chat can participate.  

Q. If users can earn credits for Peer Review, won’t more items get passed randomly?  How will you manage this?
A. Customer Support will actively monitor for users who frequently vote incorrectly, and the CS team will take appropriate action.  We also believe that a larger voting pool will allow for less biased and more accurate voting.

Q. Will Peer Review voting performance be individually reviewed?
A. Yes, Customer Support will actively monitor individual Peer Review performance and behavior.

Q. Will I be banned from participating in Peer Review if pass or fail an item incorrectly, even though I thought my review was accurate?
A. We’re not going to be monitoring for users who incorrectly review items occasionally, or Creators who have items re-rated occasionally. We are looking for serial offenders and flagrant abuse.

Q. How many promo credits can I earn each day?
A. You can earn 5 promo credits for the first 500 reviews you conduct in each a day.

Q. Won’t this flood the economy with promo credits?
A. Any additional promo credits added to the economy will be directly offset by the new channel (adding products in Peer Review to the wishlist, once Peer Review is passed) for users to purchase the newest content our Creator community publishes.

Additionally, a user can only receive a maximum of 2,500 promo credits per day, which means reviewing 500 products (which is A LOT!) The likelihood that these promo credits would be spent on one specific Creator’s products is small. Our data indicates that a large percentage of users who spend promo credits convert to spending real credits, so we believe the end result will be favorable for Creators.

Q. What happens if a product continually gets skipped?
A. The overall process will remain the same. If something stays in Peer Review too long, it will go into the CS queue for review. However, since more people will be reviewing, we expect the number of skipped products will go down.

Q. Will non-Access Pass (AP) holders be able to review AP products?
A. No. Only AP users can review AP-rated products.

Q. What about ‘special order’ products that are intended only for the Creator or specific user who asked for it to be made? Will a reviewer be able to add these products to their Wishlist if they are hidden?
A. No. Hidden products should never be for sale, and they should never be visible in anyone’s Wishlist.

The goal of allowing a reviewer to add a product to their Wishlists is so that they can bookmark the product for later purchase. We believe this is an added benefit for Creators as they gain new customers.

Q. How will product availability and catalog searches change for users who have been on the site at least 30 days?
A. Nothing will change. Users who have been on the site for 30 days or more will experience the same product availability that they experience today. Once an item passes Peer Review, it becomes available in the catalog. Note that this only affects catalog search and browse results. If a new user finds a link to a product that’s new, they’ll still be able to buy it.

Q. Why can’t products developed by Pro Creators with a proven track record require fewer reviews to pass into the catalog?
A. We are exploring the possibility of requiring fewer reviews for higher Tier Creator. However, we want to launch the changes announced here first, and then closely monitor the effects before making any further decisions. We want to minimize the potential negative impact of requiring fewer reviews to pass a product into the catalog would also mean fewer reviews to fail a product.

Share on Facebook0Share on Google+0Tweet about this on TwitterPin on Pinterest0Share on Tumblr0
171 Comments