Verified vs unverified reviews: the battle continues

Thumbs-up reviewOn Travelbug, we like to foster an environment where travellers help keep each other informed. We want our users to have great accommodation experiences, and we ask them to place reviews on places they’ve stayed. We prompt them with an email after their stay and we show an accommodation provider’s rating in search results and on their listing page. I outlined this stuff in my blog post of September last year, and none of this is terribly different to any other site. What is different is that we currently allow reviews by people who have not booked on Travelbug to place a review with us, but should this change?

When you book with Travelbug, then place a review after your stay, your review is badged as a “verified” review. That helps indicate it’s from someone we know has stayed and we reckon it therefore holds more weight. Here’s an example of an Auckland hotel with both types of reviews. We only used verified reviews to count towards our recent 100% guest reviewed awards, for instance.

We still allow those who didn’t book their stay with us to place a review from us (as long as it is based on actual experience with the property) and we do this to make the site a useful place to research and book accommodation.

However lately we have been a little concerned that in allowing these non-bookers to review on our site, we might be actually doing the opposite and skewing reviews towards the negative end of the spectrum. We wondered if those who were placing reviews with us might be visiting Travelbug purely to place a negative review on a place they were particularly unhappy with. So we took a look at the numbers.

The numbers show that, over the last 12 months, unverified reviews are over twice as likely to be thumbs-down reviews. Here are verified reviews first:

And here are unverified reviews:

So while 6% of those who we know have booked with us place a thumbs-down review, 14% of the ‘unknowns’ place a review with a big fat downward thumb.

The question this raises then is this: should we change our rules and only allow those who have made a booking with us place a review, thereby making the reviews automatically more authentic and trustworthy? Or should we keep it open for anyone and everyone to place a review regardless of whether they booked on Travelbug?

To help us decide, we’re running a poll over on our Travelbug Facebook page, so take a look and place your vote. Internal Trade Me polling is pretty strongly skewed in one direction, so I’m keen to see what our readers, followers and likers have to say.

Hop on over, place a vote and help shape the future of Travelbug accommodation reviews! I’ll keep you posted on the results, meanwhile here’s a cartoon from the awesome XKCD to leave you with:


6 thoughts on “Verified vs unverified reviews: the battle continues

  1. When you calculate your award system I feel you should use all those with “Travel Bug Verified” in your calculation.

    Accommodations that use the bookit calendar on their own website can show with a lot more quality reviews on Travel Bug but do not get included in the award calculation.

    It could appear strange that those achieving a high thumbs up review rate do not show as the winner.

    Since Travel Bug and Bookit are owned by Trade Me it would encourage more in the industry to use the bookit calendar on their own sites. Trade me would be the winner.

    Bookit system has much merit as it saves a small operator expensive merchant banking fees involved with credit cards.

    • Hi guys, thanks for your comment. The 2012 Travelbug Awards were calculated using only verified reviews anyway, we felt that was only fair. And reviews from bookings from you own website – if you use BookIt – will appear as “verified” on Travelbug, so there is only good news for you there too. That is certainly another advantage of using BookIt on your own site on top on the cost savings you mention. Cheers – Daniel

  2. Yeah Thanks a bunch!
    Now we have lost all our positive reviews, much appreciated NOT!
    Now when people look at our site on Travelbug they are much more likely NOT to book!
    A Lose Lose situation for both of us!
    Surely it should be up to the individual listing/member whether they allow reviews from guests that havent booked through Book It!?
    You coulda asked us first!!
    We await your response with some interest.

    • Hi Tom, sorry to hear that this change has hurt your listing. We can’t have different rules for different operators and we took the path that made it fairest for all. If you feel that you have had a review removed that should be verified as having come through BookIt/Travelbug please let us know and provide your proof and we’ll happily reinstate it. I see you have a booking arriving soon, they’ll be able to review and have it badged as verified. All new bookings through your own website taken through BookIt will likewise be eligible. We did canvas opinion through our social media channels and across Trade Me, and it was a fairly overwhelming 2-1 in favour of locking down to only those we can verify. When we make these changes we know they will never please everyone, and though the feedback thus far has been mainly positive, we are once again sorry that this has impacted you in a negative way.

  3. All our reviews were placed by clients who had booked through our previous booking agent “Resbook” we then changed too “Bookit” when Resbook changed there billing arrangements.
    You mentioned a booking arriving soon…. we only received this booking because of our positive reviews!!!!
    We were never asked or informed via email or any other method that you were considereing changing the review rules!!
    Surely there is room for unbadged reviews? WE WANT OUR POSITIVE REVIEWS BACK!!! you are punishing our business just because our reviews were placed by clients who had booked through our previous booking agent….As you can guess not a happy client here!!

    • Hi Tom, as I said we knew this wouldn’t be perfect for all operators but on balance felt the fairest. This way there is certainty for all reviews and that will be good in the long term. The reviews were placed on Travelbug and so were ours to keep on or take off, but I understand that will be of little comfort to you. There’s no point in a system where we say that unbadged reviews are bad for one site but OK on another, so we’ve gone for consistency. Give me a call on 0508 BOOKIT if you want to chat.

Post a comment

Your email address will not be published. Required fields are marked *

11 − 5 =