The PhoCusWright Travel Innovation Summit offers thirty travel technology companies the opportunity to pitch a panel of judges comprised of industry leaders on their customer value proposition, technology platform and business model.
photo credit: Louish PixelBased on the number of punks trying to stuff the ballot box, there's big money to be won or lost betting on the 2011 PhoCusWright Travel Innovation Summit - just like betting on the companies themselves...
Leveraging the wisdom of crowds (attendees of PhoCusWright are supposed to be smart people, right?) and in the interest of having some fun, everyone following the conference on social media were invited to predict the top six winners before the actual presentation sessions started. Think of it like handicapping a horse race.
But before we review the results of the handicapping exercise, a bit of unanticipated editorial commentary on the process is necessary…
[Climbs onto Soap Box...]
At first glance, the raw results showed two clear favorites in the Emerging and Established company categories, with the Startup group looking like a tight four-way competition. Unfortunately, upon closer scrutiny, it seems that certain “fans” of various demonstrators (you know who you are) felt that their faves needed a little boost in the voting.
How could I tell? Well, they only voted for their company once – not for 2nd 3rd or 4th place as the other people did… Yes, individuals from multiple presenters were either lazy or stupid in their approach to astroturfing the vote tallies. The singular votes represented a whopping 51% of the total votes cast – a material number with a highly atypical voting pattern that was clearly differentiated by the other respondents.
Morons. Didn’t they know that a cardinal rule of astroturfing is to blend in with the crowd?
By sheer coincidence, I just authored a three-part series for Tnooz on how black-hat fake review optimizers combine social media and SEO techniques to undermine the reliability of travel reviews sites. Take a look at the second installment: Fake Review Optimization – How black hat masters beat the travel system
So here’s the moral of my sermon.
Considering that the motivation exists to game a goofy poll that has no bearing on the outcome of a judged competition, one can only imagine what steps these reprobates are willing to take when real sales volume and profit are on the line. More evidence travel marketers and their technology partners might not be as trustworthy as one would hope.
There is an excellent case for punishing this behavior by posting the names of these groups on a wall of shame. I won’t – the actions may have been undertaken by a single individual and not condoned by the company. They get a pass this time, but one never knows when I might need a good example of social media abuse for a speaking engagement in the future…
Sparing you the gory details, let’s just say an adjustment was made to mitigate the impact of these highly irregular votes. Does this mean the scrubbing process was perfect? No. Just like with review spam, it is very possible the smarter / less lazy ballot stuffers went undetected {sigh…}
Does this action invalidate the reliability of the survey? In short, no. The survey never claimed a scientific sample or process. The results were always to be based on what I call Trustularity – results that look reasonable and superficially appear to have authority based on their presentation. I merely took measures deemed necessary to clean the data to improve the veracity of the results.
This is exactly why PhoCusWright went to a 100% judging process – In the past, audience votes (which had to be done in person, with only one vote per terminal during a defined time frame) were skewed toward larger companies with more personnel attending the conference.
It was a smart move to eliminate the audience component. Next year, I’ll see what I can do to create a more reliable method to gauge authentic audience sentiment. And maybe add interactive parimutuel betting…
[Descends from Soap Box...] continue reading →
