While Steam Spy can offer useful insight into overall release trends and large-scale sale fluctuations, the accuracy of its information and its utility to developers and players alike have both been called into question in the past. But overall, is the third-party statistic tracker beneficial to game developers?
PCGamesN queried several developers to find out, in their own experiences, how accurate Steam Spy has been, how useful or harmful its information can be, and if they think the site should let some developers opt out of public tracking.
The full story is a worthwhile read for any developer curious about the application of Steam Spy's information or looking to compare their own experiences with the website against those of many other individuals within the game development community.
On the note of accuracy, most developers polled said that Steam Spy’s data on how many people own a game is fairly correct, within a margin of error, for many Steam games.
Sergey Galyonkin, who runs Steam Spy, told PCGamesN that 98 percent of the numbers should be correct, at least within the provided margin of error, though information on games without 30,000 sales “should be discounted as completely inaccurate” due to a small sample size.
Many of the comments gathered from developers fell along those lines; people who made games that sold above that line said that Steam Spy’s data was accurate, while developers that sold just a thousand or so copies of their game saw numbers that significantly overestimated their game’s actual sales.
So what about Steam Spy’s actual utility? Again, developers were divided but mostly didn’t seem to have a problem with the information offered by the website. Most spoken too agreed that, while the data can be interesting, knowing how to correctly interpret the meaning of the information is the most important thing.
“Despite the shortcomings of the service, it's still useful as long as you're willing to accept that the numbers you see don't reflect a true picture of sales,” said SkyScrappers developer James Parker. “Over time, I think Steam Spy's algorithms will improve, and I have no problem with the estimations made, but it's what people do with them that's often the bigger problem.”
Be sure to head over to PCGamesN to read the story in its entirety, which is filled with many other direct quotes from developers both on their own experiences and on the utility, accuracy, and benefit of Steam Spy as a whole.