An Examination of the Current Inadequacy of Video Game Reviews

For seemingly the entire history of the industry, reviews have been such a prominent aspect of the video game community. However, over time, video game review scores have shown incredible amounts of inflation, which has subsequently limited the effectiveness of video game reviews as a whole. Additionally, game review scores have shown increased influence over the video game industry for both gamers and developers themselves, specifically through the proliferation of Metacritic scores. Overall, the video game review system at large needs an overhaul for it to be truly effective. Specifically, once we are able to improve the inflated scoring system as well as the style of critique, we will be able to make video game reviews much more worthwhile for both consumers and developers alike.

Especially when compared to other forms of artistic media, video games have shown a consistent trend towards inflated review scores. Overall, there are a few general ideas as to why video games reviews are overly positive. One common theory is that studios and publishers often pressure reviewers to rate their games highly. Particularly, “small, up-and-coming website[s]” do not want to alienate game publishers in hopes of gaining “more exclusive stories, more swag, plugs, and a plethora of perks” (Burch, 2007). Although this theory might make sense for large, blockbuster-type games, it is not necessarily an idea that works for all video games. In reality, this type of pseudo-reward system would only apply to larger publishing studios who have the money and capability to make inflating reviews worthwhile. Thus, we need a hypothesis that can explain this trend over the entire gaming spectrum, from small indie games to multi-million dollar franchises.

On the other hand, a universal— and more plausible—explanation for review score inflation is that the game reviewing sector has unintentionally adopted a grading scale similar to the education system. In both the “worlds of school and video game reviewing, a 70% is average, an 80% is passing, and a 50% is failing” (Burch, 2007). Despite there being no apparent reason for the adoption of this scale, it is still heavily pervasive in the game reviewing community.  Ultimately, educational-style grading proves to be rather limiting for artistic media like video games because by using this comparison, anything that receives a score lower than a 60% is equivalent to a failing grade. Because there is nothing to truly differentiate between a “2 out of 10” and a “5 out of 10” score—as they are both considered failing marks—video game reviews have essentially limited themselves to a critical scale ranging from 7 to 10. This limited scale is especially apparent in the average reviews scores of IGN and Gamespot. As of 2006, “IGN’s average review score was a whopping 8.0,” while Gamespot had an average review score of 7.0” (Snow, 2006). One major consequence of this inflated review scale is that such a scale only promotes mediocrity. If games do not receive much punishment for their mistakes, then there is less of an incentive to try to improve them. Unless a game is considered bad in most aspects, it can receive a score above a 7 fairly easily, while games receiving lower scores are deemed failures.  However, video games should not follow a grading system subliminally based on passing or failing, for they are simply too complex to critique in such a way. Video games span across several different genres and often have completely different goals (typically constrained by budget and time). For instance, a game can be particularly ambitious in trying to be innovative, but it may ultimately receive poor reviews for poor execution.

One good example of this innovation dynamic is the 2010 American football game, Backbreaker. Unlike traditional football games like Madden, Backbreaker revolves around a realistic physics engine that determines player animations dynamically (as opposed to stringing together pre-determined animations together). Reviewers heavily criticized the game for a lack of single-player depth and laggy online play; however, despite this, the game still had redeeming aspects that prevented it from being a complete failure. For example, Kotaku praised the game’s physics-based engine, exclaiming that “dragging a defender the last 10 yards of a kickoff return touchdown run” was an “all time football gaming highlight, without a doubt” (Good, 2010). Overall, the game would end up receiving an average score of “54 out of 100” on Metacritic ("Backbreaker for Xbox 360 Reviews"). Unfortunately, most gamers would not likely buy a game (like Backbreaker) if it received a score lower than a “7,” even if it was somewhat decent. Because the lower portion of the review scale (i.e. the portion ranging from 1-6) is lumped together into one category, there is little to differentiate between games that are complete garbage with ones that have some redeeming qualities. On the opposite end of the spectrum, because “good” games are lumped together on the upper end of the scale, it is also difficult to distinguish between decent games that do not make many mistakes with those that are true masterpieces. In the end, this prevents video game reviews from doing their job in the first place—helping consumers deciding which games they should buy and which games they should not. Although, the inflation of review scores themselves is certainly a problem, the actual text of video game reviews could use an overhaul as well.

Another major problem that game reviewers often have is that they usually only provide a summary report of a game’s several parts, rather than actually providing a stimulating critique. To be more specific, reviews often contain meaningless diction such as “good, bad, awesome, lame, or cool.” The problem with such vague language is that it does not get at the heart of why something might be terrible or fantastic” in the first place (Portnow, 2012). As both game reviewers and gamers, we need to be able to use a critical language that can compare the many aspects that relate between various games. Once we develop this critical language, then we might be able to specifically determine the roots of what truly makes a good game, as a game is not simply good just because it is “cool.”

In some cases, this overly simplistic style of review can lead to rather comical results, representing how unproductive video game reviews often are. A more specific example of arbitrary game reviewing is in IGN’s review of Pokémon Omega Ruby and Alpha Sapphire. While the game was commended for several updates, including its gorgeous 3D graphics, IGN would ultimately give the game a 7.8 out of 10 rating. Notoriously, one of the listed criticisms of was that there was “too much water.” Because the geographical setting of the game (which is a remake) was “a fictional region consisting of many islands and waterways, the review’s commentary on there being ‘too much water’ instantly become a point of contention among IGN members and fans of the game at large” ("Too Much Water", 2014). In this case, the idea of the game having “too much water,” although comical, takes the game out of context and does not provide any sort of critique on the experience of the game as a whole. By simply saying there is too much water to traverse and that there are too many water Pokémon to fight, the reviewer is only giving mere summary points. Instead, the review would be more beneficial by explaining possible balancing issues or easy exploitations found in the game; however, the context of this statement ends sounding like a complaint that there are too many guns in a first-person shooter. It may be true, but it is not necessarily a point that should reduce the game’s score. If it truly is a negative point, then the reviewer should at least explain the reason why it is so. To a certain extent, this explains the somewhat arbitrary nature of video game reviews. They are highly subjective, and it is especially difficult to define what separates individual points or decimals on a large rating scale. In reality, video game reviews should only give a general idea to overall opinion on the game—they should guide the consumer in a general direction (and potentially warn them about something like an overhyped game that fell short) but ultimately let them make the decision. Regrettably, this is not always the case.

Despite its mechanical approach to aggregated review score averages, Metacritic scores often holds an unnecessary amount of influence within the video game industry. For those who do not know, “Metacritic is an aggregation website that rounds up review scores for all sorts of media, including video games […] averages them out using a mysterious weighting formula” to spew out a number called a “Metascore” (Schreier 2012). Unfortunately, Metacritic scores are too often a major influence in determining a game developer’s eventual payout, specifically when it comes to bonuses. One high-profile example of Metacritic’s influence relates to the 2010 release, Fallout: New Vegas, developed by Obsidian Entertainment and published by Bethesda Softworks. Supposedly, according to Obsidian’s contract, the entire company would receive a $1 million bonus (which worked out to be about $14,000 per employee) if the game achieved a certain Metacritic score. Unfortunately, “Obsidian missed out on being paid […] by Bethesda for Fallout: New Vegas because it failed to hit a Metacritic average of 85. (Tragically, it currently sits at 84.)” (MacDonald, 2012). This concept of Metacritic affecting the outcome of video game studios is occurring at a surprising rate when it shouldn’t be happening at all. Because Metacritic can indirectly determine how much money a studio makes, Metascores now affect the video games industry in two aspects. Firstly, because developers’ salaries are linked with the review scores of a game, they are more likely to be pressured to make a game that will earn high scores instead of making a “good” game, and these two are not always the same. For example, developers might try to cram in more gameplay hours into a game to increase its supposed value. They might try to add other modes, such as multiplayer modes, even if they are unnecessary in order to make the game seem broader in scope. In general, by trying to artificially increase a games score, whether intentionally or not, will ultimately spread a game too thin. Secondly, Metacritic’s influence might also encourage reviewers themselves to give higher reviews out of pity. Although it is understandable why reviewers would do this on a human level, it is still bad reviewing practice and further contributes to the trend of score inflation. In the end, this potentially creates a never-ending cycle in which reviewers’ expectations decrease while publishers’ expectations increase—both of which further push the average score of video games upwards.

Game review scores themselves should only serve as a general guide for community opinion, but it must not represent a definite marker for success. Because video games are inherently based on personal experience and personal interaction, video game reviews will naturally have a subjective slant to them. If games were objectively good or bad, then we would not see such drastically varying scores on sites like Metacritic. As such, it is impractical—possibly even immoral—to determine a video game developer’s pay on what Metascore they receive. Besides, it is impossible to determine what the practical difference between an “84” and an “85” is anyways. Of course, this is not a fault of Metacritic itself, and there is little Metacritic can do to fix this problem. In the end, publishers need to realize that better games will not come out of trying to force good review scores. Instead, good games should come as a natural process of development.

Now that we have discussed the current problems associated with video game reviews, we need to strive to find possible solutions. As we just recently discussed, the problems associated with Metacritic’s influence are, in principle, easy to fix. This fix just has to come from the side of publishers. I hope that over time, publishers will treat their developers with greater fairness and respect. Although gamers and reviewers can certainly voice their opinions on the problems with Metascores, there is little they can do to cause change directly. This brings to my other possible solutions for the video game review sector. Firstly, we need to change the way we look at review scores. Although the utopian idea to get rid of review scores entirely might seem nice, people ultimately love putting numbers on things. In the end, numerical scores themselves will likely always be a part of the industry. Assuming this notion, we need to look at alternatives for scoring games. One common proposal is for the games industry to adopt the “five star system” that has worked consistently for TV, movies, games. Additionally, with this system, one star rating does not always equate to the worst game possible, and a five star rating does not equate to the best game possible. In this situation, a five star rating system would help to allow for subjectivity and general scores (as opposed to undefined decimal points) that would work well in the context of games. Another possibility, although likely supplementary to numerical review systems, would be to categorize games into a “buy/rent/don’t bother” system, which would immediately tell consumers whether or not a game would be worthwhile to spend a large sum of cash on—which is especially crucial when games are upwards of $60 (Portnow, 2012). Of course, reviews in general should see general deflation in order for them to become more useful. This may be a long and gradual process, but if enough reviewers start to give more realistic review scores, then perhaps progress will ensue eventually.

Lastly, both gamers and reviewers need to change the way reviews are written, read, and discussed. If game reviewers begin to adopt a more critical language, then reviews themselves will be able to target what specifically makes a game good or bad. If reviewers begin “drilling down into why we hold the opinions [they] hold and find ways to express that, rather than just delivering a review full of opaque personal feelings,” then reviews will become much more useful for the consumer (Portnow, 2012). Besides, the root purpose of video game reviews should be to act as a guide for the consumer to help them determine what games they might enjoy playing. Additionally, if game reviewers take a more critically deep perspective, then perhaps game developers will be able to learn from their mistakes as well as the mistakes from others. Finally, as gamers, we need to take a different perspective on game reviews ourselves. Instead of just looking at the score and moving on, we also need to participate in the critical discussion by reading the reviews themselves. There seems to be a general tendency for gamers to be upset about a poor review score for the game they love without actually reading the review itself. Most of the time, the text of a review itself will explain much more about the pros and cons of a game than an arbitrary number.

Overall, the reviewing of video games has unintentionally caused interesting controversy within the industry as a whole. Especially when compared with other artistic industries, reviews of video games are heavily inflated. Even though there is no definitive cause to this inflation, it is still greatly apparent throughout the industry as a whole. In addition to improving the overall scale for rating games (either by introducing a 5-star system or reducing inflation of scores) reviewers should also seek to improve the way in which they review games. Instead of just providing mere summary, reviewers should strive to provide knowledgeable critique and further explain why a game deserves the score it does. Ultimately, this controversy will only be reduced when we begin to understand video game reviews for their original intended purpose. Game reviews are subjective by nature, and they should be treated as such. A game loved by one reviewer might be totally detested by another. In the end, game reviews should not be so strict and objective in the gaming world. Instead, reviews should simply act as a map towards finding games that you will truly enjoy playing.                            

References

Backbreaker for Xbox 360 Reviews. (n.d.). Retrieved December 8, 2014, from http://www.metacritic.com/game/xbox-360/backbreaker

Burch, A. (2007, March 14). Why video game reviews suck: Part two. Retrieved November 30, 2014.

Good, O. (2010, June 2). Backbreaker Review: The Challenger Crashes. Retrieved December 8, 2014.

MacDonald, K. (2012, July 16). Is Metacritic Ruining the Games Industry? Retrieved December 8, 2014.

Portnow, J. [Extra Credits]. (2012, May 16). Extra Credits: Game  Reviews [Video File]. Retrived December 8, 2014, from http://youtu.be/bb3HQlFmfds.

Schreier, J. (2013, April 11). Metacritic Matters: How Review Scores Hurt Video Games. Retrieved December 5, 2014.

Snow, B. (2006, August 7). IGN, Gamespot review score inflation revealed. Retrieved November 30, 2014.

Too Much Water. (2014, November 20). Retrieved December 8, 2014, from http://knowyourmeme.com/memes/too-much-water