Christian Ress is the founder of PlaytestCloud, an online playtesting service for mobile games.

Freshly arrived in Australia and New Zealand, Assassin's Creed: Identity is finally bringing the third-person action-adventure experience of the Assassin's Creed series into a pocket-sized format. In this series of articles I apply the tools of playtesting and user research to soft-launched games as an exercise in game design. My goal is to find out what parts of the games already work well, and in what parts players are getting lost and the game still needs work. Without spoiling too much, it's a close call in this one.

My methodology is to invite a few players to play these games in a usual usability testing setting, with screen recording, a think-aloud protocol, and a survey after the test and then analyze the results. As fellow playtesting advocate Vin St. John points out, selecting the right target audience is crucial when doing playtests. I recruited players aged 20-30 who played at least one previous PC or console version of Assassin's Creed and who are also active gamers on phones or tablets.

Virtual joysticks: Console vs. Mobile

The players first contact with the game is through a tutorial explaining the controls. Assassin's Creed: Identity offers two control schemes: virtual joysticks to control movement and camera angle, and a tap-to-move scheme.

The joysticks operate in different modes depending on how many fingers are touching the screen. When one finger is touching the Movement Area of the screen, a virtual joystick that can be used for movement is displayed. When the player puts down a second finger anywhere on the screen, a second virtual joystick for looking around and steering the character appears. However, when a single finger is placed in a small area on the right of the screen only the camera can be moved. Confused yet? So were the players when they first encountered the controls.

The second navigation approach is a tap-to-move system where the Assassin navigates autonomously to the selected point in the world. This can be triggered with a single tap on the screen when no other fingers are touching the screen. Except for the buttons in the lower right the whole screen can be used for tap-to-move. I often saw players accidentally trigger movements through this mode when tapping objects, NPCs, the map or other non-interactive elements of the UI like the avatars of the quest list.

Both in my playtests and in the App Store reviews the control scheme was heavily criticized. The playtesters showed a lack of confidence in being able to move where they wanted to and feared accidentally climbing up walls or causing auto-navigation to a far away, yet visible, point on the screen. Even during the tutorial, they couldn't reliably find the invisible box in which the camera movement mode was activated. They expected to be able to change the camera angle by dragging anywhere on the right half of the screen.

The main problem the playtesters had with the joystick controls were related to turning quickly, e.g. to turn around to attack a guard, because the camera did not following fast enough in those cases. This caused the players to overshoot their target direction and get frustrated because they couldn't control the Assassin in the way they intended.

One possible reason for the player's confusion is that Assassin's Creed uses an unfamiliar virtual joystick scheme. Current third-person games like N.O.V.A. 3 or Dead Trigger 2 use the left joystick for forward/backward movement and strafing, similar to the WASD controls on the PC. Assassin's Creed instead maps the equivalent of the arrow keys on the left joystick, enabling forward/backward movement and turning. The MMORPG Order & Chaos uses a control scheme similar to Assassin's Creed, with the difference that the camera follows the character closely when turning and mapping the "backwards" motion to walking backwards instead of turning around. While AC: Identity presents a control scheme similar to the the console versions, switching to the more common mobile joystick layout used in N.O.V.A. 3 or trying out the more restrictive joystick controls Order & Chaos uses would be worth testing.

As for the tap-to-move system, players consistently failed to remember that a tap anywhere on the screen would cause their character to move there. They didn't expect it to get activated when they tapped on or close to the quest objectives, mini map, and other buttons. One possibility would be to make this feature an opt-in mode, like in World of Warcraft, or reduce the area of the screen that's receptive for tap-to-move to the center of the screen.

Immersion vs. Menus

AC: Identity departs from the open world approach taken in the previous console and PC versions and bases the entire gameplay around separate and only loosely connected missions. This mission based design has become commonplace in mobile games and gives players the opportunity to experience small, independent parts of a game even when they only have two minutes on their hands.

Upon close inspection of the App Store description it's revealed that the missions are actually procedurally generated based on 16 different mission types:

If done right, this will increase the replayability a lot and is certainly something we don't see in many mobile games. Will the playtesters notice that missions are changing every time?

Let's take a look at the mission selection screen:

Each mission displays the difficulty, indicated by the green and red bars on the right, a name, and the type. When players were presented with this screen after returning to the main menu from their last mission they wondered what mission they just played and what the next mission should be. In their 15-30 minute play session they did not realize that the missions they see in the main menu changed and were confused what they should play next to see new content. To them there didn't seem to be a clear progression path. 

Another point of confusion presented itself in the Mission completed screen players are shown after finishing a mission:

"Retry" and "Return to Animus" (Animus is the main menu) are the only two options, with no way to continue playing. In total it takes at least 9 taps and around 25 seconds to start the next mission. All of our playtesters were confused by the options offered on this screen and were trying to figure out how to play the next mission. Eventually they figured out that they have to return to the main menu, navigate through three mission selection screens, confirm the mission and skill assignments and choose a mercenary to continue playing.

They however hadn't once considered using the Retry functionality during their playtest, because playing the same mission did not sound intriguing to them at all. In my post-playtest discussion with the testers I asked them if they realized that missions are generated randomly by the game. They didn't, and then they wondered if the Retry button would have generated a new mission for them, or if it would repeat the mission they already played.

There are two problems here:

A proper fix for the first issue is not as simple as adding a "Next mission" button. Assassin's Creed: Identity puts value on upgrading the Assassin's skills and changing their equipment, and according to the App Store reviews people are quite into the customization options. An option would be to add buttons to edit the skill tree and change the gear right from the Next mission screen, indicating new skill points and items with a badge. This is a common pattern on mobile devices and appeals to the tendency of people to get rid of badges, which could result in an increase of players exploring the skill tree and equipment customization.

With this in place, tapping the Next mission button could take players to a new mission, possibly of higher difficulty than the current mission, otherwise using the same options the player selected for the current level.

Players will run into the second issue, not knowing which missions they already played, every time they re-open the game after a longer pause. In this instance I would favor a solution that introduces the concept of procedurally generated missions to the players and makes them aware that every time they open the game they'll be presented with fresh and unique missions. This could be further supported by adding a "dice" button that will randomly generate mission, allowing players to generate new missions at will.

App Store Observations

Soft-launched games provide the very interesting opportunity to gather lots of data, including, for better or worse, App Store reviews. We've read through all 422 (as per Oct 20) reviews in the App Stores of Australia and New Zealand and identified a number of common topics.

Ahead of all of them were crashes and lag which are not unexpected for a soft-launched version. However, in the game's App Store description Ubisoft states the game is only compatible with the iPhone 5/iPad 3 or newer -- a message many players didn't seem to have seen because the reviews mentioning crashes and lags were usually by players who tried playing on unsupported devices. Given the importance of App Store ratings it should be considered to show an in-game warning to players of older devices about the lack of compatibility of their device.

In general, players were stoked about an authentic third-person Assassin's Creed version for iOS. In line with our playtest the controls got criticized and players complained about a lack of a broader story connecting the missions, and nobody seemed to have realized that missions are procedurally generated.

Less frequent requests were for being able to play without an internet connection and to play with a female assassin.

What's next?

I hope Ubisoft and Blue Byte can address the many points raised by the players. A game based around procedurally generated missions is a welcome sight, and hopefully one players will learn to embrace. It'd be interesting to learn the reasons behind their decision against a story-based gameplay and hear how their current UI design resulted from the playtesting sessions they conducted. If someone from the Identity team is reading this, maybe they can shed some light!

As for this article, I hope it's been interesting to get a look at independent playtesting results of a highly anticipated game like Assassin's Creed: Identity. Any feedback to improve this series is greatly appreciated.