Rococo VR: Dealing with ghosts in User Interface

TLDR:

How the project started

After we finished our first game at Black River Studios, the G11: Invasion, the team had an ideation meeting where many ideas for new games were given. They were discussed and analysed until one of them was chosen to be our next game, which ended up being Finding Monsters Adventure (aka FMA).

One of the firsts BRS' ideation process

During FMA mobile development, a new technology was presented to us: the Gear VR. And we decided to use one of the ideas given on that ideation meeting to make a Technical Art Demo in order to study where we could go in terms of coding, art and performance in this new platform. That idea was Rococo VR, a mystery murder game in a sophisticated and elegant era. The team felt that this experience would be perfect for VR. It was simple, artistic and had a lot of challenges. While core tech team was focused on working on it, the art team was only able to work during their spare time, since FMA was the main priority. To learn more about the tech side of Rococo VR, check this article made by our former Lead Engineer, Rafael Ferrari, here.

Rococo's concept art

The art style is inspired on rococo silhouettes from 18^th century. We chose to use the black and white to make it look more dramatic. The environment feels mysterious, elegant, and the asymmetric ornaments give it a special touch. Also to learn more about the art inspiration, check this article from Thais Weiller, our former producer. The first idea was to make a detective-like game in this style, enabling the player to move around the scene, collecting clues and solving puzzles, but since it was a tech demo, the gameplay was simplified.

Rococo final look

Ghosting

The Black & White art style was beautiful, we were all very glad about it but it also made us learn our first VR lesson as well: Ghosting.

But what is Ghosting? Far from been an ectoplasmic phenomenon, it's rather a slow transition between highly contrasting colors, such as black and white. The pixels of the phone used on Gear VR can't process the color transition as fast as our head moves, so we can see a ghost-like trail if we move our head left and right fast enough.

How ghosting seems like if you were moving your head from left to right and then to down. Obs.: That gif is but a simulation, you can only experience ghosting by using your VR device.

It's also known as Black Smear (you'll find more references in the internet under this name). Pixel values can be in the range of 0 and 255, 0 being completely black and 255 being full white. When a pixel has the value 0 and the game requests it to be 255, due to screen limitations, it can't reach that value in one frame. Instead, it will be a value in between causing the ghosting.

To go from black to white, all screen RGB colors needs to go from 0 to 255, and it takes more time than one frame.

VR requires high frame rates to deliver a good experience. Gear VR devices, for instance, updates the screen 60 times per second. That means it takes about 16 milliseconds to update the screen. Even that time is not enough to avoid ghosting. It takes more than 16ms to turn a pixel from completely black to completely white, that's when ghosting happens.

Displays with low persistence can make that transition faster than the ones with high persistence. Persistence is the amount of time it takes for a pixel to fade out completely. Imagine it's like if you have a torch on fire. If the fire is burning at it's maximum capacity (255) and you need to turn it off (0), it takes a time to do so, it won't happen in one frame. Low persistence in this case, means that the time that it will take for the fire to turn off is short.

The lower the persistence, the better. All currently Gear VR devices have low persistence display. Even with that, it's still slow to our human eyes. On high persistence displays, ghosting is much more noticeable.

Oculus Rift and HTC Vive also have it, but since they use a desktop of high performance, they are able to perform software corrections to decrease ghosting effect, so players will hardly see it. If you want to learn more technically about it, this is a great article for it.

Working around it

In this experience, we had to display to the players the character's line, as if his thoughts had a bubble speech, so the player would read it and choose if that character was suspicious enough to be blamed. To do so, we first tried to display like this:

It didn't work for two reasons: if the players move their head, ghosting would spoil legibility and also, that text could appear over any part of the game's geometry, so its color would make it disappear in some areas. User Interface in this project has presented its challenge. We had to make the texts look legible and avoid ghosting. Here are the things that we did and worked for us:

Reducing contrast

In order to make the transition between black and white smoother and faster, the first thing done was to never use 100% white nor 100% black, we reduced the distance between the colors. The phone used on Gear VR has an OLED screen and although it has high quality and great resolution, the contrast also makes the light color feels like it's bleeding out from the edges of the pixels. So, we used a smaller range of shades:

Container

We decided to make a background for texts. That's quite a simple idea but, that enable us to work out the ghosting inside it, we didn't have to worry about either or not the text would appear on a specific part of the scenery. The text had a light color and it was contained inside a dark background that reminded a "rococó bubble speech". We also added a visual effect to make it look more ethereal. That effect is very soft and as it has a transition, it helps in making the ghosting smoother when player looks around.

Strokes and shadows

Since ghosting happens because of the high contrast between colors, we chose to add strokes (with 50% of opacity of the contrasting color) and shadow to the most of UI assets to create an intermediary color. While wearing Gear VR, that stroke is hardly seen and the asset seems smoother.

This made the assets look smoother.

ALWAYS test with the device

This seems like an obvious tip, but I've met a lot of people that design their layouts, place them in Unity and say "that's probably good". Working in VR forces us to create a build for every change we do. We, UI designers, need to check camera distance, legibility, usability, experience, flow and even, in some cases like this one, the ghosting.

In Conclusion

Gear VR is a great platform and although it doesn't have an algorithmic correction for ghosting yet (like Oculus Rift and probably HTC Vive as well), it's the easier and most practical way to show around a game, like many new developers and studios do. The best thing to draw player's attention is by having a remarkable art. It doesn't need to be triple A, but a good management of the game's color palette will make the player more likely to download it.

That's what happened to Rococo VR. It's only an experience, not a full game yet, but its art, its details and its color palette draws a lot of attention. We took it to several dev events and we received great feedbacks. The best part for me is that most of the people could read and understand the game. I hope this text could help the reader and don't forget that Rococo VR is free and available on Oculus Store (Concepts session).

Special thanks to Phil Lira who helped me to explain in a didactic way what ghosting is :)