Conjuring a gestural spellcasting system for VR

What should it feel like to be a wizard?

That’s what I had to figure out when I began prototyping IRONSTRIKE, my VR co-op action roguelite. Adding magic was a frequent request from the players of IRONSTRIKE’s prequel, Ironlights, so I knew from the beginning that I wanted to include some kind of spellcasting system. But it couldn’t just be a matter of selecting spells from a glorified menu. I wanted to go beyond radial menus or one-step gestures and design something that took real advantage of VR.

Most importantly, I wanted something that rewarded skill.

We can trace my inspiration fairly easily. See, I’m a nerd. And I read a LOT of cheap fantasy books as a kid. What made the spellcasters in these books so cool wasn’t (just) that they could throw fireballs around. It’s that they were skillful experts of the arcane arts, manipulating complex energies by practicing intricate gestures and memorizing mystical incantations. (They were nerds too!)

So, I decided that I wanted a magic system that:

I started by trying two very different approaches. Spoiler: both were doomed.

Failed Approach #1: Tracing Runes

I started with a simple idea: tracing a line through the air.

(I failed to save my old prototypes, so here’s a mockup.)

This has some nice features. You can make the lines twist into all sorts of shapes, from very simple (e.g. drawing an X) to extremely complex (intricate runes). You could make the lines into cool 3D shapes. You could have two lines, one for each hand!

And even better: what if we only showed the player the *next portion* of the line they’re supposed to trace, rather than showing the entire complex path? That way, expert players who’ve memorized the entire spell can rush through it, while newbies can keep tracing it bit by bit until they learn it by heart!

Frankly, I loved this idea. I had a vision of players waving their hands through intricate convoluted gestures, but doing it so fast you can barely see the structure, amassing magical energy and then blasting it toward their foes.

Cool! But it fell apart during playtesting.

The problem came from combining lightning-fast hand motions with precise line dimensions. Whenever we tried to cast a spell quickly, we’d repeatedly end up “missing” the line by a fraction of an inch, requiring us to go back and retrace the missed section, only to miss the line again a second later. This felt incredibly frustrating, and the frustration didn’t seem to get better with experience. Sure, at some level this was probably a “skill issue”, but it was still a problem if it felt this bad in practice.

At first I thought we could fix it by subtly tweaking the implementation. We tried a LOT of variations. For instance, what if we removed any sharp corners from the spell line? Or dynamically reduced the precision required during tracing? Or tweaked the tracing-detection algorithm in a dozen different ways?

A few times, we were successful at making fast spellcasting viable! But this involved making the system so lenient that the best strategy was to flail your hands around wildly. Kind of the opposite of the “elite wizard” vibe we were aiming for.

Failed Approach #2: Direction Sequences

Okay, forget the whole line-tracing complex-shape thing. What if the whole spell were just a series of directional swipe motions? Move your wand up, then down, then diagonal, then U-U-D-D-L-R-L-R and *poof* you’ve got a fireball!

Can’t get simpler than that! And it still had the potential to reward memorization and skillful execution. But it came with a few problems. For one thing, you can cast any spell by drawing rapid tight circles (i.e. hitting every direction) as fast as possible. (This can be mitigated in various ways, but then it starts getting complicated again.) For another, this system doesn’t use space in a meaningful way. It feels more like inputting a code than casting a spell. It doesn’t make me feel like a wizard.

You could make this work, but I wasn’t thrilled by the idea. Thankfully, however, this inspired the mechanic that actually worked:

Synthesis: Line Crossing

It turns out that if you combine the two approaches above, they join to form the perfect mechanic! Sort of!

We needed a mechanic that was more strict than just swiping in a given direction, but more lenient than hitting a certain point on a path. So, why not merge the two ideas: make the player swipe in a given direction near a certain point?

Well, that sounds easy to implement! Define a spell as a series of line segments, and require the player’s motion to intersect each of those line segments in the correct direction. We can define an “intersection” either in 2D (by projecting both the line segments and the player’s hand motion vector onto a 2D plane) or in 3D (by giving the player a long wand / beam extending from their hand and detecting when it hits the line segment).

And... it works!

It’s easy to explain and teach to new players:

It can easily expand to accommodate lots of complex gestures:

And, with practice, you can cast complex spells quickly and smoothly:

I can tweak the lengths of the line segments to control how precise the motions need to be at any point, but mindlessly flailing your hands around won’t cast these spells effectively.

This system encourages players to memorize the spells and to practice the physical skill of casting them quickly. Here’s some footage just recorded by an IRONSTRIKE player to show off their spellcraft in combat:

This spellcasting system has been a great success so far. Casters represent about a quarter of players in any given run of IRONSTRIKE, usually playing the role of support or artillery. And there’s a ton of room for expanding the system with more spells, since it’s easy to author new gestures of any complexity

When I look back at this system, it’s interesting to note how the key elements (requiring a certain directional swipe / at a certain position in front of the player) are similar to those of one of VR’s biggest hits, Beat Saber. That’s not intentional, but that kind of convergent evolution suggests to me that these might be useful ingredients for other game mechanics. I’d be curious to learn about other systems that use a similar approach in the future!

- @E_McNeill