The new grammar of television (and games): Lessons from an avocado
Designer and academic Dan Hill takes an in-depth look at a game about an avocado, drawing fascinating insight about interactivity in both television and video games.
![](https://eu-images.contentstack.com/v3/assets/blt740a130ae3c5d529/bltba62518415cda0e2/652fe6ddbc479f8697ef691f/default-cubic.png?width=1280&auto=webp&quality=95&format=jpg&disable=upscale)
Part three of a four-part series, in which a new game about an avocado and a young inventor sketches out new ideas for both television and video games.
Attention-seeking cameras
So whatâs going on within Avo? The direct manipulation mentioned earlier, and the sense that this lends of âtouching TV itselfâ, is in part enabled by a game engine (presumably Unity) and its ability to drop interactions on top of 3D environments mapped onto pre-filmed sequences. In other words, you control the Avo in Avo by moving a real-time animation over a set of filmed physical backdrops and scenes, of Billieâs studio, or of other locations. By apparently moving Avo freely in that world, you lean-in to the point of being-in. Avo exists on a separate layer super-imposed onto the set, yet thanks to real-time lighting effects and 3D scanning the bumpy green protagonist feels situated within, rather than upon. And so do you. This apparent freedom, albeit necessarily limited, moves Avo beyond Bandersnatch, with the latter really only a stack of pre-filmed elements piled up in an somewhat editable sequence. One ends up feeling rather disengaged, particularly when the experience is overlaid with the cynicism drenching the narrative, and thus it is essentially lean-out.
In this balance of lean-in, lean-out, it is a quite different experience to traditional console games, like Call of Duty or NBA 2K, say, which stress the lean-in of engagement with sometimes overly long lean-back moments in the extravagant cut-scenes: exposition and spectacle in Call of Duty; half-time commentary in NBA 2K. (Iâd love to see the analytics on how often such cut scenes are skipped.) With Avo, this is a more rat-a-tat relationship, with shorter cut scenes dropped almost seamlessly in and out of lean-in gameplay elements, and thus a more engaged rhythm of lean-in, lean-out.
This dynamic capability is underpinned by a fascinating approach to camerawork. In gameplay, the backdrops are filmed sequences, without Avo but with the young inventor Billie and other active elements. These are necessarily shot beforehand. Yet due to Avoâs real-time freedom of movementâthe thing is under your finger, after allâwhich could involve numerous camera angles, Playdeo have created software that essentially picks the best camera to present in real-time.
If Avo is walking along a long planar space, like a propped-up plank of wood between two tables, this might be best viewed side-onâa sort of sawn-off epic view that David Lean might tend towards, were he given a shed and an avocado rather than a desert and Peter OâToole. Yet as you reach a decision-point, you, the player, need to see whatâs ahead, so the optimum camera position there might switch to overhead, at an angle, to enable a glimpse of whatâs around the corner. Having drawn a dotted line in that direction, for Avo to follow, the best camera might switch to one behind Avo, watching the fruit totter off. (The sound design, cleverly, does the same trick.)
Because Avo has CGI-level freedom of movement, any of these events could happen nextâor you could turn around and head back up the plank to some beans you thought might be in another directionâhence a more distant view from behind, along the length of the plank, entirely perpendicular to the original David Lean shot. And all that could happen in a few seconds. Or not.
Playdeoâs software understands each of these cameras, their various viewpoints, their efficacy in terms of game elements (beans to collect, or objects important to the plot), or their sheer aesthetic qualitiesâdoes it look good, essentially. At any one time, each of the cameras is assigned a value based on a rapid assessment of these facets, trading off function against aesthetics. And the winning camera gets the edit. Again, in real-time, and in response to the subtle nudges your fingers are making.
The software suspends a kind of net around you and Avo, a space of possibility defined by the accumulated views from these attention-seeking cameras, and allocates camera time based on the best possible decision at that point.
Playdeoâs Ruby Bell:
âAs a player, youâre dynamically editing between cameras. Weâve had to really consider the language of film, and the language of editing.â
The traditional movement of the virtual cameras in games has complete fluidity, as it is moving around in a space generated by CGI. Yet that freedom of movement, and potential edit, usually conforms to a series of typical positions, to simplify things: third person shooter; first person shooter; panning panorama, as in sports games NHL or NBA, and so on. But Avoâs approach, based on hybridizing a small fruit-shaped CGI element into 3D mapped CGI spaces, overlaid onto pre-shot backdrop sequences or views, is somehow more filmic, more TV.
As such, it sits between interactive and non-interactive entertainment, and creates a new space; integrated, and not opposed, as is usually assumed (as say with in this piece at Gamasutra) or executed, flitting between interactive spaces and cut-scene sequences. Avo tries to do both, simultaneously. It asks questions about a real-time cinematography, and the notion of an algorithmic director/editor making decisions in real-time. You can find numerous research papers studying the idea of interactive cinematography, yet not many examples of genuine combinations of interactive and cinematic cameras as with Avo. Aalborg Universityâs Paolo Burelli writes, in âGame Cinematography: From Camera Control to Player Emotionsâ