Sponsored By

Twitch's new filtering options are fueled by automatic live video indexing

Twitch has implemented a deep learning-based video indexing platform that filters live gameplay into categories based on in-game factors without any additional input from streamers or staff.

Alissa McAloon, Publisher

August 18, 2017

2 Min Read

Twitch has introduced a number of new filtering options to Overwatch and Hearthstone streams that allow viewers to find games based on factors like hero choice or number of wins.

But what makes those filters particularly interesting is the fact that neither Twitch staff nor streamers will have to lift a finger to file live gameplay into its appropriate category.

Rather, tech picked up in Twitch’s acquisition of ClipMine converts visual elements into metadata on the fly and instantly categorizes each stream. 

In practice, ClipMine’s technology is being used to scan over streaming gameplay, isolate elements like objects, text, and logos, and use that information to automatically categorize and tag livestreams. 

"Video game streams have a very rich structure that has been difficult to exploit for the purpose of enabling content discovery," explains a statement from ClipMine director of engineering Zia Syed. 

"By employing computer vision and machine learning developed at ClipMine, we have been able to recreate that structure in a reliable, scalable and cost effective way. This enables us to match creators and viewers in a very precise manner opening up new ways of content discovery on Twitch."

The result is a deep learning-based video indexing platform that can filter live gameplay into categories based on things like player rank, character choice, and game mode without requiring any additional input from the streamers themselves.

Twitch had previously introduced a similar feature to League of Legends that sorted streams based on player ranking and hero selection, but that data required the streamer to link their Twitch and Battle.net accounts ahead of time. Twitch's director of integration success JT Gleason notes that the feature was received well, but caused players to want similar discovery tools for other popular Twitch games. 

“Since Overwatch and Hearthstone are also among our most popular competitive games whose players take to Twitch to improve their skills by watching others play, we focused on how we can improve discovery for them,” said Gleason. “Our new metadata filters now make it easy to find more granular aspects of gameplay that previously required a lot more searching.”

About the Author(s)

Alissa McAloon

Publisher, GameDeveloper.com

As the Publisher of Game Developer, Alissa McAloon brings a decade of experience in the video game industry and media. When not working in the world of B2B game journalism, Alissa enjoys spending her time in the worlds of immersive sandbox games or dabbling in the occasional TTRPG.

Daily news, dev blogs, and stories from Game Developer straight to your inbox

You May Also Like