Twitch's new filtering options are fueled by automatic live video indexing

Twitch has implemented a deep learning-based video indexing platform that filters live gameplay into categories based on in-game factors without any additional input from streamers or staff.

Twitch has introduced a number of new filtering options to Overwatch and Hearthstone streams that allow viewers to find games based on factors like hero choice or number of wins.

But what makes those filters particularly interesting is the fact that neither Twitch staff nor streamers will have to lift a finger to file live gameplay into its appropriate category.

Rather, tech picked up in Twitch’s acquisition of ClipMine converts visual elements into metadata on the fly and instantly categorizes each stream. 

In practice, ClipMine’s technology is being used to scan over streaming gameplay, isolate elements like objects, text, and logos, and use that information to automatically categorize and tag livestreams. 

"Video game streams have a very rich structure that has been difficult to exploit for the purpose of enabling content discovery," explains a statement from ClipMine director of engineering Zia Syed. 

"By employing computer vision and machine learning developed at ClipMine, we have been able to recreate that structure in a reliable, scalable and cost effective way. This enables us to match creators and viewers in a very precise manner opening up new ways of content discovery on Twitch."

The result is a deep learning-based video indexing platform that can filter live gameplay into categories based on things like player rank, character choice, and game mode without requiring any additional input from the streamers themselves.

Twitch had previously introduced a similar feature to League of Legends that sorted streams based on player ranking and hero selection, but that data required the streamer to link their Twitch and accounts ahead of time. Twitch's director of integration success JT Gleason notes that the feature was received well, but caused players to want similar discovery tools for other popular Twitch games. 

“Since Overwatch and Hearthstone are also among our most popular competitive games whose players take to Twitch to improve their skills by watching others play, we focused on how we can improve discovery for them,” said Gleason. “Our new metadata filters now make it easy to find more granular aspects of gameplay that previously required a lot more searching.”

Latest Jobs

Double Fine Productions

Hybrid, San Francisco CA, USA
Senior Systems Programmer

Purdue University

West Lafayette, IN, USA
Clinical Assistant Professor in Game Development

Digital Extremes

Lead AI Programmer
More Jobs   


Explore the
Advertise with
Follow us

Game Developer Job Board

Game Developer


Explore the

Game Developer Job Board

Browse open positions across the game industry or recruit new talent for your studio

Advertise with

Game Developer

Engage game professionals and drive sales using an array of Game Developer media solutions to meet your objectives.

Learn More
Follow us


Follow us @gamedevdotcom to stay up-to-date with the latest news & insider information about events & more