Twitch's new filtering options are fueled by automatic live video indexing

Twitch has introduced a number of new filtering options to Overwatch and Hearthstone streams that allow viewers to find games based on factors like hero choice or number of wins.

But what makes those filters particularly interesting is the fact that neither Twitch staff nor streamers will have to lift a finger to file live gameplay into its appropriate category.

Rather, tech picked up in Twitch’s acquisition of ClipMine converts visual elements into metadata on the fly and instantly categorizes each stream. 

In practice, ClipMine’s technology is being used to scan over streaming gameplay, isolate elements like objects, text, and logos, and use that information to automatically categorize and tag livestreams. 

"Video game streams have a very rich structure that has been difficult to exploit for the purpose of enabling content discovery," explains a statement from ClipMine director of engineering Zia Syed. 

"By employing computer vision and machine learning developed at ClipMine, we have been able to recreate that structure in a reliable, scalable and cost effective way. This enables us to match creators and viewers in a very precise manner opening up new ways of content discovery on Twitch."

The result is a deep learning-based video indexing platform that can filter live gameplay into categories based on things like player rank, character choice, and game mode without requiring any additional input from the streamers themselves.

Twitch had previously introduced a similar feature to League of Legends that sorted streams based on player ranking and hero selection, but that data required the streamer to link their Twitch and Battle.net accounts ahead of time. Twitch's director of integration success JT Gleason notes that the feature was received well, but caused players to want similar discovery tools for other popular Twitch games. 

“Since Overwatch and Hearthstone are also among our most popular competitive games whose players take to Twitch to improve their skills by watching others play, we focused on how we can improve discovery for them,” said Gleason. “Our new metadata filters now make it easy to find more granular aspects of gameplay that previously required a lot more searching.”