Researchers have developed a new AI-Model to Classify Video Games Based on Their Covers

Video games have been a popular and most engaging form of entertainment for decades. Their importance in our lives peaked amid the COVID-19 lockdown. But have you ever wondered that the catchy covers of video games and stylized textual descriptions play a huge role in creating an impression on the gamers and potential customers? Besides, they also try to convey important information about the genre of video games. For instance, if we see a video game cover featuring a football, we conclude that the video game is a football based game. Or if the cover has a military built man holding a sniper, the video game is shooting themed RPG game.

Researchers often attempt to categorize video game genres based on its cover and textual description. However, it is quite a challenging task for them because

• There is a wide variety of video game genres, many of which are not concretely defined.
• Video game covers vary in terms of colors, styles, textual information, and more, even for games of the same genre.
• Cover designs and textual descriptions may differ due to many external factors such as country, culture, target reader populations, etc.

A pair of researchers from Western Kentucky University has developed a solution for this conundrum. Yuhang Jiang and Lukun Zheng, in a research paper titled, ‘Deep learning for video game genre classification‘, explain how they created a large training database and used it to train an AI system that classifies games from its cover.

According to arxiv preprint, the approach involves two steps:

(1) A neural network is trained on the classification task for each modality.

(2) Intermediate representations are extracted from each network and combined in a multimodal learning step.

To create the initial database, first, the researchers took 50,000 video game titles with their cover images, description texts, title texts, and genre info, from IGDB.com, a video game database. Collectively, the original dataset originally had 21 genres, which was later churned out into 15 different genres, like adventure, arcade, fighting, and strategy, among others. For simplification purposes, games with more than one genre were labeled under one genre. According to the researchers, developers and other AI-researchers can use this dataset for various other studies like “text recognition from images, automatic topic mining, and so on”. Hence, they plan to make this dataset available for the public soon.

After collecting such a vast dataset, the researchers used it to train two types of deep learning models that can recognize texts and images. These include five image-based models (MobileNet-V1, MobileNet-V2, Inception-V1, Inception-V2, and ResNet-50) and two text-based models (recurrent neural networks (RNN) with Long Short-Term Memory (LSTM)) using deep transfer learning methods utilizing the information from existing pre-trained models. The image-based deep learning approach was made for the game covers, while text-based deep learning approach was developed using textual descriptions.

Upon completion of their training, they discovered that the text-based model worked much better than the image-recognizing model. Afterwards, the researchers tested each model to determine which one worked best. They found that the text-based models fared better than image-based ones. Additionally, they had also developed a third deep learning hybrid model made from information from both modalities (text and image) are then combined using the concatenation method. This hybrid model outperformed the other models.

All the experiments were executed with Python in Google Colab using an Nvidia Tesla K80 GPU. The researchers used Keras (an open-source deep learning library) with the TensorFlow backend.

The researchers’ effort is not complete yet as there are a wide variety of video game genres that are not clearly defined. The researchers suggested that to solve this task on a better level of performance, more efforts are needed on the creation of better data sets and the development of more sophisticated models.

However, they are confident that the new video game-genre categorizing model can be of massive help for gamers to find the perfect game for themselves. Even, video game sellers can leverage it to categorize and better organize their physical stores for customers to find their desired games easily.

You can read the full paper here for more information.