NCSoft's AI Can Transform You Into An Anime Character

Millions of people around the world watch anime, which is a style of Japanese animation made for television shows or films. Each anime has an epic storyline that consists of amazing characters that come together to create a new reality and makes anime special. Every anime is a different world where anything is possible. This makes fans want to step into the shoes of an anime character and slay Titans or talk with Death Gods, but you have to look the part first. This is where NCSoft's new AI algorithm steps in.

Researchers of the video game company, that created Guild Wars among other games, have used machine learning in order to transform you into an anime character. Jun-Ho Kim, Minjae Kim, Hyeonwoo Kang, and Kwanghee Lee created the program using Generative Adversarial Networks (GANs) that are trained using an unsupervised approach. It also makes use of an Adaptive Layer-Instance Normalisation (AdaLIN) function that enables the model to "flexibly control the amount of change in shape and texture by learned parameters depending on datasets".

The anime portraits. (Source: NCSoft)

The anime portraits. (Source: NCSoft)

As you can see, the experimental results are very good not only in terms of how close the character and the original user look like but also how the generated character looks like an authentic anime character. This is thanks to the "novel method" that was used to generate the images. Previously, the models used consisted of fixed network architectures and hyper-parameters. This means that throughout the training process, the model did not change the hyper-parameters' set values, which could result in worse results. This is why the researchers implemented a new, 'learnable' normalisation function. The NCSoft research team also used Class Activation Mapping (CAM) in its model. CAM is where an image is separated into layers and specific features are used in the classification process. For example, in a model used to identify different breeds of dogs, using the fur as a classification feature is not very useful as it is all very similar. Instead, the model will use other, more distinguishing features such as eyes. Using these class activation maps, the same program can also be used to transform cats into dogs, or horses into zebras so the possibilities are endless!

Dogs become cats. (Source: NCSoft)

Dogs become cats. (Source: NCSoft)

While it may not be the biggest feat that AI can be used to turn a human into a spitting image of themselves but as an anime character, there is a greater accomplishment. That is the fact that it can be done so accurately shows the power of AI and machine learning, and this concept can be used in many other applications such as transforming animals as shown before.

The code of the project is on GitHub and the researchers also published a paper, so click on the links if you would like to see them.

Thumbnail Source: ‘Kaguya-same: Love is War’