Learnings from creating AI for the game industry

Sometimes you need to be in the right place at the right time. For us, it was an event hosted by JIC, the South-Moravian innovation centre, which experimented with connecting people and companies from the creative industry to those from the tech space. During the event, we met representatives of a local AAA game studio and talked about artificial intelligence and its potential for game development. That was the beginning of a journey that a year later got us a long-term project at one of the world's largest game publishers.

Today I want to reflect on the experience of the last two years exploring, researching and building AI tools within the game development industry and community.

Listen and learn, don't pitch

The Kapnetix team has a lot of experience with AI projects across various industries. The video game industry is quite unique. It's a rapidly growing industry that is somewhat insular and has evolved its own culture, conferences and networks.

There are several conferences and events in the gaming industry focused specifically on developers, publishers and other industry professionals. Kapnetix had the opportunity to visit the Game Developers Conference in San Francisco. These events provide opportunities to network, showcase work, and learn about the latest trends and technologies in the field.

Breaking into the industry takes a bit of time. A lot of respect is gained from the work done in the industry. Most "about me" slides were a name and a list of the titles that person had worked on. We did have some exposure to game development through a colleague, which helped, but we weren't up to date with how the industry works these days. Conferences are a good place to start exploring and learning. I typically reach out to a few people and, instead of pitching what we do, express a desire to learn. I managed to have a few coffee conversations and naturally the person would ask what I was working on. I gave a very short pitch and asked what they thought about the project.

The lesson learnt is that breaking into the community is hard, especially when you are not a gamer. On the other hand, an honest desire to learn can slowly get you connected. Weeks later, this led to discovery meetings with professionals from EA, Industrial Light and Magic and Disney.

Working with a large game publisher

I read a post from Andrew Chen at a16z sharing some reasons why it's hard for AI companies to break into the gaming industry. One point said:

Large publishers often provide tech to their internal studios. They'll partner to learn about AI, but will try to build in-house. Is your tech defensible?

We landed a behemoth of a publisher and the experience has been really good. Maybe surprisingly good. The team openly shared their processes, in our case how they run motion capture pipelines. I never had a sense that they were trying to learn and then build in-house, but it is a valid point. How is our tech defensible?

In my view, defensibility comes down to three things: the uniqueness of the architecture and approach, the quality of data required, and the level of accuracy needed for the solution to be production ready.

We most certainly did not take a model from GitHub or Hugging Face and apply it to mocap data. Not that we didn't try, but we were nowhere near the quality professionals produce. While we read a lot of papers, we took them only as inspiration for how to approach certain parts of the cleanup process. To get the accuracy we needed, we built the architecture from the ground up. That took a year of research and development.

The challenge with data in our case was that world-class mocap data is hard to get hold of outside the top studios. We trained on raw and cleaned pairs of motion capture footage from various parts of the process.

Which runs into another point Andrew mentioned:

They also care that their models are trained on data that's safe from a copyright perspective. There's lots of hoops to jump through.

Studios view the data they create as an asset. Using a model trained on their data for us to monetise is likely to be quickly dismissed. So what do you do? Without data, you don't have a model, which means you don't have a product.

One solution is to create your own. In our case, we hired a studio that some of the top studios and publishers outsource to. The key element for us was the experience of the motion capture team. We needed the best possible quality.

The lesson learned is that publishers are open to working with startups like ourselves, but the level of quality they need for production is really high. You also shouldn't rely on every publisher sharing their data for model training.

Focusing on quality and time spent rather than saved cost

Anyone following the video gaming community knows that buggy, low quality or rushed games get ripped to shreds in reviews and forums. That directly impacts sales.

Studios have been adjusting much of their development process to mitigate that. In many cases it's a balancing act between time and quality. While automation has been mostly welcomed, there are still cases where developers want people to spend extra time and care on the job.

This helped us focus on creating AI that solves the boring parts of the process. Things that don't require much creativity, but still cost an immense amount of time.

Conclusion

AI has the potential to transform the gaming industry, providing developers with tools to create better experiences. To succeed in this space, companies need to be open to learning about the industry and to consider what actually matters to developers, studios and publishers.