PlayStation live games from Haven Studios may use PS5 machine learning

Sony's first-party PlayStation game dev Haven Studios is looking to use machine learning for its upcoming live games.

Today Sony completed its acquisition of Haven Studios, a new startup dev team created by Assassin's Creed and Mirror's Edge alum Jade Raymond. Haven is currently helping lead the charge in Sony's big new live service game plan which will see 12 first-party PlayStation live games released onto the market by 2025.

What's interesting is that Haven Studios wants to use the PlayStation 5's machine learning (ML) capabilities to help power its live games. Back in late June, Logan Olson, who's part of Haven's Creative ML team, announced he was building a group of engineers to help build ML solutions.

So why use machine learning? The AI-based tech can add new realism to any game and it can help streamline game scenes, improve facial capture/animation,

For example, Insomniac Games used machine learning inference to provide a significant boost to in-game visuals in Spider-Man Miles Morales. Using the ML inference, Insomniac is feeding the AI algorithm data in real-time that's running on the PlayStation 5. The PS5's 7nm+ AMD SoC is compatible with ML inference.

• Note: There's two parts to AI-based machine learning: training and inference. Training is done by supercomputers. Developers feed code into huge datacenters and the AI trains the data. Once the data is ready, it can be distributed and ready via inference.

Sony's AI division and has been working closely with PlayStation first-party devs to utilize and iterate on ML technology for quite some time.

"Sony AI, which we established last year, has begun a collaboration with PlayStation that will make game experiences even richer and more enjoyable. By leveraging reinforcement learning, we are developing Game AI Agents that can be a player's in-game opponent or collaboration partner," Sony CEO Kenichiro Yoshida said in the report.

"Sony AI and SIE are jointly developing AI technologies that can make game experiences even richer and more enjoyable," reads the press release.

Sony's Visual Arts Service Group (VASG), which is responsible for a significant portion of PlayStation games' motion capture, is also using ML and AI tech to help streamline facial captures for upcoming games.

Microsoft is also using ML in creative ways. The company established Direct ML, a new API that's part of DirectX 12 that's specifically optimized for machine learning. Microsoft has used ML to get next-gen games like Flight Simulator to run on older Xbox One hardware.

DirectML -Xbox Series X and Xbox Series S support Machine Learning for games with DirectML, a component of DirectX. DirectML leverages unprecedented hardware performance in a console, with Xbox Series X benefiting from over 24 TFLOPS of 16-bit float performance and over 97 TOPS (trillion operations per second) of 4-bit integer performance on Xbox Series X. Machine Learning can improve a wide range of areas, such as making NPCs much smarter, providing vastly more lifelike animation, and greatly improving visual quality.

Check below for the Haven Studios job listings for clues on what they are working on:

Senior Tools Programmer

We are looking for an experienced programmer to develop ML-powered creative tools for AAA games. You will be responsible for designing and implementing the tools' user experience. Your goal is making ML driven creative tools that are intuitive and fun to use. Our team uses a variety of software and it will be up to you to prototype interfaces that meet the user where they are, whether that's a web app or plugin for Blender. If you are passionate about generative ML and AI-human collaborative creation then this role is for you.

• Design and implement the user experience for ML-powered tools

Senior Software Developer (Games/Graphics for ML)

We are looking for a Senior Software Developer to work with our ML researcher team to deliver AI-powered creative tools. In this key role, you will use your background in games and graphics to inform the ML researcher's work and how it applies to game development. You will be the cross-disciplinary glue that binds the ML research to the game development team's workflow. If you are comfortable working across a broad range of applications and excited to learn new ones, this is a great opportunity to push the boundaries of how generative ML is applied to game development.

• Work with the ML researcher to move models from experimental prototypes to robust applications used by the production team;
• Graphics subject domain expert on the team responsible for guiding the ML researcher towards problems and solutions that apply to real-time 3D game development;
• Work with the tools prototyper to transform ML models and notebooks into features used by the dev team.