OpenAI officially announced the first acquisition! Open source "Minecraft" 8-member top team joins, betting on AI agents

Source: Xinzhiyuan

Author: Editor: Taozi is so sleepy

OpenAI officially announced the first acquisition ever.

Just now, OpenAI acquired Global illumination, a start-up company that produced an open source version of "Minecraft", for an unknown amount.

According to the announcement, the entire Global Illumination team will join OpenAI and focus on the development of ChatGPT in the future.

Global illumination, co-founded by Thomas Dimson, Taylor Gordon and Joey Flynn, is a company that uses AI to build creative tools, infrastructure and digital experiences.

What does OpenAI's acquisition imply?

In this regard, Jim Fan, a senior research scientist at Nvidia, expressed great curiosity: "What's the next step? Running a multi-agent civilization simulation on GPT-5? Maybe "My World" is all that is needed to build AGI? "

"Eight King Kong" joined OpenAI

Founded in 2021, Global Illumination is a New York-based digital products company.

Most recently, the company developed its most popular product, the open-source multiplayer online role-playing game "Biomes."

project address:

All along, Global Illumination's work has focused on vision-oriented products and services.

In other words, OpenAI may be expanding more image/video functions for ChatGPT, or to build Dall-E 2 image generation services, or to launch a text-video product to compete with startup Runway's Gen- 2 competition.

On the other hand, the company's naming is also different.

The term "Global Illumination" has been used in computer graphics for many years to describe algorithms for creating realistic lighting effects on 3D objects.

One of the original algorithms for global illumination was proposed in 1986 by Caltech researcher James Jim Kajiya.

The company consists of 8 members and 3 founders. What's interesting is that everyone's avatar on the official website uses pixel villains.

Some netizens said that this move is an excellent "acquisition" of OpenAI, through acquisitions to recruit new talents.

In the OpenAI announcement, it is also listed that the Global Illumination team has made contributions to famous companies such as Instagram, Facebook, YouTube, Google, Pixar, Riot Games, etc.

Let's take a look together, what is the strength of the team behind OpenAI from this acquisition?

Thomas Dimson

Thomas Dimson is CEO of Global illumination.

Before starting the company, he spent most of his career on Instagram.

In 2013, Thomas joined Instagram as Chief Engineer, and 5 years later, he was promoted to Director of Engineering.

He is known as the "No. 16 engineer" of Ins, "the original creator of the Instagram content ranking algorithm", and officially resigned in 2020.

In the 7 years of working on Instagram, he has played an important role in Instagram's iterative platform's discovery algorithm, such as information ranking algorithm, exploration ranking, and story ranking.

In addition, he also invented products such as story voting stickers, Hyperlapse, emojineering, and was named one of the ten most creative people in business by Fast Company.

In an article written for Future last year, Thomas revealed that the team he led developed Instagram's personalized content ranking/recommendation system, replacing the default reverse chronological recommendation.

Taylor Gordon

Taylor Gordon is the Chief Technology Officer of Global Illumination.

Taylor has over 20 years of programming experience, 8+ years of computer science education, and 10+ years of experience working at leading technology companies.

He has rich experience in many laboratories Facebook AI, Instagram, YouTube, Google, Microsoft.

In 2019, he was mainly doing computer vision research at Facebook AI.

Likewise, after joining Instagram in 2015, he led the development of the feed rankings, focusing on machine learning and rankings for infrastructure components.

In 2012, as a YouTube machine learning engineer, he was mainly responsible for video recommendation and led the refactoring of the core infrastructure ranking facility.

While at Microsoft, designed and built production algorithms for in-site links in Bing search results. At Google, responsible for internal core services and infrastructure development analytics, and monitoring software.

Joey Flynn

Joey Flynn is Chief Product Officer at Global Illumination.

He himself has also worked in Facebook and Ins, and is known as one of the top designers of Facebook by the outside world.

In 2010, he joined Facebook and led some important projects, such as Chat Heads, and Facebook's mobile software Home.

In 2014, Flynn conceived and built Slingshot, a Facebook messaging app.

Alexei Karpenko

Alexei Karpenko himself has worked at Facebook, Midnox, and Apple.

Andrew Top

Andrew Top is the technical lead at Cobalt, the app platform behind the YouTube TV app, and has been with YouTube for seven years.

In 2012, he joined the team as a graphics lead, implementing their respective proprietary graphics API subsets of the GLES graphics API for PlayStation 3, PlayStation 4, and WiiU.

He also designed the Cobalt graphics rendering stack to enable faster animation frame rates on embedded devices than Chrome.

Andrew's other contributions include:

-Designed a high-level layout of the main browser components, Java engine, Web API (including DOM), network engine and graphics.

-Designed and implemented a cross-platform low-latency web-based solution to watch YouTube's 3D 360 spherical videos on TVs and VR headsets.

• Systems designed to monitor and analyze performance issues.

Nick Cooper

Nick Cooper has worked at Artifact, Ins, Google.

Ian Silber

According to his homepage, Ian Silber also worked in Artifact and Ins.

Brandon Wang

The only Chinese researcher in the team, who worked on the research of game animation at Riot Games and Pixar.

He is also very good at real-time rendering, physically based rendering, and Networked Gameplay.

Below are some of the projects designed by Brandon Wang.

Open source version of "My World", you can run it with a browser

"Biomes" is an open source sandbox MMORPG (massively multiplayer online role-playing game) built for the web.

Just open the browser, you can build, gather, play mini-games and more.

As for OpenAI, in fact, as early as 2022, a brand new VPT "video pre-training model" was proposed, which allows AI to learn to build stone pickaxes from scratch in "Minecraft".

The entire process takes a hardcore player at least 20 minutes to complete, and a total of 24,000 operations are required.

Research address:

Specifically, the researchers first collected a wave of data annotation outsourcers playing games, including videos and records of keyboard and mouse operations.

Overview of the VPT method

Then, use these data to make an inverse dynamics model (inverse dynamics model, IDM), so as to infer how the keyboard and mouse move when each step in the video is in progress.

In this way, the whole task becomes much simpler, and only requires much less data than before to achieve the goal.

The influence of base model training data on fine-tuning

After 70,000 hours of training, OpenAI's behavioral cloning model can do things that other models can't.

For example, how to cut down trees to collect wood, how to use wood to make sticks, and how to use sticks to make tables. And this set of things requires a more skilled player to operate for less than 50 seconds.

In addition to being a table, the model can swim, hunt, and eat.

There is even a show operation of "running, jumping and building", that is, when taking off, put a brick or wooden block under your feet, and you can build a pillar by jumping and jumping. This is a compulsory course for hardcore players.

"Run and Jump" simple version

Making a diamond pickaxe, which is more difficult, requires a long and complicated series of subtasks.

To make this task tractable, the researchers reward the agent for each item in the sequence.

They found that RL policies trained with traditional methods received almost no rewards, never learned to collect logs, and rarely collected sticks.

In stark contrast, the VPT model fine-tuned on human data can not only learn how to make a diamond pickaxe, but also achieve human-level success in collecting all items.

Make a Stone Pickaxe

In this regard, some netizens think that OpenAI Gym playgrounds is a "My World".

Before applying these systems in the real world, it's a good idea to practice them in a virtual world and see what happens. The speed of training may also be faster. Looking forward to seeing some amazing videos of Minecraft speeding up in a few months.

The startup's engineering team comes from a high-level design team, and I think ChatGPT will be greatly upgraded soon.

Others think that from what Global Illumination has done. Their product capabilities are very much needed by OpenAI, but combined with the Stanford AI town a few days ago, this kind of sandbox game with high degree of freedom and low operating cost combined with AI will have infinite possibilities, so OpenAI may value their members The product capabilities may also be purely based on this high-degree-of-freedom game.

References: