OpenAI is obviously also ramping up its own robotics efforts. Last week, Caitlin Kalinowski, who previously led virtual and augmented reality headset development at Meta, announced on LinkedIn that she was joining OpenAI to work on hardware, including robotics.
Lachy Groom, a friend of OpenAI CEO Sam Altman and an investor and co-founder of Physical Intelligence, joins the team in the conference room to discuss the business side of the plan. Groom wears an expensive hoodie and looks remarkably young. He points out that physical intelligence has many avenues to pursue a breakthrough in robot learning. “I just had a call with Kushner,” he says, referring to Joshua Kushner, founder and managing partner of Thrive Capital, who led the startup’s seed investment round. He is also of course the brother of Donald Trump’s son-in-law, Jared Kushner.
A few other companies are now seeking the same type of breakthrough. Skild, founded by roboticists at Carnegie Mellon University, raised $300 million in July. “Just as OpenAI created ChatGPT for language, we are building a general-purpose brain for robots,” says Deepak Pathak, CEO of Skild and assistant professor at CMU.
Not everyone is sure this can be achieved in the same way OpenAI cracked the AI language code.
There simply is no Internet-scale repository of robot actions similar to the text and image data available for LLM training. Achieving a breakthrough in physical intelligence could require an exponential amount of data anyway.
“Words in sequence are, dimensionally speaking, a tiny toy compared to all the movements and activities of objects in the physical world,” says Illah Nourbakhsh, a roboticist at CMU who is not involved with Skild. “The degrees of freedom we have in the physical world are much more than just letters of the alphabet.”
Ken Goldberg, a UC Berkeley academic who works on applying AI to robots, warns that enthusiasm for the idea of a robotics revolution powered by data as well as humanoids is reaching peak levels. hype-worthy proportions. “To achieve expected performance levels, we will need ‘good old-fashioned engineering’, modularity, algorithms and measurements,” he explains.
Russ Tedrake, a computer scientist at the Massachusetts Institute of Technology and vice president of robotics research at the Toyota Research Institute, says the success of LLMs has caused many roboticists, including himself, to rethink their research priorities and focus on finding ways to continue robotics learning on a more ambitious scale. But he admits that formidable challenges remain.
