Agentic Frameworks, Robots with Touch and AI Game Engines – Live and Learn #55

✨ Quote ✨

Allegedly it takes 30 days to build a new habit, so you could pick one each month and be a running meditating ice-bathing early-rising journaling duolingo-maxing überman by the end of the year. But it never quite works like that, does it?

– Nat Eliason - (source)

📖 Booknotes 📖

The Web of Meaning by Jeremy Lent. A good read on the problems of the current, pre-dominantly capitalistic worldview based on competition and exploitation of the natural world. The book proposes a different worldview, combining modern insights from diverse fields such as neuroscience and systems thinking with ideas from ancient wisdom and books such as the "Tao Te Ching". Lent's new worldview is built on cooperation and symbiosis, focusing on the whole and interconnectedness. Lent argues that adopting this new view could help us fix many of the global scale problems and the ecological catastrophes we face in the 21st century while giving us meaning and something bigger than ourselves to contribute to.

🖇️ Links 🖇️

First Generalist Robot Policy by Physical Intelligence. To have robots that can work in real-world situations, they need to have a general understanding of the world. Furthermore, robots need to be able to extend this understanding easily to novel environments and tasks and maybe even different hardware architectures, without requiring the addition of large amounts of novel training data. The startup Physical Intelligence has created the first policy going in this direction. It is still a far cry from their stated goal of a truly general learning policy, but it is a giant step in the right direction and a pretty impressive proof of concept. The videos of their dual robot arms folding and stacking laundry are interesting to watch and I am looking forward to what they come up with next.

Robots with "Touch" by Meta FAIR. Meta is creating robotics that can sense the world through their fingertips. This can help robots finely control the pressure exerted on an object. Which in turn should help them to grasp objects much more delicately manipulating them in ways closer to what humans can do and across a wider range of forces. It is remarkable how much our hands can do with ease, without us even thinking about it. We can pick up an egg, a spoon, a power tool, or a piece of paper without creasing it and a large part of these abilities is adjusting the grabbing force necessary to pick up these things. Robots might be able to do the same, with new sensors such as those proposed by Meta.

What happens when Robots dont need us anymore documentary by Emily Chang. With progress in the field of robotics drastically accelerating, the questions about what we do once everything is automated become more loud and serious. This short Bloomberg documentary with Emily Chang dives into these questions head-on with interviews of leading roboticists in companies like Boston Dynamics. My favorite part was at the end though, where she asks random people on the street what they think about having robots everywhere and what that would mean for humanity and their personal lives.

Oasis Game Engine Demo by Decart AI. This is another demo, where instead of a game engine producing the game an AI does it. Every frame, all the controls, all the physics, and interactions are "dreamt" up in real-time by machine learning. In the words of the creators: "Oasis takes in user keyboard and mouse input and generates real-time gameplay, internally simulating physics, game rules, and graphics." Their demo is an entirely AI-generated Minecraft clone. It frankly looks quite bad. Yet, that this is possible at all is insane. You can even try it online in the browser. The insane engineering feat and novel thing they achieved is the speed of their video generation. Because this is a game it has to be real-time and achieving that is terribly hard and compute-intensive. But they somehow built it, outperforming other video generation models, like those of Runway by a factor of ten (while also generating coherent game physics and incorporating player controls somehow).

Magentic One: A Generalist Multi-Agent System by Microsoft. This blog post and accompanying technical report detail a new agentic system developed by Microsoft. They are aiming to enable AI to do complex tasks, end to end. The authors themselves write: "The future of AI is agentic. AI systems are evolving from having conversations to getting things done—this is where we expect much of AI's value to shine." Their system Magentic One employs a multi-agent architecture, which means it has several agents with specific capabilities, that work together to execute a goal, coordinated by a lead agent called the orchestrator. This architecture is modular and can be extended to use more capable and different models with novel capabilities to solve different tasks, which makes this approach adaptable for the future. At the moment, many companies are diving head-first into creating AI agents. Hubspot and Salesforce come to mind and now Microsoft is among them too. The answer to AI's 600 billion dollar question begins to crystalize.

Sam Altman Interview by Y Combinator. To me, the craziest bit in this interview is that Sam Altman casually puts his estimate for achieving AGI at 2025. That's next year. WTF. He is saying something like "We know what we need to do, we now just have to go and do it and we have the team to do this fast". It seems like the consensus among OpenAI employees is something very similar. Essentially all they need to do is scale the approach inherent in their o1 model and that might be enough to unlock AGI. We live in truly strange times and I wonder if this prediction will come true and we have AGI next year...

🌌 Travel 🌌

In one week, I'll be leaving Berlin for quite some time to come. The plan currently is to go to South America and buy a bicycle there. Let's see how that goes 😊

🎶 Song 🎶

Kasoy by FKJ

Youtube Music

---

That's all for this time. I hope you found this newsletter useful, beautiful, or even both!

Have ideas for improving it? As always please let me know.

Cheers,

– Rico