More advanced Robotics, Neuralink Podcast and SAM 2 – Live and Learn #48

✨ Quote ✨

When you're forced to be simple, you're forced to face the real problem. When you can't deliver ornament, you have to deliver substance.

– Paul Graham - (source)

📖 Book Notes 📖

The Unicorn Project by Gene Kim. I swallowed this book whole because it's written like a novel with an intriguing story where a company is struggling to organize its software development department. The pain of the engineers in the book, trying to rebel against the shortsighted business practices of the company is something that I can personally relate to, having worked in companies with similarly malfunctioning org structures before. The story is great in and of itself but the book is teaching real world concepts about how to create awesome environments for software engineers to do great work in. It's a great read and I took a lot of notes from it, but see for yourself.

🖇️ Links 🖇️

Fireside Chat with Marc Zuckerberg + Jensen Huang at SIGGRAPH. To me it's always fun to listen to people who run big companies about what they are doing and why they are doing it. In this talk, AI was a big topic (of course), especially in relation to how Meta is open-sourcing Llama and what that does to the industry. What struck me the most about their conversation, is how the two seem like really good friends, which makes me wonder how much of the tech industries' deals are cut at private dinner parties.

SAM 2 by Meta. Meta has released their newest Segmentation Model => Segment Anything 2. The results are beyond impressive, enabling accurate real-time segmentation in videos. This to me is nuts and some of the demos people build with it and the implications for surveillance technology are crazy. But there are also a lot of good use cases, in factories, automation, robotics, etc. You can try out the model and play around with it on the web yourself to see what it can do.

Figure 02 Update by Figure. Figure just announced "the most advanced humanoid robotics hardware platform" ever. Figure is constantly pushing what is possible with robotics, targeting factory jobs in pilot plants with BMW already. Them announcing their newest, upgraded design is interesting to see. Though the whole thing to me looks more like a shiny commercial than a technical demo of new breakthrough capabilities. But they have to get the word out and their PR department is doing a good job at that.

Human Level Tennis Playing Robot by Deepmind. Deepmind has built a robot that can play table tennis at an intermediate human level. The robot can win against human opponents and people actually enjoy playing with it. To me, simply watching the robot play is oddly satisfying. There is something about a disembodied arm swinging a table tennis racket around, that just makes me smile. But to be honest, what they did here is crazy. They essentially built a reinforcement learning policy for a real-world game, with all the messiness that the real world has. And it... works. It seems to even be generalizing against playstyles and opponents that it has not seen before. Stuff like this makes me think that we are much closer to universally usable robots than many people think.

The Rot Economy by Ed Zitron. I stumbled across a series of blog posts by Ed Zitron, which are all trying to tackle the same question: How can it be, that the internet feels sort of like it is getting worse over time, even though the technology that everybody is using now is vastly more powerful than it has ever been? The answer is the "Rot Economy". It's a fascinating read and I can highly recommend it. There's also a piece of Zitron on how Google Search is dying and who is killing it and how ads and ad-based businesses are looting the internet in general.

Robocasa by Nvidia and University of Texas. In this paper, the team found a way of generating a large variety of kitchen environments using generative AI. They provide thousands of 3D models and textures that can be used to do training simulations for real-world robots to learn tasks from. The method they developed can automatically alter a lot of kitchen parameters for a more robust transfer of trained behaviors into the real world. It's just crazy to see that they essentially built digital replicas of interactable kitchens. Only so that they can use those for training, and therefore scale training virtually to thousands of robots at the same time. It's going further into Nvidia's Omniverse direction and I like to see what else will come in the next few years from this type of research.

Neuralink Team Interview by Lex Fridman. Lex's podcast is by far my favorite and he is known for his long-form content, but this podcast is long even for his standards. Clocking in at over 8 hours it's insane. Ok, it's really multiple conversations stitched together, but all of them are fascinating because he does a deep dive into what Neuralink is doing, and how, and why they are creating a crazy future. The idea of generating new qualia alone is worth pondering. And them diving into the UI/UX problems of designing the user interface of a neural implant is just super interesting. I listened to the whole thing (at 2x speed) and can only recommend it.

🌌 Traveling 🌌

The summer in Berlin has come to an end for me... but it was wonderful and over way too quickly. Right now I am sitting on a little rooftop in Lagos, Portugal, enjoying the beautiful view and the little architectural details of the city. It's way too warm here, but there's an ocean nearby so I am happy 😊

🎶 Song 🎶

Burning by Cory Wong

Youtube Music | Spotify

---

That's all for this time. I hope you found this newsletter useful, beautiful, or even both!

Have ideas for improving it? As always please let me know.

Cheers,

– Rico