Discover more from be radical Briefing
The (Possible) AI Training Dilemma
Read on to see why Training Day might be an AI’s nightmare…
I’m Pascal Finette, co-founder of be radical – and this is our weekly Briefing. We share our latest insights, analysis, and articles we read; all focussed on the future of technology and business. Just like a good banana, it’s easy to digest. nutritious and yummy.
Decode. Disrupt. Transform.
AI, specifically Large Language Models (LLMs), are trained on very large datasets. In the case of OpenAI’s ChatGPT, it looks like eight experts trained on 220 billion parameters each, stacked together like a tower made of Lego bricks. In essence, AI companies train their AI models on most of the publicly available text on the Internet – books, articles, discussions, and source code. In other words: Human-generated content (as most of the content on the public Internet is human-generated).
So far, so good. But what do you do when AI itself generates more and more content on the Internet? Point in case: Stack Overflow. The site is the de-facto standard Q&A site for programmers – software engineers worldwide flock to Stack Overflow to find solutions for gnarly coding problems. Coding AIs, such as Microsoft’s GitHub Copilot, and the code-generating features in popular LLMs, such as ChatGPT, are trained on the rich content generated by tens of thousands of volunteers. Stack Overflow experienced a 14% decline in traffic in March this year, a trend continues to hold as more and more developers shun the site and instead rely on their AI assistants.
With fewer programmers caring to ask and, as importantly, answer questions – the body of new knowledge AIs can be trained on diminishes. In turn, AI’s knowledge becomes frozen in time.
It gets worse. If not having new data to train models on wasn’t bad enough, AIs being trained on their output have been shown to degrade in their quality rapidly. When the Internet is flooded with AI-generated content, which, judging from the early signals, isn’t far away, models will train on this content and exhibit irreversible defects that gradually exacerbate across generations. AIs experience similar degenerative defects like weakening a species’ genetic pool when inbreeding occurs.
Outside of AI-generated content, which seemingly started to become the norm, humans, who are specifically paid to do “human work,” are starting to figure out that they can cheat and have AIs do their work – with the result that even the content we assumed was human-generated comes from an AI.
This is not to say that this future is inevitable, but it requires attention and effort. (via Pascal)
The Thin Wisps of Tomorrow
Some People in the HackerNews Community Exhibit Existential AI Angst. HackerNews, the news bulletin and community by startup accelerator program and investment fund Y Combinator, is somewhat the de-facto place for startup founders and tech talent to come together and discuss. A recent (anonymous) post by a software engineer on his fear of AI destroying his career (and thus financial security) stoked nearly 200 comments on the impending AI apocalypse (or not).
Even Google Itself Doesn’t Trust Its Own AI. The company warned its employees not to use Bard (its LLM) with any sensitive data as it might leak this data in future interactions with other users. To add insult to injury, Google also warned its developers that code generated by Bard might be buggy and/or bloated, and the cost of fixing the code might be higher than the assumed cost savings of using Bard in the first place — brave new world.
Brands and Influencers Go Virtual. What do luxury-good-empire LVMH, NIKE, and Epic Games have in common? They all collaborate to bring offline brands into virtual worlds. Epic Games has cornered the virtual world market with its blockbuster title Fortnite and the ever-growing Unreal Engine development tool ecosystem – and brands are eager to play. Meanwhile, virtual influencers such as Miquela, with her 2.8 million followers on Instagram, or Lu do Magalu, clocking in at 6.3 million Instagram fans, are competing for eyeballs and attention from real-world celebrities. Talking of which – real-world celebrities are rushing to create (and monetize) virtual clones of themselves. Up next? Consumers replacing their personal attention with AIs consuming and interacting with all the synthetic brand and celebrity content. “Have your AI talk to my AI.” will become the default.
What We Are Reading
💁♀️ How Technology Can Help Us Become More Human Digital technologies can aid in human development if we take the time to harness them for a positive purpose. The article explores the concept of human development across different stages of life and how we can use technology proactively rather than trying to keep up with the pace of change. Jane ⇢ Read
A Simple Way to Introduce Yourself Do you like the way you introduce yourself? This framework helps you structure a confident introduction touching on your present, past, and future. Mafe ⇢ Read
🪴 The Extraordinary Green Promise Of A Tiny Molecule The climate crisis is also an engagement crisis. Realizing the potential of green hydrogen as part of a global solution to the former depends critically on our ability to find local solutions to the latter. Jeffrey ⇢ Read
🇸🇪 Sweden wants to build an entire city from wood It’s encouraging to hear of concepts that challenge the norm with such promising new alternatives. A combination of ambition and advantages is driving it. Julian ⇢ Read
🧑🔧 Deep Tech - The New Wave By seizing the opportunities presented by Deep Tech, Latin America can overcome existing challenges and progress toward a more sustainable and inclusive future. Pedro ⇢ Read
🤯 Stop Calling Each New Disaster “The New Normal” Let’s all stop talking about the “New Normal” – it’s not just a meaningless platitude by now, it actively hurts our future prospects by shifting the Overton window. Pascal ⇢ Read
Around The Horn
Metformin, a common and cheap diabetes drug, can prevent long COVID.
Build your own synthetic embryo.
Humans have pumped so much water out of the ground that it changed the tilt of the Earth.
In case you ever need to write a cover letter for a job – here’s how AI can do it for you.
The ever-amazing Ben Evans on Apple’s vision for the Vision Pro.
A crazy deep dive into why watching movies on a plane using your pricey Apple Vision Pro might not be that great.
Some Fun Stuff
Celebrating the iconic napkin idea format on a digital level – from messiness to world-changing ideas. 💡
Did you know you can comment on this post by simply hitting reply? Your response lands directly in our inbox – and we love to hear from you. 🤗
Please help us spread the word – we’d love to share our weekly Briefing with more people. Please share the Briefing with a few of your friends and have them sign up here.