🌐 LLMs are a Superpower for Robots

+ Northvolt IPO, labour shortages and the CIA's venture fund share their model for public-private tech partnerships

Oct 30, 2023

This week;

Robots+LLMs: Language models have created an explosion of new applications
How to fund physical assets through SPVs
Peak fossil fuels?
The In-Q-Tel playbook: On public-private technology partnerships
Northvolt IPO: Why it’s going public in Sweden vs the US

Is the West in a labour workforce crisis?

This week I spoke to a founder whose software replaces a specific, physical job in warehouses. They’re getting traction because warehouses simply can’t find people who are willing and able to do that job.

This is nothing new. In energy, logistics, agriculture and defence, 70% of the seed-stage startups I speak to report severe labour shortages across their industry. These are the “staffing shortages” announced on delayed flights and trains, and the reason the company’s line will often ring out if you call to have a heat pump or solar panel installed.

Despite these shortages presenting a massive opportunity for physical automation startups, in most cases, this is not part of the company’s core narrative. It should be; this skills gap is a long-term structural reality in these industries.

Demographic: More people are retiring than entering the workforce. People are living longer and reproduction rates have been falling at an alarming rate.
Social: For many young people, physical work does not confer the same social status as service jobs (ironically, being an electrician or heat pump installer often pays better).

Transforming infrastructure requires people to build it, and this issue is political as well as economic. While politicians and the electorate may not be talking about it, there is a growing opportunity for founders who can solve these issues.

📣Want to meet awesome frontier tech builders?

Of course you do. The first group of speakers are confirmed for the RealTech Conference on November 22nd in London. More coming soon.

You’ll be hearing from the founders of:

Modern Synthesis - materials platform for fashion
Dexory - robots for warehouse digital twins
Hoxton Farms - fats for synthetic meat
Optimal Ag - using AI to protect against food scarcity

Join us - If you’d like to attend and have not attended or signed up before, please apply here. Invites will be going out in the next week.

⚡️Energy and Climate

It’s warming up, with October on track to be the warmest on record by Climate Brink

Northvolt’s potential $20bn IPO in Sweden by Financial Times (paywall)

Northvolt and Volvo Cars select Gothenburg for gigafactory

Northvolt is a battery maker founded in 2015 by two ex-Tesla employees
The company designs and produces lithium-ion batteries. They currently have four manufacturing plants across Europe and JVs with Volvo, BMW and VW who own 20% of the company
The company has prepared to file and has operationalised its internal processes to be public market-ready whenever "markets recover”, hopefully in 2024
Interestingly, the company wants to go public on Sweden’s main exchange
Europe’s main market has a depth of capital to support companies in the energy transition. As we covered last week, Europe is far ahead (v RoW) on the energy transition, with the Nordics being global leaders

Funding physical assets using Project Finance + SPVs by HV

We might be reaching peak fossil fuels by Carbon Brief

Global CO2 emissions could peak as soon as 2023, IEA reveals.

🌍 Policy and Geopolitics

The In-Q-Tel playbook for public-private technology partnerships by In-Q-Tel

Public-private partnerships are crucial for getting innovative commercial technologies into government agencies to address major challenges like climate change, civil protection and increasing sovereign industrial capacity
However, there is often a disconnect between startups and government due to differing perspectives and incentives
IQT has developed a playbook to building successful partnerships:
- Three-Way Envisioning: Understand perspectives of government, startups, and investors
- Architectural Thinking: Map technologies to identify capability gaps
- Q-Brainstorming: Adapt commercial products creatively
- Two-Way Winning: Ensure mutual public-private benefit
- Achieving Tech Adoption: Involve stakeholders early and facilitate feedback.

🦾 Manufacturing and Robotics

Primer: Controlling robots with LLMs by PCH Innovations

LLMs have found rapid adoption in robotics, LLMs allow instructions to be provided in natural language at a high level of abstraction.
They can then convert these instructions into executable tasks through zero-shot learning using their broad understanding derived from web-scale training data. This enables rapid adaptation to new scenarios without retraining, as well as conversational interactions with users
LLMs are writing custom reward functions which are used in reinforcement learning models to teach the robots new tasks
Four key areas for LLM integration in robotics: user interaction, scene understanding through VLMs, planning/execution by outputting code, and reflection on actions taken
Vision-language models (VLMs) can translate visual data into textual descriptions, allowing robots to understand environments and locate objects using just a camera

Nvidia just released Eureka!, which uses LLMs for robot dexterity

“The AI agent taps the GPT-4 LLM and generative AI to write software code that rewards robots for reinforcement learning. It doesn’t require task-specific prompting or predefined reward templates — and readily incorporates human feedback to modify its rewards for results more accurately aligned with a developer’s vision”

Eureka! This week, Nvidia, released a paper showing impressive robot dexterity using an LLM
Eureka outperforms human-written rewards on over 80% of tasks, improving performance by over 50% on average
This is an impressive low-level robotic control, which until recently people thought LLMs wouldn’t be able to solve

The number of LLM + robotic uses is exploding, as per this thread ⬇

An interesting Twitter🧵 with numerous new applications of LLMs + robotics

increased product complexity = engineering coordination complexity
opportunity for companies building workflows to minimise coordination costs