Two 24-Year-Old Dropouts Aim to Disrupt NVIDIA: AI Inference Chips Are Becoming the Next Trillion-Dollar Bet
TechFlow Selected TechFlow Selected
Two 24-Year-Old Dropouts Aim to Disrupt NVIDIA: AI Inference Chips Are Becoming the Next Trillion-Dollar Bet
If you believe in the future of AI agents, you must bet on inference chips.
Navigating Web3 tides with focused insightsOrganized & Compiled: TechFlow
Guests: EJ and Josh, Hosts of Limitless Podcast
Podcast Source: Limitless Podcast
Original Title: If You Believe in AI, You Have to Bet on This
Air Date: July 7, 2026
Key Takeaways
The core argument of this Limitless Podcast episode is straightforward: everyone is focused on AI training, but the real next asymmetric bet is AI inference chips. The two hosts start with a startup called Etched, founded by two 24-year-old college dropouts, specializing in ASIC inference chips for the Transformer architecture. They have raised over $800 million, signed customer contracts exceeding $1 billion, and their investor list includes Peter Thiel, Jane Street, TSMC, and longevity enthusiast Brian Johnson.
But Etched is just the entry point. What the episode really discusses is the structural opportunity across the entire inference sector: more than two years ago, inference accounted for only one-third of chip demand, with training making up two-thirds. Now these numbers have flipped; inference accounts for two-thirds, training for one-third, and by the end of the year, it may skew towards 80% inference. NVIDIA holds about 75% of the chip share, but its GPUs are not optimized for inference. This is where startups have an opportunity to break in.
Etched's approach is not to make a better chip, but to redesign the entire rack system. They found that NVIDIA GPUs have a real utilization rate of only 30-40% on inference tasks, so they optimized the entire system to 80-90% utilization, while cutting power consumption by 75% by lowering voltage. The cost is hard-coding: they are betting that the Transformer architecture will continue to dominate; if the model architecture changes in the future, the chips become useless. Josh calls this an existential bet, while EJ believes this bet has already won for five years.
The latter half of the episode discusses how to participate via the public market: Cerebras is already listed, MediaTek is up 180% this year, Broadcom is designing TPUs for Google, and OpenAI's Jalapeno chip taped out in nine months. If AI agents start running autonomously for hours or even days, token consumption will grow exponentially, and inference chips are the inevitable path.
Highlights Summary
On Inference vs. Training
- "Everyone is still focused on training, thinking NVIDIA GPUs are the ultimate answer. But inference is the real profit center. Anthropic is rumored to be profitable this quarter due to inference margins."
- "More than two years ago, inference demand accounted for about one-third, training for two-thirds. Now it's exactly the reverse; inference accounts for two-thirds, training for one-third. By the end of this year, inference might approach 80%. NVIDIA holds 75% of the share, but their chips are actually not optimized for inference. This shows there's nothing else available in the market."
- "Chinese companies don't have NVIDIA GPUs; they rely on inference optimization to bring models to 90% of the level of US frontier models. Inference has no moat; NVIDIA has no moat in inference."
On Etched's Technical Breakthroughs
- "They are not making a better chip. They are making the entire rack system, increasing inference utilization from 30-40% to 80-90%. If you spend $50,000 to $150,000 on a machine and it only uses 30-40% of its real capability, you'd be very annoyed."
- "Power equals voltage squared. They cut the voltage in half, meaning power consumption decreases by 75%. You need less electricity to achieve the same intelligence output."
- "They use Bitcoin mining ASICs as an analogy. ASICs are specialized computers for specific mathematical problems, with efficiency that can be orders of magnitude higher."
On Team and Execution
- "They work in two shifts in Bangalore and the US, 12-hour shifts each, truly operating 24 hours a day. They are inside TSMC factories, calling in the middle of the night to test chips, seeing whether the wafers light up green or red."
- "Google has TPUs, Amazon has Trainium, but these people won't sleep in the factory. Etched's life or death rests entirely on this chip; if Google's TPU loses, Google is still Google."
On the Bet on Transformer Architecture
- "They hard-coded the entire computation graph into the silicon. If Andrej Karpathy comes up with a new architecture next year that isn't Transformer, Etched's chips become useless."
- "From GPT-2 to today, Transformer has ruled for five years. OpenAI, Anthropic, Google, Meta are all using it. No alternative architecture is visible in the short term."
On OpenAI's Jalapeno Chip
- "OpenAI and Broadcom together taped out Jalapeno in nine months. It takes nine months to have a baby; they also gave birth to a Jalapeno."
- "OpenAI's approach is different from Etched. They didn't hard-code Transformer, but deeply optimized the chip and rack system for GPT. They own the models, know what prompts users will send, so they can optimize the entire inference path."
- "OpenAI previously had deals with Cerebras too. Cerebras just went public, stock price fell 35.5%, but this isn't a sector problem, it's a pricing problem."
On Investment Logic
- "Brian Johnson's exact words: A few years ago two college dropouts told me they could accelerate longevity research by building a faster AI chip. That was their entire argument. If your chip spits out tokens faster, you can solve research problems faster."
- "Jane Street, Peter Thiel, TSMC's own venture fund all invested. TSMC investing in you isn't just giving you money; they are saying, we want to manufacture your chips."
- "This is an asymmetric bet placed in front of everyone. Everyone is looking at memory bottlenecks, power shortages, but forgot that the true source of margins for AI labs after listing is inference."
On NVIDIA and Vertical Integration
- "Do not rule out NVIDIA. They acquired Groq for $20 billion. Jensen knows clearly what is happening. Big companies are hard to turn around, but they are moving."
- "Apple M-series chips are the model of vertical integration. If you can connect chips, software, and models all together, the efficiency of the product line will change completely."
What is Inference: From Prefill to Decode
EJ: At Limitless, we are always looking for alpha, looking for opportunities around the corner. This weekend everyone is setting off fireworks, we are reading about a company called Etched. This company wants to change forever the way we look at inference.
Let's start with the basics. You use Claude or ChatGPT, write a prompt, hit enter, and the answer comes out. But what happens on the backend? Your prompt is sent to the server, where there are a bunch of AI chips on the server rack, usually NVIDIA GPUs. This GPU does one thing first: reads your entire prompt and processes it. This is called prefill. Then it retrieves the entire conversation context from memory, including all your previous prompts and the information you told it, then starts generating the response token by token. This is called decode. This is the entire process of inference.
Most people are still focused on training, thinking NVIDIA GPUs are the ultimate answer. But we see a trend emerging: Google is building its own accelerators, Amazon is building its own, Cerebras just went public, Groq was acquired by NVIDIA. Etched is the most aggressive among them.
Josh: There is a key difference between training and inference. Training is a one-time event, usually taking several months. When you hear that a company is training a new GPT or Claude model, that's training. Inference is completely another matter. And there is an interesting number: more than two years ago, inference demand accounted for about one-third, training for two-thirds. Now it's exactly the reverse; inference accounts for two-thirds, training for one-third. By the end of this year, inference might approach 80%.
This is very telling, because NVIDIA holds about 75% of the chip share, but their GPUs are not optimized for inference. NVIDIA's share is rising, and inference demand is also rising, but NVIDIA's chips are not good at this. This proves only one thing: there are no other available options in the market. No one has really figured out how to mass-produce custom inference chips. So the entire market can only buy NVIDIA. But companies like Etched are emerging, attempting to solve this gap.
EJ: You could even say, Chinese companies have already proven this path works. They don't have NVIDIA GPUs, yet they can train models reaching 90% of the capability of US frontier models. They rely on extreme innovation in inference. So inference is the next real battlefield, and on this battlefield NVIDIA has no moat.
Etched's Two 24-Year-Old Founders
EJ: Etched was founded by two 24-year-olds. They started doing this three years ago, betting on not making a general-purpose GPU, but rather an ASIC chip completely targeted at the Transformer architecture. Now they have secured over $800 million in financing, with customer contracts exceeding $1 billion. In early tests, their server racks have already reached state-of-the-art levels in latency, power consumption, and inference workloads.
You might think: this is a private company, how do I invest? Indeed, you can't invest directly. But there are paths in the public market to get exposure to this sector. We will talk about that later. First, let's talk about why this company could get such a high valuation before officially launching its product.
The answer is simple: they are not making a chip. They are making the entire rack system. This is their core argument. They observed the actual way inference works, looked at NVIDIA GPU performance, and found that GPUs have a real utilization rate of only 30-40% on inference. You spend $50,000 to $150,000 on a machine, and it only uses 30% of its real capability. You would be very annoyed. So they didn't just design a better chip, but designed the entire system that can be put into data centers, pulling inference utilization to 80-90%.
They mainly did two things. First, they found a way to achieve the same intelligence output with lower voltage. Around the Transformer architecture, they redesigned the entire chip. Power equals voltage squared; they cut the voltage in half, meaning power consumption decreases by 75%. At the data center level, this means you can save tens of millions of dollars in electricity and cooling costs.
Second, because the entire system is designed for inference rather than training, they optimized the entire process of prefill and decode. NVIDIA GPUs are general-purpose computing devices, capable of training, inference, graphics rendering, etc. Etched does only one thing, but does it to the extreme.
Josh: Here is a particularly sharp analogy. They directly took Bitcoin mining ASICs as inspiration. Bitcoin mining machines are specialized computers designed for specific mathematical problems, with efficiency that can be orders of magnitude higher. What Etched is doing is essentially this logic: targeting the specific mathematical problems of Transformer inference, making the specialized chip to the extreme.
The Team Sleeping in TSMC Factories
EJ: What is impressive about this company is not just the technology, but the execution style. They are already collaborating with TSMC. They convinced TSMC their technology was good enough to warrant tape-out. Half the team is in Bangalore, half in the US, working two shifts, 12 hours each per day, truly operating 24 hours around the clock. They got the opportunity to test chips at TSMC. They call the factory in the middle of the night to see whether the chips on the wafers light up green or red. This job is hard enough to make people quit.
This point is interesting compared to Google and Amazon. Google has TPUs, Amazon has Trainium, but these big companies won't let employees sleep in the factory. For them, if the chip project loses, the company still operates as usual. Etched is different; its life or death rests entirely on this chip. So when they recruit, they recruit the kind of people who are excited because the success or failure of the company hinges on the chip.
Josh: This is also why they could achieve this step within three years. The normal process takes one and a half to two years; they managed to compress it out.
What the People on the Investor List Are Betting On
EJ: Look at their investor list, and you will understand what problem these people are solving. Brian Johnson, the guy obsessed with longevity who measures a hundred metrics every day, why is he on the cap table? Brian Johnson himself posted a tweet: "A few years ago two college dropouts told me they could accelerate longevity research by building a faster AI chip." This is his entire logic. If your chip can spit out tokens faster, you can solve research problems faster. Drug discovery, protein folding, disease mechanisms—all require AI inference. Whoever is faster, wins.
Jane Street is also there, one of the best quantitative hedge funds in the world. Peter Thiel. TSMC also invested through its own venture fund. TSMC investing in you isn't just giving money; they are also saying: "We want to manufacture your chips." This signal is very strong.
Josh: Longevity, finance, semiconductor manufacturing, people from these completely unrelated fields are putting money into the same company, indicating that what they see is not some small opportunity, but that the improvement in underlying computing efficiency will change all industries relying on AI.
The Existential Bet on Transformer Architecture
Josh: But there is a boundary condition here I hadn't fully realized before. They made a very specific bet, betting on the Transformer architecture. From GPT-2 to today, all frontier models run on Transformer. Its core mechanism is next token prediction, learning through latent space recursion. But Etched's chips are hard-coded for this architecture. If the architecture changes, for example, if someone comes up with something better than Transformer, these chips would need a large part of the technology stack rebuilt. Is this an existential risk, or an acceptable bet?
EJ: You are very right. They hard-coded the computation graph into the silicon. If Andrej Karpathy releases a brand new architecture next year, not Transformer, Etched's chips become useless. But the benefit is also obvious: if you bet right, the efficiency improvement could be 10x to 50x. From GPT-2 to now, Transformer has ruled for five years. OpenAI, Anthropic, Google, Meta are all using it. No alternative architecture is visible in the short term. So although this bet is extreme, it is not crazy probabilistically.
EJ: Compare again to OpenAI's own Jalapeno chip. OpenAI did not hard-code Transformer, but deeply optimized the chip and rack system for GPT. Why can they do this? Because they own the models. They know what prompts users will send, know how to load tokens. They vertically integrated chips, models, and distribution. This is actually safer than Etched's pure third-party route. If Etched wants to achieve the same level of optimization, either they get acquired by some Frontier lab, or they can only serve multiple clients, but cannot achieve extreme optimization for a single model.
Josh: And OpenAI together with Broadcom taped out Jalapeno in nine months. Nine months, exactly enough to have a baby. They gave birth to a Jalapeno.
EJ: OpenAI is not playing this for the first time. They previously had deals with Cerebras too. Cerebras just went public, stock price fell 35.5%, performing poorly since the beginning of the year. But this is not a sector problem, it is a pricing problem. Many people in the market feel Cerebras was wrongly killed.
Inference is the New Moat
EJ: The reason we made this episode is because many people haven't realized: inference has become the new moat for training better models, and also the new moat for sending tokens to large numbers of users.
Most people are still stuck at the stage of "you use LLMs like using Google". Possibly less than 1% of people on Earth have actually run an autonomous agent for more than an hour. But the trend is clear: soon you will have large numbers of AI models working autonomously for you for hours or even days. This will burn massive amounts of tokens. You want them to spit out as many tokens as possible per unit of time, because you can get answers faster, finish work faster, and defeat competitors faster.
To achieve this, you need a completely different chip architecture. And NVIDIA, the big brother of all GPU architecture companies, has not yet solved the problem of inference efficiency.
Anthropic is rumored to be profitable this quarter due to inference margins. Everyone is focused on training costs, but the profit center for AI labs after listing is inference. Training is upfront investment, inference is continuous cash flow.
Paths in the Public Market
EJ: You might say, Etched is a private company, I can't invest in it. But there are paths in the public market. Cerebras is already listed. MediaTek is helping design these specialized chips, up 180% this year. Broadcom is designing TPUs for Google, up about 10% today. Groq was acquired by NVIDIA, reportedly at a price of $20 billion. NVIDIA is not unaware of this trend; they are just turning around.
Josh: If Apple M-series chips are the model, you know vertical integration can redo a product line completely. Now OpenAI is doing Jalapeno, Google is doing TPU, Amazon is doing Trainium, everyone is doing ASIC. This trend has just begun.
EJ: This is an asymmetric bet placed in front of everyone. Everyone is looking at memory bottlenecks, very important. Everyone is looking at power shortages, also very important. But many people forgot, when these AI labs list, the true source of margins is inference. If you believe in agents, believe in the future of autonomous work, you must bet on inference chips. This is not an option.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
深潮TechFlow
