We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.
Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!
But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general.
Here are the articles for the week ending 09 August 2026:
1. The Disaggregation of Inference Compute – Eugene Ng
Recurrent Neural Networks (RNNs) process and convert a sequential data input into a specific sequential data output, one step at a time. Because RNNs read one word at a time, training is slow and cannot be parallelized. Worse, as sequences grow long, the learning signal either explodes into instability or vanishes towards zero, and the model increasingly struggles to connect distant words. In short, RNNs are slow, inefficient, and expensive to train, and become increasingly forgetful over distance.
Transformers ended the waiting and unlocked parallelism. Instead of reading one word at a time and passing notes down a slow chain, self-attention lets every word look at every other word at once, and positional encoding keeps track of the order the words came in. Attention decides what matters. Positional encoding preserves order…
…Attention has one non-negotiable rule. Every token must compare itself with every other token. It is how each word figures out which others give it meaning…
…Ten tokens, one hundred comparisons. One thousand tokens, a million. A hundred thousand tokens, ten billion. The work grows with the text squared.
As the context size increases, the attention weight matrix grows quadratically, meaning the number of scores that must be computed and stored grows significantly faster. This leads to higher memory usage and computational cost, making it challenging to scale transformers to very long sequences efficiently…
…Fixes have come in three variations. Compute it smarter. Compute less of it. Or replace it. The first two focus on treating the symptoms. The third goes after the disease…
…The quadratic wall is real, and no fix has removed it cleanly. So, the frontier settled on hybrids for now. Cheap linear layers do most of the work. A few real attention layers stay to catch what the cheap layers miss. The frontier has settled on hybrids, a truce, not a cure.
Because the quadratic problem was never fully solved at the software architectural level, the industry has been forced to keep compensating with more expensive HBM and more GPU FLOPs at the problem, plus incremental software fixes, especially as workloads shift from training to inference and agentic use cases…
…CPUs are built to finish one task fast. GPUs are built to finish many tasks per second. CPUs are great for serial computations, but are architecturally poorly suited for matrix multiplications, which involve numerous parallel calculations.
LLMs are mostly matrix multiplications by arithmetic. NVIDIA did not make GPUs good at matrix multiplication by accident. Matrix multiplication turned out to look exactly like the problem GPUs were already built to solve.
GPUs, which are excellent at parallel computations, have rapidly displaced CPUs in this initial AI infrastructure buildout phase…
…AI LLM workloads started primarily with training, as models are trained, and are increasingly shifting to inference and agentic workloads…
…Inference workloads can be split into prefill and decode.
Prefill (prompt processing) is compute-bound. It processes the entire input prompt in parallel, builds the KV cache for attention, and is dominated by large matrix multiplications.
Decode (token generation) is memory-bound. It generates one output token at a time sequentially and reuses and grows the KV cache on every step autoregressively. It is dominated by memory reads of model weights and KV state. The bottleneck is memory bandwidth.
Each token requires reading the entire model’s weights and the full KV cache from memory, then performing relatively light arithmetic on them. The bottleneck is bandwidth, specifically the latency of getting weights and cache to the compute unit.
The GPU’s FLOPs sit idle waiting for memory. Using a GPU to decode at a batch size of 1 is a Ferrari in a parking lot: expensive silicon and memory doing almost nothing.
Continuous batching can improve throughput by interleaving many independent sequences, but it cannot eliminate the fundamentally low arithmetic intensity of decode, especially for latency-sensitive or single-stream workloads.
Agentic workloads compound this issue further. The industry’s software response is to run both phases on the same chip and optimize for throughput. But the issue remains. One cannot batch your way out of a workload that is inherently sequential. Decode looks like a compute problem but behaves like a memory problem.
AI inference is no longer a single workload that can be served efficiently by a single type of accelerator or memory. AI inference is a dual-stage workload, with very different resource demands for each. The same knife should not be used to cut everything…
…GPUs address this hardware architecture flaw with fast, expensive HBM stacked next to the compute die. Stacking to increase capacity causes thermal, yield, and capacity issues. DRAM chips generate heat, and it is becoming more difficult to remove it effectively…
…There are physical limits to how much thinner DRAM can be shaved and how much higher the shaved layers can be stacked vertically with TSVs (Through-Silicon Vias) and micro-bumps to make HBM.
Yield is also becoming difficult to achieve as layer counts increase. For a given stack yield, total yield decreases as layer count increases…
…Inference compute remains memory-bound on decode and batch size one. While HBM is a lousy memory, unfortunately, it is the best solution we have right now that can address this structural memory wall problem.
In short, HBM is bit inefficient, power inefficient, and bandwidth inefficient. The memory makers likely know it too and recognize that HBM is not the final answer and will not solve the memory wall problem…
…The next best solution is to redesign the hardware to better address the inherent software flaws. One way that is increasingly being adopted by the industry is to disaggregate the inference compute workload of prefill, decode, and orchestration/execution…
…While GPUs have been great for training, they are increasingly poorly suited for inference, especially on decode. If inference hardware becomes increasingly disaggregated into prefill and decode and is even less suited for agentic AI, where workloads are dominated by orchestration, GPUs could become less dominant.
Prefill on compute-dense silicon (i.e., NVIDIA and AMD GPUs, custom ASICs). Decode on bandwidth-dense silicon (i.e., Groq LPU, Cerebras WSE, SambaNova RDU, etc). Then orchestrate both from a CPU (i.e., NVIDIA, Intel, AMD, Graviton CPUs) and carry the KV cache on the fabric.
It makes sense to further disaggregate the AI inference hardware stack. Numerous launches to disaggregate prefill and decode have been announced over the last few months by the majors, including NVIDIA, AWS, Intel, AMD, with Groq, Cerebras, and SambaNova for decode-specialized chips.
2. Inside Google’s $200bn Wall Street finance machine for Anthropic – Ryan McMorrow
Google has assembled one of the largest infrastructure financing programmes in history to supply more than $150bn of artificial intelligence chips to Anthropic…
…To support the relentless surge in demand for the AI chips, Google, Broadcom and Wall Street investors have each taken on different pieces of the financial risk.
Google guarantees the data centres. Broadcom commits to buying the chips and helps finance them. Apollo and Blackstone provide much of the private-credit capital that purchases the hardware before leasing it to Anthropic…
…In June, the first tranche of TPU hardware passed from Google through Broadcom into this financing blender. A special-purpose vehicle known as Compute SPV paid $35bn for roughly 1GW of the AI hardware, representing around 1mn TPUs, according to people familiar with the matter.
The SPV’s cash came from three tranches of debt anchored by Apollo and Blackstone. Broadcom, in effect, guaranteed the two senior tranches by agreeing to cover any shortfall if Anthropic stopped paying and the SPV could not sell the hardware for enough to make the senior investors whole.
The arrangement, known as residual value support, covers about $30bn of the $35bn financing, with Broadcom’s exposure declining as Anthropic makes its lease payments…
…Financing the chips solved only half of Google’s problem. The company also needed enough powered data centres to house them. “We have a schedule and we’re looking for capacity that will fit the schedule,” the Google executive said. “Crypto miners with excess capacity were helpful.”
It has helped transform several crypto miners with secured power into a new breed of AI infrastructure developers, with a small outfit called TeraWulf the first to land a Google backstop to add a 360MW data centre on its campus in upstate New York.
Google guaranteed the lease payments on the Anthropic-bound site, which Morgan Stanley packaged into a construction bond that in October raised $3.2bn to get it built…
…People familiar with the matter said the Big Tech company had so far backstopped 10 developments with 2.4GW of power for TPUs. Google’s guarantees put it on the hook for as much as $44bn if all the leases go bad, though it marks the liability at $815mn on its balance sheet. It could also step into the leases itself.
3. Drug Discovery Has No Magic Wands – Daphne Koller
To understand where AI fits, it helps to decompose drug discovery into its three essential stages:
1. Disease-to-mechanism: Identifying a biological mechanism — a pathway, a target, a molecular interaction — where therapeutic intervention will alter the course of disease in humans.
2. Mechanism-to-drug: Creating a molecular intervention in the right therapeutic modality — a small molecule, antibody, siRNA, gene therapy — that achieves the desired mechanistic effect with acceptable safety and pharmacological properties.
3. Drug-to-patient: Designing a clinical development program that identifies the right patients and assesses the molecule’s effects — beneficial as well as adverse.
The vast majority of AI work in drug discovery has focused on stage 2…
…More than 90% of drugs that enter clinical trials fail — a dismal statistic that has barely improved in several decades. In the large majority of cases, the molecule was engineered just fine. The mechanism it targeted was wrong. We are doing a pretty good job at manufacturing keys, but they are generally for the wrong locks. Even if AI lets us make better keys at an accelerating pace, that won’t improve our ability to identify the right locks. The real bottleneck in making a novel medicine is disease understanding: identifying a biological mechanism whose modification actually changes the course of disease in patients. That, far more than molecular design, is where drug discovery succeeds or fails.
This mechanistic understanding is a rare commodity. And because no-one likes to fail in the clinic, we are seeing industry trends that are truly destructive. There are currently 38 targets that have over 50 programs against each of them — slightly better keys for those few locks where we have strong conviction. How many variants of GLP-1 do we really need? Even worse than this misallocation of capital is the disservice to patients: the number of novel targets the industry advances each year fell from ~100 in 2015 to about 30 in 2024. That collapse is the far bigger cost: the inability to help the hundreds of millions of people for whom medicine currently offers nothing…
…The challenge is that human biology is incredibly complex, spanning multiple interconnected biological layers — DNA, protein, cells, multi-cellular environments, entire organisms. Individual components respond dynamically to even subtle changes in related components or in the environment. Moreover, biology wasn’t engineered; it is the result of billions of years of messy, stochastic evolution, which produced staggering variation — countless genes, cell types, states, and contexts, each behaving in its own way. There is too much of it, too idiosyncratic, to reason about in the abstract. You have to measure it…
…Which brings up the greatest data challenge. While some processes are conserved across all forms of life, others are far more specific. The folding of a single protein is a self-contained process, highly conserved — closer to physics than to biology; this allows protein folding models to be trained on sequences collected across thousands of species. Metabolism involves at least a dozen distinct cell types and might be conserved across mammals. Brain function and dysfunction involves dozens of distinct cellular identities; and these processes are exquisitely specialized to humans: rodents do not get Alzheimer’s disease; non-human primates do not recapitulate ALS. The diseases where we have made the least progress tend to be precisely those that are most human-specific, and therefore those for which the data is most expensive to collect, least available, and most fraught with ethical constraints…
…But agentic iterated optimization relies on a fundamental attribute: agents thrive when there is a fast, accurate, and cheap scorecard to evaluate progress. If you give a sufficiently smart model an instant feedback loop, it will grind against that benchmark until it wins. This is why coding assistants and molecular design tools advanced so rapidly — the feedback is cheap, accurate, and fast…
…Drug development is the exact opposite. The ultimate scorecard — whether a drug actually provides therapeutic benefit to a patient — cannot be captured well by computational models or high-throughput assays. The only true ground truth is a human clinical trial. This feedback loop currently takes years, costs millions, and is strictly bound by human ethics and living biology. It is the ultimate slow feedback loop, and no amount of compute or process optimization can change this…
…Some have argued that the most important AI unlock in drug discovery is in the third stage — drug-to-patient — taking a drug candidate through preclinical testing and clinical trials. This is the fourth AI Magic Wand: reduce the time and cost of this very expensive phase, and drug discovery becomes faster and cheaper. Sadly, if you accelerate a pipeline full of drugs aimed at the wrong mechanisms, all you get is faster failures…
… An AI-enabled, deep mechanistic understanding of a disease enables the identification of novel clinical readouts that serve three distinct purposes: selecting the patients most likely to respond, confirming that the drug is hitting its intended target, and detecting early and reliable signals that it is actually modifying disease biology. Together, these allow trials to enroll the right patients, read out faster, and catch failures earlier — changes that transcend clinical trial operations, transforming the trial design itself. This capability is inseparable from solving the disease-understanding problem; they are one and the same. Better trials, in the end, are downstream of better biology…
…To fulfill the promise of AI for the millions of patients lacking any meaningful treatment, we must direct our efforts toward the problem that really matters: the identification of biological mechanisms with disease-transforming clinical benefit. This is arguably the hardest problem in drug discovery, because the only conclusive test of whether we have correctly identified a novel biological mechanism is a human clinical trial. There are multiple other paths in this space with shorter timelines and clearer near-term proof points. Those paths are shorter because the problems are more tractable: the feedback loops are faster and the benchmarks are cleaner. But a shorter path to a smaller destination is still a smaller destination — process improvements for problems we already know how to solve.
4. The 1970s: Warren Buffett’s Defining Decade – Dirtcheapstocks
In the early 1970’s the Nifty Fifty were all the rage. Investors would seemingly pay any multiple for blue chip growth stocks (funny how history repeats itself).
Polaroid was selling for 91x earnings.
McDonald’s sold for 86x earnings…
…Then the music stopped.
The market was down nearly 50% from its highs in 1972.
That’s when Buffett got busy buying…
…Buffett’s most important purchase in the early part of the 1970’s was Blue Chip Stamps as the zero cost float provided leverage for other investments. He added to his position throughout the decade. Some of the prices paid were absurdly low…
…Buffett bought O&M throughout 1973 and 1974.
His basis valued the business at $29mm.
Ogilvy’s enterprise value was roughly the same as its market cap. So, Buffett was buying the shares at ~3x EBIT.
O&M grew its operating profit at a 23% CAGR from 1970 to 1974.
Buffett’s shares doubled in value within 2 years of his purchase…
…Interpublic fell 73% from its 1972 highs when Buffett began buying.
Buffett’s basis valued the business at a $25mm market cap. The enterprise value was only $19mm. Interpublic earned $14.8mm of operating income in 1973.
Buffett’s investment was up 10x in 10 years!..
…Berkshire’s book value compounded at a 30% CAGR from 1973 to 1985…
…The 1970’s made Buffett, but he was also perfectly prepared for the opportunity.
He didn’t stretch to buy businesses at lofty valuations. He waited for the prices to come to him.
This was a difficult time for American business, but it was hardly unprecedented. I believe there is a reasonable probability that businesses get this cheap again.
5. Ways to think about token pricing – Benedict Evans
There are only two things you can say with certainty about token prices: we’re in a supply crunch, and this is unstable. All of the variables are in play, and the market will get shaken out over the next few years to arrive at a new equilibrium. Right now we have a lot of frantic analysis of ‘time to power’, but the question at the end of that remains whether the foundation models have sustainable pricing power, strategic leverage and value capture, or whether they become low-margin commodity infrastructure providers. At the moment, I think every dynamic we can see points to the latter…
…First, how many people will pay to be at the top right of the curve – to be at the frontier? At one extreme there are already use cases that already work just fine with a small, old, perhaps open source model that runs for ‘free’ on-prem or on your phone; at the other extreme there will be some that get better results from the latest, most expensive frontier model, consuming lots of tokens for lots of money; and then there will be many that are somewhere in between…
…Second, does the frontier keep moving significantly? This is obviously the most basic science question in AI: how long does the frontier keep getting better, how long does that keep needing more and more compute, and does that continue to happen at a rate that keeps it ahead of downward pricing pressure from efficiency and capacity gains?…
…Third, will there still be fierce competition between frontier models? Does the field shrink to fewer and fewer frontier models, perhaps with network effects emerging? Do frontier models diverge, with different models having much clearer leads in different fields? That could be another path to sustainable pricing power…
…Fourth, how much of the value from those high-end use cases is captured by the frontier model itself? How much needs to be wrapped in tooling, process, proprietary data, go-to-market, networks, support, and everything else associated with a traditional software company, even if you do need the big expensive frontier model underneath? Can that model do the whole thing, or is the model, no matter how good, still a piece of infrastructure that you use to make the actual product?…
…Meanwhile, there is structural uncertainty at the early stages of every big new technology, but the uncertainty now is different, because we don’t have a good theoretical understanding of why these models work so well and so we don’t know how much better they can get. In 1995, we didn’t know how the internet would evolve but we knew that there were less than 100m PCs on earth (and they were expensive) and that telcos couldn’t give everyone FTTH next year; in 2010 we didn’t know what the next iPhone would be but we knew it wouldn’t have retinal projection. We knew the physical limits in ways we don’t really know with LLMs. Next month a new approach could cut inference compute needs by 90%, or double demand, or both…
…That makes mobile data a more fruitful comparison here. Mobile networks have marginal cost for capacity, and like AI they had an enormous surge in usage 15 years ago, that overwhelmed capacity and had carriers scrambling to add capacity and rebalance their pricing. Meanwhile, selling bits looks superficially similar to selling tokens: it’s an opaque measure of marginal cost that doesn’t map in any transparent or intuitive way to use cases or value, and needs to be replaced with bundles of some kind. But most importantly, in the last 20 years cellular data traffic has risen by several orders of magnitude, and this has become an enormous industry, with annual revenue of a trillion dollars and capex of $200 billion, but the stocks have gone nowhere, and all the value was captured by other people further up the stack. This, of course, is one of the core questions for AI: is this going to be low-margin commodity infrastructure with all the value captured by other people further up the stack?…
…However, these examples do tell us, empirically, that something can be very important, very expensive, change the world, and be full of very sophisticated science and engineering, and yet have a wide range of possible outcomes. There isn’t one inevitable path here: you can have price equilibrium at high margins and at low margins, and with and without market concentration, and you can’t hand-wave that away by talking about AGI and saying “you don’t understand exponentials!”
However, if one thread in everything I’ve written above is how much we don’t yet know, the other thread is that every path to foundation models having market dominance, strategic leverage, value capture, winner-takes-all effects, or anything else other than becoming commodity infrastructure, requires something to change.
Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google) and Amazon (parent of AWS). Holdings are subject to change at any time.