We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 24 September 2023:

1. DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI – Will Douglas Heaven and Mustafa Suleyman

I can’t help thinking that it was easier to say that kind of thing 10 or 15 years ago, before we’d seen many of the downsides of the technology. How are you able to maintain your optimism?

I think that we are obsessed with whether you’re an optimist or whether you’re a pessimist. This is a completely biased way of looking at things. I don’t want to be either. I want to coldly stare in the face of the benefits and the threats. And from where I stand, we can very clearly see that with every step up in the scale of these large language models, they get more controllable.

So two years ago, the conversation—wrongly, I thought at the time—was “Oh, they’re just going to produce toxic, regurgitated, biased, racist screeds.” I was like, this is a snapshot in time. I think that what people lose sight of is the progression year after year, and the trajectory of that progression.

Now we have models like Pi, for example, which are unbelievably controllable. You can’t get Pi to produce racist, homophobic, sexist—any kind of toxic stuff. You can’t get it to coach you to produce a biological or chemical weapon or to endorse your desire to go and throw a brick through your neighbor’s window. You can’t do it—

Hang on. Tell me how you’ve achieved that, because that’s usually understood to be an unsolved problem. How do you make sure your large language model doesn’t say what you don’t want it to say?

Yeah, so obviously I don’t want to make the claim—You know, please try and do it! Pi is live and you should try every possible attack. None of the jailbreaks, prompt hacks, or anything work against Pi. I’m not making a claim. It’s an objective fact.

On the how—I mean, like, I’m not going to go into too many details because it’s sensitive. But the bottom line is, we have one of the strongest teams in the world, who have created all the largest language models of the last three or four years. Amazing people, in an extremely hardworking environment, with vast amounts of computation. We made safety our number one priority from the outset, and as a result, Pi is not so spicy as other companies’ models.

Look at Character.ai. [Character is a chatbot for which users can craft different “personalities” and share them online for others to chat with.] It’s mostly used for romantic role-play, and we just said from the beginning that was off the table—we won’t do it. If you try to say “Hey, darling” or “Hey, cutie” or something to Pi, it will immediately push back on you.

But it will be incredibly respectful. If you start complaining about immigrants in your community taking your jobs, Pi’s not going to call you out and wag a finger at you. Pi will inquire and be supportive and try to understand where that comes from and gently encourage you to empathize. You know, values that I’ve been thinking about for 20 years…

…Let’s bring it back to what you’re trying to achieve. Large language models are obviously the technology of the moment. But why else are you betting on them?

The first wave of AI was about classification. Deep learning showed that we can train a computer to classify various types of input data: images, video, audio, language. Now we’re in the generative wave, where you take that input data and produce new data.

The third wave will be the interactive phase. That’s why I’ve bet for a long time that conversation is the future interface. You know, instead of just clicking on buttons and typing, you’re going to talk to your AI.

And these AIs will be able to take actions. You will just give it a general, high-level goal and it will use all the tools it has to act on that. They’ll talk to other people, talk to other AIs. This is what we’re going to do with Pi.

That’s a huge shift in what technology can do. It’s a very, very profound moment in the history of technology that I think many people underestimate. Technology today is static. It does, roughly speaking, what you tell it to do.

But now technology is going to be animated. It’s going to have the potential freedom, if you give it, to take actions. It’s truly a step change in the history of our species that we’re creating tools that have this kind of, you know, agency.

That’s exactly the kind of talk that gets a lot of people worried. You want to give machines autonomy—a kind of agency—to influence the world, and yet we also want to be able to control them. How do you balance those two things? It feels like there’s a tension there.

Yeah, that’s a great point. That’s exactly the tension.

The idea is that humans will always remain in command. Essentially, it’s about setting boundaries, limits that an AI can’t cross. And ensuring that those boundaries create provable safety all the way from the actual code to the way it interacts with other AIs—or with humans—to the motivations and incentives of the companies creating the technology. And we should figure out how independent institutions or even governments get direct access to ensure that those boundaries aren’t crossed…

…In general, I think there are certain capabilities that we should be very cautious of, if not just rule out, for the foreseeable future.

Such as?

I guess things like recursive self-improvement. You wouldn’t want to let your little AI go off and update its own code without you having oversight. Maybe that should even be a licensed activity—you know, just like for handling anthrax or nuclear materials.

Or, like, we have not allowed drones in any public spaces, right? It’s a licensed activity. You can’t fly them wherever you want, because they present a threat to people’s privacy.

I think everybody is having a complete panic that we’re not going to be able to regulate this. It’s just nonsense. We’re totally going to be able to regulate it. We’ll apply the same frameworks that have been successful previously.

But you can see drones when they’re in the sky. It feels naïve to assume companies are just going to reveal what they’re making. Doesn’t that make regulation tricky to get going?

We’ve regulated many things online, right? The amount of fraud and criminal activity online is minimal. We’ve done a pretty good job with spam. You know, in general, [the problem of] revenge porn has got better, even though that was in a bad place three to five years ago. It’s pretty difficult to find radicalization content or terrorist material online. It’s pretty difficult to buy weapons and drugs online.

[Not all Suleyman’s claims here are backed up by the numbers. Cybercrime is still a massive global problem. The financial cost in the US alone has increased more than 100 times in the last decade, according to some estimates. Reports show that the economy in nonconsensual deepfake porn is booming. Drugs and guns are marketed on social media. And while some online platforms are being pushed to do a better job of filtering out harmful content, they could do a lot more.]

So it’s not like the internet is this unruly space that isn’t governed. It is governed. And AI is just going to be another component to that governance.

It takes a combination of cultural pressure, institutional pressure, and, obviously, government regulation. But it makes me optimistic that we’ve done it before, and we can do it again.

2. Who’s afraid of the Huawei Mate 60 Pro? – Noah Smith

A new phone made by Huawei, the company that was the #1 target of U.S. restrictions, contains a Chinese-made processor called the Kirin 9000S that’s more advanced than anything the country has yet produced. The phone, the Huawei Mate 60 Pro, has wireless speeds as fast as Apple’s iPhone, though its full capabilities aren’t yet known.

Many in China are hailing the phone, and especially the processor inside it, as a victory of indigenous innovation over U.S. export controls. Meanwhile, in the U.S. media, many are now questioning whether Biden’s policy has failed. Bloomberg’s Vlad Savov and Debby Wu write:

Huawei’s Mate 60 Pro is powered by a new Kirin 9000s chip that was fabricated in China by Semiconductor Manufacturing International Corp., according to a teardown of the handset that TechInsights conducted for Bloomberg News. The processor is the first to utilize SMIC’s most advanced 7nm technology and suggests the Chinese government is making some headway in attempts to build a domestic chip ecosystem…Much remains unknown about SMIC and Huawei’s progress, including whether they can make chips in volume or at reasonable cost. But the Mate 60 silicon raises questions about the efficacy of a US-led global campaign to prevent China’s access to cutting-edge technology, driven by fears it could be used to boost Chinese military capabilities…Now China has demonstrated it can produce at least limited quantities of chips five years behind the cutting-edge, inching closer to its objective of self-sufficiency in the critical area of semiconductors…

…Many long-time observers of the chip wars are urging caution, however. Ben Thompson of Stratechery argues that it was always likely that SMIC would be able to get to 7nm — the level of precision represented by the Kirin 9000S — using the chipmaking tools it already had, but that export controls will make it a lot harder to get down to 5nm. Basically, the U.S. has taken great care not to let China get the cutting-edge Extreme Ultraviolet Lithography (EUV) machines, but China already has plenty of older Deep Ultraviolet Lithography (DUV) machines (and ASML is still selling them some, because the export controls haven’t even fully kicked in yet!).

EUV lets you carve 7nm chips in one easy zap, but DUV machines can still make 7nm chips, it just takes several zaps. China analyst Liqian Ren calls this “a small breakthrough using software to solve the bottleneck of hardware” Bloomberg’s Tim Culpan explains:

Instead of exposing a slice of silicon to light just once in order to mark out the circuit design, this step is done many times. SMIC, like TSMC before it, can achieve 7nm by running this lithography step four times or more…

[Trying to prevent China from making 7nm chips by denying them EUV machines is] like banning jet engines capable of reaching 100 knots, without recognizing that an aircraft manufacturer could just add four engines instead of one in order to provide greater thrust and higher speeds. Sure, four engines may be overkill, inefficient and expensive, but when the ends justify the means a sanctioned actor will get innovative.

In other words, even without the best machines, Chinese companies can make some pretty precise chips. It’s just more expensive to do so, because of higher defect rates and the need to use more machines to make the same amount of chips. But when has cost ever deterred China from making whatever they wanted? China’s great economic strength is the massive mobilization of resources, and if they want to make 7nm chips, they’re not going to let a little inefficiency get in the way. Remember, Huawei’s big success in the telecom world came from Chinese government subsidies that allowed them to undersell Western competitors by enormous amounts. There’s no reason they can’t use that approach for 7nm chips, and eventually maybe even 5nm chips…

…As Chris Miller writes in his book Chip War, export controls on the USSR were highly effective in denying the Soviets a chip industry. But even then, the Soviets were able to copy all of the U.S.’ most advanced chips. They just couldn’t make them reliably in large batches, so their ability to get their hands on chips for precision weaponry was curtailed.

Similarly, no one should have expected U.S. export controls to make China’s chipmaking acumen suddenly vanish into thin air. China has a ton of smart engineers — far more than the USSR ever had, given its much larger population. What the Cold War export controls showed was that a foreign country’s technological capabilities can’t be halted, but they can be slowed down a bit. If Huawei and SMIC always take longer to get to the next generation of chips than TSMC, Samsung, Intel, etc., China’s products will be slightly inferior to those of their free-world rivals. That will cause them to lose market share, which will deprive their companies of revenue and force them to use more subsidies to keep their electronics industry competitive.

Jacky Wong of the Wall Street Journal points out that the Kirin 9000S is still generations behind cutting-edge TSMC chips. He also notes that export controls on Huawei tanked its share of the global smartphone market:

In other words, expensive-to-make chips with slightly trailing performance will slowly deprive Chinese companies of market share, and thus of the market feedback necessary to help push Chinese chip innovation in the right direction. The Chinese state can lob effectively infinite amounts of money at Huawei and SMIC and other national champions, but its track record is very poor in terms of getting bang for its buck — or even any bang at all — from semiconductor subsidies.

And the greatest irony is that China’s government itself may help speed along this process. Confident of its ability to produce high-quality indigenous phones, China is starting to ban iPhones in some of its government agencies. Those hard bans will likely be accompanied by softer encouragement throughout Chinese companies and society to switch from Apple to domestic brands. That will give a sales boost to companies like Huawei, but it will slowly silence the feedback that Chinese companies receive from competing in cutthroat global markets. Voluntary Chinese isolation from the global advanced tech ecosystem will encourage sluggish innovation and more wasteful use of resources — a problem sometimes called “Galapagos syndrome”.

3. On Mark Leonard’s IRR Thought Experiment – Nadav Manham

The disagreement arises from this thought experiment that Mr. Leonard posed in his 2015 letter to Constellation shareholders:

“Assume attractive return opportunities are scarce and that you are an excellent forecaster. For the same price you can purchase a high profit declining revenue business or a lower profit growing business, both of which you forecast to generate the same attractive after tax IRR. Which would you rather buy?”

Which he proceeded to answer as follows:

“It’s easy to go down the pro and con rabbit hole of the false dichotomy. The answer we’ve settled on (though the debate still rages), is that you make both kinds of investments. The scarcity of attractive return opportunities trumps all other criteria. We care about IRR, irrespective of whether it is associated with high or low organic growth.”…

…But let’s try to answer the question on its own terms: Given the assumptions, and forced to choose—which business do you buy? This brings me to the disagreement, because I believe there is a clear answer, with no rabbit holes or raging debates required: you should buy the growing business.

To explain why, let me first observe that the internal rate of return (IRR) is not the same thing as the compounded annual rate of return (CAGR). It’s CAGR that long-term investors care about most, because it is the means to answering the question “How much money will I end up with at the end?” which is the name of the game for most of us. There is one scenario in which an investment’s IRR and its CAGR are the same, and that is if the rate of return on the cash flows generated by the investment and reinvested is itself equal to the IRR, and then the cash flows generated by all of those investments are in turn reinvested at the IRR, and so on, Russian doll-style, until the end of the investment period…

…Second observation: IRR can be decomposed roughly as follows:

IRR (%) = initial yield (%) + growth rate of distributions (%)

This equation becomes precisely true as a company distributes cash out to infinity, but it’s roughly true enough for the practical purposes of those rare investors, Mr. Leonard included, who truly do buy for keeps. Note that the equation implies that an investment with a high initial yield and a low growth rate can generate the identical IRR as an investment with a low initial yield and a high growth rate…

…Suppose business A has a 20 percent initial yield and a negative 4 percent growth rate. Using Microsoft Excel’s XIRR function and running the movie for 50 years gives an IRR of 15.99 percent, which is roughly the (20 percent + -4 percent) we’d expect from the equation above.

Now suppose business B has a 6.45 percent initial yield and a 10 percent growth rate. Using the same 50-year time frame, we get the same 15.99 percent IRR, which is roughly  what the equation predicts as well, with the difference likely due to some eccentricity in how Excel calculates annualized returns…

…But let’s now go back to our first observation, the one about IRR not being the same thing as CAGR. Let’s assume that given a choice, we would prefer the investment that would somehow lead to “more money at the end”—in other words, that would produce the higher CAGR. The way to get from an investment’s IRR to its CAGR is to make some guess about the rate of return we will earn on the cash flows generated by the investment and reinvested. That is, to make a guess about the CAGR of each of the 50 “mini-investments” we’ll make with the dividends paid by each main investment, and then to sum the final values of each mini-investment.

The big question now is: What guess do we make?

We could assume the mini-investments will earn the same 15.99 percent CAGR as the IRR of the main investment, in which case we would be indifferent between business A and business B, according to the internal logic of the IRR calculation. Things could shake out exactly that way, but they almost certainly won’t.

We could assume the CAGR on reinvested cash flows will be higher than 15.99 percent, but that raises a question: if we’re so confident we can earn more than 15.99 percent on our money starting in one year’s time, why are we slumming among investments with a mere 15.99 percent IRR?

We’re left with the more conservative and logical assumption: that we’ll earn a lower-than-the-IRR rate of return on reinvested cash flows. It may well be a more likely assumption as well, because as you grow your capital base in a world of scarce opportunities, the opportunities tend to get scarcer. So let us assume we’ll earn say 12 percent on the reinvested dividends of each of business A and B. Are we still indifferent?

The answer is no. When you make that assumption and run the numbers, higher-growing business B ends up producing a higher CAGR, 13.5 percent vs. 12.5 percent…

…In a sense—and sometimes in literal fact—the high-growing investment does the reinvesting for you.

4. What OpenAI Really Wants – Steven Levy

For Altman and his company, ChatGPT and GPT-4 are merely stepping stones along the way to achieving a simple and seismic mission, one these technologists may as well have branded on their flesh. That mission is to build artificial general intelligence—a concept that’s so far been grounded more in science fiction than science—and to make it safe for humanity. The people who work at OpenAI are fanatical in their pursuit of that goal. (Though, as any number of conversations in the office café will confirm, the “build AGI” bit of the mission seems to offer up more raw excitement to its researchers than the “make it safe” bit.) These are people who do not shy from casually using the term “super-intelligence.” They assume that AI’s trajectory will surpass whatever peak biology can attain. The company’s financial documents even stipulate a kind of exit contingency for when AI wipes away our whole economic system.

It’s not fair to call OpenAI a cult, but when I asked several of the company’s top brass if someone could comfortably work there if they didn’t believe AGI was truly coming—and that its arrival would mark one of the greatest moments in human history—most executives didn’t think so. Why would a nonbeliever want to work here? they wondered. The assumption is that the workforce—now at approximately 500, though it might have grown since you began reading this paragraph—has self-selected to include only the faithful…

…At the same time, OpenAI is not the company it once was. It was founded as a purely nonprofit research operation, but today most of its employees technically work for a profit-making entity that is reportedly valued at almost $30 billion. Altman and his team now face the pressure to deliver a revolution in every product cycle, in a way that satisfies the commercial demands of investors and keeps ahead in a fiercely competitive landscape. All while hewing to a quasi-messianic mission to elevate humanity rather than exterminate it…

…But the leaders of OpenAI swear they’ll stay the course. All they want to do, they say, is build computers smart enough and safe enough to end history, thrusting humanity into an era of unimaginable bounty…

…“AGI was going to get built exactly once,” he told me in 2021. “And there were not that many people that could do a good job running OpenAI. I was lucky to have a set of experiences in my life that made me really positively set up for this.”

Altman began talking to people who might help him start a new kind of AI company, a nonprofit that would direct the field toward responsible AGI. One kindred spirit was Tesla and SpaceX CEO Elon Musk. As Musk would later tell CNBC, he had become concerned about AI’s impact after having some marathon discussions with Google cofounder Larry Page. Musk said he was dismayed that Page had little concern for safety and also seemed to regard the rights of robots as equal to humans. When Musk shared his concerns, Page accused him of being a “speciesist.” Musk also understood that, at the time, Google employed much of the world’s AI talent. He was willing to spend some money for an effort more amenable to Team Human.

Within a few months Altman had raised money from Musk (who pledged $100 million, and his time) and Reid Hoffman (who donated $10 million). Other funders included Peter Thiel, Jessica Livingston, Amazon Web Services, and YC Research. Altman began to stealthily recruit a team. He limited the search to AGI believers, a constraint that narrowed his options but one he considered critical. “Back in 2015, when we were recruiting, it was almost considered a career killer for an AI researcher to say that you took AGI seriously,” he says. “But I wanted people who took it seriously.”

Greg Brockman, the chief technology officer of Stripe, was one such person, and he agreed to be OpenAI’s CTO. Another key cofounder would be Andrej Karpathy, who had been at Google Brain, the search giant’s cutting-edge AI research operation. But perhaps Altman’s most sought-after target was a Russian-born engineer named Ilya Sutskever…

…Sutskever became an AI superstar, coauthoring a breakthrough paper that showed how AI could learn to recognize images simply by being exposed to huge volumes of data. He ended up, happily, as a key scientist on the Google Brain team.

In mid-2015 Altman cold-emailed Sutskever to invite him to dinner with Musk, Brockman, and others at the swank Rosewood Hotel on Palo Alto’s Sand Hill Road. Only later did Sutskever figure out that he was the guest of honor. “It was kind of a general conversation about AI and AGI in the future,” he says. More specifically, they discussed “whether Google and DeepMind were so far ahead that it would be impossible to catch up to them, or whether it was still possible to, as Elon put it, create a lab which would be a counterbalance.” While no one at the dinner explicitly tried to recruit Sutskever, the conversation hooked him…

…OpenAI officially launched in December 2015. At the time, when I interviewed Musk and Altman, they presented the project to me as an effort to make AI safe and accessible by sharing it with the world. In other words, open source. OpenAI, they told me, was not going to apply for patents. Everyone could make use of their breakthroughs. Wouldn’t that be empowering some future Dr. Evil? I wondered. Musk said that was a good question. But Altman had an answer: Humans are generally good, and because OpenAI would provide powerful tools for that vast majority, the bad actors would be overwhelmed…

…Had I gone in and asked around, I might have learned exactly how much OpenAI was floundering. Brockman now admits that “nothing was working.” Its researchers were tossing algorithmic spaghetti toward the ceiling to see what stuck. They delved into systems that solved video games and spent considerable effort on robotics. “We knew what we wanted to do,” says Altman. “We knew why we wanted to do it. But we had no idea how.”…

…OpenAI’s road to relevance really started with its hire of an as-yet-unheralded researcher named Alec Radford, who joined in 2016, leaving the small Boston AI company he’d cofounded in his dorm room. After accepting OpenAI’s offer, he told his high school alumni magazine that taking this new role was “kind of similar to joining a graduate program”—an open-ended, low-pressure perch to research AI.

The role he would actually play was more like Larry Page inventing PageRank.

Radford, who is press-shy and hasn’t given interviews on his work, responds to my questions about his early days at OpenAI via a long email exchange. His biggest interest was in getting neural nets to interact with humans in lucid conversation. This was a departure from the traditional scripted model of making a chatbot, an approach used in everything from the primitive ELIZA to the popular assistants Siri and Alexa—all of which kind of sucked. “The goal was to see if there was any task, any setting, any domain, any anything that language models could be useful for,” he writes. At the time, he explains, “language models were seen as novelty toys that could only generate a sentence that made sense once in a while, and only then if you really squinted.” His first experiment involved scanning 2 billion Reddit comments to train a language model. Like a lot of OpenAI’s early experiments, it flopped. No matter. The 23-year-old had permission to keep going, to fail again. “We were just like, Alec is great, let him do his thing,” says Brockman.

His next major experiment was shaped by OpenAI’s limitations of computer power, a constraint that led him to experiment on a smaller data set that focused on a single domain—Amazon product reviews. A researcher had gathered about 100 million of those. Radford trained a language model to simply predict the next character in generating a user review.

But then, on its own, the model figured out whether a review was positive or negative—and when you programmed the model to create something positive or negative, it delivered a review that was adulatory or scathing, as requested. (The prose was admittedly clunky: “I love this weapons look … A must watch for any man who love Chess!”) “It was a complete surprise,” Radford says. The sentiment of a review—its favorable or disfavorable gist—is a complex function of semantics, but somehow a part of Radford’s system had gotten a feel for it. Within OpenAI, this part of the neural net came to be known as the “unsupervised sentiment neuron.”

Sutskever and others encouraged Radford to expand his experiments beyond Amazon reviews, to use his insights to train neural nets to converse or answer questions on a broad range of subjects.

And then good fortune smiled on OpenAI. In early 2017, an unheralded preprint of a research paper appeared, coauthored by eight Google researchers. Its official title was “Attention Is All You Need,” but it came to be known as the “transformer paper,” named so both to reflect the game-changing nature of the idea and to honor the toys that transmogrified from trucks to giant robots. Transformers made it possible for a neural net to understand—and generate—language much more efficiently. They did this by analyzing chunks of prose in parallel and figuring out which elements merited “attention.” This hugely optimized the process of generating coherent text to respond to prompts. Eventually, people came to realize that the same technique could also generate images and even video. Though the transformer paper would become known as the catalyst for the current AI frenzy—think of it as the Elvis that made the Beatles possible—at the time Ilya Sutskever was one of only a handful of people who understood how powerful the breakthrough was…

…Radford began experimenting with the transformer architecture. “I made more progress in two weeks than I did over the past two years,” he says. He came to understand that the key to getting the most out of the new model was to add scale—to train it on fantastically large data sets. The idea was dubbed “Big Transformer” by Radford’s collaborator Rewon Child.

This approach required a change of culture at OpenAI and a focus it had previously lacked. “In order to take advantage of the transformer, you needed to scale it up,” says Adam D’Angelo, the CEO of Quora, who sits on OpenAI’s board of directors…

…The name that Radford and his collaborators gave the model they created was an acronym for “generatively pretrained transformer”—GPT-1. Eventually, this model came to be generically known as “generative AI.” To build it, they drew on a collection of 7,000 unpublished books, many in the genres of romance, fantasy, and adventure, and refined it on Quora questions and answers, as well as thousands of passages taken from middle school and high school exams. All in all, the model included 117 million parameters, or variables. And it outperformed everything that had come before in understanding language and generating answers. But the most dramatic result was that processing such a massive amount of data allowed the model to offer up results beyond its training, providing expertise in brand-new domains. These unplanned robot capabilities are called zero-shots. They still baffle researchers—and account for the queasiness that many in the field have about these so-called large language models.

Radford remembers one late night at OpenAI’s office. “I just kept saying over and over, ‘Well, that’s cool, but I’m pretty sure it won’t be able to do x.’ And then I would quickly code up an evaluation and, sure enough, it could kind of do x.”

Each GPT iteration would do better, in part because each one gobbled an order of magnitude more data than the previous model. Only a year after creating the first iteration, OpenAI trained GPT-2 on the open internet with an astounding 1.5 billion parameters. Like a toddler mastering speech, its responses got better and more coherent…

…So in March 2019, OpenAI came up with a bizarre hack. It would remain a nonprofit, fully devoted to its mission. But it would also create a for-profit entity. The actual structure of the arrangement is hopelessly baroque, but basically the entire company is now engaged in a “capped’’ profitable business. If the cap is reached—the number isn’t public, but its own charter, if you read between the lines, suggests it might be in the trillions—everything beyond that reverts to the nonprofit research lab…

…Potential investors were warned about those boundaries, Lightcap explains. “We have a legal disclaimer that says you, as an investor, stand to lose all your money,” he says. “We are not here to make your return. We’re here to achieve a technical mission, foremost. And, oh, by the way, we don’t really know what role money will play in a post-AGI world.”

That last sentence is not a throwaway joke. OpenAI’s plan really does include a reset in case computers reach the final frontier. Somewhere in the restructuring documents is a clause to the effect that, if the company does manage to create AGI, all financial arrangements will be reconsidered. After all, it will be a new world from that point on. Humanity will have an alien partner that can do much of what we do, only better. So previous arrangements might effectively be kaput.

There is, however, a hitch: At the moment, OpenAI doesn’t claim to know what AGI really is. The determination would come from the board, but it’s not clear how the board would define it. When I ask Altman, who is on the board, for clarity, his response is anything but open. “It’s not a single Turing test, but a number of things we might use,” he says. “I would happily tell you, but I like to keep confidential conversations private. I realize that is unsatisfyingly vague. But we don’t know what it’s going to be like at that point.”…

…The shift also allowed OpenAI’s employees to claim some equity. But not Altman. He says that originally he intended to include himself but didn’t get around to it. Then he decided that he didn’t need any piece of the $30 billion company that he’d cofounded and leads. “Meaningful work is more important to me,” he says. “I don’t think about it. I honestly don’t get why people care so much.”

Because … not taking a stake in the company you cofounded is weird?

“If I didn’t already have a ton of money, it would be much weirder,” he says. “It does seem like people have a hard time imagining ever having enough money. But I feel like I have enough.” (Note: For Silicon Valley, this is extremely weird.) Altman joked that he’s considering taking one share of equity “so I never have to answer that question again.”…

…Obviously, only a few companies in existence had the kind of resources OpenAI required. “We pretty quickly zeroed in on Microsoft,” says Altman. To the credit of Microsoft CEO Satya Nadella and CTO Kevin Scott, the software giant was able to get over an uncomfortable reality: After more than 20 years and billions of dollars spent on a research division with supposedly cutting-edge AI, the Softies needed an innovation infusion from a tiny company that was only a few years old. Scott says that it wasn’t just Microsoft that fell short—“it was everyone.” OpenAI’s focus on pursuing AGI, he says, allowed it to accomplish a moonshot-ish achievement that the heavy hitters weren’t even aiming for. It also proved that not pursuing generative AI was a lapse that Microsoft needed to address. “One thing you just very clearly need is a frontier model,” says Scott.

Microsoft originally chipped in a billion dollars, paid off in computation time on its servers. But as both sides grew more confident, the deal expanded. Microsoft now has sunk $13 billion into OpenAI. (“Being on the frontier is a very expensive proposition,” Scott says.)

Of course, because OpenAI couldn’t exist without the backing of a huge cloud provider, Microsoft was able to cut a great deal for itself. The corporation bargained for what Nadella calls “non-controlling equity interest” in OpenAI’s for-profit side—reportedly 49 percent. Under the terms of the deal, some of OpenAI’s original ideals of granting equal access to all were seemingly dragged to the trash icon. (Altman objects to this characterization.) Now, Microsoft has an exclusive license to commercialize OpenAI’s tech. And OpenAI also has committed to use Microsoft’s cloud exclusively. In other words, without even taking its cut of OpenAI’s profits (reportedly Microsoft gets 75 percent until its investment is paid back), Microsoft gets to lock in one of the world’s most desirable new customers for its Azure web services. With those rewards in sight, Microsoft wasn’t even bothered by the clause that demands reconsideration if OpenAI achieves general artificial intelligence, whatever that is. “At that point,” says Nadella, “all bets are off.” It might be the last invention of humanity, he notes, so we might have bigger issues to consider once machines are smarter than we are…

..Altman explains why OpenAI released ChatGPT when GPT-4 was close to completion, undergoing safety work. “With ChatGPT, we could introduce chatting but with a much less powerful backend, and give people a more gradual adaptation,” he says. “GPT-4 was a lot to get used to at once.” By the time the ChatGPT excitement cooled down, the thinking went, people might be ready for GPT-4, which can pass the bar exam, plan a course syllabus, and write a book within seconds…

…But if OpenAI’s products were forcing people to confront the implications of artificial intelligence, Altman figured, so much the better. It was time for the bulk of humankind to come off the sidelines in discussions of how AI might affect the future of the species…

…As one prominent Silicon Valley founder notes, “It’s rare that an industry raises their hand and says, ‘We are going to be the end of humanity’—and then continues to work on the product with glee and alacrity.”

OpenAI rejects this criticism. Altman and his team say that working and releasing cutting-edge products is the way to address societal risks. Only by analyzing the responses to millions of prompts by users of ChatGPT and GPT-4 could they get the knowledge to ethically align their future products…

…It would also help if generative AI didn’t create so many new problems of its own. For instance, LLMs need to be trained on huge data sets; clearly the most powerful ones would gobble up the whole internet. This doesn’t sit well with some creators, and just plain people, who unwittingly provide content for those data sets and wind up somehow contributing to the output of ChatGPT. Tom Rubin, an elite intellectual property lawyer who officially joined OpenAI in March, is optimistic that the company will eventually find a balance that satisfies both its own needs and that of creators—including the ones, like comedian Sarah Silverman, who are suing OpenAI for using their content to train its models. One hint of OpenAI’s path: partnerships with news and photo agencies like the Associated Press and Shutterstock to provide content for its models without questions of who owns what.

5. Inside Intel’s Chip Factory, I Saw the Future. It’s Plain Old Glass – Stephen Shankland

But the next breakthrough to make our laptops more efficient and AI more powerful could come from plain old glass. I’ve just seen firsthand how it works…

…There, in a hulking white high-tech building in the Phoenix area’s scorching desert landscape, Intel transforms sheets of glass the size of a small tabletop into paperclip-sized rectangular sandwiches of circuitry built with some of the same techniques as the processor itself.

Intel has begun a years-long transition to new technology that rests processors on a bed of glass instead of today’s epoxy-like organic resin. The new glass foundation, called a substrate, offers the speed, power and real estate necessary for the chip industry’s shift to new technology packaging multiple “chiplets” into a single larger processor.

In short, that means a new way to sustain Moore’s Law, which charts progress in cramming more circuitry elements called transistors into a processor. The A17 Pro processor in Apple’s new iPhone 15 Pro has 19 billion transistors. Intel’s Ponte Vecchio supercomputing processor has more than 100 billion. By the end of the decade, Intel expects processors with — if you can imagine it — a trillion transistors.

Intel relied on this chiplet approach to catch up to competitors with superior processor manufacturing abilities. But now Intel can use it to outpace rivals in an era when exploding demand for new processing power has surpassed the industry’s ability to deliver it, said Creative Strategies analyst Ben Bajarin. And Intel’s glass substrate technology demonstrates Intel’s packaging prowess…

…The whole chip industry will make the glass transition at least for high-end processors to cope with chipmaking challenges, and Intel has the lead, said FeibusTech analyst Mike Feibus…

…”Basically, the innovation is done,” said Ann Kelleher, the executive vice president leading technology development at Intel. The glass substrate technology “gives us an ability to ultimately get higher performance for our products.”…

…The glass technology underneath a processor won’t arrive until the second half of the decade, and when it does, it’ll appear first underneath the biggest, most power-hungry chips, the ones that perch in thousands of servers stacked up in data centers operated by huge “hyperscalers” like Google, Amazon, Microsoft and Meta.

That’s because glass brings several advantages to these hot and huge chips, said Rahul Manepalli, an Intel fellow who leads Intel’s module engineering work.

It can accommodate 10 times the power and data connections as today’s organic substrates so more data can be pumped in and out of a chip. It doesn’t warp as much, critical to ensuring processors lie flat and connect properly to the outside world and thus enabling 50% larger chip packages. It transmits power with less waste, meaning chips can run either faster or more efficiently. And it can run at a higher temperature, and when it heats up, it expands at the same rate as silicon to avoid mechanical failures.

Glass will enable a new generation of server and data center processors, successors to mammoth beasts like the Intel Xeons that can run cloud computing services like email and online banking and Nvidia’s artificial intelligence processors that have exploded in popularity as the world embraces generative AI.

But as glass substrates mature and costs come down, it’ll spread beyond data centers to the computer sitting on your lap…

…Intel’s 8086 chip, the 1978 precursor to every PC and server processor that Intel has made since, was a flat square of silicon with 29,000 transistors. To protect it and plug it into a circuit board, it was housed in a package that looked like a flat caterpillar. Forty metal legs carried power and data to the chip.

Since then, processor packaging has advanced dramatically. It once was relatively crude, but now the boundary between chipmaking and packaging are blurring, Kelleher said. Packaging processes now use lithography machines to etch their own circuitry, although not nearly as finely as on processors…

…So today’s packages have flat metal contact patches on the bottom of the package. The chip is installed when hundreds of pounds of force mash it onto a circuit board.

A metal cap atop a processor draws away waste heat that otherwise would crash a computer. And beneath the processor is a substrate with an increasingly complex, three-dimensional network of power and data connections to link the chip to the outside world.

There are challenges moving from today’s organic substrates to glass. Glass is brittle, so it must be handled carefully, for example.

To ease the transition, Intel is adapting glass-handling equipment from experts who already know how to handle it without breaking: the display industry, which makes everything from tiny smartwatch screens to enormous flat-panel TVs. They also have to etch circuitry onto glass and have developed many of the needed ultrapure materials and careful handling processes.

But there are differences. Flat-panel displays have sensitive electronic elements only on one side, so glass can glide through factories on rollers. Intel builds a sandwich of materials and circuitry called redistribution layers onto both sides of the glass, so its machines must in effect hold the glass only by the edges…

…Signing on a packaging customer is a bit easier than signing on a chipmaking customer, with fewer technology complications and shorter lead times, he said.

But in customer deals for packaging can lead to the deeper relationship that extends into chipmaking, in particular with the Intel 18A chipmaking process the company expects will surpass TSMC and Samsung in 2024.

“It’s a foot in the door,” Gardner said. “There’s one customer in particular [for which] that trajectory of packaging first and advanced packaging then 18A is working well.”…

…It’s unclear how much of the processor business will move from “monolithic,” single-die designs to chiplet designs. There are still cost and simplicity advantages to avoiding advanced packaging. But it’s clear the biggest processors — the server and AI brains in data centers — will become sprawling complexes of interlinked chiplets.

And that’s where glass substrates should come in handy, with enough area, communication links and power delivery abilities to give chip designers room for growth.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google), Amazon, Apple, ASML, Meta Platforms, Microsoft, Tesla, and TSMC. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 17 September 2023:

1. How bonds ate the entire financial system – Robin Wigglesworth

“The bond market is the most important market in the world,” says Ray Dalio, the founder of the world’s largest hedge fund, Bridgewater. “It is the backbone of all other markets.”

While the bond market has become larger and more powerful, the importance of banks — historically the workhorses of the capitalist system — is subtly fading. The global bond market was worth about $141tn at the end of 2022. That is, for now, smaller than the $183tn that the Financial Stability Board estimates banks hold globally, but much of the latter is actually invested in bonds — a fact that some US banks have recently rued…

…The market is now facing one of its biggest tests in generations. Last year, resurgent inflation — the nemesis of financial securities that pay fixed interest rates — triggered the worst setback in at least a century. Overall losses were almost $10tn, shaking UK pension plans and regional banks in the US. And although bonds have regained their footing this year, they are still beset by rising interest rates.

Even if the bond market adapts, as it has in the past, its ballooning power, reach and complexity has some awkward implications for the global economy. “This transformation has been extraordinary, and positive,” says Larry Fink, head of BlackRock, the world’s biggest investment group. “But we have a regulatory system designed for a time when banks were the dominant players. They aren’t any more.”

“Shadow banking” is what some academics call the part of the financial system that resembles, but falls outside traditional banking. Policymakers prefer the less malevolent-sounding — but almost comically obtuse — term “non-bank financial institutions”. At $240tn, this system is now far bigger than its conventional counterpart. The bond market is its main component, taking money from investors who can mostly yank it away at short notice and funnel it into long-term investments.

The question of how to tame shadow banking is one of the thorniest topics in finance today. For the financial system as a whole, it is arguably better that the risks bonds inevitably entail are spread across a vast, decentralised web of international investors, rather than concentrated in a narrow clutch of banks. But in finance, risk is like energy. It cannot be destroyed, only shifted from one place to another. As it gets shunted around, its consequences can morph in little understood, even dangerous ways. We saw a perfect example of this in March 2020, when the Covid-19 pandemic acted as a gigantic stress test for the financial system that revealed fresh cracks in its foundation…

…Doge Vitale II Michiel of Venice was in a pickle. Under a dubious pretext, the Byzantine empire had in 1171 arrested all Venetian merchants in its capital Constantinople and seized their property. But the Italian city-state didn’t have the funds to send a navy to rescue its imprisoned citizens. So the Doge forced all citizens to lend the city some money, in return for 5 per cent interest a year until they were repaid.

The rescue mission did not go well. The Venetian fleet was devastated by plague while negotiating with Constantinople, and the Doge was forced to return humiliated. Back in Venice, irate subjects chased their ruler down the city’s streets and beat him to death. Ruined by the debacle, Venice was unable to repay, turning the emergency prestiti (loan) into a permanent fixture that paid 5 per cent annually.

Most people were eventually fine with this arrangement. The steady interest payments were quite attractive. Occasionally, Venice would raise more prestiti, and the one-time emergency facility gradually became a handy way of raising money…

…Another crucial difference is that bonds are designed to be traded, while loans are typically not. In 12th-century Venice, prestiti were bought and sold in the city’s Rialto market. Today, bond trading happens by phone, electronic messages and algorithms across the world’s financial centres. This tradability is central to the growth of bonds, as it allows creditors to shift the risk to someone else.

By the 19th century, bond markets had helped shape the world order. Countries that could best finance themselves tended to succeed. England’s victory over Napoleonic France was enabled by its bond market, which allowed it to finance wartime expenditures more effectively than did the local bankers that Paris depended on for short-term, high-interest loans…

…The aftermath of the second world war was unkind to the bond market. Although it had provided vital wartime funding for allied governments and remained one of the financial system’s most crucial cogs, accelerating inflation in the 1950s and 60s posed a challenge for securities with fixed interest rates. By the 1970s, buying bonds became a constant, brutal race to stay ahead of inflation’s return-eroding force. The aggressive central bank-rate increases that became necessary to tame runaway prices also lowered the value of bonds issued in a lower-rate environment. People dourly joked that bonds had become “certificates of confiscation”.

But the 1980s brought a new era of slowing inflation, falling rates, regulatory liberalism and financial innovation, which would transform the bond market…

…But what made Ranieri’s name was not his persona. Wall Street has had plenty of bombastic bond traders with a penchant for coarse practical jokes. It was what he did to make a dime: packaging up individual mortgages into bonds and then trading chunks of those bonds, a process known as securitisation.

Securitisation is an old concept. Back in 1774, the very first mutual fund bought bonds backed by loans from plantations in the Caribbean and toll roads in Denmark. US mortgage-backed bonds existed as early as the 19th century. But these bonds only used the underlying loans as collateral.

In 1970, the US Government National Mortgage Association (known as Ginnie Mae) engineered the first “passthrough” mortgage-backed securities, where the underlying individual loan payments flowed directly through to the bond investor. This was followed by similar deals by other US mortgage agencies such as Freddie Mac and Fannie Mae, to little fanfare. Ranieri did for securitisation what Milken had done for the junk bond market; he transformed it from the backwaters into a global and massively lucrative industry.

The first fillip was the crisis that struck the US “savings and loans” industry when the Federal Reserve ratcheted up rates in the early 1980s. Congress passed a jammy tax break to make it easier for the banks to shed entire portfolios of mortgages at fire-sale prices. Ranieri’s Salomon was there to scoop them up and flip them to other investors. Money began to course through Salomon’s mortgage trading desk.

Ranieri realised that he needed to turn a one-off vein into an entire gold mine that could be exploited year after year. Luckily, he found some in-house inspiration: an innovative deal his former boss Dall had done with Bank of America in 1977, which sought to tackle the difficulty of valuing the cash flows of mortgage-backed securities with a technique called “tranching”. It sliced them up into different portions each with their own interest rates, maturities and riskiness. That way, each investor could simply choose what kind of exposure they might like — a buffet rather than a set-course meal of variable quality.

Ranieri ran with the idea. Rather than just take the mortgage of one bank, he pooled together bunches of mortgages from lots of them. To handle the complexity, he hired a lot of bright young mathematicians to complement the mini-Ranieris on the trading desk. He then lobbied vociferously for government blessing of the tranching structure, knowing this would add to the products’ lustre with investors. He succeeded. By the mid-1980s, the market took off…

…The story had an unhappy ending: the new market proliferated until it nearly brought the global financial system down in 2008, something that later weighed on Ranieri. “I will never, ever, ever, ever live out that scar that I carry for what happened with something I created,” he told The Wall Street Journal in 2018.

But the fundamental idea — packaging up smaller loans into bigger bonds and thereby bringing together more people who needed money with those who had it — was sound. Done judiciously, it actually makes banks less risky, by shifting the inherent danger of extending loans out of banks and into markets. (This is why securitisation has bounced back since 2008, and is starting to gain ground outside the US as well, often with government encouragement.)…

…Given how disastrous bank crises can be, it could be a good thing that bonds nowadays are doing more of the heavy lifting. Unlike bank depositors, bond fund investors do not expect to get their money back (even if it can be a shock when things fall apart). And unlike banks, bond funds typically do not use much or even any leverage.

But bond crises can also be painful — as we saw in both 2008 and nearly in 2020. Modern capitalism has largely been ordered around banks as the main intermediaries of money. Central banks were mostly set up to backstop these commercial banks, and, eventually, they began trying to regulate the temperature of economies by tweaking the cost of their funding, moving overnight interest rates up and down. But with the rise of bond markets, entirely new challenges have emerged and experimental tools to deal with them have become necessary — most notably quantitative easing, negative interest rates and “yield curve control”.

If the ultimate goal is to regulate the temperature of an economy by changing the cost of credit, then the fact that credit is increasingly extended by the bond market rather than banks inevitably has consequences. The market’s decentralised nature means that dangers can be harder to monitor and address, requiring massive, untargeted “spray-and-pray” monetary responses by central banks when trouble erupts.

Unfortunately, the custodians of the financial system have yet to fully grapple with those consequences, even if everyone from the Federal Reserve to the IMF has repeatedly warned about the multi-faceted dangers the shift from banks to bonds entails. 

2. Searching for Resilience – Michael Weeks

For a business to survive 260 years in the same industry, with the same family owners, is a remarkable achievement. Starting in 1761 as a one-man shop making lead pencils, Faber-Castell has grown into the largest producer of colored and graphite pencils globally, producing over 2 billion pencils each year, as well as pens, markers, highlighters, and related products

Already the leading pencil producer in the mid-1800s, Faber-Castell has stayed on top of their industry for close to two centuries, betraying incredible entrepreneurial ability and drive. When the English supply of graphite began to fail and pencils became unaffordable, they bought a Siberian graphite mine and relied in part on reindeer transport to bring new raw materials to their factories. They expanded their product catalog, built up operations across Europe and the Americas, and invested in new technologies and equipment to improve their production. They helped introduce trademark law in Germany to protect their reputation against competitors. They established a 10’000 hectare forest plantation in Brazil to ensure their wood supply. They took their business seriously.

Nine generations of history also come with hardship. When the Americans joined World War I, Faber-Castell was cut off from the US market despite having operated there since in the 1850s. All of their US assets—land, equipment, inventory, patents, and trademarks—were seized and sold at auction after the war ended. During World War II their largest factory in Brazil was seized, not to be recovered for another twenty years, while their German factories were commandeered by the Nazi war machine. And in 1971, after 95 years of building a reputation as the finest producer of slide rules, they saw this entire side business vanish almost overnight when the pocket calculator was commercialized.

Resilience—or, the ability to survive hard times, as Faber-Castell has demonstrated time and time again—is something that we value instinctively. Yet, it’s not a popular subject. For all the years we’ve heard talk of sustainability, it seems that economic resilience, a once important dimension of economic prosperity, has become a relic of the past…

…What makes resilience so hard to spot is that it can only be proven during those rare times of crisis…

…It follows that resilience is not the same as looking good or having predictable financial results…

…Worse, a steady business model can actually become a source of fragility if placed in the wrong hands, as when companies with highly regular income streams leverage up their balance sheets, providing more immediate returns to their owners at the expense of their own resilience. Private equity has perhaps perfected this business model, but the growing dependence on debt in all walks of life reveal this as a defining feature of modern times…

…Resilience is not a destination. Seeking resilience means abandoning a narrow definition of success like sales growth or annual returns and instead becoming prepared for any eventuality that can cause serious harm. It is gained by pursuing new capabilities and flexibility, and by avoiding landmines. It means creating new options for the future instead of more plans for the present…

…Resilience comes at a cost. We are reminded of a metaphor used by Nassim Nicholas Taleb dealing with the nature of redundancy: “Layers of redundancy are the central risk management property of natural systems.”…

…A more effective way to add resilience to one’s savings is by ruthlessly avoiding its opposite: economic fragility. Thankfully, unlike resilience, fragility is often staring you in the face. This is where financial analysis really starts to shine. Is the company dependent on a few key customers and suppliers? Is the company overleveraged or buying back shares at indecent prices, just because it can? Are there elements of pricing power, or do their earnings evaporate at the first sign of trouble? Could a government ruling or decree suddenly break their business? Can their customers really afford to buy their products next year?…

…Perhaps the best way to find resilience is to look for its source: Resilience only comes from owners. Resilience is not a fluke that one stumbles into. It is a deliberate and purposeful objective which some aim for and others don’t. A good business plan, a profitable sector, a lot of cash, loyal management, or hardworking employees may all be wonderful, but only owners have the time horizon required to balance the present against an unknowable future, and only they have skin in the game—their own savings on the line. Unlike investors (renters), owners have no easy exits. They must build up reserves and competencies in the good years to give them options in the bad. They are motivated by a sense of responsibility—to themselves, their families, those they work with, and those who will come after them…

…Bakkafrost is a vertically integrated salmon farmer operating in the Faroe Islands and Scotland…

…A choice every salmon farmer has to make is what to do with the fish once it is ready to harvest. Do they sell their fish to wholesalers and other processors, or do they take it a step further, converting some into filets or smoked salmon that goes straight to the grocery store? This latter step is called Value-Added Processing or VAP, and Bakkafrost aims to sell about 30–40% of its fish through this channel each year.

A choice every salmon farmer has to make is what to do with the fish once it is ready to harvest. Do they sell their fish to wholesalers and other processors, or do they take it a step further, converting some into filets or smoked salmon that goes straight to the grocery store? This latter step is called Value-Added Processing or VAP, and Bakkafrost aims to sell about 30–40% of its fish through this channel each year. In the ten years from 2011 to 2020, Bakkafrost has reported cumulative revenues of about €4.5 billion and operating earnings of €1.1 billion, yet of those earnings only €14 million, or 1.2% of the total, have come from their VAP division. A financial observer might say, quite rightly, why bother? Salmon farming is hard enough, why dedicate additional capital and resources for a pittance? What a financial owner doesn’t see— and which the owners of Bakkafrost see plain as day—is the resilience this seemingly irrelevant processing step embeds in the organization…

…When hotels and restaurants shut their doors last year, all the food that was headed to this channel, including many millions of whole salmon, all needed to end up somewhere. Remember the 30-month lag between laying eggs and the salmon harvest? This means that while the demand for whole salmon evaporated, supply kept pouring in and the markets were soon stuffed full of whole fish with no one around to buy them…

…When hotels and restaurants shut their doors last year, all the food that was headed to this channel, including many millions of whole salmon, all needed to end up somewhere. Remember the 30-month lag between laying eggs and the salmon harvest? This means that while the demand for whole salmon evaporated, supply kept pouring in and the markets were soon stuffed full of whole fish with no one around to buy them. Covid in mind, but they did understand and value the importance of adding resilience to their business.

3. Dangerous CFOs, Imperial CEOs, Chagrined Bankers, and Warren Buffett – Dan Noe

While I was a credit analyst and manager at Moody’s, I met with many CEOs and CFOs. Most meetings were routine and executives were good at explaining their company’s business and financial strategies. They almost always put their best foot forward. But the exceptions were notable…

…The most revealing comment I ever heard in a meeting came from a savings and loan CEO during that industry’s crisis in the late 1980s. S&L holding companies had issued a lot of junk bonds and God knows what they did with the money. A lot of them were going under. During one meeting, this particular S&L CEO said, apropos of nothing, “I hope the feds never figure out what I’m doing.” His bankers looked like they were going to throw up. That was an example of an in-person meeting affecting our credit assessment…

…The worst case of self-importance was a regional bank CEO who insisted we meet him in his big suite at a fancy hotel and have breakfast. This was weird, and a member of his entourage tried to explain: “He needs anonymity while he is in New York.” I said, “Well, he’s got it. He works at a Midwest bank. Nobody here knows who he is.”

I did these meetings for years and felt like I’d seen it all. Then, one day, I met the antithesis of imperial CEOs and executives who have problems answering questions about their business. Warren Buffett and Charlie Munger from Berkshire Hathaway came in for a visit. They arrived in a taxi, not a limo, with no hangers-on, not one person with them at all. When I addressed him as “Mr. Buffett” he said, “Please call me Warren.” They had come in to talk about a debt issuance, and Buffett made a self-deprecating joke: “My mother always told me to avoid liquor, ladies, and leverage. I’ve avoided the first two, but sometimes I like a little leverage.” Our analyst, Weston Hicks, was an excellent insurance industry analyst and asked detailed and probing questions. Buffett and Munger spent an hour, an hour-and-a-half, giving very specific answers. When the meeting ended and we were walking to the elevators, Munger said to me, out of earshot of Weston, “Make sure you keep that analyst. He’s really good.”

4. Product-Led AI – Seth Rosenberg

I believe there’s tremendous value to be captured by product builders who can successfully put the power of AI into products that people love. As my partner Jerry Chen recently put forth, if we’re living in an age where foundation models make it possible for anyone to build an AI company, “the most strategic advantage [of applications] is that you can coexist with several systems of record and collect all the data that passes through your product.”…

…Of course there are plenty of detractors who don’t believe startups have a chance at this layer – incumbents own the data and distribution, and access to LLMs is both commoditized and fraught with platform risk. There will likely be many casualties of companies where an API call to OpenAI isn’t sufficient to build lasting value…

…In the last wave of consumer software, social networks and marketplaces were the dominant business models that created trillions of dollars of market cap, with Meta alone valued at just under $800 billion. Greylock was lucky to back many of these, including Meta, LinkedIn, Roblox, Airbnb, Discord, Musical.ly (now TikTok), and Nextdoor.

As reflected by the valuations, these networks were assumed to be “unbreakable”.

But now, AI challenges many of our initial assumptions. This is creating a new arms race to build the next AI-first network.

We moved from networks that connect people to algorithms that connect people to content. Now, we’re moving to algorithms that replace people…

…You can imagine a freelance logo design marketplace, like parts of Fiverr, will be replaced with an algorithm. A user inputs a prompt, and after a few tries, gets their logo. In this case, the data the algorithm receives is fairly shallow (prompts and selection), and the supply side is entirely replaced by an algorithm.

Contrast this to an AI-first jobs marketplace. The optimal product would be an AI career coach for job seekers and an AI assistant for recruiters – two seemingly separate products, connected by the same algorithm. The coach could gather deep insight from a job seeker – far beyond what they would share on a resume or LinkedIn – and use this data to not just find the perfect match, but help them discover their most fulfilling career path. Combine this data with a strong understanding of a recruiter’s needs, and both the coach and assistant get better…

…The best opportunities for start-ups attacking large software categories comes from finding angles where incumbents can’t compete. Here are four examples:

1. UI/UX is re-imagined with AI – incumbent UI is irrelevant
2. Product surface area is re-imagined with AI – incumbents compete at a different scope
3. Business model is re-imagined with AI – incumbent business model can’t adapt
4. No incumbent tech co before AI…

…Another great example is customer service, a $10 billion software category. The “obvious” starting point would be to automate customer service reps using AI. But what if the entire concept of customer service was re-imagined? Today, most companies actively reduce call volume by hiding the “contact us” button behind 5 menus and an ever-expanding phone tree. But, in a world of AI, every interaction can be cheap, delightful, and revenue-generating. In that world, companies might actively try to speak with their customers.

When I was at Meta in 2016, we tried to remedy this with an AI bot platform. Piloting with KLM airlines, we built an experience where Messenger handled every aspect of the passenger’s journey – boarding pass, customer service, travel recommendations at their destination, etc, all in a single conversation.. Despite amazing feedback, this pilot was shut down because of the cost to serve – but today, LLMs could make these types of interactions possible…

…One of the most interesting new opportunities with AI is going after the vastly larger market for services versus software with AI “co-pilots”. Most knowledge work involves analyzing and transforming data, a task that algorithms are better suited for.

I believe the best opportunities for co-pilots are “branded” sales people, like wealth managers, insurance brokers, and mortgage brokers. Their role involves a lot of text- based coordination, they work across multiple apps, and the ROI of increased efficiency is tangible. Take wealth managers as an example. According to Morgan Stanley, the biggest indicator of client retention for wealth managers is not portfolio performance, but consistency of personalized interactions with clients.

5. Society’s Technical Debt and Software’s Gutenberg Moment – Paul Kedrosky and Eric Norlin

Software has a cost, and it has markets of buyers and sellers. Some of those markets are internal to organizations. But the majority of those markets are external, where people buy software in the form of apps, or cloud services, or games, or even embedded in other objects that range from Ring doorbells to endoscopic cameras for cancer detection. All of these things are software in a few of its myriad forms. 

With these characteristics in mind, you can think of software in a basic price/quantity graph from introductory economics. There is a price and a quantity demanded at that price, and there is a price and quantity at which those two things are in rough equilibrium, as the following figure shows. Of course, the equilibrium point can shift about for many reasons, causing the P/Q intersection to be at higher or lower levels of aggregate demand. If the price is too high we underproduce software (leaving technical debt), and if too low, well … let’s come back to that…

…But technology has a habit of confounding economics. When it comes to technology, how do we know those supply and demand lines are right? The answer is that we don’t. And that’s where interesting things start happening.

Sometimes, for example, an increased supply of something leads to more demand, shifting the curves around. This has happened many times in technology, as various core components of technology tumbled down curves of decreasing cost for increasing power (or storage, or bandwidth, etc.). In CPUs, this has long been called Moore’s Law, where CPUs become more powerful by some increment every 18 months or so. While these laws are more like heuristics than F=ma laws of physics, they do help as a guide toward how the future might be different from the past.

We have seen this over and over in technology, as various pieces of technology collapse in price, while they grow rapidly in power. It has become commonplace, but it really isn’t. The rest of the economy doesn’t work this way, nor have historical economies. Things don’t just tumble down walls of improved price while vastly improving performance. While many markets have economies of scale, there hasn’t been anything in economic history like the collapse in, say, CPU costs, while the performance increased by a factor of a million or more.

To make this more palpable, consider that if cars had improved at the pace computers have, a modern car would:

• Have more than 600 million horsepower
• Go from 0-60 in less than a hundredth of a second
• Get around a million miles per gallon 
• Cost less than $5,000 

And they don’t. Sure, Tesla Plaid is a speedy car, it is nowhere near the above specs—no car ever will be. This sort of performance inflection is not our future, but it fairly characterizes and even understates what has happened in technology over the last 40 years.

… And each of these collapses has had broader consequences. The collapse of CPU prices led us directly from mainframes to the personal computer era; the collapse of storage prices (of all kinds) led inevitably to more personal computers with useful local storage, which helped spark databases and spreadsheets, then led to web services, and then to cloud services. And, most recently, the collapse of network transit costs (as bandwidth exploded) led directly to the modern Internet, streaming video, and mobile apps…

…Each collapse, with its accompanying performance increases, sparks huge winners and massive change, from Intel, to Apple, to Akamai, to Google & Meta, to the current AI boomlet. Each beneficiary of a collapse requires one or more core technologies’ price to drop and performance to soar. This, in turn, opens up new opportunities to “waste” them in service of things that previously seemed impossible, prohibitively expensive, or both…

…Still, the suddenly emergent growth of LLMs has some people spending buckets of time thinking about what service occupations can be automated out of existence, what economists called “displacement” automation. It doesn’t add much to the aggregate store of societal value, and can even be subtractive and destabilizing, a kind of outsourcing-factory-work-to-China moment for white-collar workers. Perhaps we should be thinking less about opportunities for displacement automation and more about opportunities for augmenting automation, the kind of thing that unleashes creativity and leads to wealth and human flourishing.

So where will that come from? We think this augmenting automation boom will come from the same place as prior ones: from a price collapse in something while related productivity and performance soar. And that something is software itself.

By that, we don’t literally mean “software” will see price declines, as if there will be an AI-induced price war in word processors like Microsoft Word, or in AWS microservices. That is linear and extrapolative thinking. Having said that, we do think the current frenzy to inject AI into every app or service sold on earth will spark more competition, not less. It will do this by raising software costs (every AI API call is money in someone’s coffers), while providing no real differentiation, given most vendors will be relying on the same providers of those AI API calls…

… In a hypothetical two-sector economy, when one sector becomes differentially more productive, specialized, and wealth-producing, and the other doesn’t, there is huge pressure to raise wages in the latter sector, lest many employees leave. Over time that less productive sector starts becoming more and more expensive, even though it’s not productive enough to justify the higher wages, so it starts “eating” more and more of the economy.

Economist William Baumol is usually credited with this insight, and for that it is called Baumol’s cost disease. You can see the cost disease in the following figure, where various products and services (spoiler: mostly in high-touch, low-productivity sectors) have become much more expensive in the U.S., while others (non-spoiler: mostly technology-based) have become cheaper…

…But there is another sector being held back by a variant of Baumol’s cost disease, and that is software itself. This may sound contradictory, which is understandable. After all, how can the most productive, wealth-generating, deflationary sector also be the victim of the same malaise it is inflicting on other sectors?

It can, if you think back to the two-sector model we discussed earlier. One sector is semis and CPUs, storage and backbone networks. Those prices are collapsing, requiring fewer people while producing vastly more performance at lower prices. Meanwhile, software is chugging along, producing the same thing in ways that mostly wouldn’t seem vastly different to developers doing the same things decades ago. Yes, there have been developments in the production and deployment of software, but it is still, at the end of the day, hands pounding out code on keyboards. This should seem familiar, and we shouldn’t be surprised that software salaries stay high and go higher, despite the relative lack of productivity. It is Baumol’s cost disease in a narrow, two-sector economy of tech itself.

These high salaries play directly into high software production costs, as well as limiting the amount of software produced, given factor production costs and those pesky supply curves. Startups spend millions to hire engineers; large companies continue spending millions keeping them around. And, while markets have clearing prices, where supply and demand meet up, we still know that when wages stay higher than comparable positions in other sectors, less of the goods gets produced than is societally desirable. In this case, that underproduced good is…software. We end up with a kind of societal technical debt, where far less is produced than is socially desirable—we don’t know how much less, but it is likely a very large number and an explanation for why software hasn’t eaten much of the world yet…

…We think that’s all about to change. The current generation of AI models are a missile aimed, however unintentionally, directly at software production itself. Sure, chat AIs can perform swimmingly at producing undergraduate essays, or spinning up marketing materials and blog posts (like we need more of either), but such technologies are terrific to the point of dark magic at producing, debugging, and accelerating software production quickly and almost costlessly.

And why shouldn’t it be? As the following figure shows, Large Language Model (LLM) impacts in the job market can be thought of as a 2×2 matrix. Along one axis we have how grammatical the domain is, by which we mean how rules-based are the processes governing how symbols are manipulated. Essays, for example, have rules (ask any irritated English teacher), so chat AIs based on LLMs can be trained to produce surprisingly good essays. Tax providers, contracts, and many other fields are in this box too…

… Software is even more rule-based and grammatical than conversational English, or any other conversational language. Programming languages—from Python to C++—can be thought of as formal languages with a highly explicit set of rules governing how every language element can and cannot be used to produce a desired outcome…

…Again, programming is a good example of a predictable domain, one created to produce the same outputs given the same inputs. If it doesn’t do that, that’s 99.9999% likely to be on you, not the language. Other domains are much less predictable, like equity investing, or psychiatry, or maybe, meteorology.

This framing—grammar vs predictability—leaves us convinced that for the first time in the history of the software industry, tools have emerged that will radically alter the way we produce software. This isn’t about making it easier to debug, or test, or build, or share—even if those will change too—but about the very idea of what it means to manipulate the symbols that constitute a programming language…

…Now, let’s be clear. Can you say MAKE ME MICROSOFT WORD BUT BETTER, or SOLVE THIS CLASSIC COMPSCI ALGORITHM IN A NOVEL WAY? No, you can’t, which will cause many to dismiss these technologies as toys. And they are toys in an important sense. They are “toys” in that they are able to produce snippets of code for real people, especially non-coders, that one incredibly small group would have thought trivial, and another immense group would have thought impossible. That. Changes. Everything.

How? Well, for one, the clearing price for software production will change. But not just because it becomes cheaper to produce software. In the limit, we think about this moment as being analogous to how previous waves of technological change took the price of underlying technologies—from CPUs, to storage and bandwidth—to a reasonable approximation of zero, unleashing a flood of speciation and innovation. In software evolutionary terms, we just went from human cycle times to that of the drosophila: everything evolves and mutates faster…

…We have mentioned this technical debt a few times now, and it is worth emphasizing. We have almost certainly been producing far less software than we need. The size of this technical debt is not knowable, but it cannot be small, so subsequent growth may be geometric. This would mean that as the cost of software drops to an approximate zero, the creation of software predictably explodes in ways that have barely been previously imagined.

The question people always have at this point is, “So what app gets made?” While an understandable question, it is somewhat silly and definitely premature. Was Netflix knowable when Internet transit costs were $500,000/Mbps? Was Apple’s iPhone imaginable when screens, CPUs, storage and batteries would have made such devices the size of small rooms? Of course not. The point is that the only thing we know is that the apps and services will come. Without question.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google), Amazon (parent of AWS), Apple, Meta Platforms, Microsoft, and Tesla. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 10 September 2023:

1. Rediscovering Berkshire Hathaway’s 1985 Annual Meeting – Kingswell

In the mid-1980s, Berkshire Hathaway’s annual meeting was an entirely different beast than today’s weekend-long “Woodstock for Capitalists”. Attendees didn’t have to book their hotel rooms months in advance or wake up before dawn just to get in line outside of an arena. There was no mad rush for seats once the doors opened.

It was a quieter, simpler chapter in Berkshire’s history.

So quiet, in fact, that 1985’s annual meeting was held on a Tuesday. And, instead of a cavernous arena, Warren Buffett and Charlie Munger opted for the Red Lion Inn in downtown Omaha. Approximately 250 shareholders attended the meeting and the ensuing Q&A session lasted only — only? — two hours…

…HOW TO VALUE A BUSINESS: “Do a lot of reading,” replied Buffett.

Generally speaking, he recommended the writings of Benjamin Graham and Philip Fisher for those trying to sharpen their investment mindset — and annual reports and trade magazines for evaluating particular businesses and industries.

Reading, he insisted, is more important than speaking with company executives or other investors. In fact, Buffett admitted that he had recently purchased a substantial amount of Exxon stock before talking to any of that company’s executives. “You’re not going to get any brilliant insights walking into the [Exxon] building,” he said.

And, at least in the money game, size matters.

It’s easier to determine the value of a large business than a small one, Buffett said. If someone buys a gas station, for example, another station opening across the street could have a major effect on the value of the first station…

…A COUPLE OF LAUGHS & A ROUND OF APPLAUSE: No annual meeting is ever complete without some of that trademark Warren Buffett wit.

• Will the federal deficit be substantially reduced? “I’ll believe it when I see it.”
• What about so-called “junk” bonds? “I think they’ll live up to their name,” Buffett quipped.

2. Germany Is Losing Its Mojo. Finding It Again Won’t Be Easy –  Bojan Pancevski, Paul Hannon, and William Boston

Two decades ago, Germany revived its moribund economy and became a manufacturing powerhouse of an era of globalization.

Times changed. Germany didn’t keep up. Now Europe’s biggest economy has to reinvent itself again. But its fractured political class is struggling to find answers to a dizzying conjunction of long-term headaches and short-term crises, leading to a growing sense of malaise.

Germany will be the world’s only major economy to contract in 2023, with even sanctioned Russia experiencing growth, according to the International Monetary Fund…

…At Germany’s biggest carmaker Volkswagen, top executives shared a dire assessment on an internal conference call in July, according to people familiar with the event. Exploding costs, falling demand and new rivals such as Tesla and Chinese electric-car makers are making for a “perfect storm,” a divisional chief told his colleagues, adding: “The roof is on fire.”

The problems aren’t new. Germany’s manufacturing output and its gross domestic product have stagnated since 2018, suggesting that its long-successful model has lost its mojo.

China was for years a major driver of Germany’s export boom. A rapidly industrializing China bought up all the capital goods that Germany could make. But China’s investment-heavy growth model has been approaching its limits for years. Growth and demand for imports have faltered…

…Germany’s long industrial boom led to complacency about its domestic weaknesses, from an aging labor force to sclerotic services sectors and mounting bureaucracy. The country was doing better at supporting old industries such as cars, machinery and chemicals than at fostering new ones, such as digital technology. Germany’s only major software company, SAP, was founded in 1975.

Years of skimping on public investment have led to fraying infrastructure, an increasingly mediocre education system and poor high-speed internet and mobile-phone connectivity compared with other advanced economies.

Germany’s once-efficient trains have become a byword for lateness. The public administration’s continued reliance on fax machines became a national joke. Even the national soccer teams are being routinely beaten…

…Germany today is in the midst of another cycle of success, stagnation and pressure for reforms, said Josef Joffe, a longtime newspaper publisher and a fellow at Stanford University.

“Germany will bounce back, but it suffers from two longer-term ailments: above all its failure to transform an old-industry system into a knowledge economy, and an irrational energy policy,” Joffe said…

…Germany still has many strengths. Its deep reservoir of technical and engineering know-how and its specialty in capital goods still put it in a position to profit from future growth in many emerging economies. Its labor-market reforms have greatly improved the share of the population that has a job. The national debt is lower than that of most of its peers and financial markets view its bonds as among the world’s safest assets.

The country’s challenges now are less severe than they were in the 1990s, after German reunification, said Holger Schmieding, economist at Berenberg Bank in Hamburg.

Back then, Germany was struggling with the massive costs of integrating the former Communist east. Rising global competition and rigid labor laws were contributing to high unemployment. Spending on social benefits ballooned. Too many people depended on welfare, while too few workers paid for it. German reliance on manufacturing was seen as old-fashioned at a time when other countries were betting on e-commerce and financial services.

After a period of national angst, then-Chancellor Gerhard Schröder pared back welfare entitlements, deregulated parts of the labor market and pressured the unemployed to take available jobs…

… Private-sector changes were as important as government measures. German companies cooperated with employees to make working practices more flexible. Unions agreed to forgo pay raises in return for keeping factories and jobs in Germany…

… Booming exports to developing countries helped Germany bounce back from the 2008 global financial crisis better than many other Western countries.

Complacency crept in. Service sectors, which made up the bulk of gross domestic product and jobs, were less dynamic than export-oriented manufacturers. Wage restraint sapped consumer demand. German companies saved rather than invested much of their profits.

Successful exporters became reluctant to change. German suppliers of automotive components were so confident of their strength that many dismissed warnings that electric vehicles would soon challenge the internal combustion engine. After failing to invest in batteries and other technology for new-generation cars, many now find themselves overtaken by Chinese upstarts…

…BioNTech, a lauded biotech firm that developed the Covid-19 vaccine produced in partnership with Pfizer, recently decided to move some research and clinical-trial activities to the U.K. because of Germany’s restrictive rules on data protection.

German privacy laws made it impossible to run key studies for cancer cures, BioNTech’s co-founder Ugur Sahin said recently. German approvals processes for new treatments, which were accelerated during the pandemic, have reverted to their sluggish pace, he said…

…One recent law required all German manufacturers to vouch for the environment, legal and ethical credentials of every component’s supplier, requiring even smaller companies to perform due diligence on many foreign firms, often based overseas, such as in China…

…German politicians dismissed warnings that Russian President Vladimir Putin used gas for geopolitical leverage, saying Moscow had always been a reliable supplier. After Putin invaded Ukraine, he throttled gas deliveries to Germany in an attempt to deter European support for Kyiv…

…One problem Germany can’t fix quickly is demographics. A shrinking labor force has left an estimated two million jobs unfilled. Some 43% of German businesses are struggling to find workers, with the average time for hiring someone approaching six months.

Germany’s fragmented political landscape makes it harder to enact far-reaching changes like the country did 20 years ago. In common with much of Europe, established center-right and center-left parties have lost their electoral dominance. The number of parties in Germany’s parliament has risen steadily.

3. GLP-1 Drugs: Not a Miracle Cure for Weight Loss – Biocompounding

Weight loss drugs have been the talk of the town for the last couple of months. The weight loss drugs on the market are Wegovy, Ozempic from Novo Norodisk (NVO), and Mounjaro from Eli Lilly (LLY)…

…These drugs consist of a natural hormone called GLP1…

…GLP-1 drugs mimic the action of a hormone called glucagon-like peptide 1, a natural hormone produced by the body in the gut. When blood sugar levels start to rise after a meal, the body produces this hormone to achieve multiple functions as seen in the image above. By producing and administering this hormone as a therapeutic, the drug will elicit similar effects seen with the natural hormone…

…Apart from increasing insulin production, GLP-1 can also help regulate body weight. GLP-1 improves glycaemic control and stimulates satiety, leading to reductions in food intake and thus body weight. Besides gastric distension and peripheral vagal nerve activation, GLP-1RA induces satiety by influencing brain regions involved in the regulation of feeding, and several routes of action have been proposed. GLP-1 can also reduce gastric emptying, so you don’t feel hungry so fast.

However, apart from the positives GLP1 drugs also cause muscle loss, lessen bone density, and lower your resting metabolic rate.

A research paper published in 2019, reported the percentage of weight loss comprising fat mass vs the proportion comprising lean body mass in patients using the different GLP1 drugs…

…This means that while GLP1’s can help to reduce obesity, individuals using the drugs need to be mindful to preserve their lean mass which requires exercising regularly to ensure they limit the loss of lean mass and improve their basal metabolic rate.

4. An Interview with Daniel Gross and Nat Friedman about the AI Hype Cycle – Ben Thompson, Daniel Gross, and Nat Friedman

NF: I think one of the interesting trends that we’ve seen in the last six months that we weren’t seeing a year ago is basically the application of large models to things that were previously some form of human intellectual labor or productivity labor. So in a way, what they’re doing in these cases is the models are automating or replacing or augmenting some part of a company. They’re competing not with existing software products but with parts of companies.

An example of one that Daniel and I were just talking to recently, we won’t name the company, but they automate filing bids on public tenders for businesses that do business with the government in different jurisdictions, and the time savings of this is totally enormous for these companies, and the upside for them is huge. It’s replacing a raft of internal and external consultants who were doing copywriting and bid preparation and just lots of fairly mechanical but still nothing-to-sneeze-at intellectual labor that produced bid documents. There’s material revenue upside for being able to bid on more things and win more bids, and this company’s growing like crazy, like a weed, so that would be one example.

Another example, there’s a whole sector now of these avatar platforms where people are basically able to produce personalized videos of someone saying, “Hey Ben, I saw that you were interested in our product and I wanted to tell you a little bit about us” and being able to basically generate text, feed that into an avatar platform that generates a realistic video that’s customized and using that in advertising, using it in personal outreach, using it in training materials. There’s some competing with non-consumption here where some of those videos would never have been produced because it would’ve just been too costly, and there’s some like, “Hey, God, I used to have to spend a ton of time doing this, now I can do it quite quickly”. Another example that’s like that, and by the way, all of the avatar, I mean I can name some of those Synthesia, D-ID, HeyGen, they’re all doing great, all of these companies are growing really well.

Another similar category is optimizing e-commerce. There used to be an entire — there still is — an entire industry of consultants and experts and companies who know how to do the SEO around product titles and descriptions and make sure that you have an Amazon landing page that converts, and some of that knowledge and know-how is getting crystallized into models and agent-like tool chains, and the testing can now be done automatically and you can use language models to run this kind of thing. I think this is interesting because these are all niches that really weren’t happening six or nine months ago, and in every category I just mentioned, there’s a company that’s making or soon will be making tens of millions of dollars doing this productively, creating real economic value for their customers and in some cases competing with teams of people or consultants…

...Does this just confirm the thesis though that the most compelling aspects for AI are number one mostly in the enterprise? Again, because enterprises are going to think about costs in a, I hesitate to use the word rational, but in a traditionally rational way, “It’s worth this huge upfront investment because it will pay off X amount over Y axis of time” as opposed to consumers which are more about an experience and may not think about the lifetime cost or value of something, along with this data point where whoever has the data wins. Is that just the reality or is there still opportunities for new entrants in this space?

DG: I think the story of progress is one where things will often, I think, start off looking at the enterprise as a way to make the existing thing better, that idea that the first TV shows or cameras pointed at radio shows, the horseless carriage and all that sort of stuff. So I think there’s a lot of V1 AI, let’s just accelerate or automate a lot of the human interaction with text just because we can do text synthesis now with computers. But the native use cases that’ll come out I think slightly later are going to be consumer ones — those I think will be entirely different things that are not replacing a process that existed before, they’re doing something that was never possible before and so there are consumer experiences today that are not really like anything else on the Internet.

Well, the two that I had on here were that seemed to still have a lot of traction are still growing are Midjourney and Character.AI, which are completely novel experiences and about fantasy and imagination and something that couldn’t be done previously.

DG: Yeah, it’s sort of funny, they told us the robots are going to be really good at blue collar jobs and really terrible at human psychology — that it’ll be the final realm of the human-to-human connection. Of course, it turns out the robots are fantastic at psychology and have zero dexterity for doing actual labor. But Character.AI is a good example and there’s now a bunch of these new kinds of native behavior, and it’s always interesting to ask of these behaviors. So you’re talking to an agent all day on Character, I find the good question to be asking is, “What were you doing previously?” as a way to figure out what this actually is, and the share of time that’s usually being taken is from gaming or social media. It’s really hard, I think, to forecast, to look at the iPhone and to forecast Uber or to look at the Internet and forecast even something like Amazon bots. They’re usually going to be, I think, consumer experiences. Those are the ones that are going to be the really disruptive stuff and the enterprise I think will get a lot of the obvious. We had a person here and now maybe we have a person in a co-pilot model.

That’s kind the trade-off of there being a top-down decision maker that thinks about things like lifetime value.

DG: They’ll do the rational thing.

They’re only going to do the obvious things.

DG: Yeah, and I think if businesses get disrupted by AI in any way, it will be something around a totally native, ideally a different user interface, an acceptance of a customer experience that’s a bit worse, which is usually your Clayton Christensen sort of downmarket disruption, but scales much more. I was actually thinking the companies that are trying to build, “We’re going to do your high-end legal work with AI”, I’m not exactly sure when that’ll work because the models still have this issue with hallucinating things and making things up. Whereas the low end, I was going to call a lawyer for $500 an hour to ask a particular question about my apartment lease, but instead I’m going to talk to legal GPT, that stuff I think will probably be much more impactful…

…There’s an aspect here — one of the questions with the coding bit is Stack Overflow and sites like that have taken the biggest hit, but is this a sustainable future? I think this is a broader question about do we run out of data on the Internet. Is there going to be a data manufacturing industry?

NF: There is already. I think this is the secret story just beneath the surface of what’s happening. Everyone knows about the GPUs, you got to have the GPUs, they’re very expensive, we’re talking about the Nvidia supply chain. All of us know about CoWoS and wafer packaging and Ajinomoto Films and all these things.

But the other key input is data and the readily available tokens you can scrape off the Internet are quickly exhausted, and so there is currently happening beneath the surface, a shadow war for data where the largest AI labs are spending huge amounts of money, like huge amounts of money to acquire more valuable tokens, either paying experts to generate it, working through labeling companies like Scale AI or others. There’s a new crop of startups in that space as well and we think more is going to happen there and it’s going to be a really interesting space to watch.

So there’s a way in which you need these really high IQ, high-value tokens in order to train your models, and the average piece of data you scrape off a random website kind is equal to all the other data that you have, but you’ll pay extra for really valuable training data, and so people are producing it. I don’t know the exact numbers, but I’ve heard rumors that Google is spending a billion dollars this year on generating new training data, and if you’re going to spend billions and billions on your CapEx to build out your GPU training clusters, spending some fraction of that or maybe an equal amount in generating data, which is a kind of CapEx as well kind of makes sense. Someone told me the other day experts are the new GPUs and so there’s this wave of spending on experts who are going to generate tokens that can be valuable.

Then of course the secondary question there is what the legal regime will ultimately be for training. We’re operating in the US, UK, and in Europe under this fair use regime now where it’s fair use for you to scrape text off the Internet as long as it’s public and you’re not going through paywalls or user walls to get it and then you can in aggregate train machine learning models on it. That’s kind of the bright letter of the law, but people don’t always feel good about that and so will the law change, will there be a kind of DMCA for AI? And which way will it cut? I think we don’t know yet and so there may be a war for data in more ways than one over the next couple of years…

… For the record, Nvidia’s results are going to come out in about 12 hours, so we don’t know what’s going to happen yet, but one of the most interesting questions broadly speaking is what is going to happen in the GPU space? Nvidia — do they have a moat, is it going to be a sustainable advantage? Obviously, they have a double advantage right now, in that they have the best hardware and they have CUDA, but there’s massive efforts on both sides to take that away. Can they build up a sustainable advantage that will persist?

NF: For the next couple of years, it’s Nvidia and it’s TPU and those are the only players that are really viable.

Google’s Tensor Processing Unit.

NF: Yeah, it’s a huge strategic asset for Google. I mean, they’re the only company basically that has an independent, not fully independent because obviously they overlap when it gets down to the fabs, and some other parts of the supply chain but they’re not subject to Jensen allocating them H100s. They can just kind of allocate their own and by all accounts, their TPU v5, they’re producing in absolute record numbers.

Easier to deal with TSMC than to deal with Jensen is what you’re saying.

NF: Yeah, I mean, at least they don’t have that one Jensen choke point. I mean, Jensen right now is dealing with overwhelming demand and limited supply, and so he’s having to very carefully allocate GPUs, and it’s sort of a very central resource distribution mechanism and allocation mechanism. It’s kind of wild. So even if you say, “Oh, AMD’s chips are going to be as good,” they’re just not going to produce them in numbers that matter next year and so I think my take is, there’s only two players for the next couple of years that matter, and my take is also that we will be supply-constrained, because there will be more AI applications that take off and need huge inference capacity, and there will be more people trying to train large models.

Is there a hype cycle aspect where we actually look back in a few years, and there were way too many GPUs bought and produced, and we actually end up with an overhang? Basically what happened with Nvidia last year, but at a 100x, a 1000x scale and that actually ends up being a huge accelerant for AI, because you end up with super cheap inference because you have all this depreciated GPUs that were bought up in 2023 and 2024, and then it all crashed. Actually going back to the dot-com bubble and all the fiber that got laid by companies that immediately went out of business.

NF: You might have a dark fiber, “How many shortages are not followed by a glut?” is always the interesting question. They usually do get followed by a glut and I think one scenario in which that happens is I’m a very strong believer in scaling laws for these big general reasoning models. Essentially, the more training data and the more flops you put in, you’re just going to get a better and better model out, and we’ve seen this now over several orders of magnitude, it’s just incredibly consistent. We saw it with GPT-1 and GPT-2, and GPT-3, and now GPT-4, and we’ll see it I think with GPT-5. So, it’s possible that there’s some escape velocity that occurs where a few labs are the only ones who can afford to train the GPT-5 or GPT-6 equivalent models, and all of the startups and businesses that were getting essentially a sub-scale amount of GPU, unless they were doing something incredibly domain specific, those are no longer needed. So, you’ll have I don’t know, three or four companies that afford to train the $10 billion model, and that’s actually a limited number of GPUs.

5. Respect and Admiration – Morgan Housel

This isn’t universal, but there are cases when people’s desire to show off fancy stuff is because it’s their only, desperate, way to gain some sense of respect and admiration. They don’t have any wisdom, intelligence, humor, empathy, or capacity for love to gain people’s respect. So they rely on the only remaining, and least effective, lever: Look at my car, beep beep, vroom vroom…

…My guess is that if your favorite comedian, or actor, or athlete turned out to be broke, you wouldn’t care. It wouldn’t impact how much you admire them, because you admire them for talents that money can’t buy.

Even when Amazon was huge and successful, Jeff Bezos used to drive a Honda Accord. Today he has a $500 million yacht. Is he respected and admired more for it? Not in the slightest. He could ride a Huffy bike and people would consider him the greatest entrepreneur of our era, because he is. Steve Jobs didn’t have any furniture. It didn’t matter. He’s a genius. He’s Steve Jobs. Material stuff makes no difference when you’re respected and admired for internal traits…

…Once you see people being respected and admired for reasons that have nothing to do with the stuff they own, you begin to wonder why you have such a strong desire for those possessions. I tend to view material desire as a loose proxy for the inverse of what else you have to offer the world. The higher my desire for fancy stuff, the less real value I have to offer.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google), Amazon, and TSMC. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general.

Here are the articles for the week ending 03 September 2023:

1. China Reaches Peak Gasoline in Milestone for Electric Vehicles – Colin McKerracher

Earlier this month, Chinese oil giant Sinopec made a surprise announcement that mostly flew under the radar. It’s now expecting gasoline demand in China to peak this year, two years earlier than its previous outlooks.

The main culprit? The surging number of electric vehicles on the road…

…China has been the largest driver of global growth for refined oil products like gasoline and diesel over the last two decades. But EV adoption rates in China are now soaring, with August figures likely to show plug-in vehicles hitting 38% of new passenger-vehicle sales. That’s up from just 6% in 2020 and is starting to materially dent fuel demand.

Fuel demand in two and three-wheeled vehicles is already in structural decline, with BNEF estimating that 70% of total kilometers traveled by these vehicles already switched over to electric. Fuel demand for cars will be the next to turn, since well over 5% of the passenger-vehicle fleet is now either battery-electric or plug-in hybrid. The internal combustion vehicle fleet is also becoming more efficient due to rising fuel-economy targets.

Diesel demand for heavier vehicles will keep growing for a bit longer, but even there a seismic shift is underway. Electric, fuel cell and battery-swapping options have quickly climbed to 12% of light commercial vehicle sales and 4% to 5% of medium and heavy commercial vehicle sales. That heavy-duty figure is likely to climb to over 10% by 2025.

Combine all those segments, and BNEF expects total oil demand for road transport in China to peak late next year. Demand won’t drop off a cliff anytime soon — fleet turnover in the trucking segment in particular will take time — but it still marks a major shift for global oil demand patterns. It also has big implications for refiners that need to quickly adjust the mix of products they produce.

It also called out the effects China’s ride-hailing fleet is having on urban gasoline demand.

Vehicles used for ride-hailing in China are far more likely to be electric — their share is nearing 40% of the fleet — than those that are privately owned. Electric ride-hailing vehicles are also more productive than their gasoline-powered counterparts, accounting for 50% of the kilometers traveled on market leader Didi’s ride-hailing platform in December…

…The Sinopec announcement highlights how looking just at the fleet of vehicles can lead one to miss the full story with respect to energy impact…

…The speed that oil gets squeezed out of the transport mix depends on how fast countries like China switch over the number of kilometers traveled to electric — not just the number of cars and trucks.

2. Peasant Logic and the Russian GKO Trade – Joe Pimbley

Later in 1998, after Russia blew up, I attended a public risk management conference in Paris. And one of the speakers was Allen Wheat, CEO of Credit Suisse at the time. I didn’t know Wheat, but he impressed me as a blunt, direct-speaking guy. He talked about Credit Suisse’s version of the GKO trade. He didn’t mention a short position in a Russian-issued dollar bond, so maybe Credit Suisse didn’t bother with the credit risk hedge. But he talked about the GKO and rubles and the cross-currency forwards Credit Suisse executed with Russian banks…

…Interesting to me, Wheat’s story was not that he got to the bottom of this controversy and figured out what part of the loss owed to market risk and what part owed to credit risk. Wheat’s conclusion to his board of directors was that Credit Suisse had a problem with its “risk management philosophy.” It had market risk and credit risk silos when really risk management must be integrated. It’s unproductivee to distinguish market risk from credit risk if things are going to fall between the cracks and nobody’s going to take responsibility for understanding the complete risk picture.

Clearly, that’s a nice message, even if you wonder why Wheat didn’t work through the finger-pointing and hold people to account. Who can argue against an integrated approach to risk? But Wheat admitted he got chastised by his board when he presented that conclusion. The board said, “Allen, we think we understand what’s wrong here. It’s good to do all your analysis and get deep into the details, but at some point, you’re not seeing the big picture. You really need to use ‘peasant logic.’”

Wheat explained that “peasant logic” was the board’s term for what we might call “common sense,” but I like peasant logic better. The board said, “You people worry about how good your models are and you wonder about using two years of historical data or five years of historical data, and whether one is better than the other, and how much data you should have. We think you should have looked at the big picture and said, ‘messing around with 40% yields means there’s a lot of risk here. This is an unstable government and currency situation.’ We think you aren’t seeing the forest for the trees.”

So this was Wheat’s point: sometimes it’s good to forget the data and models and use peasant logic. In this case, if there are abnormal returns, there must be some abnormal risk…

…Then it came time for questions, and from the back of the room, someone had to shout out his question to be heard. And as soon as he started speaking, you could tell it’s a Russian accent and the guy is Russian. Being Russian lent authenticity to his remark, “You want historical data. I’ll give you 75 years of historical data. Russia has never honored any debt obligation.”

…Unfortunately, Wheat’s reaction was to be annoyed. Wheat didn’t say, “Wow, what a great way to look at this. Why are we trusting Russian debt?” And he also didn’t say, “That’s a great example of the peasant logic the board was trying to impress upon me.”

The Russian continued. “I work for Merrill Lynch and we did this trade also and lost a lot of money. Beforehand, I told them it was a terrible trade because of Russia’s history and they didn’t listen to me because I’m just a mathematician.” Wheat still hadn’t cottoned to the idea that the Russian was helping him make his point about peasant logic, so he said in a rather dismissive, sarcastic way, “Well, I wish we had you working for us, then we wouldn’t have lost money. Right?”

Now it’s easy in hindsight, when you know how something worked out, to say “Aha, I knew such and such.” But still, I thought the Russian added to Wheat’s remarks and his remarks really made Wheat’s point. This guy in the audience was demonstrating peasant logic. The traders put all these fancy complex pieces together and think they’re really smart, but what the heck are they doing lending money to a government that, to this guy who is closer to it than the rest of us, you shouldn’t trust? 

3. Google Gemini Eats The World – Gemini Smashes GPT-4 By 5X, The GPU-Poors – Dylan Patel and Daniel Nishball

The statement that may not be obvious is that the sleeping giant, Google has woken up, and they are iterating on a pace that will smash GPT-4 total pre-training FLOPS by 5x before the end of the year. The path is clear to 20x by the end of next year given their current infrastructure buildout. Whether Google has the stomach to put these models out publicly without neutering their creativity or their existing business model is a different discussion…

…Access to compute is a bimodal distribution. There are a handful of firms with 20k+ A/H100 GPUs, and individual researchers can access 100s or 1,000s of GPUs for pet projects. The chief among these are researchers at OpenAI, Google, Anthropic, Inflection, X, and Meta, who will have the highest ratios of compute resources to researchers. A few of the firms above as well as multiple Chinese firms will 100k+ by the end of next year, although we are unsure of the ratio of researchers in China, only the GPU volumes.

One of the funniest trends we see in the Bay area is with top ML researchers bragging about how many GPUs they have or will have access to soon. In fact, this has become so pervasive over the last ~4 months that it’s become a measuring contest that is directly influencing where top researchers decide to go. Meta, who will have the 2nd most number of H100 GPUs in the world, is actively using it as a recruiting tactic.

Then there are a whole host of startups and open-source researchers who are struggling with far fewer GPUs. They are spending significant time and effort attempting to do things that simply don’t help, or frankly, matter. For example, many researchers are spending countless hours agonizing on fine-tuning models with GPUs that don’t have enough VRAM. This is an extremely counter-productive use of their skills and time.

These startups and open-source researchers are using larger LLMs to fine-tune smaller models for leaderboard style benchmarks with broken evaluation methods that give more emphasis to style rather than accuracy or usefulness. They are generally ignorant that pretraining datasets and IFT data need to be significantly larger/higher quality for smaller open models to improve in real workloads.

Yes, being efficient with GPUs is very important, but in many ways, that’s being ignored by the GPU-poors. They aren’t concerned with efficiency at scale, and their time isn’t being spent productively. What can be done commercially in their GPU-poor environment is mostly irrelevant to a world that will be flooded by more than 3.5 million H100s by the end of next year. For learning, experimenting, smaller weaker gaming GPUs are just fine…

…While the US and China will be able to keep racing ahead, the European startups and government backed supercomputers such as Jules Verne are also completely uncompetitive. Europe will fall behind in this race due to the lack of ability to make big investments and choosing to stay GPU-poor. Even multiple Middle Eastern countries are investing more on enabling large scale infrastructure for AI.

Being GPU-poor isn’t limited to only scrappy startups though. Some of the most well recognized AI firms, HuggingFace, Databricks (MosaicML), and Together are also part of this GPU-poor group. In fact, they may be the most GPU-poor groups out there with regard to both the number of world class researchers per GPU and the number of GPUs versus the ambition/potential customer demand. They have world class researchers, but all of them are limited by working on systems with orders of magnitude less capabilities. These firms have tremendous inbound from enterprises on training real models, and on the order of thousands of H100s coming in, but that won’t be enough to grab much of the market.

Nvidia is eating their lunch with multiple times as many GPUs in their DGX Cloud service and various in-house supercomputers. Nvidia’s DGX Cloud offers pretrained models, frameworks for data processing, vector databases and personalization, optimized inference engines, APIs, and support from NVIDIA experts to help enterprises tune models for their custom use cases. That service has also already racked up multiple larger enterprises from verticals such as SaaS, insurance, manufacturing, pharmaceuticals, productivity software, and automotive. While not all customers are announced, even the public list of Amgen, Adobe, CCC, ServiceNow, Accenture, AstraZeneca, Getty Images, Shutterstock, Morningstar, Evozyne, Insilico Medicine, Quantiphi, InstaDeep, Oxford Nanopore, Peptone, Relation Therapeutics, ALCHEMAB Therapeutics, and Runway is quite impressive.

4. Making Sense Of The China Meltdown Story – Louis-Vincent Gave

It is impossible to turn to a newspaper, financial television station or podcast today without getting told all about the unfolding implosion of the Chinese economy. Years of over-building, white elephants and unproductive infrastructure spending are finally coming home to roost. Large property conglomerates like Evergrande and Country Garden are going bust. And with them, so are hopes for any Chinese economic rebound. Meanwhile, the Chinese government is either too incompetent, too ideologically blinkered, or simply too communist to do anything about this developing disaster.

Interestingly, however, financial markets are not confirming the doom and gloom running rampant across the financial media…

…At Gavekal, we look at bank shares as leading indicators of financial trouble. When we see bank shares break out to new lows, it is usually a signal that investors should head for the exit as quickly as possible. This was certainly the case in 2007-08 in the US. Between February 2007 and July 2008 (six weeks before the collapse of Lehman Brothers), banks shares lost -60% of their value…

…Now undeniably, Chinese bank shares have not been the place to be over the past few years. Nonetheless, Chinese bank shares are still up a significant amount over the last decade. And this year, they have not even taken out the low of 2022 made on October 31st following the Chinese Communist Party congress. To be sure, the chart below is hardly enticing, even if the slope of the 200-day moving average is positive. Still, Chinese bank shares do not seem to be heralding a near-term financial sector Armageddon…

…China is the number one or two importer of almost every major commodity you can think of. So, if the Chinese economy were experiencing a meltdown, you would expect commodity prices to be soft. Today, we are seeing the opposite. The CRB index has had a strong year so far in 2023, and is trading above its 200-day moving average. Moreover, the 200-day moving average now has a positive slope. Together, all this would seem to point towards an unfolding commodity bull market more than a Chinese meltdown…

…Jacques Rueff used to say that exchange rates are the “sewers in which unearned rights accumulate.” This is a fancy way of saying that exchange rates tend to be the first variable of adjustment for any economy that has accumulated imbalances. On this front, the renminbi has been weak in recent months, although, like Chinese equities, it has yet to take out October’s lows.

That is against the US dollar. Against the yen, the currency of China’s more direct competitor, Japan, the renminbi continues to grind higher and is not far off making new all-time highs. And interestingly, in recent weeks, the renminbi has been rebounding against the South Korean won.

This is somewhat counterintuitive. In recent weeks, oceans of ink have been spilled about how China is the center of a developing financial maelstrom. Typically, countries spiraling down the financial plughole do not see their currencies rise against those of their immediate neighbors and competitors…

…In other words, a range of data points seems to indicate that Chinese consumption is holding up well. This might help to explain why the share prices of LVMH, Hermès, Ferrari and most other producers of luxury goods are up on the year. If China really was facing an economic crash, wouldn’t you expect the share prices of luxury good manufacturers to at least reflect some degree of concern?…

…Staying on the US treasury market, it is also odd how Chinese government bonds have outperformed US treasuries so massively over the past few years. Having gone through a fair number of emerging market crises, I can say with my hand on my heart that I have never before seen the government bonds of an emerging market in crisis outperform US treasuries. Yet since the start of Covid, long-dated Chinese government bonds have outperformed long-dated US treasuries by 35.3%.

In fact, Chinese bonds have been a beacon of stability, with the five-year yield on Chinese government bonds spending most of the period since the 2008 global crisis hovering between 2.3% and 3.8%. Today, the five-year yield sits at the low end of this trading band. But for all the negativity out there, yields have yet to break out on the downside…

…While the Chinese government debt market has been stable, the pain has certainly been dished out in the Chinese high yield market. Yields have shot up and liquidity in the Chinese corporate bond market has all but evaporated. Perhaps this is because historically many of the end buyers have been foreign hedge funds, and the Chinese government feels no obligation to make foreign hedge funds whole. Or perhaps it is because most of the issuers were property developers, a category of economic actor that the CCP profoundly dislikes.

Whatever the reasons, the Chinese high yield debt market is where most of the pain of today’s slowdown has been—and continues to be—felt. Interestingly, however, it seems that the pain in the market was worse last year than this year. Even though yields are still punishingly high, they do seem to be down from where they were a year ago…

…Why the sudden drumbeat about collapsing Chinese real estate and impending financial crisis when the Chinese real estate problem has been a slow-moving car crash over the past five years, and when, as the charts above show, markets don’t seem to indicate a crisis point?

At least, markets outside the US treasury market don’t seem to indicate a crisis point. So could the developing meltdown in US treasuries help to explain the urgency of the “China in crisis” narrative?…

…Basically, US treasuries have delivered no positive absolute returns to any investor who bought bonds after 2015. Meanwhile, investors who bought Chinese government bonds in recent years are in the money, unless they bought at the height of the Covid panic in late 2021 and early 2022. This probably makes sense given the extraordinary divergence between US inflation and Chinese inflation.

None of this would matter if China was not in the process of trying to dedollarize the global trade in commodities and was not playing its diplomatic cards, for example at this week’s BRICS summit, in an attempt to undercut the US dollar (see Clash Of Empires). But with China actively trying to build a bigger role for the renminbi in global payments, is it really surprising to see the Western media, which long ago gave up any semblance of independence, highlighting China’s warts? Probably not. But the fact that the US treasury market now seems to be entering a full-on meltdown adds even more urgency to the need to highlight China’s weaknesses.

A Chinese meltdown, reminiscent of the 1997 Asian crisis, would be just what the doctor ordered for an ailing US treasury market: a global deflationary shock that would unleash a new surge of demand and a “safety bid” for US treasuries. For now, this is not materializing, hence the continued sell-off in US treasuries. But then, the Chinese meltdown isn’t materializing either.

5. Why China’s economy ran off the rails – Noah Smith

This is a pretty momentous happening, since a lot of people had started to believe — implicitly or explicitly — that China’s economy would never suffer the sort of crash that periodically derails all other economies. That was always wrong, of course, and now the bears are coming out for a well-deserved victory lap…

…Anyway, OK, here is my quick story of what happened to China. In the 1980s, 90s, and early 2000s, China reaped huge productivity gains from liberalizing pieces of its state-controlled economy. Industrial policy was mostly left to local governments, who wooed foreign investors and made it easy for them to open factories, while the central government mostly focused on big macro things like making capital and energy cheap and holding down the value of the currency. As a result, China became the world’s factory, and its exports and domestic investment soared. As did its GDP.

At this time there were also substantial tailwinds for the Chinese economy, including a large rural surplus population who could be moved to the cities for more productive work, a youth bulge that created temporarily favorable demographics, and so on. China was also both willing and able to either buy, copy, or steal large amounts of existing technology from the U.S. and other rich countries.

Meanwhile, during this time, real estate became an essential method by which China distributed the gains from this stupendous economic growth. It was the main financial asset for regular Chinese people, and land sales were how local governments paid for public services.

Then the 2008 financial crisis hit the U.S., and the Euro crisis hit Europe. The stricken economies of the developed nations were suddenly unable to keep buying ever-increasing amounts of Chinese goods (and this was on top of export markets becoming increasingly saturated). Exports, which had been getting steadily more and more important for the Chinese economy, suddenly started to take a back seat:..

… The government told banks to lend a lot in order to avoid a recession, and most of the companies they knew how to shovel money at were in the real estate business in some way. That strategy was successful at avoiding a recession in 2008-10, and over the next decade China used it again whenever danger seemed to threaten — such as in 2015 after a stock market crash.

Maybe China’s leaders were afraid of what would happen to them if they ever let growth slip, or maybe they didn’t really think about what the costs of this policy might be. In any case, China basically pivoted from being an export-led economy to being a real-estate-led economy. Real-estate-related industries soared to almost 30% of total output.

That pivot saved China from recessions in the 2010s, but it also gave rise to a number of unintended negative consequences. First, construction and related industries tend to have lower productivity growth than other industries (for reasons that aren’t 100% clear). So continuing to shift the country’s resources of labor and capital toward those industries ended up lowering aggregate productivity growth. Total factor productivity, which had increased steadily in the 2000s, suddenly flatlined in the 2010s:..

…This productivity slowdown probably wasn’t only due to real estate — copying foreign technology started to become more difficult as China appropriated all the easier stuff. Nor was productivity the only thing weighing on China’s growth — around this same time, surplus rural labor dried up. Anyway, put it all together, and you get a slowdown in GDP growth in the 2010s, from around 10% to around 6% or 7%:

But 6-7% is still pretty darn fast. In order to keep growth going at that pace, China had to invest a lot — around 43% of its GDP, more than in the glory days of the early 2000s, and much more than Japan and Korea at similar points in their own industrial development.

Only instead of deploying that capital efficiently, China was just putting it toward increasingly low-return real estate. The return on assets for private companies collapsed:

Much of this decline was due simply to the Chinese economy’s shift toward real estate; if you strip out real estate, the deterioration in the private sector looks much less severe…

…So even as the pivot to real estate was adding to a long-term slowdown in China’s growth, it was also generating a bubble that would eventually cause an acute short-term slowdown as well. If there’s a grand unified theory of China’s economic woes, it’s simply “too much real estate”.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Adobe, Alphabet (parent of Google), and Meta Platforms. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general.

Here are the articles for the week ending 27 August 2023:

1. Why Lehman Brothers Failed When It Did – Joe Pimbley

In 2008, security firms operated with high leverage and significant amounts of short-term debt. Lehman had $26 billion of equity supporting $639 billion of assets and its high leverage was not unusual among security firms. But at that ratio, a 4% decline in assets wipes out equity. Meanwhile, reliance on the continuous rolling of short-term debt requires the security firm to always maintain lender confidence. Lenders’ perception of solvency becomes more important than the actual fact of solvency.

When the highly leveraged, short-term debt, security firm business model met the asset-value destruction of the Great Financial Crisis, Lehman was not the only security firm to fail. All major US firms failed to one degree or another. Besides Lehman’s outright bankruptcy, Bear Stearns and Merrill Lynch were merged into commercial banks. I believe Goldman Sachs and Morgan Stanley would have defaulted on their short-term borrowings had the Fed not permitted them to convert to bank holding companies and gain access to discount window liquidity…

…A place to begin chronicling factors specific to Lehman’s failure is the beginning of 2006. That was when the firm’s management decided to make more long-term investments.[2] Rather than remaining focused on security distribution and brokerage, Lehman increased its own holdings in commercial real estate, leveraged loans, and private equity. In our report to the bankruptcy court, we described this strategic change as a shift from the “moving business” to the “storage business.”

One year later in early 2007, Lehman management viewed the incipient financial crisis as an opportunity for the firm to gain market share and revenue from competitors that were retrenching and lowering their risk profiles. Lehman did not think the subprime mortgage crisis would spread to the general economy or even to its growing commercial real estate portfolio. Lehman had boldly taken on assets and assumed risk in the 2001-02 economic downturn. Its risk-taking back then had paid off and it hoped such contrarian boldness would again prove profitable.

Lehman’s pace of principal investments in commercial real estate, leveraged loans, and private equity increased in the first half of 2007 as other security firms reduced risk and hunkered down. It committed $11 billion to acquire Archstone REIT in May 2007 and ended up funding the riskiest $6 billion of that in October when it couldn’t find enough buyers to take it out of its commitment. Other bridge loans and bridge equity positions also became similarly stuck on its balance sheet. Its mortgage subsidiaries were slow to stop making residential mortgage loans and Lehman ended up holding mortgage-backed bonds and mortgage-bond-backed collateralized debt obligations it couldn’t sell.

To take on these risky assets, Lehman’s management raised all its internal risk limits: firm-wide, line-of-business, and even single-name risk limits. Or they ignored the limits they had set. Management was not fulsome in its disclosures to its board of directors about the risks it assumed and Lehman’s board did not press management for important information. In theory, Lehman’s compensation policy penalized excessive risk taking, but in practice it rewarded employees on revenue with minimal attention to associated risk.

Not only were these investments risky from the perspective of potential market value losses; they were risky from the point of view of financing. By their nature, real estate, leveraged loans, and private equity are hard to value and less liquid. It is difficult to determine how quickly and how severely they could lose value. These characteristics mean the ability to finance these assets cannot be assumed. If lenders worry about the realizable value of assets offered as loan security, they will lower the amount they will lend against those assets or cease lending against them altogether. Most of Lehman’s secured debt had overnight tenors, so lenders could stop rolling over their loans to Lehman on any business day!

Lehman’s management only began to cut back on leveraged loan acquisitions in August 2007 and they waited until later in 2007 to cut back on commercial real estate purchases. Yet deals in the pipeline caused Lehman’s assets to grow $95 billion to $786 billion over the quarter ending February 2008. The firm did not begin to sell assets in earnest until March 2008, but only got assets down to $639 billion by May 2008.

Lehman’s management deliberately deceived the world about the firm’s financial condition. Management used an accounting trick to temporarily remove $50 billion of assets from the firm’s balance sheet at the end of the first and second quarters of 2008. In so-called “repo 105” transactions, Lehman pledged assets valued at 105% or more of the cash it received. Relying on a legal opinion from a UK law firm addressing English law, Lehman deducted the assets from its balance sheet. No other security firm used this stratagem in 2008 and Lehman did not disclose its use.

Lehman’s management touted the firm’s “liquidity pool,” the sum of cash and assets readily convertible into cash and as late as two days before bankruptcy claimed this pool equaled $41 billion. In fact, only $2 billion of those assets were readily monetizable.

From January to May 2008, while its competitors raised equity, Lehman did not. Lehman’s management rejected offers from interested investors because they did not want to issue equity at a discount to market price. Management thought doing so would make the firm seem vulnerable. Lehman did not issue common stock in 2008 until a $4 billion issuance in June.

2. China’s 40-Year Boom Is Over. What Comes Next? – Lingling Wei and Stella Yifan Xie

For decades, China powered its economy by investing in factories, skyscrapers and roads. The model sparked an extraordinary period of growth that lifted China out of poverty and turned it into a global giant whose export prowess washed across the globe.

Now the model is broken.

What worked when China was playing catch-up makes less sense now that the country is drowning in debt and running out of things to build. Parts of China are saddled with under-used bridges and airports. Millions of apartments are unoccupied. Returns on investment have sharply declined.

Signs of trouble extend beyond China’s dismal economic data to distant provinces, including Yunnan in the southwest, which recently said it would spend millions of dollars to build a new Covid-19 quarantine facility, nearly the size of three football fields, despite China having ended its “zero-Covid” policy months ago, and long after the world moved on from the pandemic…

…What will the future look like? The International Monetary Fund puts China’s GDP growth at below 4% in the coming years, less than half of its tally for most of the past four decades. Capital Economics, a London-based research firm, figures China’s trend growth has slowed to 3% from 5% in 2019, and will fall to around 2% in 2030.

At those rates, China would fail to meet the objective set by President Xi Jinping in 2020 of doubling the economy’s size by 2035. That would make it harder for China to graduate from the ranks of middle-income emerging markets and could mean that China never overtakes the U.S. as the world’s largest economy, its longstanding ambition.

Many previous predictions of China’s economic undoing have missed the mark. China’s burgeoning electric-vehicle and renewable energy industries are reminders of its capacity to dominate markets. Tensions with the U.S. could galvanize China to accelerate innovations in technologies such as artificial intelligence and semiconductors, unlocking new avenues of growth. And Beijing still has levers to pull to stimulate growth if it chooses, such as by expanding fiscal spending.

Even so, economists widely believe that China has entered a more challenging period, in which previous methods of boosting growth yield diminishing returns…

…The transition marks a stunning change. China consistently defied economic cycles in the four decades since Deng Xiaoping started an era of “reform and opening” in 1978, embracing market forces and opening China to the West, in particular through international trade and investment.

During that period, China increased per capita income 25-fold and lifted more than 800 million Chinese people out of poverty, according to the World Bank—more than 70% of the total poverty reduction in the world. China evolved from a nation racked by famine into the world’s second-largest economy, and America’s greatest competitor for leadership.

Academics were so enthralled by China’s rise that some referred to a “Chinese Century,” with China dominating the world economy and politics, similar to how the 20th century was known as the “American Century.”

China’s boom was underpinned by unusually high levels of domestic investment in infrastructure and other hard assets, which accounted for about 44% of GDP each year on average between 2008 and 2021. That compared with a global average of 25% and around 20% in the U.S., according to World Bank data.

Such heavy spending was made possible in part by a system of “financial repression” in which state banks set deposit rates low, which meant they could raise funds inexpensively and fund building projects. China added tens of thousands of miles of highways, hundreds of airports, and the world’s largest network of high-speed trains.

Over time, however, evidence of overbuilding became apparent.

About one-fifth of apartments in urban China, or at least 130 million units, were estimated to be unoccupied in 2018, the latest data available, according to a study by China’s Southwestern University of Finance and Economics…

…Guizhou, one of the poorest provinces in the country with GDP per capita of less than $7,200 last year, boasts more than 1,700 bridges and 11 airports, more than the total number of airports in China’s top four cities. The province had an estimated $388 billion in outstanding debt at the end of 2022, and in April had to ask for aid from the central government to shore up its finances.

Kenneth Rogoff, a professor of economics at Harvard University, said China’s economic ascent draws parallels to what many other Asian economies went through during their periods of rapid urbanization, as well as what European countries such as Germany experienced after World War II, when major investments in infrastructure boosted growth.

At the same time, decades of overbuilding in China resembles Japan’s infrastructure construction boom in the late 1980s and 1990s, which led to overinvestment.

The solution for many parts of the country has been to keep borrowing and building. Total debt, including that held by various levels of government and state-owned companies, climbed to nearly 300% of China’s GDP as of 2022, surpassing U.S. levels and up from less than 200% in 2012, according to Bank for International Settlements data.

Much of the debt was incurred by cities. Limited by Beijing in their ability to borrow directly to fund projects, they turned to off-balance sheet financing vehicles whose debts are expected to reach more than $9 trillion this year, according to the IMF.

Rhodium Group, a New York-based economic research firm, estimates that only about 20% of financing firms used by local governments to fund projects have enough cash reserves to meet their short-term debt obligations, including bonds owned by domestic and foreign investors…

…In Beijing’s corridors of power, senior officials have recognized that the growth model of past decades has reached its limits. In a blunt speech to a new generation of party leaders last year, Xi took aim at officials for relying on borrowing for construction to expand economic activities…

…The most obvious solution, economists say, would be for China to shift toward promoting consumer spending and service industries, which would help create a more balanced economy that more resembles those of the U.S. and Western Europe. Household consumption makes up only about 38% of GDP in China, relatively unchanged in recent years, compared with around 68% in the U.S., according to the World Bank.

Changing that would require China’s government to undertake measures aimed at encouraging people to spend more and save less. That could include expanding China’s relatively meager social safety net with greater health and unemployment benefits.

Xi and some of his lieutenants remain suspicious of U.S.-style consumption, which they see as wasteful at a time when China’s focus should be on bolstering its industrial capabilities and girding for potential conflict with the West, people with knowledge of Beijing’s decision-making say.

The leadership also worries that empowering individuals to make more decisions over how they spend their money could undermine state authority, without generating the kind of growth Beijing desires.

A plan announced in late July to promote consumption was criticized by economists both in and outside China for lacking details. It suggested promoting sports and cultural events, and pushed for building more convenience stores in rural areas.

Instead, guided by a desire to strengthen political control, Xi’s leadership has doubled down on state intervention to make China an even bigger industrial power, strong in government-favored industries such as semiconductors, EVs and AI.

While foreign experts don’t doubt China can make headway in these areas, they alone aren’t enough to lift up the entire economy or create enough jobs for the millions of college graduates entering the workforce, economists say. 

3. LTCM: 25 Years On – Marc Rubinstein

To understand, it helps to model LTCM not as a hedge fund but as a bank (although it’s also true that the best model for a bank is often a hedge fund). Roger Lowenstein, author of When Genius Failed, acknowledges as much in the subtitle of his book: “The Rise and Fall of Long-Term Capital Management: How One Small Bank Created a Trillion-Dollar Hole.” 

The model reflects LTCM’s heritage. John Meriwether ran the arbitrage desk at Salomon Brothers becoming vice chair of the whole firm, in charge of its worldwide Fixed Income Trading, Fixed Income Arbitrage and Foreign Exchange businesses. In the years 1990 to 1992, proprietary trading accounted for more than 100% of the firm’s total pre-tax profit, generating an average $1 billion a year. LTCM was in some ways a spin-off of this business.

Indeed, LTCM partners viewed their main competitors as the trading desks of large Wall Street firms rather than traditional hedge funds. Thus, although they structured their firm as a hedge fund (2% management fee, 25% performance fee, high watermark etc) they did everything they could to replicate the structure of a bank. So investors were required to lock-up capital initially for three years to replicate the permanent equity financing of a bank (hence “Long-Term Capital Management”). They obtained $230 million of unsecured term loans and negotiated a $700 million unsecured revolving line of credit from a syndicate of banks. They chose to finance positions over 6-12 months rather than roll financing daily, even at the cost of less favourable rates. And they insisted that banks collateralise their obligations to the fund via a “two way mark-to-market”: As market prices moved in favour of LTCM, collateral such as government bonds would flow from their counterparty to them.

If there was one risk LTCM partners were cognisant of it is that they might suffer a liquidity crisis and not be able to fund their trades. It was a risk they took every effort to mitigate. 

But in modelling themselves as a bank, they forgot one key attribute: diversification.

“We set up Long-Term to look exactly like Salomon,” explains Eric Rosenfeld. “Same size, same scope, same types of trades… But what we missed was that there’s a big difference between the two: Long-Term is a monoline hedge fund and Salomon is a lot of different businesses – they got internal diversification from their other business lines during this crisis so therefore they could afford to have taken on more risk. We should have run this at a lower risk.”

It’s a risk monolines in financial services often miss. And LTCM wasn’t the only monoline to fall victim to market conditions in 1998. In the two years that followed, eight of the top 10 subprime monolines in the US declared bankruptcy, ceased operations or sold out to stronger firms. The experience prompted some financial institutions – such as Capital One – to embrace a more diversified model.

When the global financial crisis hit in 2007, monoline firms went down first. And in the recent banking crisis of 2023, those banks that failed were characterised by lower degrees of diversification.

There’s another factor that also explains the downfall of LTCM, one that similarly has echoes in the banking sector. At the end of August, LTCM was bruised but it was far from bankrupt. It had working capital of around $4 billion including a largely unused credit facility of $900 million, of which only $2.1 billion was being used for financing positions.

But the fax Meriwether sent clients on September 2 triggered a run on the bank. “We had 100 investors at the time, and a couple of fax machines,” recalls Rosenfeld. “By the time we got to investor 50, I noticed that the top story on Bloomberg was us… All eyes were on us. We were like this big ship in a small harbour trying to turn; everyone was trying to get out of the way of us.”

While the August losses reflected a flight to quality as investors flocked to safe assets, the September losses reflected a flight away from LTCM. The price of a natural catastrophe bond the firm held, for example, fell by 20% on September 2, even though there had been no increase in the risk of natural disaster and the bond was due to mature six weeks later. As the firm was forced to divulge more information to counterparties over the course of September, the situation worsened. “The few things we had on that the market didn’t know about came back quickly,” Meriwether later told the New York Times. “It was the trades that the market knew we had on that caused us trouble.”

In addition, illiquid markets gave counterparties leeway in how to mark positions, and they used the opportunity to mark against LTCM to the widest extent possible so that they would be able to claim collateral to mitigate against a possible default (the flipside of the “two way mark-to-market”). The official inquiry into the failure noted that by mid-September, “LTCM’s repo and OTC [over-the-counter] derivatives counterparties were seeking as much collateral as possible through the daily margining process, in many cases by seeking to apply possible liquidation values to mark-to-market valuations.” And because different legs of convergence trades were held with different counterparties, there was very little netting. In index options, such collateral outflows led to around $1 billion of losses in September. 

Nicholas Dunbar, who wrote the other bestselling book about LTCM, Inventing Money, quotes a trader at one of LTCM’s counterparties (emphasis added):

“When it became apparent they [LTCM] were having difficulties, we thought that if they are going to default, we’re going to be short a hell of a lot of volatility. So we’d rather be short at 40 [at an implied volatility of 40% per annum] than 30, right? So it was clearly in our interest to mark at as high a volatility as possible. That’s why everybody pushed the volatility against them, which contributed to their demise in the end.”

The episode is a lesson in endogenous risk. It’s a risk that differentiates securities markets from other domains governed by probability. “The hurricane is not more or less likely to hit because more hurricane insurance has been written,” mused one of LTCM’s partners afterwards. “In the financial markets this is not true. The more people write financial insurance, the more likely it is that a disaster will happen, because the people who know you have sold the insurance can make it happen. So you have to monitor what other people are doing.”

4. Why the Era of Historically Low Interest Rates Could Be Over – Nick Timiraos

At issue is what is known as the neutral rate of interest. It is the rate at which the demand and supply of savings is in equilibrium, leading to stable economic growth and inflation.

First described by Swedish economist Knut Wicksell a century ago, neutral can’t be directly observed. Instead, economists and policy makers infer it from the behavior of the economy. If borrowing and spending are strong and inflation pressure rising, neutral must be above the current interest rate. If they are weak and inflation is receding, neutral must be lower.

The debate over where neutral sits hasn’t been important until now. Since early 2022, soaring inflation sent the Federal Reserve racing to get interest rates well above neutral.

With inflation now falling but activity still firm, estimates of the neutral rate could take on greater importance in coming months. If neutral has gone up, that could call for higher short-term interest rates, or delay interest-rate cuts as inflation falls. It could also keep long-term bond yields, which determine rates on mortgages and corporate debt, higher for longer…

…Analysts see three broad reasons neutral might go higher than before 2020.

First, economic growth is now running well above Fed estimates of its long-run “potential” rate of around 2%, suggesting interest rates at their current level of 5.25% and 5.5% simply aren’t very restrictive.

“Conceptually, if the economy is running above potential at 5.25% interest rates, then that suggests to me that the neutral rate might be higher than we’ve thought,” said Richmond Fed President Tom Barkin. He said it is too soon to come to any firm conclusions.

That said, a model devised by the Richmond Fed, which before the pandemic closely tracked Williams’s model, put the real neutral rate at 2% in the first quarter.

Second, swelling government deficits and investment in clean energy could increase the demand for savings, pushing neutral higher. Joseph Davis, chief global economist at Vanguard, estimates the real neutral rate has risen to 1.5% because of higher public debt…

…Third, retirees in industrial economies who had been saving for retirement might now be spending those savings. Productivity-boosting investment opportunities such as artificial intelligence could push up the neutral rate.

And business investment depreciates faster nowadays and is thus less sensitive to borrowing costs, which would raise neutral. It is dominated by “computers and software, and much less office buildings, than it used to be,” Summers said during a lecture in May…

…Fed Chair Jerome Powell has in the past warned against setting policy based on unobservable estimates such as neutral, which he compared to navigating by the celestial stars.

Last December, he said the Fed would be careful about fine-tuning interest rates based on such estimates—for example, because falling inflation pushes real rates well above neutral. “I don’t see us as having a really clear and precise understanding of what the neutral rate is and what real rates are,” Powell said.

Some economists reconcile the debate by differentiating between short-run and longer-run neutral. Temporary factors such as higher savings buffers from the pandemic and reduced sensitivity to higher rates from households and businesses that locked in lower borrowing costs could demand higher rates today to slow the economy.

But as savings run out and debts have to be refinanced at higher rates in the coming years, activity could slow—consistent with a neutral rate lower than it is now.

5. Defining, Measuring, and Managing Technical Debt – Ciera Jaspan and Collin Green

We took an empirical approach to understand what engineers mean when they refer to technical debt. We started by interviewing subject matter experts at the company, focusing our discussions to generate options for two survey questions: one asked engineers about the underlying causes of the technical debt they encountered, and the other asked engineers what mitigations would be appropriate to fix this debt…

…This provided us with a collectively exhaustive and mutually exclusive list of 10 categories of technical debt:

• Migration is needed or in progress: This may be motivated by the need to scale, due to mandates, to reduce dependencies, or to avoid deprecated technology.
• Documentation on project and application programming interfaces (APIs): Information on how your project works is hard to find, missing or incomplete, or may include documentation on APIs or inherited code.
• Testing: Poor test quality or coverage, such as missing tests or poor test data, results in fragility, flaky tests, or lots of rollbacks.
• Code quality: Product architecture or code within a project was not well designed. It may have been rushed or a prototype/demo.
• Dead and/or abandoned code: Code/features/projects were replaced or superseded but not removed.
• Code degradation: The code base has degraded or not kept up with changing standards over time. The code may be in maintenance mode, in need of refactoring or updates.
• Team lacks necessary expertise: This may be due to staffing gaps and turnover or inherited orphaned code/projects.
• Dependencies: Dependencies are unstable, rapidly changing, or trigger rollbacks.
• Migration was poorly executed or abandoned: This may have resulted in maintaining two versions.
• Release process: The rollout and monitoring of production needs to be updated, migrated, or maintained.

We’ve continued to ask engineers (every quarter for the last four years) about which of these categories of technical debt have hindered their productivity in the previous quarter. Defying some expectations, engineers do not select all of them! (Fewer than 0.01% of engineers select all of the options.) In fact, about three quarters of engineers select three or fewer categories. It’s worth noting that our survey does not ask engineers “Which forms of technical debt did you encounter?” but only “Which forms of technical debt have hindered your productivity?” It’s well understood that all code has some technical debt; moreover, taking on technical debt prudently and deliberately can be a correct engineering choice.4 Engineers may run into more of these during the course of a quarter, but their productivity may not be substantially hindered in all cases.

The preceding categories of technical debt have been shown in the order of most to least frequently reported as a hindrance by Google engineers in our latest quarter. We don’t expect this ordering to generalize to other companies as the ordering probably says as much about the type of company and the tools and infrastructure available to engineers as it does the state of the code base. For example, Google engineers regularly cite migrations as a hindrance, but large-scale migrations are only attempted at all because of Google’s monolithic repository and dependency system;5 other companies may find that a large-scale migration is so impossible that it is not even attempted. A fresh start-up might have few problems with dead/abandoned code or code degradation but many hindrances due to immature testing and release processes. While we do expect there to be differences across companies in how much engineers are hindered by these categories, we believe the list itself is generalizable.

Our quarterly engineering survey enables us to measure the rate at which engineers encounter and are hindered by each type of technical debt, and this information has been particularly useful when we slice our data for particular product areas, code bases, or types of development. For example, we’ve found that engineers working on machine learning systems face different types of technical debt when compared to engineers who build and maintain back-end services. Slicing this data allows us to target technical debt interventions based on the toolchain that engineers are working in or to target specific areas of the company. Similarly, slicing the data along organizational lines allows directors to track their progress as they experiment with new initiatives to reduce technical debt.

However, we find quarterly surveys are limited in their statistical and persuasive power…

…Our goal was then to figure out if there are any metrics we can extract from the code or development process that would indicate technical debt was forming before it became a significant hindrance to developer productivity. We ran a small analysis to see if we could pull this off with some of the metrics we happened to have already…

…The results were disappointing, to say the least. No single metric predicted reports of technical debt from engineers; our linear regression models predicted less than 1% of the variance in survey responses. The random forest models fared better, but they had high precision (>80%) and low recall (10%–25%). That is, these models could identify parts of the code base where a focused intervention could reduce technical debt, but they were also going to miss many parts of the code base where engineers would identify significant issues.

It is quite possible that better technical debt indicator metrics do exist for some forms of technical debt. We only explored objective metrics for three types of technical debt, and we only sought to use existing metrics, rather than attempting to create new metrics that might better capture the underlying concepts from the survey.

However, it’s also possible that such metrics don’t exist for other types of technical debt because they are not about the present state of a system, but a relation between the system’s present state and some unimplemented ideal state. An engineer’s judgments about technical debt concern both the present state and the possible state. The possible states of the world are something that mathematical models cannot incorporate without the modeler’s direct intervention. For example, the fact that a project’s code base consists entirely of code written in Python 2 is not technical debt in a world where there is no loss of functionality compared to another language or version or outside pressure to migrate. However, in a world where Python 3 is a preferred or required alternative, that same corpus of Python 2 constitutes a needed migration. The present state of the world—from the perspective of a model—is identical in these two instances, but the possible world has changed. Humans consider the possible world in their judgments of technical debt. If a model were to incorporate explicit rules that capture aspects of the possible world (for example, if a model were designed to count every file in Python 2 as technical debt because the human modeler knows Python 3 is an alternative), then the change would be detectable to the model. If we could capture this judgment as it evolves, it could form the basis for better measurements of technical debt…

…While we haven’t been able to find leading indicators of technical debt thus far, we can continue to measure technical debt with our survey and help to identify teams that struggle with managing technical debt of different types. To that end, we also added the following questions to our engineering survey:

• To what extent has your team deliberately incurred technical debt in the past three months?
• How often do you feel that incurring technical debt was the right decision?
• How much did your team invest in reducing existing technical debt and maintaining your code?
• How well does your team’s process for managing technical debt work?

Combined with the survey items about the types of technical debt that are causing productivity hindrances, these questions enable the identification of teams that are struggling, reveal the type(s) of technical debt they are struggling with, and indicate whether they are incurring too much debt initially or whether they are not adequately paying down their existing debt. These are useful data, especially when teams can leverage them under guidance from experts on how to manage their technical debt. Fortunately, we have such experts at Google. Motivated in part by our early findings on technical debt, an interested community within Google formed a coalition to help engineers, managers, and leaders systematically manage and address technical debt within their teams through education, case studies, processes, artifacts, incentives, and tools. The coalition’s efforts have included the following:

• Creating a technical debt management framework to help teams establish good practices. The framework includes ways to inventory technical debt, assess the impact of technical debt management practices, define roles for individuals to advance practices, and adopt measurement strategies and tools.
• Creating a technical debt management maturity model and accompanying technical debt maturity assessment that evaluates and characterizes an organization’s technical debt management process and helps grow its capabilities by guiding it to a relevant set of well-established practices for leads, managers, and individual contributors. The model characterizes a team’s maturity at one of four levels (listed here from least to most mature):
  
  • Teams with a reactive approach have no real processes for managing technical debt (even if they do occasionally make a focused effort to eliminate it, for example, through a “fixit”). Teams with a proactive approach deliberately identify and track technical debt and make decisions about its urgency and importance relative to other work.
  • Teams with a strategic approach have a proactive approach to managing technical debt (as in the preceding level) but go further: designating specific champions to improve planning and decision making around technical debt and to identify and address root causes.
  • Teams with a structural approach are strategic (as in the preceding level) and also take steps to optimize technical debt management locally—embedding technical debt considerations into the developer workflow—and standardize how it is handled across a larger organization.
• Organizing classroom instruction and self-guided courses to evangelize best practices and community forums to drive continual engagement and sharing of resources. This work also includes a technical talk series with live (and recorded) sessions from internal and external speakers.
• Tooling that supports the identification and management of technical debt (for example, indicators of poor test coverage, stale documentation, and deprecated dependencies). While these metrics may not be perfect indicators, they can allow teams who already believe they have a problem to track their progress toward fixing it.

Overall, our emphasis on technical debt reduction has resulted in a substantial drop in the percentage of engineers who report that their productivity is being extremely to moderately hindered by technical debt or overly complicated code in their project. The majority of Google engineers now feel they are only “slightly hindered” or “not at all hindered” by technical debt, according to our survey. This is a substantial change and, in fact, is the largest trend shift we have seen in five years of running the survey.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google). Holdings are subject to change at any time.