We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 14 April 2024:

1. Perplexity is ready to take on Google – Alex Heath and Aravind Srinivas

What’s it like on the frontlines of the AI talent war right now?

I made mistakes in chasing the wrong people. Recently there was a really senior backend engineer who ended up joining X.AI. He was talking to us, too.

I was talking to Patrick Collison for advice on this, and he said, “Why are you even in this race? Why are you trying to compete with these people? Go after people who want to actually build the stuff that you’re building and don’t chase AI clout.”

There are a lot of good engineers who are applying to us and Anthropic and OpenAI and X.AI and Character.ai. These are the top five choices of AI startups. And people normally just go to the highest bidder. Whoever has the highest valuation will be able to win this race all the time because, on paper, you’re always going to be able to offer the same amount of shares but the dollar value is going to be much higher…

...Have you taken any kind of lesson away from the Gemini diversity scandal? I saw you recently integrated photo generation into Perplexity.

Factfulness and accuracy is what we care about. Google has many other cultural things that they care about, and that’s why they made their products that way. They should only prioritize one aspect, which is giving an accurate answer. They don’t do that for whatever reasons. They have all these other people in the room trying to make decisions.

If I learned one thing, it’s that it’s better to be neutral. Don’t try to have any values you inject into the product. If your product is an answer engine, where people can ask questions and get answers, it better respond in a scholarly way. There’s always a nerd in your classroom who’s just always right, but you don’t hate them for having a certain political value, because they are just going to give you facts. That’s what we want to be. And Google’s trying to be something different. That’s why they got into trouble.

What are you hearing generally about the state of Google from people there right now?

The researchers are still pretty excited about what they’re doing. But the product team messes up their releases. The Gemini product team was fine-tuning all these models to put in the product. There’s a lot of bureaucracy, basically.

I know Sergey Brin being there is making things faster and easier for them. You might have seen the video that was circulating of him being at some hackathon. He brushed it [the Gemini diversity scandal] off as just some kind of a small bug, right?

It’s not a small bug. It’s actually poor execution. The image generation thing is actually very easy to catch in testing. They should have caught it in testing. When you consider Google as the place for de facto information and correctness, when they make mistakes it changes the way you perceive the company…

…How much of your tech is in-house versus fine-tuning all these models that you work with? What’s your tech secret sauce?

In the beginning, we were just daisy-chaining GPT-3.5 and Bing. Now, we post-train all these open-source models ourselves. We also still use OpenAI’s model.

We are never going to do the full pre-training ourselves. It’s actually a fool’s errand at this point because it takes so much money to even get one good model by pre-training yourself. There are only four or five companies that are capable of doing that today. And when somebody puts out these open-source models, there’s no reason for you to go and recreate the whole thing.

There is a new term that has emerged in this field called post-training. It’s actually like fine-tuning but done at a much larger scale. We are able to do that and serve our models ourselves in the product. Our models are slightly better than GPT-3.5 Turbo but nowhere near GPT-4. Other than Anthropic and Gemini, nobody has actually gotten to that level yet.

How are you doing to solve AI hallucination in your product? Can you?

The reason why we even have sources at the top of the answer is because we want to make sure that users have the power to go verify the answer. We precisely tell you which link to go to versus showing ten blue links and you not being sure which to read.

The other way is constantly improving the authority of which sources we use to cite the answer and then getting rid of the bad ones. When you don’t have sufficient information, it’s better to say you don’t know rather than saying something you made up.

2. Book Summary: Our Investing Strategy, who does the market smile upon – Made In Japan

He goes by the name of Tatsuro Kiyohara, who was the CIO of Tower Investment Management, which ran the flagship K-1 fund that compounded 20% annually during his 25-year run (that’s 9300%). Compare this to the TOPIX which did an annualized return of roughly 3%.

But its not just the numbers that he posted that were inspiring, the journey to get there was a tumultuous one that would be almost impossible for us to replicate. He is built differently. Who else is willing to pour in almost their entire net worth when the fund is down -72% in an attempt to save his fund not just for his sake, but for the clients that decided to stick with him amid all the redemptions?..

…During his stint in New York, his clients included the familiar hedge funds we’d all heard of. One of which was Julian Robertson’s Tiger Management. Meeting Tiger was perhaps the first instance he decided that he wanted to do something with hedge funds, he would spend his days talking stock at their office. Tiger appreciated him too – one time he realized that Tiger was short a stock, Kawasaki Steel, and also realized Nomura was attempting to ‘promote’ the stock (what you call ‘pump’ these days), he almost had to fight with Tiger to convince exiting and that the stock was going up regardless of fundamentals which they finally obliged. This stock 3xed not long after. Not a surprise that he was invited to Tiger’s annual party to be awarded best salesman of the year…

…The game is to figure out when you’re in the minority. This doesn’t always mean that because everyone is bullish, that you being bullish isn’t a variant perception. If for example the market expects a company to grow by 10% a year for the next 5 years, and you believe it will be more like 30%, you are still in the minority.

It is more difficult to thus have a good investment idea in large caps because its harder to figure out what’s priced in.

From an opportunity cost standpoint for an individual investor, the return on time is so much better researching cheap micro & small-caps with the potential to grow. If you had an hour with the CEO of a large company or a small one the likelihood you will get more valuable insights (your alpha) is much higher in the latter. In general, you also won’t lose much if you’re wrong here, these companies aren’t priced for growth anyway. For him personally, he tries to avoid investing in stocks that have already gone up as much as possible…

…One of his first investments with the fund that falls into this archetype was Nitori, which is a household name today but when he first invested nobody touched it. It was a Hokkaido-based company, and the furniture market in Japan was shrinking. What he realized though was that the market was very fragmented, and he saw Nitori as the one to take market share over time with an exceptional founder. Which proved to be correct. His investment in NItori 10xed all the while the market halved. The lesson here was that even with a shrinking market if you find the right company, you can generate strong returns. No doubt there are some diamonds in the rough…

…In the end, 90% of investing in Small/Micro-caps is about Management
Heres what he looks for:

1. Operators with a growth mindset
2. Talented employees that are aligned with the CEO’s Vision and Mission
3. Doesn’t get crushed by competitors
4. A widening core competence (read = Moat) as it grows
5. Not in an industry where you’re pulling forward demand (There is a finite pile of demand that once absorbed will be gone, he points out the Japanese M&A industry as one)
6. The management is impeccable with their word that is, they do what they say
7. Companies that have positive feedback loops…

…With one company, every time he requested a meeting with Investor Relations, the CEO showed up every time without fail which he found strange. Eventually, though this business got caught in an accounting scandal and went under. Maybe if the CEO shows up too readily you need to be careful. Another business had zero interest in doing IR, showing up in their factory uniform, and wasn’t too friendly. One day however they show up in suits! This business also didn’t too well…

…There are mavericks among them. Founder of Zensho Holdings (the operator of Beef bowl chain Sukiya). This was an unpopular stock that IPOd. The first thing he saw when visiting the founder’s office was a bench press. He was ‘ripped like Popeye’. At the meeting, all he talked about was how superior a food a ‘beef bowl’ (Gyuudon) was. If Japanese people had eaten enough beef bowls and benchpressed enough Japan wouldn’t have lost the war (LOL).

One time one of Kiyohara-sans employees told this founder he’s also been going to the gym, he immediately challenged said employee and tested him with a ‘lariat’ a type of wrestling tackle. The key is to find a CEO who knows his wrestling moves.

The IR was also interesting. In its mid-term plan, they even included the P/E multiple the company should be trading at in 5 years…

…As he once reflected on his portfolio, he realized that the business and its management was like looking at himself in the mirror. If assessing management correctly is the key to investing, also understand that there is a self-selection bias. If the Founder and CEO is that much more brilliant than you – you won’t even realize how brilliant he is. That said you’d never invest in a ‘dumb’ CEO, so ultimately you end up selecting people ‘on your level’. This it appears, to be the reality for investing in microcaps…

…He was adamant about that whether large or small – he had no interest in buying an expensive business. If the P/E was too high, that was a pass.

Over the years he tried building various growth models and realized this had almost no benefit to making money so just stopped. Was a waste of time.

He screens for such companies by finding a high net cash ratio which was just net cash over market cap. (So basically net-nets).

He also liked to invert the problem, by looking at the current P/E of the stock you can figure out the kind of earnings growth it implied. No rocket science here – he tries to figure out the Terminal Multiple with a Perpetual Growth model. For example, if the risk-free rate is 2% and the P/E is 10x that implies a terminal growth of -8.2% all else equal if earnings growth was -3.1% instead then the P/E should be 20x. (Yes this is all negative growth)

The 40-year average of the risk-free rate is about 1.7% in Japan so that sounds fair to him. Also, he says, forget about the concept of equity risk premium – this is just a banker’s term to underwrite uncertainty. If you’re uncertain just model that into your earnings projections…

…He doesn’t look at P/B where Investors may be calculating the liquidation price which can be inaccurate.

The point he thinks many miss – if a company is loss-making would the hypothetical buyer buy the assets at face value? And say that the business is a decent profitable business – no one’s going to be looking at the P/B they’ll be fixated on the P/E…

…Risks of small/micro caps:

• Illiquidity discount
• Many businesses are small suppliers to a much larger company
• It operates in an industry with low entry barriers
• Limited talent within the organization
• Succession issues and nepotism – the son of the owner can be a dumb ass
• Because no one notices them – the likelihood of a fraud/scandal is higher
• When the owner retires, this person may pay out a massive retirement bonus
• Because it’s an owner-operator and harder to take over, no one keeps them in check and may screw up
• When there is an accounting fraud, the damage will be large
• They don’t have the resources to expand overseas
• They have an incentive to keep their valuation low to minimize the inheritance tax.

3. Three reasons why oil prices are remarkably stable – The Economist

Shouldn’t oil prices be surging? War has returned to the Middle East. Tankers in the Red Sea—through which around 12% of seaborne crude is normally shipped—are under attack by Houthi militants. And opec, a cartel of oil exporters, is restricting production. Antony Blinken, America’s secretary of state, has invoked the spectre of 1973, when the Yom Kippur war led to an Arab oil embargo that quadrupled prices in just three months. But oil markets have remained calm, trading mostly in the range of $75 and $85 per barrel for much of last year…

…Oil production is now less concentrated in the Middle East than it has been for much of the past 50 years. The region has gone from drilling 37% of the world’s oil in 1974 to 29% today. Production is also less concentrated among members of OPEC… That is partly because of the shale boom of the 2010s, which turned America into a net energy exporter for the first time since at least 1949…

…Another reason for calm is opec members’ ample spare production capacity (ie, the amount of oil that can be produced from idle facilities at short notice)…

…America’s Energy Information Administration (eia) estimates that opec’s core members have around 4.5m barrels per day of spare capacity—greater than the total daily production of Iraq…

…The world still has a big appetite for oil: according to the eia demand hit a record in 2023 and will be higher still in 2024, thanks in part to growth in India. But that is unlikely to push prices much higher. Global growth is not at the levels seen in the early 2000s. China, long the world’s biggest importer of oil, is experiencing anaemic economic growth. Structural changes to its economy also make it less thirsty for the stuff: next year, for example, half of all new cars sold in the country are expected to be electric.

4. How We’ll Reach a 1 Trillion Transistor GPU – Mark Liu and H.S. Philip Wong

All those marvelous AI applications have been due to three factors: innovations in efficient machine-learning algorithms, the availability of massive amounts of data on which to train neural networks, and progress in energy-efficient computing through the advancement of semiconductor technology. This last contribution to the generative AI revolution has received less than its fair share of credit, despite its ubiquity.

Over the last three decades, the major milestones in AI were all enabled by the leading-edge semiconductor technology of the time and would have been impossible without it. Deep Blue was implemented with a mix of 0.6- and 0.35-micrometer-node chip-manufacturing technology. The deep neural network that won the ImageNet competition, kicking off the current era of machine learning, was implemented with 40-nanometer technology. AlphaGo conquered the game of Go using 28-nm technology, and the initial version of ChatGPT was trained on computers built with 5-nm technology. The most recent incarnation of ChatGPT is powered by servers using even more advanced 4-nm technology. Each layer of the computer systems involved, from software and algorithms down to the architecture, circuit design, and device technology, acts as a multiplier for the performance of AI. But it’s fair to say that the foundational transistor-device technology is what has enabled the advancement of the layers above.

If the AI revolution is to continue at its current pace, it’s going to need even more from the semiconductor industry. Within a decade, it will need a 1-trillion-transistor GPU—that is, a GPU with 10 times as many devices as is typical today…

…Since the invention of the integrated circuit, semiconductor technology has been about scaling down in feature size so that we can cram more transistors into a thumbnail-size chip. Today, integration has risen one level higher; we are going beyond 2D scaling into 3D system integration. We are now putting together many chips into a tightly integrated, massively interconnected system. This is a paradigm shift in semiconductor-technology integration.

In the era of AI, the capability of a system is directly proportional to the number of transistors integrated into that system. One of the main limitations is that lithographic chipmaking tools have been designed to make ICs of no more than about 800 square millimeters, what’s called the reticle limit. But we can now extend the size of the integrated system beyond lithography’s reticle limit. By attaching several chips onto a larger interposer—a piece of silicon into which interconnects are built—we can integrate a system that contains a much larger number of devices than what is possible on a single chip…

…HBMs are an example of the other key semiconductor technology that is increasingly important for AI: the ability to integrate systems by stacking chips atop one another, what we at TSMC call system-on-integrated-chips (SoIC). An HBM consists of a stack of vertically interconnected chips of DRAM atop a control logic IC. It uses vertical interconnects called through-silicon-vias (TSVs) to get signals through each chip and solder bumps to form the connections between the memory chips. Today, high-performance GPUs use HBM extensively…

…With a high-performance computing system composed of a large number of dies running large AI models, high-speed wired communication may quickly limit the computation speed. Today, optical interconnects are already being used to connect server racks in data centers. We will soon need optical interfaces based on silicon photonics that are packaged together with GPUs and CPUs. This will allow the scaling up of energy- and area-efficient bandwidths for direct, optical GPU-to-GPU communication, such that hundreds of servers can behave as a single giant GPU with a unified memory. Because of the demand from AI applications, silicon photonics will become one of the semiconductor industry’s most important enabling technologies…

…We can see the trend already in server GPUs if we look at the steady improvement in a metric called energy-efficient performance. EEP is a combined measure of the energy efficiency and speed of a system. Over the past 15 years, the semiconductor industry has increased energy-efficient performance about threefold every two years. We believe this trend will continue at historical rates. It will be driven by innovations from many sources, including new materials, device and integration technology, extreme ultraviolet (EUV) lithography, circuit design, system architecture design, and the co-optimization of all these technology elements, among other things.

Largely thanks to advances in semiconductor technology, a measure called energy-efficient performance is on track to triple every two years (EEP units are 1/femtojoule-picoseconds).

In particular, the EEP increase will be enabled by the advanced packaging technologies we’ve been discussing here. Additionally, concepts such as system-technology co-optimization (STCO), where the different functional parts of a GPU are separated onto their own chiplets and built using the best performing and most economical technologies for each, will become increasingly critical.

5. The illusion of moral decline – Adam Mastroianni

In psychology, anything worth studying is probably caused by multiple things. There may be lots of reasons why people think morality is declining when it really isn’t.

• Maybe people say that morality is declining because they think it makes them look good. But in Part I, we found that people are willing to say that some things have gotten better (less racism, for instance). And people still make the same claims when we pay them for accuracy.
• Maybe because people are nice to you when you’re a kid, and then they’re less nice to you when you’re an adult, you end up thinking that people got less nice over time. But people say that morality has declined since they turned 20, and that it’s declined in the past four years, and all that is true for old people, too.
• Maybe everybody has just heard stories about how great the past is—like, they watch Leave It to Beaver and they go “wow, people used to be so nice back then.” But again, people think morality has declined even in the recent past. Also, who watches Leave It to Beaver?
• We know from recent research that people denigrate the youth of today because they have positively biased memories of their own younger selves. That could explain why people blame moral decline on interpersonal replacement, but it doesn’t explain why people also blame it on personal change.

Any of these could be part of the illusion of moral decline. But they are, at best, incomplete.

We offer an additional explanation in the paper, which is that two well-known psychological phenomena can combine to produce an illusion of moral decline. One is biased exposure: people pay disproportionate attention to negative information, and media companies make money by giving it to us. The other is biased memory: the negativity of negative information fades faster than the positivity of positive information. (This is called the Fading Affect Bias; for more, see Underrated ideas in psychology).

Biased exposure means that things always look outrageous: murder and arson and fraud, oh my! Biased memory means the outrages of yesterday don’t seem so outrageous today. When things always look bad today but brighter yesterday, congratulations pal, you got yourself an illusion of moral decline.

We call this mechanism BEAM (Biased Exposure and Memory), and it fits with some of our more surprising results. BEAM predicts that both older and younger people should perceive moral decline, and they do. It predicts that people should perceive more decline over longer intervals, and they do. Both biased attention and biased memory have been observed cross-culturally, so it also makes sense that you would find the perception of moral decline all over the world.

But the real benefit of BEAM is that it can predict cases where people would perceive less decline, no decline, or even improvement. If you reverse biased exposure—that is, if people mainly hear about good things that other people are doing—you might get an illusion of moral improvement. We figured this could happen in people’s personal worlds: most people probably like most of the people they interact with on a daily basis, so they may mistakenly think those people have actually become kinder over time.

They do. In another study, we asked people to answer those same questions about interpersonal replacement and personal change that we asked in a previous study, first about people in general, and then about people that they interact with on a daily basis. When we asked participants about people in general, they said (a) people overall are less moral than they were in 2005, (b) the same people are less moral today than in 2005 (personal change) and (c) young people today are less moral than older people were in 2005 (interpersonal replacement). Just as they did before, participants told us that morality declined overall, and that both personal change and interpersonal replacement were to blame.

But we saw something new when we asked participants about people they know personally. First, they said individuals they’ve known for the past 15 years are more moral today. They said the young folks they know today aren’t as moral as the old folks they knew 15 years ago, but this difference was smaller than it was for people in general. So when you ask people about a group where they probably don’t have biased exposure—or at least not biased negative exposure—they report less moral decline, or even moral improvement.

The second thing that BEAM predicts is that if you turn off biased memory, the illusion of moral decline might go away. We figured this could happen if you asked people about times before they were born—you can’t have memories if you weren’t alive. We reran one of our previous studies, simply asking participants to rate people in general today, the year in which they turned 20, the year in which they were born, 20 years before that, and 40 years before that.

People said, basically, “moral decline began when I arrived on Earth”:

Neither of these studies mean that BEAM is definitely the culprit behind the illusion of moral decline, nor that it’s the only culprit. But BEAM can explain some weird phenomena that other accounts can’t, and it can predict some data that other accounts wouldn’t, so it seems worth keeping around for now.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google) and TSMC. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 07 April 2024:

1. China’s capitalist experiment – Michael Fritzell

I just read a great new book by analyst Anne Stevenson-Yang. It’s called Wild Ride and is available for pre-order on Amazon.

The book tells the story of China’s economic miracle from the late 1970s until today － how Deng Xiaoping’s reforms unleashed a wave of entrepreneurship and led to China’s economy becoming one of the largest in the world.

However, it also discusses some of the system’s fragilities and how the country now seems to be turning inwards again…

…China under Mao Zedong was a closed-off, repressive society. Meat was a once-in-a-week luxury. Cooking was done outside. And personal freedoms were more or less non-existent…

…After Mao died in 1976, a power struggle ensued. Ultimately, Mao’s former ally, Deng Xiaoping, emerged victorious from this struggle. One of his first tasks was to open up the economy to the outside world. For this, he would need hard currency.

Practical considerations took priority in those early years. When Deng Xiaoping travelled to the United States in 1979, he ordered an inventory of all hard currency in China’s banks. He came up with only US$38,000 － hardly enough to pay for his delegation.

This was a low point for the Chinese economy. Deng recognized that China needed exports. Japan, Korea, and Taiwan became wealthy by promoting the export of manufactured goods. So Deng adopted a twin strategy of promoting exports in special economic zones while shielding ordinary Chinese from foreign cultural influences…

..Deng’s special economic zones were newly incorporated entities acting as quasi-governments. What made them different was that their managers were rewarded by meeting targets focused on the scale of capital investment and gross tax revenues…

…Initially, foreign influence was kept at bay. Foreign nationals were required to live in special compounds, use separate medical facilities, and even use special currencies. Romantic relationships between foreigners and Chinese were forbidden as well.

The special economic zones in the Southern parts of the Guangdong province, such as Shenzhen, were particularly successful. One of the reasons was that they were near port facilities. But perhaps even more importantly, they had access to financial powerhouse Hong Kong, with its banks and talented entrepreneurs. While, of course, having access to hundreds of millions of workers from inland provinces.

In fact, Shenzhen became a model for the China that was about to develop. It was the first city to abolish the food coupon system, thus allowing residents to buy food with their own money. And residents were soon allowed to lease their own land…

…Another important part of Deng’s reforms was allowing farmers to grow whatever they pleased after meeting some quota. They could then sell any surplus in newly established markets. This unleashed immense rural income growth of 12% per year throughout the 1980s.

A similar system was later introduced to state-owned enterprises as well. They were now allowed to retain profits, either for reinvestment or pay them out as bonuses to employees. Managers suddenly realized they had incentives to increase revenues and profits, and some became wealthy…

…But beneath the surface, discontent was growing. Students were devouring books brought in from overseas. They were clamoring not only for economic gains but also for political reforms. By 1987, Beijing students regularly held marches from the university districts to Tian’anmen Square to protect against political restrictions…

…The crackdown on the student demonstrations in Beijing in June 1989 led to a significant political shift. For two years after the massacre, the country closed off, and dissidents were hunted down and jailed. Anyone who participated in the protests was either disappeared, jailed, demoted or unable to attend university or get a good job.

After the student protests, the Communist Party shifted its strategy to maintaining control. It upped its propaganda efforts, conveying that if the party were to collapse, China would end up in total anarchy.

In the aftermath of Tian’anmen, a communication system was established that improved the party’s control over the provinces. Tax collection and audits were tightened, and a criminal detection and surveillance system was developed…

…One of Deng’s buzzwords during this era was “to get rich is glorious” (致富光荣). You no longer had to be ashamed of pursuing wealth; it was promoted from the top down.

The Communist Party bet that as long as people felt their livelihoods improved, they would not rock the boat. The restive students who protested at Tian’anmen Square would now focus on economic opportunity rather than spiritual dissatisfaction.

After his come-back in the early 1990s, Deng picked out young talent Zhu Rongji to push for further reforms. In a long list of achievements, Zhu Rongji managed to:

• Cut the government bureaucracy in half
• Privatize housing
• Sell off 2/3 of the companies in the state sector
• Unify the dual currencies used prior to 1994
• Introduce a nationwide tax system
• Take control of the appointment of all provincial-level governors…

…After the reforms of the 1990s, China’s economic growth really took off. Exporters in China’s coastal regions benefitted from the country’s admission into the WTO, and Chinese returnees started businesses left and right…

…It was also during the 2000s that the property boom really kicked into high gear. In the late 1990s, Zhu Rongji instituted reforms that allowed state-owned enterprises to sell worker housing back to tenants for a pittance. As prices rose throughout the 2000s, tenants now held significant household equity, which they could then leverage to buy new, even fancier, commodity housing.

A change in the tax structure also incentivized local governments to promote construction. In the mid-1990s, the central government established its own offices to collect taxes directly. In other words, local governments had less ability to raise taxes themselves, instead relying on remittances from the central government. Local governments thus became cash-poor.

To fund their spending programs, they instead set up local government financing vehicles (LGFVs), which used land as collateral for borrowing. And since they were government entities, they were seen as quasi-sovereign borrowers enjoying full access to loans from state banks. Over time, the number of LGFVs grew to over 10,000. They operate urban infrastructure, subway systems, water and gas utilities, etc. Some of them are profitable, but many of them are not…

…The privatization of China’s housing market, which provided collateral for new loans, created one of the biggest credit booms the world has ever seen. Later on, in just five years, more credit was created than the entire value of the US banking system…

…After the Great Financial Crisis of 2008, the Communist Party leadership unleashed a CNY 4 trillion stimulus program that brought forward demand for infrastructure and spending targets.

At this point, it was already becoming clear that the capital stock for infrastructure was starting to exceed those of most other developing or even developed economies. By 2012, China had 8x the length of highways per unit of GDP as that of Japan. At the time, more than 70% of China’s airports were failing to cover their own costs, even though such costs tend to be modest…

…Meanwhile, with the state pushing for big stimulus packages, the government increasingly directed economic resources. Concepts such as “advance of the state, retreat of the private sector” (国进民退) became more common, reflecting a shift in the economy away from private sector entrepreneurship…

…And indeed, with the emergence of Xi Jinping, the state has started to reassert control. State companies are now receiving most of the loans from China’s banks. State media is now talking of “national rejuvenation”, trying to unite the country around nationalist sentiment and acceptance of a “moderately prosperous lifestyle” (小康社会). This is a clear break from the era of Deng Xiaoping’s reforms when getting rich was perhaps the greatest virtue in life…

…Further, she believes that a Russia-Iran-China bloc is currently being formed and that China’s financial system could serve as a bedrock for trade within the bloc:

“If, however, China were someday to shrink its network of trading partners to other dictatorships like Russia and North Korea, its dedicated financial system could become the principal one used for trade among those nations.”

In other words, Anne believes that China is withdrawing from its informal pact with Western nations about open trade, with the experiment in Western-style capitalism that commenced in 1979 over. The Chinese economy is now morphing into a different system, one where the state reigns supreme and will become an influential partner in a new trading bloc formed by China’s current geopolitical allies.

2. 20 Lessons From 20 Years of Managing Money – Ben Carlson

1. Experiences shape your perception of risk. Your ability and need to take risk should be based on your stage in life, time horizon, financial circumstances and goals.

But your desire to take risk often trumps all that, depending on your life experiences. If you worked at Enron or Lehman Brothers or AIG or invested with Madoff, your appetite for risk will be forever altered.

And that’s OK as long as you plan accordingly.

2. Intelligence doesn’t guarantee investment success. Warren Buffett once wrote, “Investing is not a game where the guy with the 160 IQ beats the guy with the 130 IQ. Once you have ordinary intelligence, what you need is the temperament to control the urges that get other people into trouble in investing.”

I’ve met so many highly educated individuals who are terrible investors. They can’t control their emotions because their academic pedigree makes them overconfident in their abilities.

Emotional intelligence is the true sign of investment smarts.

3. No one lives life in the long-term. Long-term returns are the only ones that matter but you have to survive a series of short-terms to get there.

The good strategy you can stick with in those short-terms is preferable to the perfect strategy you can’t stick with…

…9. The biggest risks are always the same…yet different. The next risk is rarely the same as the last risk because every market environment is different.

On the other hand, the biggest mistakes investors make are often the same — timing the market, recency bias, being fearful when others are fearful and greedy when others are greedy and investing in the latest fads.

It’s always a different market but human nature is the constant…

…16. Experience is not the same as expertise. Just because you’ve been doing something for a long time doesn’t mean you’re an expert.

I know plenty of experienced investors who are constantly fighting the last war to their own detriment.

How many people who “called” the 2008 crash completely missed the ensuing bull market? All of them?

How many investment legends turn into permabears the older they get becasue they fail to recognize how markets have changed over time?

Loads of investment professionals who have been in the business for many years make the same mistakes over and over again…

…18. There is a big difference between rich and wealthy. Lots of rich people are miserable. These people are not wealthy, regardless of how much money they have.

There are plenty of people who wouldn’t be considered rich based on the size of their net worth who are wealthy beyond imagination because of their family, friends and general contentment with what they have.

19. Optimism should be your default. It saddens me to see an increasing number of cynical and pessimistic people every year.

I understand the world can be an unforgiving place and things will never be perfect but investing is a game where the optimists win.

3. 8 Google Employees Invented Modern AI. Here’s the Inside Story – Steven Levy

EIGHT NAMES ARE listed as authors on “Attention Is All You Need,” a scientific paper written in the spring of 2017. They were all Google researchers, though by then one had left the company…

…Recurrent neural networks struggled to parse longer chunks of text. Take a passage like Joe is a baseball player, and after a good breakfast he went to the park and got two hits. To make sense of “two hits,” a language model has to remember the part about baseball. In human terms, it has to be paying attention. The accepted fix was something called “long short-term memory” (LSTM), an innovation that allowed language models to process bigger and more complex sequences of text. But the computer still handled those sequences strictly sequentially—word by tedious word—and missed out on context clues that might appear later in a passage. “The methods we were applying were basically Band-Aids,” Uszkoreit says. “We could not get the right stuff to really work at scale.”

Around 2014, he began to concoct a different approach that he referred to as self-attention. This kind of network can translate a word by referencing any other part of a passage. Those other parts can clarify a word’s intent and help the system produce a good translation. “It actually considers everything and gives you an efficient way of looking at many inputs at the same time and then taking something out in a pretty selective way,” he says. Though AI scientists are careful not to confuse the metaphor of neural networks with the way the biological brain actually works, Uszkoreit does seem to believe that self-attention is somewhat similar to the way humans process language.

Uszkoreit thought a self-attention model could potentially be faster and more effective than recurrent neural nets. The way it handles information was also perfectly suited to the powerful parallel processing chips that were being produced en masse to support the machine learning boom. Instead of using a linear approach (look at every word in sequence), it takes a more parallel one (look at a bunch of them together). If done properly, Uszkoreit suspected, you could use self-attention exclusively to get better results…

…The transformer crew set about building a self-attention model to translate text from one language to another. They measured its performance using a benchmark called BLEU, which compares a machine’s output to the work of a human translator. From the start, their new model did well. “We had gone from no proof of concept to having something that was at least on par with the best alternative approaches to LSTMs by that time,” Uszkoreit says. But compared to long short-term memory, “it wasn’t better.”

They had reached a plateau—until one day in 2017, when Noam Shazeer heard about their project, by accident. Shazeer was a veteran Googler—he’d joined the company in 2000—and an in-house legend, starting with his work on the company’s early ad system. Shazeer had been working on deep learning for five years and recently had become interested in large language models. But these models were nowhere close to producing the fluid conversations that he believed were possible.

As Shazeer recalls it, he was walking down a corridor in Building 1965 and passing Kaiser’s workspace. He found himself listening to a spirited conversation. “I remember Ashish was talking about the idea of using self-attention, and Niki was very excited about it. I’m like, wow, that sounds like a great idea. This looks like a fun, smart group of people doing something promising.” Shazeer found the existing recurrent neural networks “irritating” and thought: “Let’s go replace them!”

Shazeer’s joining the group was critical. “These theoretical or intuitive mechanisms, like self-attention, always require very careful implementation, often by a small number of experienced ‘magicians,’ to even show any signs of life,” says Uszkoreit. Shazeer began to work his sorcery right away. He decided to write his own version of the transformer team’s code. “I took the basic idea and made the thing up myself,” he says. Occasionally he asked Kaiser questions, but mostly, he says, he “just acted on it for a while and came back and said, ‘Look, it works.’” Using what team members would later describe with words like “magic” and “alchemy” and “bells and whistles,” he had taken the system to a new level.

“That kicked off a sprint,” says Gomez. They were motivated, and they also wanted to hit an upcoming deadline—May 19, the filing date for papers to be presented at the biggest AI event of the year, the Neural Information Processing Systems conference in December. As what passes for winter in Silicon Valley shifted to spring, the pace of the experiments picked up. They tested two models of transformers: one that was produced with 12 hours of training and a more powerful version called Big that was trained over three and a half days. They set them to work on English-to-German translation.

The basic model outperformed all competitors—and Big earned a BLEU score that decisively shattered previous records while also being more computationally efficient. “We had done it in less time than anyone out there,” Parmar says. “And that was only the beginning, because the number kept improving.”…

…TRANSFORMERS DID NOT instantly take over the world, or even Google. Kaiser recalls that around the time of the paper’s publication, Shazeer proposed to Google executives that the company abandon the entire search index and train a huge network with transformers—basically to transform how Google organizes information. At that point, even Kaiser considered the idea ridiculous. Now the conventional wisdom is that it’s a matter of time.

A startup called OpenAI was much faster to pounce. Soon after the paper was published, OpenAI’s chief researcher, Ilya Sutskever—who had known the transformer team during his time at Google—suggested that one of its scientists, Alec Radford, work on the idea. The results were the first GPT products. As OpenAI CEO Sam Altman told me last year, “When the transformer paper came out, I don’t think anyone at Google realized what it meant.”

The picture internally is more complicated. “It was pretty evident to us that transformers could do really magical things,” says Uszkoreit. “Now, you may ask the question, why wasn’t there ChatGPT by Google back in 2018? Realistically, we could have had GPT-3 or even 3.5 probably in 2019, maybe 2020. The big question isn’t, did they see it? The question is, why didn’t we do anything with the fact that we had seen it? The answer is tricky.”

Many tech critics point to Google’s transition from an innovation-centered playground to a bottom-line-focused bureaucracy. As Gomez told the Financial Times, “They weren’t modernizing. They weren’t adopting this tech.” But that would have taken a lot of daring for a giant company whose technology led the industry and reaped huge profits for decades. Google did begin to integrate transformers into products in 2018, starting with its translation tool. Also that year, it introduced a new transformer-based language model called BERT, which it started to apply to search the year after.

But these under-the-hood changes seem timid compared to OpenAI’s quantum leap and Microsoft’s bold integration of transformer-based systems into its product line. When I asked CEO Sundar Pichai last year why his company wasn’t first to launch a large language model like ChatGPT, he argued that in this case Google found it advantageous to let others lead. “It’s not fully clear to me that it might have worked out as well. The fact is, we can do more after people had seen how it works,” he said…

…Does Google miss these escapees? Of course, in addition to others who have migrated from the company to new AI startups. (Pichai reminded me, when I asked him about the transformer departures, that industry darling OpenAI also has seen defections: “The AI area is very, very dynamic,” he said.) But Google can boast that it created an environment that supported the pursuit of unconventional ideas. “In a lot of ways Google has been way ahead—they invested in the right minds and created the environment where we could explore and push the envelope,” Parmar says. “It’s not crazy that it took time to adopt it. Google had so much more at stake.”

Without that environment: no transformer. Not only were the authors all Google employees, they also worked out of the same offices. Hallway encounters and overheard lunch conversations led to big moments. The group is also culturally diverse. Six of the eight authors were born outside the United States; the other two are children of two green-card-carrying Germans who were temporarily in California and a first-generation American whose family had fled persecution, respectively.

4. In Depth: Local Governments Struggle to Tackle Mountain of Hidden Debt – Cheng Siwei, Wang Juanjuan, Zhang Yuzhe, Ding Feng and Zhang Yukun

The central government has been trying to address the problem of LGFV debt for years, mainly through piecemeal measures that had limited success. But in July, the Politburo vowed to formulate and implement a comprehensive strategy to resolve local government hidden debts.

These off-the-books liabilities, which include LGFV bonds with implicit official backing, have accumulated over the years to around 30 trillion to 70 trillion yuan according to some estimates, and become a threat to the country’s fiscal and financial stability and sustainability.

One of the main instruments being used to repay hidden debt in this round of debt resolution is special refinancing bonds — on-balance-sheet local government bonds whose proceeds are used to repay outstanding hidden debt. Issuance has stepped up significantly since early October after the Ministry of Finance launched a special refinancing bond swap program.

From October to December, almost all provincial-level regions on the Chinese mainland issued these special refinancing bonds, raising nearly 1.4 trillion yuan to repay hidden borrowings, according to calculations by analysts at Tianfeng Securities Co. Ltd. The regions include heavily indebted Guizhou province, which topped the list with issuance of 226.4 billion yuan.

Many regions have announced plans to issue more such bonds in February and March, with planned issuances totaling more than 100 billion yuan, the Tianfeng analysts wrote in a January report.

The campaign to resolve hidden debt has tightened rules for new debt issuance and cut some localities off from their previous financing channels, depriving them of resources to pay interest on hidden debt. The proceeds of special refinancing bonds cannot be used to make interest payments.

“The core issue now is that we can’t make our interest payments,” a source who works for an economic development zone in West China told Caixin, noting that without new financing, the fiscal revenue of the region can only sustain government agencies’ day-to-day operations and preferential policies for attracting businesses. He said his local government has stopped making all other payments, including those to project developers, to ensure it can meet interest payments on outstanding LGFV debt…

…The renewed push to bring hidden debt onto the books and restructure or swap LGFV debt, however, has reinforced the belief that the central government won’t allow LGFVs to default on their bonds, reviving investor sentiment. That’s led to a surge in demand for LGFV bonds over the past few months, even as the central government has repeatedly highlighted the need to stem any renewed buildup in hidden debt…

…Although LGFV bonds are back in hot demand, tightened oversight has made it more difficult for some vehicles, especially those with heavy debt burdens, to continue issuing new debt. This has curbed growth in hidden debt to some extent, but it has added to default risks of some LGFV bonds as there is less money available to make the interest repayments.

The central government ordered provincial officials to compile a list of LGFVs owned by local authorities in their jurisdictions…

…Obtaining new bank loans has become much harder for LGFVs on the list, as banks heed the central government’s instruction to prevent new LGFV debt.

Regarding existing LGFV debt, the State Council in September issued guidance that banks, among the most important creditors of LGFVs, should become involved in debt resolution in 12 provincial-level regions with high government leverage, which include Liaoning, Heilongjiang, and Jilin, the three rustbelt provinces in Northeast China. The guidance set out that banks should focus on restructuring or swapping existing loans, high-interest non-standard debt, and other types of borrowing.

5. Conviction and Quality – Josh Tarasoff

Conviction is no doubt the foundation of long-term business ownership. How is it formed? What is it like to have it? Why does it falter? In my experience there are two distinct kinds of conviction. Explicit conviction, as I call it, comes from having figured something out. It entails a useful prediction, like “our ETA is 5pm” or “majoring in economics will lead to better career prospects than majoring in philosophy.” There is an underlying logic to it, which can be explained and used to persuade. Implicit conviction, on the other hand, is exemplified by the trust one might have in a family member, a dear friend, a close colleague, to do the right thing, to get the job done, to come through. It is felt as opposed to believed. This kind of conviction doesn’t make predictions so much as align with what is good. It doesn’t theorize about goodness but rather knows it when it sees it…

…In the context of investing, one might develop the thesis that a particular company can capture X% market share, generate Y dollars in annual revenue, achieve Z% operating margins, and therefore has an intrinsic value within a certain range. One might have high confidence because of the presence of competitive advantages and management with a very good track record. One would have a range of expected returns from owning the shares over time. All of this would fall into the explicit category.

Sooner or later, the investment would encounter a confounding surprise. Perhaps execution turns choppy, a new competitive vector emerges out of nowhere, an exogenous crisis turns the world upside down, etc. Old projections are now in doubt, previous plans and strategies are being reworked, everything is less fun. These things are actually happening all the time— something explicit conviction has a way of tuning out! Only genuine and well-placed implicit conviction, a qualitative knowing that the company will do what it needs to and ought to do, is equipped to ably traverse this kind of terrain. Unlike analysis-based explicit conviction, implicit conviction comes from something deeper than the cause and effect we perceive in the unfolding of events—it is both analytical and, crucially, intuitive (about which more later)…

…While in everyday life implicit conviction arises naturally, in the context of investing I can’t help but feel it is somewhat alien. In part, this is because few companies are truly deserving. Even so, I suspect that implicit conviction is proffered by investors even less than it ought to be. It isn’t difficult to see why the investment industry is inhospitable to implicit conviction, and why its partner rules the roost. Implicit conviction forms of its own accord and cannot be planned. It defies quantification, eliciting the charge of being too “fuzzy” to matter. Nor can it be fully captured in words. Implicit conviction is impossible to transmit from analyst to portfolio manager or from portfolio manager to client, which is highly inconvenient for the business of managing money. It is primarily personal. It is quiet. By contrast, the appeal of the explicit is clear. Explicit conviction furnishes the comfort of knowability and modeled outcomes. It projects the legitimacy of diligence and precision. It is thought to be reliably manufactured via “repeatable process.” It is clever and self assured….

…Nonetheless, because literal communication necessitates choosing a word, I will use “Quality” (capitalized to distinguish it from the ordinary sense of the term) to indicate the deeper-something on which implicit conviction is based. Using “Quality” in this way is consistent with my prior writing and pays homage to the work of Robert Pirsig, which was a formative influence.

Analysis plays an important but limited role in detecting Quality. For example, the following is a selection of (neither necessary nor sufficient) indicators that I have found to be suggestive of Quality in companies:

• “Wow” customer experiences
• Mission to solve an important problem
• Domain mastery (the best at what they do)
• First-principles-based thinking and invention
• Unlimited ambition combined with no-nonsense realism
• Overcapitalized balance sheet
• Founder mentality (life’s work).

While carefully looking for indicators like these is helpful, I think it would be a misstep to attempt to systematize the search, constructing a Grand Unified Theory of Quality and attendant comprehensive processes for finding and evaluating it. Quality emerges from the complexity of the system in action; it is in the how rather than the what. Thus, when Quality is broken down into parts and analyzed, its essence is lost. This explains why analysis alone has trouble discerning the authentic from the artificial. Moreover, Quality frozen in a theory or process cannot be recognized in sufficiently new contexts, such as in a company that is novel to one’s experience or in the same company as it evolves (they always do!).

So where does that leave us? With intuition. Well-honed intuition does what analysis cannot by perceiving Quality directly, as opposed to through an intellectual process. What I suspect is happening in the direct perception of Quality is subconscious pattern recognition, based upon a dynamic, holistic experience of the thing in question. Of course, the ability to intuitively recognize patterns in a specific domain must be earned through experience and feedback; indeed, I have found that the value of my own intuition has grown (starting at zero) over many years. Interestingly, I also find that experiencing Quality in any one domain (e.g., music or meditation, to use examples that are dear to me) can be helpful for recognizing it in other domains (including business) because Quality’s nature is universal, even as its manifestations are necessarily particular.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google) and Microsoft. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 31 March 2024:

1. Gold-Medalist Coders Build an AI That Can Do Their Job for Them – Ashlee Vance

Take the case of Cognition AI Inc.

You almost certainly have not heard of this startup, in part because it’s been trying to keep itself secret and in part because it didn’t even officially exist as a corporation until two months ago. And yet this very, very young company, whose 10-person staff has been splitting time between Airbnbs in Silicon Valley and home offices in New York, has raised $21 million from Peter Thiel’s venture capital firm Founders Fund and other brand-name investors, including former Twitter executive Elad Gil. They’re betting on Cognition AI’s team and its main invention, which is called Devin.

Devin is a software development assistant in the vein of Copilot, which was built by GitHub, Microsoft and OpenAI, but, like, a next-level software development assistant. Instead of just offering coding suggestions and autocompleting some tasks, Devin can take on and finish an entire software project on its own. To put it to work, you give it a job—“Create a website that maps all the Italian restaurants in Sydney,” say—and the software performs a search to find the restaurants, gets their addresses and contact information, then builds and publishes a site displaying the information. As it works, Devin shows all the tasks it’s performing and finds and fixes bugs on its own as it tests the code being written.

The founders of Cognition AI are Scott Wu, its chief executive officer; Steven Hao, the chief technology officer; and Walden Yan, the chief product officer…

…Wu, 27, is the brother of Neal Wu, who also works at Cognition AI. These two men are world-renowned for their coding prowess: The Wu brothers have been competing in, and often winning, international coding competitions since they were teenagers…

…Sport-coding—yes, it’s a real thing—requires people to solve puzzles and program with speed and accuracy. Along the way, it trains contestants to approach problems in novel ways. Cognition AI is full of sport-coders. Its staff has won a total of 10 gold medals at the top international competition, and Scott Wu says this background gives his startup an edge in the AI wars…

…One of the big claims Cognition AI is making with Devin is that the company has hit on a breakthrough in a computer’s ability to reason. Reasoning in AI-speak means that a system can go beyond predicting the next word in a sentence or the next snippet in a line of code, toward something more akin to thinking and rationalizing its way around problems. The argument in AI Land is that reasoning is the next big thing that will advance the industry, and lots of startups are making various boasts about their ability to do this type of work.

Devin does appear to be well ahead of the other coding assistants in many respects. You can give it jobs to do with natural language commands, and it will set off and accomplish them. As Devin works, it tells you about its plan and then displays the commands and code it’s using. If something doesn’t look quite right, you can give the AI a prompt to go fix the issue, and Devin will incorporate the feedback midstream. Most current AI systems have trouble staying coherent and on task during these types of long jobs, but Devin keeps going through hundreds and even thousands of tasks without going off track.

In my tests with the software, Devin could build a website from scratch in 5 to 10 minutes, and it managed to re-create a web-based version of Pong in about the same amount of time. I had to prompt it a couple of times to improve the physics of the ball movement in the game and to make some cosmetic changes on its websites, all of which Devin accomplished just fine and with a polite attitude…

…Exactly how Cognition AI made this breakthrough, and in so short a time, is something of a mystery, at least to outsiders. Wu declines to say much about the technology’s underpinnings other than that his team found unique ways to combine large language models (LLMs) such as OpenAI’s GPT-4 with reinforcement learning techniques. “It’s obviously something that people in this space have thought about for a long time,” he says. “It’s very dependent on the models and the approach and getting things to align just right.”

2. Geopolitics in the C-Suite – Jami Miscik, Peter Orszag, and Theodore Bunzel

But even though national security and foreign policy occasionally intruded on corporate America during that time, until very recently, few executives concerned themselves with geopolitics. In the post–Cold War world, with globalization on the march, the idea that national interests might be at odds with open markets and expanding trade came to seem alien to American executives.

But the changes that have roiled the geopolitical landscape in recent years have left an impression in C-suites around the United States. In a recent poll of 500 institutional investors, geopolitics ranked as the top risk to the global economy and markets in 2024…

…As governments lean on economic restrictions and industrial policies to achieve geopolitical ends, corporations have increasingly become both the objects and instruments of foreign policy…

…The centrality of economic competition to today’s foreign policy problems represents a qualitative break from the past. During the Cold War, for example, the United States and the Soviet Union hardly interacted economically: trade between them peaked at a paltry $4.5 billion in 1979; in recent years, the United States and China have generally traded that much every week or two, adjusting for inflation. In the post–Cold War era, U.S. foreign policy was focused on opening markets and reducing international economic barriers rather than erecting them. Era-defining crises such as the 9/11 attacks did little to change the relationship between U.S. policymakers and American corporations; if anything, the “war on terror” further solidified the idea that foreign policy was primarily concerned with security and military issues, not economics.

But in the background, global economic integration was transforming the playing field. In 1980, trade accounted for just 37 percent of global GDP. Today, that figure is 74 percent, and economies have become intertwined to a degree never seen in the twentieth century. Globalization is not new, of course; it has been a centuries-long process. What is new, however, is the emergence of great-power rivalry in a highly interconnected world. Military power still matters, but economic and technological competition have become the main battlefield of global politics. Under the so-called Washington consensus that dominated policymaking for decades, the question of where a semiconductor manufacturer would build its next factory or whether German auto companies would decide to throttle their investments in China would have seemed relatively unimportant to policymakers. Now, such questions are at the center of almost every major foreign policy debate.

Greater economic integration has also created a complex web of links between geopolitical rivals that policymakers now seek to leverage for strategic ends. This is especially true when it comes to financial and technological networks, where Washington holds a privileged position…

…But as great-power tensions have increased, so has the number of sectors caught in the fray of what Farrell and Newman call “weaponized interdependence.” Consider, for example, the way that G-7 countries have taken advantage of Russian dependence on shipping insurers based in the West, an industry that most foreign policymakers had probably never thought about before Russia’s 2022 invasion of Ukraine. To try to cap the price of Russian oil exports, the G-7 prevented these companies from insuring Russian crude oil cargoes unless they had been sold at a maximum of $60 per barrel.

Western powers are not the only ones playing this game. In 2010, after a Chinese fishing trawler and Japanese Coast Guard patrol boats collided in disputed waters, setting off a diplomatic row between Beijing and Tokyo, China banned exports to Japan of the rare-earth minerals that are critical components of batteries and electronics, thus raising costs and creating shortages for Japanese manufacturers of everything from hybrid cars to wind turbines…

…More recently, a number of American consulting firms have been caught in the middle of the complex U.S.-Saudi relationship, with Congress demanding details about their contracts with Saudi Arabia that Riyadh has forbidden them to provide.

All these dynamics are being turbocharged by an intensifying competition between the United States and China, the two countries with the largest and most globally intertwined economies. Both aim to dominate the twenty-first-century economy, which means gaining the upper hand in computing technologies, biotechnology, and clean energy. And the foreign policies of both countries are now driven by a shared desire to shape their economies in ways that reduce their vulnerability and increase their leverage. China calls this “self-reliance.” Washington calls it “de-risking.” For the United States, what it looks like in practice is expanded export controls on advanced semiconductors and manufacturing equipment, enhanced government screening of investments by U.S. companies in foreign markets, and major subsidies for industries such as electric vehicles and microchips, primarily through the Inflation Reduction Act and the CHIPS Act. In this brave new world, the secretary of commerce is as important to foreign policy as the secretaries of state and defense.

Washington is hardly alone in taking such steps. State-sponsored drives for greater self-reliance have taken hold in nearly every major economy, particularly after the supply-chain disruptions of the COVID-19 pandemic. The number of countries introducing or expanding investment screening, for example, jumped from three between 1995 and 2005 to 54 between 2020 and 2022. Meanwhile, a wave of industrial policies has increased trade barriers in an attempt to induce companies to reshore their supply chains. At the same time, the understanding of what matters to national security has also expanded, as countries seek to advance or protect everything from software and microchips to pharmaceuticals and foodstuffs.

Many of the complications of this new era are rooted in the difference between the way the public and private sectors view time horizons. Policymakers set bright lines with immediate operational implications—for example, suddenly forbidding companies from exporting or importing certain goods from certain countries. But companies need to make long-term investment decisions. Should a company set up another plant in China if there is market demand and doing so is currently allowed by law? Should a pharmaceutical company set up advanced R & D centers in mainland China or purchase a Chinese biotech firm, given the long-run trajectory of relations between Beijing and the West? Should a consumer electronics firm purchase Chinese-made chips if they are the most cost-efficient option? Answering these questions requires executives to forecast the outcomes of highly volatile political debates and policymaking choices over which they have little control. And yet whatever decisions they make have a significant effect on whether, for example, the United States can effectively “de-risk” its economic relationship with China.

The example of semiconductors is instructive. Washington is seeking to reshore semiconductor manufacturing, but the success of its flagship industrial policy, the CHIPS Act, depends only in part on how the Commerce Department distributes the legislation’s $39 billion in subsidies over the next five years. A much more important factor is whether the Taiwanese chip manufacturer TSMC will risk setting up facilities in the United States despite high costs and a relative scarcity of human capital, and whether Apple decides to buy slightly more expensive chips made by U.S. fabricators instead of less expensive ones produced in Asia. And the CHIPS Act is only one input in those decisions.

3. Get Smart: Chasing Nvidia? Don’t succumb to FOMO – Chin Hui Leong

Are you feeling left out because you missed Nvidia’s (NASDAQ: NVDA) massive stock rise?

Well, we have good news and bad news.

Let’s start with the bad news: that tightening in your chest you are feeling right now is the fear of missing out — or better known by its initials “FOMO”. 

And ladies and gentlemen, FOMO is real.

It’s that sneaky emotion which spurs you to buy a stock based on a feeling rather than proper research…

…But hang on, amid the hype — there’s good news too.

If you recognise that you are feeling FOMO, then congratulations — you have just taken the first step in recognising what you have to deal with: your runaway emotions.

The next step is to keep your emotions in check.

On the other side of FOMO, is its cousin FOJI — or the fear of joining in.

Like FOMO, FOJI is also a strong emotion.

That’s especially true for some investors who are bearing scars from 2022 when US stocks took a beating amid a punishing bear market.

These scars can emit another fear — FOJI — which is paralysing for investors.

The fear of looking stupid if you buy today only to watch the stock fall the very next day…

…Whether it is FOMO or FOJI, you won’t invest well if feelings dictate your actions.

Recognising the presence of both emotions is key…

…Beyond FOMO and FOJI, there’s JOMO or the joy of missing out.

Don’t feel down if you decide to give Nvidia a pass.

As Peter Lynch once said — you can’t kiss all the frogs to find out which will turn into a prince.

Unconvinced?

In Lynch book’s “One Up on Wall Street”, he wrote down the names of 65 stocks which returned at least 10 times their original price (he calls them 10-baggers).

Except that the fund that he ran owned NONE of them.

Before you start rolling your eyes, consider this point: Peter Lynch achieved a stunning 29% per year in annualized returns over 13 years, outpacing the benchmark S&P 500 index (INDEXSP: .INX) by more than two times…

…By sharing the list of missed winners, Lynch had a salient point to make: you do not need to be in every 10-bagger to deliver enviable returns.

4. How the richest woman in the world—mocked as a ‘miser’ in the press—helped bail out New York City during the panic of 1907 – Will Daniel

Hetty Green is remembered as the “world’s greatest miser” and the “Witch of Wall Street,” but these days, Green would likely be seen as an eccentric investing icon. After all, while she became famous for her frugal nature and gruff exterior, Green pioneered value investing strategies that have made billionaires out of many of today’s leading investors. And when the chips were down, when people really needed help, the whaling heiress turned independent investor, business tycoon, and world’s wealthiest woman often used her fortune to save the day…

…Over a three-week period after the panic began on Oct. 22, 1907, the New York Stock Exchange plummeted nearly 50% from its 1906 peak. And a year later, in 1908, Gross National Product (GNP), a measure akin to today’s Gross Domestic Product (GDP), cratered 12%. The problems for the banking system were so severe during the knickerbocker crisis that they spurred the establishment of the Federal Reserve System…

…As the situation deteriorated, John Pierpont Morgan, the American financier who founded what is now JPMorgan Chase, was eventually forced to call together a group of Wall Street’s best and brightest at the Morgan Library to help decide how to prop up the ailing economy and stock market. Hetty Green was the only woman who was invited to attend that meeting during the height of the panic…

…“I saw this situation coming,” she said, noting that there were undeniable signs of stress. “Some of the solidest men of the Street came to me and wanted to unload all sorts of things, from palatial residences to automobiles.”

Green said that she then gave The New York Central Railroad company a “big loan” after they came knocking, and that made her “sit up and do some thinking.” She decided to begin gathering as much cash as possible, understanding that a panic could be on the way…

…Green described how men came to New York from all over the country to ask for loans during the panic of 1907. But despite being labeled a “miser” throughout her life, she didn’t take advantage of the situation.

“Those to whom I loaned money got it at 6%. I might just as easily have secured 40%,” she explained…

…Usury, or charging excessive interest for a loan, was against Green’s moral code, which was born of her Quaker roots…

…Green would go on to lend the government of New York City $1.1 million at the peak of the 1907 panic, which is equivalent to roughly $33 million in today’s dollars…

…“On more than one occasion, when New York was running low on money, she would lend money to the city,” explained Charles Slack, the author of Green’s biography, Hetty: The Genius and Madness of America’s First Female Tycoon. “And she always did so at reasonable rates. She didn’t gouge or hold the city over a barrel.”

5. Transcript for Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI | Lex Fridman Podcast #416 – Lex Fridman and Yann Lecun

Lex Fridman (00:50:40) I would love to sort of linger on your skepticism around auto regressive LLMs. So one way I would like to test that skepticism is everything you say makes a lot of sense, but if I apply everything you said today and in general to I don’t know, 10 years ago, maybe a little bit less, no, let’s say three years ago, I wouldn’t be able to predict the success of LLMs. So does it make sense to you that autoregressive LLMs are able to be so damn good?

Yann LeCun (00:51:20) Yes.

Lex Fridman (00:51:21) Can you explain your intuition? Because if I were to take your wisdom and intuition at face value, I would say there’s no way autoregressive LLMs, one token at a time, would be able to do the kind of things they’re doing.

Yann LeCun (00:51:36) No, there’s one thing that autoregressive LLMs or that LLMs in general, not just the autoregressive one, but including the bird style bidirectional ones, are exploiting and its self supervised running, and I’ve been a very, very strong advocate of self supervised running for many years. So those things are a incredibly impressive demonstration that self supervised running actually works. The idea that started, it didn’t start with BERT, but it was really kind of good demonstration with this.

(00:52:09) So the idea that you take a piece of text, you corrupt it, and then you train some gigantic neural net to reconstruct the parts that are missing. That has produced an enormous amount of benefits. It allowed us to create systems that understand language, systems that can translate hundreds of languages in any direction, systems that are multilingual, so it’s a single system that can be trained to understand hundreds of languages and translate in any direction, and produce summaries and then answer questions and produce text.

(00:52:51) And then there’s a special case of it, which is the auto regressive trick where you constrain the system to not elaborate a representation of the text from looking at the entire text, but only predicting a word from the words that are come before. And you do this by constraining the architecture of the network, and that’s what you can build an auto aggressive LLM from.

(00:53:15) So there was a surprise many years ago with what’s called decoder only LLM. So since systems of this type that are just trying to produce words from the previous one and the fact that when you scale them up, they tend to really understand more about language. When you train them on lots of data, you make them really big. That was a surprise and that surprise occurred quite a while back, with work from Google, Meta, OpenAI, et cetera, going back to the GPT kind of work, general pre-trained transformers.

Lex Fridman (00:53:56) You mean like GPT2? There’s a certain place where you start to realize scaling might actually keep giving us an emergent benefit.

Yann LeCun (00:54:06) Yeah, I mean there were work from various places, but if you want to place it in the GPT timeline, that would be around GPT2, yeah.

Lex Fridman (00:54:19) Well, because you said it so charismatic and you said so many words, but self supervised learning, yes. But again, the same intuition you’re applying to saying that auto aggressive LLMs cannot have a deep understanding of the world. If we just apply that, same intuition, does it make sense to you that they’re able to form enough of a representation in the world to be damn convincing, essentially passing the original touring test with flying colors?

Yann LeCun (00:54:50) Well, we’re fooled by their fluency, right? We just assume that if a system is fluent in manipulating language, then it has all the characteristics of human intelligence, but that impression is false. We’re really fooled by it.

Lex Fridman (00:55:06) What do you think Alan Turing would say, without understanding anything, just hanging out with it?

Yann LeCun (00:55:11) Alan Turing would decide that a Turing test is a really bad test, okay? This is what the AI community has decided many years ago that the Turing test was a really bad test of intelligence.

Lex Fridman (00:55:22) What would Hans Marvek say about the larger language models?

Yann LeCun (00:55:26) Hans Marvek would say that Marvek Paradox still applies. Okay, we can pass-

Lex Fridman (00:55:32) You don’t think he would be really impressed?

Yann LeCun (00:55:34) No, of course everybody would be impressed. But it’s not a question of being impressed or not, it’s the question of knowing what the limit of those systems can do. Again, they are impressive. They can do a lot of useful things. There’s a whole industry that is being built around them. They’re going to make progress, but there is a lot of things they cannot do, and we have to realize what they cannot do and then figure out how we get there. And I’m seeing this from basically 10 years of research on the idea of self supervised running, actually that’s going back more than 10 years, but the idea of self supervised running. So basically capturing the internal structure of a piece of a set of inputs without training the system for any particular task, to learning representations.

(00:56:26) The conference I co-founded 14 years ago is called International Conference on Learning Representations. That’s the entire issue that deep learning is dealing with, and it’s been my obsession for almost 40 years now. So learning representation is really the thing. For the longest time, we could only do this with supervised learning, and then we started working on what we used to call unsupervised learning and revived the idea of unsupervised running in the early 2000s with your [inaudible 00:56:58] and Jeff Hinton. Then discovered that supervised running actually works pretty well if you can collect enough data. And so the whole idea of unsupervised, self supervised running kind of took a backseat for a bit, and then I tried to revive it in a big way starting in 2014, basically when we started FAIR and really pushing for finding new methods to do self supervised running both for text and for images and for video and audio.

(00:57:29) And some of that work has been incredibly successful. I mean, the reason why we have multilingual translation system, things to do, content moderation on Meta, for example, on Facebook, that are multilingual, that understand whether a piece of text is hate speech not or something, is due to that progress using self supervised running for NLP, combining this with transformer architectures and blah, blah, blah.

(00:57:53) But that’s the big success of self supervised running. We had similar success in speech recognition, a system called WAVE2VEC, which is also a joint embedding architecture, by the way, trained with contrastive running. And that system also can produce speech recognition systems that are multilingual with mostly unlabeled data and only need a few minutes of labeled data to actually do speech recognition, that’s amazing. We have systems now based on those combination of ideas that can do real time translation of hundreds of languages into each other, speech to speech.

Lex Fridman (00:58:28) Speech to speech, even including, which is fascinating, languages that don’t have written forms.

Yann LeCun (00:58:34) That’s right.

Lex Fridman (00:58:34) Just spoken only.

Yann LeCun (00:58:35) That’s right. We don’t go through text, it goes directly from speech to speech using an internal representation of speech units that are discrete, but it’s called Textless NLP. We used to call it this way. But yeah, so I mean incredible success there. And then for 10 years, we tried to apply this idea to learning representations of images by training a system to predict videos, learning intuitive physics by training a system to predict what’s going to happen in the video.

(00:59:02) And tried and tried and failed and failed, with generative models, with models that predict pixels. We could not get them to learn good representations of images. We could not get them to learn good representations of videos. And we tried many times, we published lots of papers on it, where they kind of sort of work, but not really great. They started working, we abandoned this idea of predicting every pixel and basically just doing the joint embedding and predicting and representation space, that works. So there’s ample evidence that we’re not going to be able to learn good representations of the real world using generative model. So I’m telling people, everybody’s talking about generative AI. If you’re really interested in human level AI, abandon the idea of generative AI…

…Yann LeCun (01:35:29) I actually made that comment on just about every social network I can, and I’ve made that point multiple times in various forums. Here’s my point of view on this, people can complain that AI systems are biased and they generally are biased by the distribution of the training data that they’ve been trained on that reflects biases in society, and that is potentially offensive to some people or potentially not. And some techniques to de-bias then become offensive to some people because of historical incorrectness and things like that.

(01:36:23) And so you can ask two questions, the first question is, is it possible to produce an AI system that is not biased? And the answer is, absolutely not. And it’s not because of technological challenges, although they are technological challenges to that, it’s because bias is in the eye of the beholder. Different people may have different ideas about what constitutes bias for a lot of things, there are facts that are indisputable, but there are a lot of opinions or things that can be expressed in different ways. And so you cannot have an unbiased system, that’s just an impossibility.

(01:37:08) And so what’s the answer to this? And the answer is the same answer that we found in liberal democracy about the press, the press needs to be free and diverse. We have free speech for a good reason, is because we don’t want all of our information to come from a unique source because that’s opposite to the whole idea of democracy and progressive ideas and even science. In science, people have to argue for different opinions and science makes progress when people disagree and they come up with an answer and consensus forms, and it’s true in all democracies around the world.

(01:37:58) There is a future which is already happening where every single one of our interaction with the digital world will be mediated by AI systems, AI assistance. We’re going to have smart glasses, you can already buy them from Meta, the Ray-Ban Meta where you can talk to them and they are connected with an LLM and you can get answers on any question you have. Or you can be looking at a monument and there is a camera in the glasses you can ask it like, what can you tell me about this building or this monument? You can be looking at a menu in a foreign language, and I think we will translate it for you, or we can do real time translation if we speak different languages. So a lot of our interactions with the digital world are going to be mediated by those systems in the near future.

(01:38:53) Increasingly, the search engines that we’re going to use are not going to be search engines, they’re going to be dialogue systems that we just ask a question and it will answer and then point you to perhaps appropriate reference for it. But here is the thing, we cannot afford those systems to come from a handful of companies on the west coast of the US because those systems will constitute the repository of all human knowledge, and we cannot have that be controlled by a small number of people. It has to be diverse for the same reason the press has to be diverse, so how do we get a diverse set of AI assistance? It’s very expensive and difficult to train a base model, a base LLM at the moment, in the future it might be something different, but at the moment, that’s an LLM. So only a few companies can do this properly.

(01:39:50) And if some of those top systems are open source, anybody can use them, anybody can fine tune them. If we put in place some systems that allows any group of people, whether they are individual citizens, groups of citizens, government organizations, NGOs, companies, whatever, to take those open source AI systems and fine tune them for their own purpose on their own data, then we’re going to have a very large diversity of different AI systems that are specialized for all of those things.

(01:40:35) I tell you, I talked to the French government quite a bit, and the French government will not accept that the digital diet of all their citizens be controlled by three companies on the west coast of the US. That’s just not acceptable, it’s a danger to democracy regardless of how well-intentioned those companies are, and it’s also a danger to local culture, to values, to language. I was talking with the founder of Infosys in India, he’s funding a project to fine tune Llama 2, the open source model produced by Meta, so that Llama 2 two speaks all 22 official languages in India, it is very important for people in India. I was talking to a former colleague of mine, Moustapha Cisse, who used to be a scientist at Fair and then moved back to Africa, created a research lab for Google in Africa and now has a new startup Co-Kera.

(01:41:37) And what he’s trying to do, is basically have LLM that speak the local languages in Senegal so that people can have access to medical information because they don’t have access to doctors, it’s a very small number of doctors per capita in Senegal. You can’t have any of this unless you have open source platforms, so with open source platforms, you can have AI systems that are not only diverse in terms of political opinions or things of that-

Yann LeCun (01:42:00) … AI systems that are not only diverse in terms of political opinions or things of that type, but in terms of language, culture, value systems, political opinions, technical abilities in various domains, and you can have an industry, an ecosystem of companies that fine tune those open source systems for vertical applications in industry. I don’t know, a publisher has thousands of books and they want to build a system that allows a customer to just ask a question about the content of any of their books, you need to train on their proprietary data. You have a company, we have one within Meta, it’s called Metamate, and it’s basically an LLM that can answer any question about internal stuff about the company, very useful.

(01:42:53) A lot of companies want this. A lot of companies want this not just for their employees, but also for their customers, to take care of their customers. So the only way you’re going to have an AI industry, the only way you’re going to have AI systems that are not uniquely biased is if you have open source platforms on top of which any group can build specialized systems. So the direction of inevitable direction of history is that the vast majority of AI systems will be built on top of open source platforms…

…Lex Fridman (02:04:21) You often say that a GI is not coming soon, meaning not this year, not the next few years, potentially farther away. What’s your basic intuition behind that?

Yann LeCun (02:04:35) So first of all, it’s not going to be an event. The idea somehow, which is popularized by science fiction and Hollywood, that somehow somebody is going to discover the secret to AGI or human-level AI or AMI, whatever you want to call it, and then turn on a machine and then we have AGI, that’s just not going to happen. It’s not going to be an event. It’s going to be gradual progress. Are we going to have systems that can learn from video how the world works and learn good representations? Yeah. Before we get them to the scale and performance that we observe in humans it’s going to take quite a while. It’s not going to happen in one day. Are we going to get systems that can have large amount of associated memory so they can remember stuff? Yeah, but same, it’s not going to happen tomorrow. There is some basic techniques that need to be developed. We have a lot of them, but to get this to work together with a full system is another story.

(02:05:37) Are we going to have systems that can reason and plan perhaps along the lines of objective-driven AI architectures that I described before? Yeah, but before we get this to work properly, it’s going to take a while. Before we get all those things to work together, and then on top of this, have systems that can learn hierarchical planning, hierarchical representations, systems that can be configured for a lot of different situation at hand, the way the human brain can, all of this is going to take at least a decade and probably much more because there are a lot of problems that we’re not seeing right now that we have not encountered, so we don’t know if there is an easy solution within this framework. So it’s not just around the corner. I’ve been hearing people for the last 12, 15 years claiming that AGI is just around the corner and being systematically wrong. I knew they were wrong when they were saying it. I called their bullshit…

…Lex Fridman (02:08:48) So you push back against what are called AI doomers a lot. Can you explain their perspective and why you think they’re wrong?

Yann LeCun (02:08:59) Okay, so AI doomers imagine all kinds of catastrophe scenarios of how AI could escape or control and basically kill us all, and that relies on a whole bunch of assumptions that are mostly false. So the first assumption is that the emergence of super intelligence is going to be an event, that at some point we’re going to figure out the secret and we’ll turn on a machine that is super intelligent, and because we’d never done it before, it’s going to take over the world and kill us all. That is false. It’s not going to be an event. We’re going to have systems that are as smart as a cat, have all the characteristics of human-level intelligence, but their level of intelligence would be like a cat or a parrot maybe or something. Then we’re going to work our way up to make those things more intelligent. As we make them more intelligent, we’re also going to put some guardrails in them and learn how to put some guardrails so they behave properly.

(02:10:03) It’s not going to be one effort, that it’s going to be lots of different people doing this, and some of them are going to succeed at making intelligent systems that are controllable and safe and have the right guardrails. If some other goes rogue, then we can use the good ones to go against the rogue ones. So it’s going to be my smart AI police against your rogue AI. So it’s not going to be like we’re going to be exposed to a single rogue AI that’s going to kill us all. That’s just not happening. Now, there is another fallacy, which is the fact that because the system is intelligent, it necessarily wants to take over. There is several arguments that make people scared of this, which I think are completely false as well.

(02:10:48) So one of them is in nature, it seems to be that the more intelligent species otherwise end up dominating the other and even distinguishing the others sometimes by design, sometimes just by mistake. So there is thinking by which you say, “Well, if AI systems are more intelligent than us, surely they’re going to eliminate us, if not by design, simply because they don’t care about us,” and that’s just preposterous for a number of reasons. First reason is they’re not going to be a species. They’re not going to be a species that competes with us. They’re not going to have the desire to dominate because the desire to dominate is something that has to be hardwired into an intelligent system. It is hardwired in humans. It is hardwired in baboons, in chimpanzees, in wolves, not in orangutans. The species in which this desire to dominate or submit or attain status in other ways is specific to social species. Non-social species like orangutans don’t have it, and they are as smart as we are, almost, right?

Lex Fridman (02:12:09) To you, there’s not significant incentive for humans to encode that into the AI systems, and to the degree they do, there’ll be other AIs that punish them for it, I’ll compete them over it.

Yann LeCun (02:12:23) Well, there’s all kinds of incentive to make AI systems submissive to humans.

Lex Fridman (02:12:26) Right.

Yann LeCun (02:12:27) Right? This is the way we’re going to build them. So then people say, “Oh, but look at LLMs. LLMs are not controllable,” and they’re right. LLMs are not controllable. But objectively-driven AI, so systems that derive their answers by optimization of an objective means they have to optimize this objective, and that objective can include guardrails. One guardrail is, obey humans. Another guardrail is, don’t obey humans if it’s hurting other humans within limits.

Lex Fridman (02:12:57) Right. I’ve heard that before somewhere, I don’t remember

Yann LeCun (02:12:59) Yes, maybe in a book.

Lex Fridman (02:13:01) Yeah, but speaking of that book, could there be unintended consequences also from all of this?

Yann LeCun (02:13:09) No, of course. So this is not a simple problem. Designing those guardrails so that the system behaves properly is not going to be a simple issue for which there is a silver bullet for which you have a mathematical proof that the system can be safe. It’s going to be a very progressive, iterative design system where we put those guardrails in such a way that the system behave properly. Sometimes they’re going to do something that was unexpected because the guardrail wasn’t right and we’re dd correct them so that they do it right. The idea somehow that we can’t get it slightly wrong because if we get it slightly wrong, we’ll die is ridiculous. We are just going to go progressively. It is just going to be, the analogy I’ve used many times is turbojet design. How did we figure out how to make turbojet so unbelievably reliable?

(02:14:07) Those are incredibly complex pieces of hardware that run at really high temperatures for 20 hours at a time sometimes, and we can fly halfway around the world on a two-engine jetliner at near the speed of sound. Like how incredible is this? It’s just unbelievable. Did we do this because we invented a general principle of how to make turbojets safe? No, it took decades to fine tune the design of those systems so that they were safe. Is there a separate group within General Electric or Snecma or whatever that is specialized in turbojet safety? No. The design is all about safety, because a better turbojet is also a safer turbojet, so a more reliable one. It’s the same for AI. Do you need specific provisions to make AI safe? No, you need to make better AI systems, and they will be safe because they are designed to be more useful and more controllable…

…Lex Fridman (02:28:45) Well, it’ll be at the very least, absurdly comedic. Okay. So since we talked about the physical reality, I’d love to ask your vision of the future with robots in this physical reality. So many of the kinds of intelligence that you’ve been speaking about would empower robots to be more effective collaborators with us humans. So since Tesla’s Optimus team has been showing us some progress on humanoid robots, I think it really reinvigorated the whole industry that I think Boston Dynamics has been leading for a very, very long time. So now there’s all kinds of companies Figure AI, obviously Boston Dynamics.

Yann LeCun (02:29:30) Unitree.

Lex Fridman (02:29:30) Unitree, but there’s a lot of them.

Yann LeCun (02:29:33) There’s a few of them.

Lex Fridman (02:29:33) It’s great. It’s great. I love it. So do you think there’ll be millions of humanoid robots walking around soon?

Yann LeCun (02:29:44) Not soon, but it’s going to happen. The next decade I think is going to be really interesting in robots, the emergence of the robotics industry has been in the waiting for 10, 20 years without really emerging other than for pre-program behavior and stuff like that. And the main issue is, again, the Moravec paradox, how do we get those systems to understand how the world works and plan actions? And so we can do it for really specialized tasks. And the way Boston Dynamics goes about it is basically with a lot of handcrafted dynamical models and careful planning in advance, which is very classical robotics with a lot of innovation, a little bit of perception, but it’s still not, they can’t build a domestic robot.

(02:30:41) We’re still some distance away from completely autonomous level five driving, and we’re certainly very far away from having level five autonomous driving by a system that can train itself by driving 20 hours like any 17-year-old. So until we have, again, world models, systems that can train themselves to understand how the world works, we’re not going to have significant progress in robotics. So a lot of the people working on robotic hardware at the moment are betting or banking on the fact that AI is going to make sufficient progress towards that…

…Yann LeCun (02:38:29) I love that question. We can make humanity smarter with AI. AI basically will amplify human intelligence. It’s as if every one of us will have a staff of smart AI assistants. They might be smarter than us. They’ll do our bidding, perhaps execute a task in ways that are much better than we could do ourselves, because they’d be smarter than us. And so it’s like everyone would be the boss of a staff of super smart virtual people. So we shouldn’t feel threatened by this any more than we should feel threatened by being the manager of a group of people, some of whom are more intelligent than us. I certainly have a lot of experience with this, of having people working with me who are smarter than me.

(02:39:35) That’s actually a wonderful thing. So having machines that are smarter than us, that assist us in all of our tasks, our daily lives, whether it’s professional or personal, I think would be an absolutely wonderful thing. Because intelligence is the commodity that is most in demand. That’s really what I mean. All the mistakes that humanity makes is because of lack of intelligence really, or lack of knowledge, which is related. So making people smarter, we just can only be better. For the same reason that public education is a good thing and books are a good thing, and the internet is also a good thing, intrinsically and even social networks are a good thing if you run them properly.

(02:40:21) It’s difficult, but you can. Because it helps the communication of information and knowledge and the transmission of knowledge. So AI is going to make humanity smarter. And the analogy I’ve been using is the fact that perhaps an equivalent event in the history of humanity to what might be provided by generalization of AI assistant is the invention of the printing press. It made everybody smarter, the fact that people could have access to books. Books were a lot cheaper than they were before, and so a lot more people had an incentive to learn to read, which wasn’t the case before.

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Apple, Alphabet (parent of Google), Microsoft, Meta Platforms, and Tesla. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 24 March 2024:

1. The future of ‘communist capitalism’ in China – Martin Wolf

What is the economic future of China? This question raises many specific issues, notably China’s persistent macroeconomic imbalances, the threat of population decline and worsening relations with important parts of the outside world, above all, an increasingly hostile US. But underneath all of these lies a deeper one: is “communist capitalism”, that seemingly self-contradicting invention of Deng Xiaoping, inexorably fading away under Xi Jinping? Will China’s regime ossify and, in the end, collapse, as the Soviet Union did?…

…Much light on this issue is shed by China’s World View, a recently published book by David Daokui Li, a distinguished Harvard-trained professor of economics, who teaches at Tsinghua University. People interested in China, be they hawks or doves, should read Li’s valuable book carefully.

Perhaps its most startling observation is that “from 980 until 1840, the beginning of China’s modern history”, income per head declined. Ancient China was in a Malthusian trap. This picture is even worse than the one shown in the work of the late Angus Maddison. Even after 1840, this grim reality did not get much brighter. Only after Deng Xiaoping’s “reform and opening up” did it change.

By freeing the private economy, relying on market forces and opening up to the world economy, Deng created the conditions for an extraordinary transformation. Yet, by repressing demands for democracy in Tiananmen Square in 1989, he also reinforced communist party control. He invented a new political economy: today’s China is the result.

Is it also sustainable? Li’s book answers a clear “yes” to this question. In essence, he argues that China’s political system should be viewed not as Soviet, but as a modernised form of the traditional Chinese imperial state. This state is paternal. It is responsible for the people, but not accountable to them, except in one fundamental way: if it loses mass support, it will be overthrown. Its job is to provide stability and prosperity. But, in doing so, it does not try to run everything from the centre. That would be crazy in so vast a country: it decentralises to local levels. The communist party should, he argues, be seen fundamentally as the national party of China.

From this perspective, the Xi regime does not represent an abandonment of the goals of the Deng era, but rather an attempt to remedy some of the problems created by its reliance on “go-go” capitalism, namely, pervasive corruption, soaring inequality and environmental damage…

…When considering the prospects for China, one should not focus mainly on the list of obvious problems — falling property prices, excessive debt, excess savings, an ageing population and western hostility. All these can be dealt with by a country with China’s human resources and growth potential, even if with difficulty.

The bigger issue is whether, in the centralising, cautious and conservative era of Xi, Deng’s move from stagnation to explosive growth is doomed to reverse back into stagnation. If people come to believe that the dynamism of the recent past has been lost for good, then there is a risk of a downward spiral of disappointed hopes. But the force of 1.4bn people wanting a better life is extremely powerful. Will anything be allowed to halt it? The answer, I suspect, is still “no”.

2. Is diversification a blessing or curse? – Chin Hui Leong

DIVERSIFICATION is good or bad for you, depending on whom you ask. Warren Buffett, the legendary investor and businessman, once said that if you know what you’re doing, it makes little sense to diversify.

But Peter Lynch, a star mutual fund manager of the 1980s, had a different approach. He believed that the more stocks you own, the better your chances of finding a winner. Lynch was famous for holding up to 1,400 stocks in his portfolio.

Here’s the surprise: They both achieved remarkable success, despite their opposing positions. What does this mean for you as an investor? Should you diversify, or not?…

…As we delve deeper into diversification, we should not lose sight of its goal to reduce risk. This is where buying businesses from unrelated industries or geographies can go wrong. In fact, investors who diversify into areas where they lack expertise are taking more risk, not less. It makes little sense to do so, says Lynch. How well you know your stocks matters more than how many sectors or regions you spread your money across.

I agree with Lynch. Diversify only if you want to boost your chances of finding more winning stocks in your portfolio.

Here is a point you shouldn’t miss: you should always be looking to learn more about new businesses and industries. As you become more knowledgeable, you can grow your portfolio with more stocks you know well, but without exceeding your limits.

Remaining humble is key. Knowing the limits of your knowledge in any new area is how you keep yourself in check. As author Carl Richards once said, risk is what’s left when you think you’ve thought of everything…

…Here’s a simple rule of thumb to help you. If you’ve been following a new company for a year, invest no more than 1 per cent of your portfolio into the stock. If it’s five years, then up to 5 per cent. You can adjust the percentage to fit your risk appetite.

The point of this strategy is to have a reference point where you can match your risk level with your knowledge level…

…Finally, investing over time helps to spread your risk over years. Don’t worry about starting small in a stock. A winning stock is only known in hindsight. Here’s the point most people miss: if a stock is destined to be a winner, the stock price rise will happen over years, if not decades…

…Here’s the final conundrum: the mark of a successful portfolio is a concentrated portfolio. How can that be? Let’s say you invested $1,000 each into 10 stocks. Each stock will make up a tenth of this $10,000 portfolio.

After five years, the first one skyrockets, increasing by 10 times and is worth $10,000, while the last one goes to zero. The other eight stocks stay the same at $1,000. Do the math and you’ll end up with $18,000 in total. The big difference is, the winning stock will comprise more than 55 per cent of the five-year old portfolio…

…As you diversify to find more winners, the best of them will naturally rise to the top – thereby concentrating your portfolio in the right set of winning stocks. That’s more than any investor can wish for.

3. China has little choice but stimulus – Ethan Wu

The near universal reaction in the west to China’s refreshed 5 per cent gross domestic product growth target: good luck with that…

…The old growth drivers — property, infrastructure and manufacturing — all face major constraints. Property’s structural decline is well known; home prices and sales keep falling. Meanwhile, infrastructure is running into the limit of high debt levels. Chinese officials were dispatched last year to prod local governments to delever. It began with easy cost cuts: withholding wages from civil servants, delaying payments to vendors, slashing city services. But more recently, the deleveraging drive has been hitting infrastructure projects already under way, as Reuters reported in January:

Increasing its efforts to manage $13 trillion in municipal debt, the State Council in recent weeks issued a directive to local governments and state banks to delay or halt construction on projects with less than half the planned investment completed in 12 regions across the country, the sources said…

…Lastly, manufacturing. Since about 2020, the credit that once flowed to the property sector has been redirected to manufacturing, especially in politically favoured sectors such as solar and electric vehicles. The year-over-year growth rate of loans to Chinese industry has risen steadily, though the level is now declining..

…This pivot back to manufacturing is “radical”, says Adam Wolfe of Absolute Strategy Research, and it has generated important victories for China. Most notably, BYD is now the world’s biggest EV maker, and China the biggest auto exporter. But it has also created an enormous oversupply of manufactured goods, which, when combined with limp demand at home, is crushing industrial margins and fuelling deflation…

…China’s manufacturing trade surplus is already huge, perhaps 2 per cent of world GDP. As Gavekal’s Yanmei Xie wrote in the FT last month, western countries sensibly fear China dumping cheap goods into export markets. A cheap renminbi heightens the threat; trade retaliation is widely anticipated. If that is right, export-led growth probably can’t be China’s escape valve.

This glum picture suggests that China may soon be forced into stimulus. Assuming the GDP target is at least somewhat binding, no sector of the Chinese economy stands ready to get growth to 5 per cent. A pick-up in consumption could do it, but we’ve heard no convincing story for why anxious consumers would suddenly become gripped by animal spirits…

…The unclear stimulus outlook has left the bulk of investors nervous, but equity outflows have at least stopped. The stock market has rallied 14 per cent since early February, but only because of ample support from the state. Value trade or value trap?

What keeps us sceptical is the fact that Chinese stocks are not loads cheaper than global stocks. After the rally, the CSI 300 trades at 13x forward earnings, versus 14x for the MSCI all-country world ex-US index. To us the risks in China stocks are much clearer than the reward.

4. Exxon Barges in on Hess Deal – Matt Levine

I, on the other hand, used to be a convertible bond investment banker, so I have somewhat more than the usual familiarity with them. I could tell you, for instance, that it is common in the US for a convertible to be done as a Rule 144A offering, meaning that the bonds are sold to large “qualified institutional buyers” (QIBs) in a private placement and then can’t be resold to retail investors. Doing a 144A deal is generally faster and cheaper than doing a public deal that is registered with the US Securities and Exchange Commission, and retail investors don’t really buy convertibles anyway.

But eventually the institutional buyers of a 144A deal will want to be able to convert their bonds into regular, publicly traded stock, so there needs to be some mechanism for turning “144A” convertibles into “registered” ones. I am old enough that, when I started as a converts banker, the way to do this was to file a registration statement with the SEC, but the modern approach is pretty much that you wait six months or a year and the convertible becomes freely tradeable as a legal matter.

As a practical matter, though, the way this works is that the bonds, when they are originally issued, have a “restrictive legend” on them saying that they can be sold only to institutional buyers, and after a year the company sends a notice to its transfer agent saying “you can take that legend off the bonds now.” And when the bonds have the legend, they can’t be freely traded; once the legend is off, they can be. Here I am pretending, as one does, that “the bonds” are pieces of paper with a legend stamped on them, but of course they are actually entries in an electronic database; what really happens is that the original bonds have a “restricted CUSIP” (the identification number that every security has), telling transfer agents and depositaries and brokers and everyone else that they can only be sold to QIBs, and then after a year the company gets them a new “unrestricted CUSIP” and they trade freely. This is not hard — it’s a phone call or an email, maybe a legal opinion — but the company has to do it…

…So for instance here is the indenture for Avid Bioservices Inc.’s 1.25% exchangeable senior notes due 2026, a convertible bond it issued in 2021.4 Section 4.06(e) of the indenture, the 94-page contract governing the bonds, says:

If, and for so long as, the restrictive legend on the Notes specified in ‎‎Section 2.05(c) has not been removed, the Notes are assigned a restricted CUSIP or the Notes are not otherwise freely tradable … as of the 370th day after the last date of original issuance of the Notes, the Company shall pay Additional Interest on the Notes at a rate equal to 0.50% per annum of the principal amount of Notes outstanding until the restrictive legend on the Notes has been removed. …

…Avid forgot, for two years, to take the restrictive legend off of its convertible. This was very understandable: Its obligation to remove the restricted legend was boring and technical and buried in Section 4.06(e) of a bond indenture that surely nobody read. It could only remove the legend a year after it issued the bonds, after everyone had stopped paying attention. And, as Avid points out, it “did not receive any notices and was not otherwise made aware” of this provision in, sure, a contract that it signed, but a very long and boring contract. (And, to be fair, the holders forgot too!) And because it completely forgot about its obligation to remove the legend, Avid also forgot to pay the 0.5% penalty interest rate for two years. And because it forgot to pay the extra interest, it created a non-curable default on the bonds: The holders can demand all of their money back, with interest, immediately, with no chance for Avid to fix the problem by removing the legend and paying the overdue interest…

…This is a bad oopsie by Avid, which probably should have put a reminder in its calendar to unrestrict the CUSIP. But it’s a clever trade by whoever this holder was: The old bonds are far out-of-the-money (that is, they’re not going to convert into stock), and Bloomberg tells me that they were trading in the high 70s as recently as a month ago (the high 80s more recently). If you had noticed Avid’s extremely technical oopsie, you could have bought the bonds at, say, 80 cents on the dollar, sent them a letter saying “we gotcha hahahaha,” and made a quick 20 points, plus interest. The holder owns “at least 25%” of the bonds (the amount required to accelerate), and there are $143.75 million of bonds outstanding; 20 points on 25% of $143.75 million is $7.2 million. Plus interest.

5. Sora, Groq, and Virtual Reality – Ben Thompson

Groq was founded in 2016 by Jonathan Ross, who created Google’s first Tensor Processing Unit; Ross’s thesis was that chips should take their cue from software-defined networking: instead of specialized hardware for routing data, a software-defined network uses commodity hardware with a software layer to handle the complexity of routing. Indeed, Groq’s paper explaining their technology is entitled “A Software-defined Tensor Streaming Multiprocessor for Large-scale Machine Learning.”

To that end Groq started with the compiler, the software that translates code into machine language that can be understood by chips; the goal was to be able to reduce machine-learning algorithms into a format that could be executed on dramatically simpler processors that could operate at very high speed, without expensive memory calls and prediction misses that make modern processors relatively slow.

The end result is that Groq’s chips are purely deterministic: instead of the high-bandwidth memory (HBM) used for modern GPUs or Dynamic Random Access Memory (DRAM) used in computers, both of which need to be refreshed regularly to function (which introduces latency and uncertainty about the location of data at a specific moment in time), Groq uses SRAM — Static Random Access Memory. SRAM stores data in what is called a bistable latching circuitry; this, unlike the transistor/capacitor architecture undergirding DRAM (and by extension, HBM), stores data in a stable state, which means that Groq always knows exactly where every piece of data is at any particular moment in time. This allows the Groq compiler to, in an ideal situation, pre-define every memory call, enabling extremely rapid computation with a relatively simple architecture.

It turns out that running inference on transformer-based models is an extremely ideal situation, because the computing itself is extremely deterministic. An LLM like GPT-4 processes text through a series of layers which have a predetermined set of operations, which is perfectly suited to Groq’s compiler. Meanwhile, token-based generation is a purely serial operation: every single token generated depends on knowing the previous token; there is zero parallelism for any one specific answer, which means the speed of token calculation is at an absolute premium…

…One of the arguments I have made as to why OpenAI CEO Sam Altman may be exploring hardware is that the closer an AI comes to being human, the more grating and ultimately gating are the little inconveniences that get in the way of actually interacting with said AI. It is one thing to have to walk to your desk to use a PC, or even reach into your pocket for a smartphone: you are, at all times, clearly interacting with a device. Having to open an app or wait for text in the context of a human-like AI is far more painful: it breaks the illusion in a much more profound, and ultimately disappointing, way. Groq suggests a path to keeping the illusion intact.

It is striking that Groq is a deterministic system running deterministic software that, in the end, produces probabilistic output. I explained deterministic versus probabilistic computing in ChatGPT Gets a Computer:

Computers are deterministic: if circuit X is open, then the proposition represented by X is true; 1 plus 1 is always 2; clicking “back” on your browser will exit this page. There are, of course, a huge number of abstractions and massive amounts of logic between an individual transistor and any action we might take with a computer — and an effectively infinite number of places for bugs — but the appropriate mental model for a computer is that they do exactly what they are told (indeed, a bug is not the computer making a mistake, but rather a manifestation of the programmer telling the computer to do the wrong thing).

I’ve already mentioned Bing Chat and ChatGPT; on March 14 Anthropic released another AI assistant named Claude: while the announcement doesn’t say so explicitly, I assume the name is in honor of the aforementioned Claude Shannon. This is certainly a noble sentiment — Shannon’s contributions to information theory broadly extend far beyond what Dixon laid out above — but it also feels misplaced: while technically speaking everything an AI assistant is doing is ultimately composed of 1s and 0s, the manner in which they operate is emergent from their training, not proscribed, which leads to the experience feeling fundamentally different from logical computers — something nearly human — which takes us back to hallucinations; Sydney was interesting, but what about homework?

The idea behind ChatGPT Gets a Computer is that large language models seem to operate somewhat similarly to the human brain, which is incredible and also imprecise, and just as we need a computer to do exact computations, so does ChatGPT. A regular computer, though, is actually the opposite of Groq: you get deterministic answers from hardware that is, thanks to the design of modern processors and memory, more probabilistic than you might think, running software that assumes the processor will handle endless memory calls and branch prediction.

In the end, though, we are back where we started: a computer would know where the bow and stern are on a ship, while a transformer-based model like Sora made a bad guess. The former calculates reality; the latter a virtual reality.

Imagine, though, Sora running on Groq (which is absolutely doable): could we have generated videos in real-time? Even if we could not, we are certainly much closer than you might have expected. And where, you might ask, would we consume those videos? How about on a head-mounted display like the Apple Vision Pro or Meta Quest? Virtual reality (my new definition) for virtual reality (the old definition).

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Apple and Meta Platforms. Holdings are subject to change at any time.

We’ve constantly been sharing a list of our recent reads in our weekly emails for The Good Investors.

Do subscribe for our weekly updates through the orange box in the blog (it’s on the side if you’re using a computer, and all the way at the bottom if you’re using mobile) – it’s free!

But since our readership-audience for The Good Investors is wider than our subscriber base, we think sharing the reading list regularly on the blog itself can benefit even more people. The articles we share touch on a wide range of topics, including investing, business, and the world in general. 

Here are the articles for the week ending 17 March 2024:

1. The Ultra-Pure, Super-Secret Sand That Makes Your Phone Possible – Vince Beiser

Spruce Pine is not a wealthy place. Its downtown consists of a somnambulant train station across the street from a couple of blocks of two‑story brick buildings, including a long‑closed movie theater and several empty storefronts.

The wooded mountains surrounding it, though, are rich in all kinds of desirable rocks, some valued for their industrial uses, some for their pure prettiness. But it’s the mineral in Glover’s bag—snowy white grains, soft as powdered sugar—that is by far the most important these days. It’s quartz, but not just any quartz. Spruce Pine, it turns out, is the source of the purest natural quartz—a species of pristine sand—ever found on Earth. This ultra‑elite deposit of silicon dioxide particles plays a key role in manufacturing the silicon used to make computer chips. In fact, there’s an excellent chance the chip that makes your laptop or cell phone work was made using sand from this obscure Appalachian backwater. “It’s a billion‑dollar industry here,” Glover says with a hooting laugh. “Can’t tell by driving through here. You’d never know it.”

In the 21st century, sand has become more important than ever, and in more ways than ever. This is the digital age, in which the jobs we work at, the entertainment we divert ourselves with, and the ways we communicate with one another are increasingly defined by the internet and the computers, tablets, and cell phones that connect us to it. None of this would be possible were it not for sand.

Most of the world’s sand grains are composed of quartz, which is a form of silicon dioxide, also known as silica. High‑purity silicon dioxide particles are the essential raw materials from which we make computer chips, fiber‑optic cables, and other high‑tech hardware—the physical components on which the virtual world runs. The quantity of quartz used for these products is minuscule compared to the mountains of it used for concrete or land reclamation. But its impact is immeasurable…

…In the mid‑1950s, thousands of miles from North Carolina, a group of engineers in California began working on an invention that would become the foundation of the computer industry. William Shockley, a pathbreaking engineer at Bell Labs who had helped invent the transistor, had left to set up his own company in Mountain View, California, a sleepy town about an hour south of San Francisco, near where he had grown up. Stanford University was nearby, and General Electric and IBM had facilities in the area, as well as a new company called Hewlett‑Packard. But the area known at the time as the Santa Clara Valley was still mostly filled with apricot, pear, and plum orchards. It would soon become much better known by a new nickname: Silicon Valley.

At the time, the transistor market was heating up fast. Texas Instruments, Motorola, and other companies were all competing to come up with smaller, more efficient transistors to use in, among other products, computers. The first American computer, dubbed ENIAC, was developed by the army during World War II; it was 100 feet long and 10 feet high, and it ran on 18,000 vacuum tubes.

Transistors, which are tiny electronic switches that control the flow of electricity, offered a way to replace those tubes and make these new machines even more powerful while shrinking their tumid footprint. Semiconductors—a small class of elements, including germanium and silicon, which conduct electricity at certain temperatures while blocking it at others—looked like promising materials for making those transistors.

At Shockley’s startup, a flock of young PhDs began each morning by firing up kilns to thousands of degrees and melting down germanium and silicon. Tom Wolfe once described the scene in Esquire magazine: “They wore white lab coats, goggles, and work gloves. When they opened the kiln doors weird streaks of orange and white light went across their faces . . . they lowered a small mechanical column into the goo so that crystals formed on the bottom of the column, and they pulled the crystal out and tried to get a grip on it with tweezers, and put it under microscopes and cut it with diamond cutters, among other things, into minute slices, wafers, chips; there were no names in electronics for these tiny forms.”

Shockley became convinced that silicon was the more promising material and shifted his focus accordingly. “Since he already had the first and most famous semiconductor research and manufacturing company, everyone who had been working with germanium stopped and switched to silicon,” writes Joel Shurkin in his biography of Shockley, Broken Genius. “Indeed, without his decision, we would speak of Germanium Valley.”

Shockley was a genius, but by all accounts he was also a lousy boss. Within a couple of years, several of his most talented engineers had jumped ship to start their own company, which they dubbed Fairchild Semiconductor. One of them was Robert Noyce, a laid‑back but brilliant engineer, only in his mid‑20s but already famous for his expertise with transistors.

The breakthrough came in 1959, when Noyce and his colleagues figured out a way to cram several transistors onto a single fingernail‑sized sliver of high‑purity silicon. At almost the same time, Texas Instruments developed a similar gadget made from germanium. Noyce’s, though, was more efficient, and it soon dominated the market. NASA selected Fairchild’s microchip for use in the space program, and sales soon shot from almost nothing to $130 million a year. In 1968, Noyce left to found his own company. He called it Intel, and it soon dominated the nascent industry of programmable computer chips.

Intel’s first commercial chip, released in 1971, contained 2,250 transistors. Today’s computer chips are often packed with transistors numbering in the billions. Those tiny electronic squares and rectangles are the brains that run our computers, the Internet, and the entire digital world. Google, Amazon, Apple, Microsoft, the computer systems that underpin the work of everything from the Pentagon to your local bank—all of this and much more is based on sand, remade as silicon chips.

Making those chips is a fiendishly complicated process. They require essentially pure silicon. The slightest impurity can throw their tiny systems out of whack.

Finding silicon is easy. It’s one of the most abundant elements on Earth. It shows up practically everywhere bound together with oxygen to form SiO2, aka quartz. The problem is that it never occurs naturally in pure, elemental form. Separating out the silicon takes considerable doing.

Step one is to take high‑purity silica sand, the kind used for glass. (Lump quartz is also sometimes used.) That quartz is then blasted in a powerful electric furnace, creating a chemical reaction that separates out much of the oxygen. That leaves you with what is called silicon metal, which is about 99 percent pure silicon. But that’s not nearly good enough for high‑tech uses. Silicon for solar panels has to be 99.999999 percent pure—six 9s after the decimal. Computer chips are even more demanding. Their silicon needs to be 99.99999999999 percent pure—eleven 9s. “We are talking of one lonely atom of something that is not silicon among billions of silicon companions,” writes geologist Michael Welland in Sand: The Never-Ending Story.

Getting there requires treating the silicon metal with a series of complex chemical processes. The first round of these converts the silicon metal into two compounds. One is silicon tetrachloride, which is the primary ingredient used to make the glass cores of optical fibers. The other is trichlorosilane, which is treated further to become polysilicon, an extremely pure form of silicon that will go on to become the key ingredient in solar cells and computer chips.

Each of these steps might be carried out by more than one company, and the price of the material rises sharply at each step. That first‑step, 99 percent pure silicon metal goes for about $1 a pound; polysilicon can cost 10 times as much.

The next step is to melt down the polysilicon. But you can’t just throw this exquisitely refined material in a cook pot. If the molten silicon comes into contact with even the tiniest amount of the wrong substance, it causes a ruinous chemical reaction. You need crucibles made from the one substance that has both the strength to withstand the heat required to melt polysilicon, and a molecular composition that won’t infect it. That substance is pure quartz.

THIS IS WHERE Spruce Pine quartz comes in. It’s the world’s primary source of the raw material needed to make the fused‑quartz crucibles in which computer‑chip‑grade polysilicon is melted. A fire in 2008 at one of the main quartz facilities in Spruce Pine for a time all but shut off the supply of high‑purity quartz to the world market, sending shivers through the industry.

Today one company dominates production of Spruce Pine quartz. Unimin, an outfit founded in 1970, has gradually bought up Spruce Pine area mines and bought out competitors, until today the company’s North Carolina quartz operations supply most of the world’s high‑ and ultra‑high‑purity quartz. (Unimin itself is now a division of a Belgian mining conglomerate, Sibelco.)

In recent years, another company, the imaginatively titled Quartz Corp, has managed to grab a small share of the Spruce Pine market. There are a very few other places around the world producing high‑purity quartz, and many other places where companies are looking hard for more. But Unimin controls the bulk of the trade.

The quartz for the crucibles, like the silicon they will produce, needs to be almost absolutely pure, purged as thoroughly as possible of other elements. Spruce Pine quartz is highly pure to begin with, and purer still after being put through several rounds of froth flotation. But some of the grains may still have what Glover calls interstitial crystalline contamination—molecules of other minerals attached to the quartz molecules.

That’s frustratingly common. “I’ve evaluated thousands of quartz samples from all over the world,” says John Schlanz, chief minerals processing engineer at the Minerals Research Laboratory in Asheville, about an hour from Spruce Pine. “Near all of them have contaminate locked in the quartz grains that you can’t get out.”

Some Spruce Pine quartz is flawed in this way. Those grains are used for high‑end beach sand and golf course bunkers—most famously the salt‑white traps of Augusta National Golf Club, site of the iconic Masters Tournament. A golf course in the oil‑drunk United Arab Emirates imported 4,000 tons of this sand in 2008 to make sure its sand traps were world‑class, too.

The very best Spruce Pine quartz, however, has an open crystalline structure, which means that hydrofluoric acid can be injected right into the crystal molecules to dissolve any lingering traces of feldspar or iron, taking the purity up another notch. Technicians take it one step further by reacting the quartz with chlorine or hydrochloric acid at high temperatures, then putting it through one or two more trade‑secret steps of physical and chemical processing.

The result is what Unimin markets as Iota quartz, the industry standard of purity. The basic Iota quartz is 99.998 percent pure SiO2. It is used to make things like halogen lamps and photovoltaic cells, but it’s not good enough to make those crucibles in which polysilicon is melted. For that you need Iota 6, or the tip‑top of the line, Iota 8, which clocks in at 99.9992 percent purity—meaning for every one billion molecules of SiO , there are only 80 molecules of impurities. Iota 8 sells for up to $10,000 a ton. Regular construction sand, at the other end of the sand scale, can be had for a few dollars per ton…

…Unimin sells this ultra‑high‑purity quartz sand to companies like General Electric, which melts it, spins it, and fuses it into what looks like a salad bowl made of milky glass: the crucible. “It’s safe to say the vast majority of those crucibles are made from Spruce Pine quartz,” Schlanz says.

The polysilicon is placed in those quartz crucibles, melted down, and set spinning. Then a silicon seed crystal about the size of a pencil is lowered into it, spinning in the opposite direction. The seed crystal is slowly withdrawn, pulling behind it what is now a single giant silicon crystal. These dark, shiny crystals, weighing about 220 pounds, are called ingots.

The ingots are sliced into thin wafers. Some are sold to solar cell manufacturers. Ingots of the highest purity are polished to mirror smoothness and sold to a chipmaker like Intel. It’s a thriving multi-billion dollar industry in 2012.

The chipmaker imprints patterns of transistors on the wafer using a process called photolithography. Copper is implanted to link those billions of transistors to form integrated circuits. Even a minute particle of dust can ruin the chip’s intricate circuitry, so all of this happens in what’s called a clean room, where purifiers keep the air thousands of times cleaner than a hospital operating room. Technicians dress in an all‑covering white uniform affectionately known as a bunny suit. To ensure the wafers don’t get contaminated during manufacture, many of the tools used to move and manipulate them are, like the crucibles, made from high‑purity quartz.

The wafers are then cut into tiny, unbelievably thin quadrangular chips—computer chips, the brains inside your mobile phone or laptop. The whole process requires hundreds of precise, carefully controlled steps. The chip that results is easily one of the most complicated man‑made objects on Earth, yet made with the most common stuff on Earth: humble sand.

The total amount of high‑purity quartz produced worldwide each year is estimated at 30,000 tons—less than the amount of construction sand produced in the United States every hour. (And even construction sand is in high demand; there’s a thriving black market in the stuff.) Only Unimin knows exactly how much Spruce Pine quartz is produced, because it doesn’t publish any production figures. It is an organization famously big on secrecy. “Spruce Pine used to be mom‑and‑ pop operations,” Schlanz says. “When I first worked up there, you could just walk into any of the operations. You could just go across the street and borrow a piece of equipment.”

NOWADAYS UNIMIN WON’T even allow staff of the Minerals Research Laboratory inside the mines or processing facilities. Contractors brought in to do repair work have to sign confidentiality agreements. Whenever possible, vice‑president Richard Zielke recently declared in court papers, the company splits up the work among different contractors so that no individual can learn too much.

Unimin buys equipment and parts from multiple vendors for the same reason. Glover has heard of contractors being blindfolded inside the processing plants until they arrive at the specific area where their jobs are and of an employee who was fired on the spot for bringing someone in without authorization. He says the company doesn’t even allow its employees to socialize with those of their competitors.

It was hard to check out Glover’s stories, because Unimin wouldn’t talk to me. Unlike most big corporations, its website lists no contact for a press spokesperson or public relations representative. Several emails to their general inquiries address went unanswered. When I called the company’s headquarters in Connecticut, the woman who answered the phone seemed mystified by the concept of a journalist wanting to ask questions.

She put me on hold for a few minutes, then came back to tell me the company has no PR department, but that if I faxed (faxed!) her my questions, someone might get back to me. Eventually I got in touch with a Unimin executive who asked me to send her my questions by email. I did so. The response: “Unfortunately, we are not in a position to provide answers at this point in time.”

2. It was never about LLM performance – Justin

The LLM community is obsessed with benchmarking model performance. Mistral released their new “flagship” model this week, and immediately focused the discussion on how it performs on “commonly used benchmarks” relative to other models:

The entire blog post (I’d recommend reading it) is just a read through of how this model performs relative to other models on benchmarks, from math and coding to multilingual capabilities…

…This tendency to fixate on benchmarks is understandable – right now, it’s basically the only semi-objective way to measure how these models stack up against each other. It’s something vendors in other spaces, like data streaming, do too. But it is dangerous because it misses the point of where this whole AI thing is going, and is a textbook product marketing anti-pattern.

In a trend that we’ve seen hundreds of times in developer tooling, the underlying LLM is not going to matter within a few years. Large Language Model performance is already highly commoditized, and will continue to head in that direction. All that will matter is the experience that you build on top of these models, and what that enables for your customers.

Let’s take a look at the ChatGPT interface. Here’s a common prompt I’ve been using for testing, asking the model to summarize the contents of an external link into a tweet thread. Unrelated aside, the responses to this prompt are virtually identical across every major LLM.

Which parts of this interface are the underlying model – GPT-4 in this case – and which are an experience built by OpenAI on top of the underlying model?

The text response, minus any formatting, is what the model generated. But the:

• Ability of the model to access and scrape content from a web page
• Context of the prompt, including setting the system as a helpful assistant
• Formatting the response, like changing the numbers to gray UI for typing the prompt
• Filepicker for attaching media to the prompt
• Prompt history
• Model switcher / picker (this one is meta)
• Ability to persist and share the model responses

…and more not show here

are all not GPT-4, they’re features built by OpenAI on top of GPT-4 to create an experience that is helpful and worth paying for. Some of these are harder to build than others – OpenAI’s secret sauce obviously isn’t the little arrow that scrolls down to the bottom of the response. ChatGPT would be nothing without GPT-4 – but the reverse may also be true!

The retort to this line of reasoning is that these chat interfaces are primarily for non-technical users, while the real money for these model providers comes from developer use cases, building LLMs into user-facing applications. I’ve worked closely with one of the major model compute providers, so this is not foreign to me. But experience matters to developers too!

OpenAI has dedicated significant resources to building a seamless developer experience beyond “docs for the model.” Here’s their playground for prompting GPT models – you can adjust parameters like temperature and penalties, plus change the system prompt to be any other style…

…For a closed source model provider like OpenAI, the difference between what is model and what is experience is academic – you’re paying for both. They are one thing. But where this really matters is in open source. Does the convergence of open source performance to closed source performance really matter if the experience of using that open source is bad?…

…The open source discussion has been too anchored on reaching performance parity with OpenAI models. This is a small piece of the puzzle. For developers looking to build applications with these open source models, and especially the pro-sumer chat use case, users need to consider the holistic experience that model providers offer. Integrating LLMs into your app is almost never going to be the “drop in” experience you see on marketing sites – and my concern is that the “open source is approaching parity with OpenAI!” narrative is not actually true in a meaningful way.

Folks working in AI can look to previous examples of this phenomenon in developer tools for guidance: A couple of years ago, I wrote about how underlying performance of production relational databases is becoming commoditized, and vendors are focusing much more on developer experience. It’s going to happen here too, the question is just when.

3. Aravind Srinivas – Building An Answer Engine – Patrick O’Shaughnessy and Aravind Srinivas

Patrick: [00:07:28] It’s really cool to think about the sequencing to get there. We’ve had search engines. Like you said, it’s a hack to get the answers. You’re building what I think of today as an answer engine. I type something in, it’s just giving the answer directly with great citation and all this other stuff we’ll talk about. And the vision you’re articulating is this question engine can anticipate the things that I want to learn about and give them to me beforehand.

And I’d love to build up towards that. So maybe starting with the answer engine, explain to us how it works. Maybe you could do this via the time line of how you’ve built the product or something. But what are the components? What is happening behind the scenes when I type something into Perplexity either a question or a search query or whatever? Walk us through in some detail the actual goings on behind the scenes in terms of how the product works itself?

Aravind: [00:08:13] Yes. So when you type in a question into Perplexity, the first thing that happens is, it first reformulates the question, it tries to understand the question better, expands the question in terms of adding more suffixes or prefixes to it, to make it more well formatted. It speaks to the question engine part. And then after that, it goes and pulls so many links from the web that are relevant to this reformulated question.

There are so many paragraphs in each of those links. It takes only the relevant paragraphs from each of those links. And then an AI model, we typically call it large language model. It’s basically a model that’s been trained to predict the next word on the Internet and fine-tuned for being good at summarization and chats.

That AI model looks at all these chunks of knowledge the bits of study that surface from important or relevant links and takes only those parts that are relevant to answering your query and gives you a very concise four or five sentence answer, but also with references. Every sentence has a reference to which webpage or which chunk of knowledge it took from which webpage and puts it at the top in terms of sources.

That gets you to a nicely formatted rendered answer, sometimes in markdown bullets, or sometimes just generic paragraphs, sometimes it has images in it. But a great answer with references or citation so that if you want to dig deeper, you can go and visit the link. If you don’t want and just read the answer and ask a follow-up, you can engage in a conversation, both modes of usage are encouraged and allowed. So this is what happens on Perplexity today.

Patrick: [00:09:51] What percent of users end up clicking beneath the summarized answer into a source webpage?

Aravind: [00:10:01] At least 10%.

Patrick: [00:10:02] So 90% of the time, they’re just satisfied with what you give them?

Aravind: [00:10:06] It depends on how you look at it. If you wanted to be 100% of the time, people always click on a link, that’s the traditional Google. And you want to be 100% of the time where people never click on links, that’s ChatGPT. We think the sweet spot is somewhere in the middle. People should click on link sometimes to go do their work there. Let’s say, you’re just booking a ticket, you might actually want to go away Expedia or something.

Let’s say you’re deciding where to go first. You don’t need to go away and read all these SEO blogs and get confused on what you want to do. You first make your decision independently with this research body that’s helping you decide. And once you finished your research and you have decided, then that’s when you actually have to go out and do your actual action of booking your ticket. That way, I believe there is a nice sweet spot of one product providing you both the navigational search experience as well as the answer engine experience together. And that’s what we strive to be doing…

…Patrick: [00:13:54] Can you explain from an insider’s perspective and someone building an application on top of these incredible new technologies, what do you think the future might look like or even what you think the ideal future would be for how many different LLM providers there are, how specialized they get to scale the primary answer, so there’s only going to be a few of them. How do you think about all this and where you think it might go?

Aravind: [00:14:16] It really depends on who you’re building for. If you’re building for consumers, you do want to build a scalable infrastructure because you do want to ask many consumers to use your product. If you’re building for the enterprise, you still want a scalable infrastructure.

Now it really depends, are you building for the people within that company who are using your product. Let’s say, you’re building an internal search engine, you only need to scale to the size of the largest organization, which is like maybe 100,000 people. And not all of them will be using your thing at one moment. You’re decentralizing it, you’re going to keep different servers for different companies and you can elastically decide what’s the level of throughput you need to offer.

But then if you’re solving another enterprise’s problem, where that enterprise is serving consumers and you’re helping them do that, you need to build scalable infrastructure indirectly at least. For example, OpenAI. Their APIs are used by us, other people to serve a lot of consumers. So unless they solve that problem themselves, they’re unable to help other people solve their problem. Same thing with AWS.

So that’s one advantage you have of actually having a first-party product that your infrastructure is helping you serve. And by doing that, by forcing yourself to solve that hard problem, whatever you build can be used by others as well. Amazon build AWS first for Amazon. And because Amazon.com requires very robust infrastructure, that can be used by so many other people and so many other companies emerged by building on top of AWS.

Same thing happened with OpenAI. They needed robust infrastructure to serve the GPT-3 developer API and ChatGPT as a product. But once they got it all right, then they can now support other companies that are building on top of them. So it really depends on what’s your end goal and who you’re trying to serve and what’s the scale of our ambition…

…Patrick: [00:19:02] And when I think about the history of the product, which I was a pretty early user of, the first thing that pops to my mind is that it solves the hallucination problem, which has become less of a problem. But early on, everyone just didn’t know how to trust these things and you solved that. You gave citations, you can click through the underlying webpages, et cetera.

I’d love you to walk through what you view the major time line product milestones have been of Perplexity dating back to its start. The one I just gave could be one example. There was this possibility, but there was a problem and you solved it, at least that was my perception as a user. What have been the major milestones as you think back on the product and how it’s gotten better?

Aravind: [00:19:41] I would say the first major thing we did is really making the product a lot faster. When we first launched, the latency for every query was seven seconds, then we actually had to speed up the demo video to put it on Twitter so that it doesn’t look embarrassing.

And one of our early friendly investors, Daniel Gross who co-invests a lot with Nat Friedman, he was one of our first testers before we even released the product. And he said, you guys should call it a submit button for a query. It’s almost like you’re submitting a job and waiting on the cluster to get back. It’s that slow.

And now we are widely regarded as the fastest chatbot out there. Some people even come and ask me, why are you only as fast as ChatGPT? Why are you not faster? And little did they realize that ChatGPT doesn’t even use the web by default. It only uses it on the browsing mode on Bing.

So for us to be as fast as ChatGPT already tells you that in spite of doing more work to go pull up links from the web, read the chunks, pick the relevant ones and use that to give you the answer with sources and a lot more work on the rendering, despite doing all the additional work, if you’re managing an end-to-end latency as good as ChatGPT that shows we have like even a superior back end to them.

So I’m most proud about the speed at which we can do things today compared to when we launched, the accuracy has been constantly going up, primarily few things. One is we keep expanding our index and like keep improving the quality of the index. From the beginning, we knew all the mistakes that previous Google competitors did, which is obsessed about the size of your index and focus less on the quality.

So we decided from the beginning we would not obsess about the size. Size doesn’t matter and index actually, what matters is the quality of your index. What kind of domains are important for AI chatbots and question-answering and knowledge workers. That is what we care about. So that decision ended up being right.

The other thing that has helped us improve the accuracy was training these models to be focused on hallucinations. When you don’t have enough information in the search snippets, try to just say I don’t know, instead of making up things. LLMs are conditioned to always be helpful, will always try to serve the user’s query despite what it has access to, may not be even sufficient to answer the query. So that part took some reprogramming, rewiring. You’ve got to go and change the ways. You can’t just solve this with prompt engineering. So we have spent a lot of work on that.

The other thing I’m really proud about is getting our own inference infrastructure. So when you have to move outside the OpenAI models to serve your product, everybody thinks, “Oh, you just train a model to be as good as GPT and you’re’ done.” But reality is OpenAI’s mode is not just in the fact that they have trained the best models, but also that they have the most cost-efficient, scalable infrastructure for serving this on a large-scale consumer product like ChatGPT. That is itself a separate layer of mode. You can build that mode, you can build.

And so we are very proud of our inference team, how fast, high throughput, low latency infrastructure we built for serving our own LLMs. We took advantage of the open source revolution, Llama and Mistral and took all these models, trained them to be very good at being great answer bots and served them ourselves on GPU so that we get better margins on our product. So all these three layers, both in terms of speed through actual product back-end orchestration, accuracy of the AI models and serving our own AI models, we’ve done a lot of work on all these things…

…Patrick: [00:28:50] Can you expand on index. You’ve referenced that a few times for those that haven’t built one or haven’t thought about this. Just explain that whole concept and the decisions that you’ve made. You already mentioned quality versus size. But just explain what it means to build an index, why it’s so important, et cetera?

Aravind: [00:29:07] Yes. So what does an index mean, it’s basically a copy of the web. The web has so many links and you want a cache, you want a copy of all those links in the database, so a URL and the contents in that URL. Now the challenge here is new links are being created every day on the web and also existing links keep getting updated on the web as well. New sites keep getting updated. So you’ve got to periodically refresh them. The URL needs to be updated in the cache with a different version of it.

Similarly, you got to keep adding new URLs to your index, which means you’ve got to build a crawler. And then how you store a URL, the contents in that URL also matters. Not every page is native HTML anymore. The web has upgraded a lot, rendering JavaScript a lot, and every domain has custom-based rendered the JavaScript. So you’ve got to build parsers. So you’ve got to build a crawler, indexer, parser and that together makes up for a great index.

Now the next step comes to retrieval, which is now that you have those index, every time you hit a query, which links do you use? And which paragraphs in those links do you use? Now that is the ranking problem. How do you figure out what is relevance and ranking? And once you retrieve those chunks, like the top few chunks relevant to a query that the user is asking, that’s when the AI model comes in. So this is a retrieve part. Now the generic part. That’s why it’s called retrieve and generic.

So once you retrieve the relevant chunks from the huge index that you have, the AI model will come and read those chunks and then give you the answer. Doing this ensures that you don’t have to keep training the AI model to be up to date. What you want the AI model to do is to be intelligent, to be a good reasoning model.

Think about this as when you were a student, I’m sure you would have written an open book exam, open notes exam in school or high school or college. What are those exams test you for? They don’t test you for rote learning. So it doesn’t give an advantage to the person who has the best memory power. It gives advantage to person who has read the concepts, can immediately query the right part of the notes, but the questions required you to think on the fly as well.

That’s what we want to design systems. It’s very different philosophy from OpenAI, where OpenAI wants this one model that’s so intelligent, so smart, you can just ask it anything. It’s going to tell you. We rather want to build a small efficient model that’s smart, capable, can reason on facts that it’s given on the fly. And this ambiguate different individuals with different names or saved as not sufficient information, not get confused about dates.

When you’re asking something about the future, say that was not yet happened. These sort of corner cases handle all of those with good reasoning capabilities yet have access to all of the world’s knowledge at an instant through a great index. And if you can do both of these together end-to-end orchestrated with great latency and user experience, you’re creating something extremely valuable. So that’s what we want to build…

…Patrick: [00:37:26] Do you think that the transformer architecture is here to stay and will remain the dominant tool or architecture for a long time?

Aravind: [00:37:33] This is a question that everybody asks in the last six years or seven years since the first transformer came. Honestly, nothing has changed. The only thing that has changed is the transformer became a mixture of experts model, where there are multiple models and not just a single model. But the core self-attention model architecture has not changed. And people say there are shortcomings, the quadratic attention, complexities there. But any solution to that incurs costs somewhere else too.

Most of the people are not aware that majority of the computation in a large transformer like GPT-3 or 4 is not even spent on the attention layer. It’s actually spent on the matrix multiplies. So if you’re trying to focus more on the quadratic part, you’re incurring costs and the matrix multiples, and that’s actually the bottleneck in the larger scaling.

So honestly, it’s very hard to make an innovation on the transformer that can have a material impact at the level of GPT-4 complex cost of training those models. So I would bet more on innovations, auxiliary layers, like retrievable augmented generation. Why do you want to train a really large model when you don’t have to memorize all the facts from the Internet, when you literally have to just be a good reasoning model?

Nobody is going to value Patrick for knowing all facts. They’re going to value you for being an intelligent person, fluid intelligence. If I give you something very new that nobody else has an experience in, are you well positioned to learn that skill fast and start doing it really well. When you hire a new employee, what do you care about? Do you care about how much they know about something? Or do you care about whether you can give them any task and they would still get up to speed and do it, which employee would you value more?

So that’s the sort of intelligence that we should bake into these models, and that requires you to think more on the data. What are these models training on? Can we make them train on something else and just memorizing all the words on the Internet? Can we make reasoning emerge in these models through a different way? And that might not need innovation on the transformer, that may need innovation more on what data you’re throwing at these models.

Similarly, another layer of innovation that’s waiting to happen is the architecture like sparse versus dense models. Clearly, mixture of experts is working, GPT-4 is a mixture of experts, Mixtral is a mixture of experts, Gemini 1.5 is a mixture of experts. So even there, it’s not one model for coding, one model for reasoning and math, one model for history that depending on your input, it’s getting routed to the right model. It’s not that spares.

Every individual tokened is routed to a different model, but it’s happening every layer. So you’re still spending a lot of compute. How can we create something that’s actually 100 humans in one company? So the company itself has aggregated so much smarter. We’ve not created the equivalent at a model layer, more experimentation on the sparsity and more experimentation on how we can make reasoning emerge in a different way is likely to have a lot more impact than thinking about what is the next transformer.

4. Training great LLMs entirely from ground up in the wilderness as a startup – Yi Tay

People always assume it’s simply a question/debate of accelerator choice (TPUs vs GPUs etc) and all GPU clusters are created equal. For us, this soon proved to be false. As we sampled across different service providers, we find that the variance of hardware quality differs vastly even for the same hardware, i.e., GPUs (H100s). Note that here, hardware refers to overall cluster quality and not necessarily the chips or accelerators per se. Just like a lottery. Basically:

Not all hardware is created equal. The variance of cluster quality across hardware providers is so high that it is literally a lottery pertaining to how much pain one would have to go through to train good models. In short, a hardware lottery in the era of LLMs.

More specifically, we’ve leased a few clusters from several compute providers, each with a range of hundreds to thousands of chips. We’ve seen clusters that range from passable (just annoying problems that are solvable with some minor SWE hours) to totally unusable clusters that fail every few hours due to a myriad of reasons. Specifically, some clusters have nodes that fail every N hour with issues ranging from cabling issues (where N is unreasonably small), GPU hardware errors etc. Even more surprisingly, every cluster across the same provider could also be vastly different in terms of how robust it was…

…Did I mention you’ll also get a different Model Flop Utilisation (MFU) for different clusters!? This was a non negligible amount of compute wasted if one is unlucky enough to find a provider with badly cabled nodes or some other issues. Systems with very sub-optimal file systems would have the MFU of training runs tank the moment a team mate starts transferring large amounts of data across clusters.

Every service provider also had different levels of support. These range from being polite to nonchalant, “chatgpt-style” canned responses to blaming the user for every single thing that goes wrong.

Overall, every single cluster we tried feels like they have their own vibe, struggles and failure modes. It was also almost as though every single cluster needed their own hot-fixes for their own set of issues – some more tolerable than others. That said, we’ve learned that fail safes are important, and finding fast hot fixes for any clusters could be key…

…We’re training our models on GPUs for the most part at Reka. Personally, I’ve used TPUs all my life when it comes to large language model training at Google pre-Reka life. CUDA and nccl were the most alien thing to me ever. (I only learned it’s pronounced “Nickel” from one of my coworkers who used to work at Nvidia lol)

I was completely taken aback by the failure rate of GPUs as opposed to my experiences on TPUs at Google. In fact, I don’t actually recall TPUs failing much even for large runs, though I was not sure if I was protected from knowing this just by the sheer robustness of the outrageously good infra and having a dedicated hardware team. In fact, the UL2 20B model (at Google) was trained by leaving the job running accidentally for a month. It never failed. If this were in GPU land, it would have failed within the first few days for sure.

That said, I think this could be more about the competency of the hardware team that manages your accelerators rather than the underlying chip. The presence of having good hardware support (from your compute provider) is important. And so much hinges on them being actually competent, reinforcing the notion of the “hardware lottery”…

…It is no secret that my favourite codebase of all time is T5X and Mesh Tensorflow (named tensors ftw) but these options quickly became not viable as 1) they don’t get as much support outside Google, 2) they are kind of deprecated and 3) they are not friendly to folks on our team that are not xooglers.

We ended up going for something vanilla, seemingly stable and more popular (i.e., pytorch) that is more accessible to most people on the team (except me lol). In my first few months, I was tripping all over pip, git, docker and all these wild life stuff. Then again, I am not 100% sure about how stable or user friendly it would be to use a google codebase externally (it would have been pretty nasty I guess).

To be very frank, I would have to say the quality of codebases externally significantly lag behind those I’ve been used to at Google. Primarily because codebase within Google tends to be written by ML rockstars themselves (e.g, Noam Shazeer, Barret Zoph, Adam Roberts, Hyung Won Chung et al.) and just feel better (e.g., superior vibes) compared to those I’ve tried externally. In particular, I found myself super annoyed with the code quality when dabbling with stuff built by other companies (some way worse than others 🤗).

5. How The Interstate Highway System Changed American Industry – Lawrence Hamtil

Signed into law in 1956 by then President Dwight Eisenhower, the Federal Highway Act created the Interstate Highway System, which would become the largest and costliest public works project in history.  Measuring almost 48,000 miles in total distance, the Interstate Highway System was completed only in 1992, more than three decades after work began, and for a total cost in today’s dollars of more than $500 billion…

…Among the beneficiaries of this huge outlay were the quarry owners and aggregate miners, who provided the gravel and rock on which the interstates were laid, the heavy machinery manufacturers who provided the graders, tractors, and steamrollers that turned those rocks into roads, and the oil and gas producers and refiners who made the gasoline and diesel that fueled the project…

…As families began to set out exploring the country on the new interstate system, restauranteurs such as Ray Kroc and Howard Johnson recognized the need to provide traveling families with predictable, familiar service.  The idea of the chain restaurant was born as interstate exit ramps guided hungry motorists to McDonald’s and Howard Johnson’s.  Families would also need places to say on longer journeys, so hotels followed restaurants in the chain model as franchises like Holiday Inn became a staple of interstate exits; early ads for the hotel underlined the value of the familiar by stating, “The best surprise is no surprise.”

The logistical flexibility provided by the interstate system also gave rise to a whole new model of retailing:  big box stores began to set up in small towns offering rich variety and low prices to consumers previously left unserved by larger retailers.  Walmart’s 1975 annual report detailed just such a model…

…Whereas not quite a century before the railroads had aided in the rise of Sears, Roebuck, and Co. as the first retailer with national reach, the interstate in the 1960s and 1970s would provide the backbone of Walmart’s logistical operations, with large distribution centers situated at critical points throughout the interstate network to facilitate inventory replenishment, as Professor Jesse LeCavalier has noted on his blog. 

Disclaimer: The Good Investors is the personal investing blog of two simple guys who are passionate about educating Singaporeans about stock market investing. By using this Site, you specifically agree that none of the information provided constitutes financial, investment, or other professional advice. It is only intended to provide education. Speak with a professional before making important decisions about your money, your professional life, or even your personal life. We currently have a vested interest in Alphabet (parent of Google), Amazon, Apple, and Microsoft. Holdings are subject to change at any time.