The Progress Paradox

If we’ve had so much progress, why isn’t everything awesome?

This essay is only a 10-minute read. It looks longer because it has footnotes with tangents. Particularly good footnotes are marked with a magnifying glass. 🔍

I. Intro

Some people think the world is falling apart while others think it's becoming great.

• The “it’s great” people point to lower child mortality, exponential GDP growth, the internet, and so on.

• The “it’s falling apart” people point to rising rates of depression, widespread financial anxiety, oppression, and a general feeling of wrongness.

What’s going on? Let's find out.

II. Some puzzling evidence

To understand what's happening, we’ll need to make sense of cases like these:

• In the last 100 years in the US, there’s been 5x growth in GDP per capita, and also an increase in deaths from suicide, alcohol, and drugs¹

• Twenty thousand years ago, some hunter-gatherers² likely had great work and social lives: they only worked 33 hrs/week³, were food secure, physically fit⁴, and got to live with close friends/family and frequently visit friends.⁵ 🔍

  • But also ~25% of their infants died in their first year

  • And they killed each other at rates ~2.5x the current US average.

• About twenty thousand years later, some working lives became worse: for example, women in rural West Texas in the 1930s did extreme quantities of physical labor (which prematurely bent their backs) in near-total isolation.⁶

  • But then rural electrification made the lives of these working women significantly easier.

We can see that things have gotten both better and worse in different ways across times and places. Let’s figure out what’s going on.

III. No single factor explains everything

Often people point to a particular factor to explain everything. While such factors often capture part of the truth, none fully explain what's going on. For example:

• Inequality: Some people do have it very tough because they lack resources that others have in abundance. But it’s not the whole picture — for example, there are affluent high schools where teen girls have 3× the rate of depression of other teens. source

• Social isolation: Social isolation is a major problem. But isolation alone doesn’t explain cases like Appalachia, which is more socially connected than some other regions while still having the largest rise in diseases of despair.⁷ 🔎

• Growth: Could capitalism, which has lifted tremendous numbers out of material poverty, just continue and make everything great? Let's examine this in depth.

IV. Is growth enough? Let’s look at some charts.

At first glance, this kind of chart seems convincing:

It’s easy to imagine that if growth continues, the correlation will hold and poorer countries will swing up and to the right, becoming wealthier and more satisfied with life.

There’s a critical detail, though: how life satisfaction is measured. In the above chart, life satisfaction was measured using the the Cantril Ladder, which is this survey question:

• Please imagine a ladder with steps numbered from zero at the bottom to 10 at the top.

• The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you.

• On which step of the ladder would you say you personally feel you stand at this time?

If respondents took “best possible life” and “worst possible life” literally, it would measure what we want. However, people don’t interpret the question that way: when asked which rung people wish they were at, half of respondents give an answer of the 8th rung or lower! (If the measure were more functional, everyone would’ve said they wanted the best possible life for them — aka the 10th rung.)

People have studied how people interpret the question, and it turns out that the Cantril Ladder elicits thoughts about power and wealth because it uses a ladder metaphor and top-to-bottom hierarchy. It is little surprise that this measure, which evokes thoughts of wealth, then correlates well with wealth. No survey method is perfect, but the Cantril Ladder appears quite misaligned with what we’d like to know.

For an alternative, we can look to Gallup’s Positive Experience Index, which asks these questions:

Now, please think about Yesterday, from the morning until the end of the day. Think about where you were, what you were doing, who you were with, and how you felt...

• Did you experience a lot of enjoyment during a lot of the day Yesterday?

• Did you feel well-rested Yesterday?

• Did you learn or do something interesting Yesterday?

• Did you smile or laugh a lot Yesterday?

• Were you treated with respect all day Yesterday?

Someone’s positive experience index is the portion of these five questions that they said yes to.

I couldn’t find a chart that mapped the positive experience index against GDP per capita, so I made one:

This is quite a different picture. The correlation with GDP per capita is present, but weak. Some poorer countries (especially in Central America and Southeast Asia) seem to be doing better than everyone else, including rich countries.

What’re we to make of the discrepancy? The positive experience index, though imperfect, seems like a better measure of thriving than the Cantril Ladder (see footnote for discussion⁸)

The positive experience index shows that there are materially poor people who are thriving, and wealthy people who are not. So, while growth may be an important factor, it’s clearly not the only ingredient that matters. (It turns out lots of things matter, and material wealth is one of them…)

V. But would enough growth do the trick?

If we were all ridiculously wealthy, could we just easily solve our problems? There are some reasons to think not:

• There are plenty of millionaires today who are miserable, despite making three orders of magnitude more than people did on average a thousand years ago.

• Given that the first 1000× in growth didn’t solve all their problems, why should we expect that more would?

There must be something more going on. In particular:

Growth provides market goods (stuff you can buy), but does not provide non-market goods (friends, love, the esteem of others, community, trust, mental health, serenity, etc).

So: growth offers some of what we need, but not all of it.

And the situation is actually worse than that: because non-market goods don’t have price signals, our economy is blind to them, and often inadvertently disrupts them. This is a surprisingly large effect.⁹ 🔎

Growth is great for providing market goods, but non-market goods also matter.

VI. A model of human thriving that resolves the paradox

What would a full solution of the paradox look like? It would need to:

1. Explain why so many different factors matter.

2. Explain why things have gotten worse in some times/places despite substantial advancements and material gains.

Here’s a model that meets these requirements:

Thriving is like a chemical reaction that requires many vital ingredients. If even one or two ingredients are missing, the reaction does not occur.

A missing ingredient cannot be made up for by abundance of other ingredients.

Here’s an illustrative sketch of the minimum ingredients that might be required for thriving:

• enough food/water/shelter/air/etc

• at least a couple of solid close relationships

• at least one adult who was consistently emotionally attuned to you while you in your childhood

• a modicum of community

• a modicum of purpose

• minimal unhealed trauma

• minimal significant chronic stress or disease

• some hope about the future

• feeling physically safe

• some economic security

• some autonomy/power

• not being trapped in a caste structure

• having some status

• decent physical fitness

• decent sleep (no racing thoughts, low noise pollution, etc)

• some good feelings (contentment, merriment, awe, curiosity, exhilaration, etc)

• …

(If you question the importance of any of these items, just imagine leading a life without ever having had any of it.)

Some of these ingredients may have become uncommon in some places, for example in the US¹⁰:

• decent sleep (~60% lack this)

• community

• solid emotionally-connected relationships (~~40% lack this)

• minimal chronic daily stressors (e.g. workplace-related) (~50% lack this)

• minimal unhealed trauma (~35% lack this)

• sufficient sense of purpose (~60% lack this)

Because some ingredients have become uncommon, the odds that any given person has all of them have declined — it’s like needing to flip a coin heads several times in a row.

This offers a solution to the progress paradox: there has been incredible progress/abundance in some ingredients, but many people are missing at least one ingredient.

Why are many people missing at least one ingredient?

• Particular ingredients have become uncommon on average

• Unequal distributions within particular ingredients

• Anti-correlations between ingredients

  • For example: people may face a choice between living in the country, where they lack materially, or moving to a city, where they get chronically stressed. So, few people might have both enough material wealth and low stress.

  • Or: those who have material abundance due to long working hours may have less time for social connection.

  • Community is often wrought from interdependent needs, which can disappear when resources are abundant.

  • It’s possible to increase the population average of every ingredient, whilst simultaneously there are fewer people who possess all ingredients.

• Adverse network effects

  • Example: people move more often for work, which shears the social fabric.

  • Example: mental health — people who have become emotionally reactive can cause their children or others to also become reactive.

• Economic externalities that damage non-market goods

• Individuals and societies don’t keep all the ingredients in mind with decisions they make, and they make lots of big/powerful decisions, so they often unintentionally destroy some of the ingredients.

Various ingredients have become abundant or scarce at particular times in particular places. This explains why history has been so mixed — with pockets of thriving and of despair — and why so many factors matter but none alone is the whole picture.

---

We can call this the “chemical reaction” model of human thriving (it differs from Maslow’s hierarchy¹¹). To make the chemical reaction model more concrete, let’s try to represent it quantitatively. We’re looking for an expression for thriving where:

• All ingredients are required

• Only a smallish amount of each ingredient is needed to do pretty well (more helps, but in diminishing degree)

Here’s a toy model that satisfies those requirements: thriving is the product of the logarithm of each ingredient:

\(\begin{gather*} \large \text{Thriving} \approx \prod_{i=1}^{n} \log{(x_i+1)} \\[2em] \normalsize \begin{aligned} &\text{where:} \\[0.5em] &x_i \text{ represents the level of each ingredient} \\ &\quad\text{(food, community, relationships, etc.)} \\[0.5em] &n \text{ is the total number of ingredients} \end{aligned} \\[2em] \text{This is a toy model; not empirically validated.} \end{gather*}\)

This formula illustrates the dynamics of the chemical reaction model: each ingredient is vital (you need at least some of it to thrive) and tapers (the benefits to more tail off due to the logarithm). Because the formula uses multiplication, any single ingredient being near zero causes the whole thing to be near zero.¹² 🔍

The model presented here is just illustrative¹³. A more complete model might codify ingredients and allow for individual differences (some people may have higher or lower thresholds for particular ingredients). The hope is that the model captures the right “shape” for the thriving function:

• There are lots of ingredients

• No ingredient can be neglected

• There are diminishing returns to each ingredient

• Some ingredients are non-market goods

• A missing ingredient cannot be substituted for by other ingredients

VII. Implications

We've resolved the paradox with a new model of human thriving. What follows? Can we use this understanding to live better lives? The model suggests some actions.

Individual actions:

• Figure out which ingredient(s) you are short of and work to obtain them!

• Don’t worry about getting more of the things you already have enough of, even if you feel an impulse to get more of them. Instead, focus on obtaining a bit more of anything you have little to none of.

Societal actions:

1. Run surveys to discover which ingredients are rare in which areas — for each city and state, build a dashboard with charts for each ingredient. Intervene to help where needed. Robust progress would look like 📈📈📈📈📈 across the board.

2. Advocate for growth in both market goods and non-market goods.

   • Historically, lefties/hippies/degrowthers/etc have emphasized non-market goods, whilst economic conservatives/libertarians/liberals have emphasized market goods.¹⁴ 🔎 However, there is no law of physics against having both!

3. Foster slack (time and attention which are not consumed by other demands). Slack allows people to produce non-market goods.¹⁵ 🔎

4. Build new ways to produce non-market goods.¹⁶ 🔎

As we depart further from our ancestral environments, we accidentally neglect some things we need. The solution is not to return to the ancestral environment — that would be impractical and have too many downsides. Instead, we should understand ourselves better and use that understanding to guide our individual and collective actions.

---

tldr

• Some ingredients of thriving have become abundant thanks to progress

• Other ingredients have become less common

• Many people are missing some of the ingredients they need

This is why many people feel there has been tremendous progress (there has been!), and simultaneously many people feel that things are wrong/bad (they are.)

1. Over the last century (1900 – 2017), the US has had >5x growth in GDP per capita:

Yet over the same time period, deaths of despair have not dropped:

Keep in mind that each death of despair is the tip of an iceberg — each implies many more people who are doing very badly, but not quite badly enough to die.

2. Much of our direct evidence of how hunter-gatherers lived is from recent work, e.g. the !Kung San who were studied extensively in the 1960s. There is archeological evidence that groups of people lived much the same way 10,000–20,000 years earlier (very similar tools, geographical locales, etc.)

3. From The !Kung San: Men, Women and Work in a Foraging Society by Richard B. Lee (p 278):

• Subsistence (e.g. gathering, hunting) — 17.1 hrs/wk

• Making & maintaining things — 6.3 hrs/wk

• Housework — 18.9 hrs/wk

The degree to which researchers count housework as work varies:

• Modern figures usually exclude housework.

• The !Kung San book’s definition of housework includes some activities, like gathering firewood, that would count as work in modern stats.

To make the !Kung figures comparable with modern statistics, we count half of !Kung housework hours as “work”, yielding 33 hrs/week.

4

• !Kung San Food security: they found it so easy to gather more food that they only bothered to keep three days of food on hand. Mongongo nuts were plentiful and nutritious. They hunted mainly to add variety to their diet, so it didn’t matter much if their hunts failed.

• Fitness: they were thin and walked five miles per day.

5. 🔎 In cities in the US, many people aspire to live in walking distance of friends and family. From this perspective, the !Kung had an admirable setup:

• There were seven water holes suitable for camping.

• Each was occupied by 10-30 people (often relatives).

• On any given day about 15% of people were off visiting another camp.

• Camps sometimes split or joined or reorganized. This could gracefully account for changing affinities, marriages, etc all while keeping everyone a walking-distance visit away.

(This seems advanced compared to modern static living arrangements)

6. See: the chapter titled “The Sad Irons” in the Path to Power by Robert Caro. P.S. if you are curious about how the world works, I recommend reading all of Caro’s work.

7. 🔎 Anecdotal counterexamples like this one have a bad rap, but they can be powerful. Sometimes it only takes a single clear case to overturn a theory. Richard Feynman gave the example of trying to understand the rules of chess by watching gameplay. You might think you’ve built up an accurate understanding, but a single counterexample (such as there being two black queens on the board at once — due to a promoted pawn) can be enough to tell you your picture is wrong or incomplete.

8. The positive experience index is undoubtedly imperfect — I can imagine issues around some cultures being more forthcoming than others, defining various emotional states differently from one another, etc. That said, there are some major points in its favor:

• The positive experience index includes multiple concrete questions covering a range of experiences/areas of life. I think that makes it more robust than vaguer measures.

• We have evidence that people interpret the Cantril Ladder in ways misaligned with what we aim to know.

Imagine meeting a wealthy person who lives on a large estate who gives a Cantril Ladder score of 8/10, but also reports that they are usually not well-rested, tend not to experience much enjoyment, and most days don’t smile/laugh much (Positive experience index: 2/5). I would take the view that they’re not doing that well, despite their Cantril Ladder score.

I don’t think either measure alone is complete enough to capture all relevant aspects of well-being. For example, some people might care tremendously about peak moments in their lives, or difficult/important achievements — neither of which would be captured well by either measure.

9. 🔎 Economists speak of externalities as if they are exceptional or unusual — but I think externalities are pervasive and massive (and often subtle/non-obvious).

For example: people seeking income move cities for work more than they used to, which tears the social fabric. This reduces the average lifetime of friendships for everyone (not just for the people that move!) This is an unintended consequence: companies are not against friendships, they just don’t think about them. This is a big deal — c.f. the loneliness epidemic.

Another example: when we started optimizing for energy efficiency, we sealed up houses — which stopped homes from receiving fresh air, inadvertently causing bad indoor air quality and health issues.

Whenever we optimize hard for something (such as growth), we accidentally mess up everything else. Most valuable things are delicate, so they’ll often be damaged when we make major changes without considering them. (Picture a grand gothic cathedral supported by flying buttresses. Now, imagine that someone starts adding and removing bricks according to some other criteria, such as their shade of gray. Because these changes are made without considering the structure of the building, it’s as if bricks are being added and removed at random. Most likely the building will become weaker, not stronger! This is why value is accidentally destroyed when people optimize for something else.)

This “gothic cathedral is more likely to be destroyed than buttressed” effect is why you’re more likely to see headlines like “Microplastics cause inflammation!” than “Microplastics cause surprise health benefits!”

10

• Sleep: A 2020 survey by The National Sleep Foundation found that only about 16% of Americans say they feel well-rested every day, with another 35% saying they feel well-rested most days.

• Emotionally-connected relationships: in 2004 General Social Survey, 25% of people reported having no confidants to discuss important matters with. I adjusted the number up to 40% because even those that do have confidants don’t necessarily have solid emotional connections with them, e.g. people (often men) who were raised not to share or be aware of their feelings, etc.

• Minimal chronic daily stressors: The American Psychological Association Stress in America survey: 60% are a 6 or above on a 10 point average stress scale.

• Minimal unhealed trauma: 30-40% of children have experienced complex trauma according to the National Child Traumatic Stress Network (And most people don’t find the resources to heal).

• Purpose: A 2018 Gallup poll found that about 4 in 10 Americans feel they have discovered a satisfying life purpose (and thus 60% have not).
  

11. Maslow’s hierarchy of needs posits sequential “stages” to fulfill, from physiological to self-actualization. In contrast, the chemical reaction model emphasizes many simultaneous and non-substitutable ingredients. It’s a bit like Maslow’s hierarchy except with everything squashed into a single level.

12. 🔎 This “product of logarithmis” metric is an example of a plural maximand — an optimization target which requires attention to all its parameters. No parameter can be neglected, and the overall result cannot be dominated by large increases in any single parameter.

Many people have developed an instinctive aversion to numeric optimization targets, but that might be because they’re mainly familiar with singular maximands that flatten important complexities. However, it is possible to design metrics that incorporate and deepen.

13. Empirically validating the chemical reaction thriving model would be difficult because it’s simultaneously proposing both: (a) what thriving is, and (b) how to measure it. I don’t think there is a good existing gold standard to compare against.

One approach could be to collect gold-standard thriving data by following people around with movie cameras, systematically documenting their lives, and interviewing them. Third parties could then watch these life vignettes and express the degree to which they think the subject is thriving. Such qualitative data is much richer and could make for a reasonable standard.

We could then see how accurately these gold standard scores correlate with a codified version of the thriving formula described in this essay (and determine if the correlations are better or worse than existing metrics such as the Cantril Ladder, Positive Experience Index, Kahneman’s Day Reconstruction Method, and so on.)

14. 🔎 There does not seem to be a movement or political party which advocates for both more market goods and more non-market goods.

• Leftists/socialists/degrowthers/etc usually focus on non-market goods. Many of them employ wishful thinking about production of market goods, the need for meritocracy, etc, which taken to extremes can lead to famine.

• Conservatives/libertarians/liberals/etc usually focus on growth in market goods (at the expense of non-market goods). Their hope is that individual autonomy/freedom will yield sufficient non-market goods, but this has proven inadequate due to negative externalities from growth and the network nature of many non-market goods (e.g. cultural norms).

• I’m interested in the synthesis — aiming for robust growth of both market and non-market goods. Anyone with me? What should we call ourselves?

15. 🔎 When people have significant slack — time and attention not consumed by other demands — they often create non-market goods for themselves and others. For example: when US women were mostly barred from the workforce, they often worked to provide non-market goods for society — organizing communities, social spaces, social relationships, events, etc. When women entered the workforce, it shifted our national allocation of labor from non-market goods towards market goods.

We could find equitable ways to generate slack while still allowing paid work.

One difficulty is a “race to the bottom” — e.g., companies that allow employees slack may be outcompeted by less generous companies. One market-based solution could be a Pigouvian tax on companies that over-dominate the time/attention/slack of their employees, as measured via survey. This could efficiently trade some market goods for non-market goods.

16. 🔎 To produce more non-market goods, and to protect them, we need to keep them in mind during decisions at all levels (companies, cities, states, nations).

In markets, we have price signals that reveal information about what people want. We may need something analogous for non-market goods. For example, so-called collective intelligence tools like Community Notes can surface commonsense views. Weaving such tools throughout our decision-making could help generate and protect non-market goods.