The Power of Ideas

In the interests of full disclosure, the following post was written by Google Notebook LM’s “blog” report writing feature. It references a paper titled “When Ideas Trump Interests,” by economist Dani Rodrik I read a while back but never got around to writing up.

Over to Notebook LM ….

Introduction: The Hidden Force in Politics

It is a common and cynical view of politics: powerful “vested interests” and special interest groups always get their way at the expense of the general public. We see it as a battle of raw power, where lobbies and elites push their agenda, and the rest of us pay the price. This perspective is so ingrained that it forms the foundation of most modern models of political economy.

But what if this view is incomplete? In his paper, “When Ideas Trump Interests,” economist Dani Rodrik challenges this conventional wisdom. He argues that before we can even talk about interests, we have to talk about “ideas”—a powerful, often overlooked force that shapes what those interests are, how actors understand the world, and what they believe is possible. This perspective doesn’t dismiss the role of powerful groups, but it places them in a new context where their influence is not a foregone conclusion.

This post will distill the most surprising and impactful takeaways from Rodrik’s argument. We will explore how ideas about our identity, our understanding of the world, and our policy imagination are the true drivers of political outcomes.

Three Surprising Ways Ideas Shape Our World

The standard political playbook focuses on interests and power, but the real story is often more complex. Here are three key insights that reveal how ideas, not just interests, shape our world.

Takeaway 1: Your ‘Interests’ Aren’t Fixed—They’re an Idea About Who You Are

The concept of “self-interest” seems simple enough—we all want what’s best for ourselves. But Rodrik argues that before anyone can pursue their interest, they must first have an idea of their “self.” Who we believe we are fundamentally determines what we value and, therefore, what we pursue.

This identity isn’t fixed or purely economic. We might see ourselves primarily as a member of a social class (‘middle class’), an ethnic group, a religion, a nation (‘global citizen’), or a profession. These identities dictate our priorities, which can easily override purely material concerns. As the source text notes, abstract ideals and moral conceptions can be powerful motivators:

“humans will kill and die not only to protect their own lives or defend kin and kith, but for an idea—the moral conception they form of themselves, of ‘who we are’”

This is a profoundly counter-intuitive point because it helps explain a wide range of “anomalous” political actions. When people vote against their immediate material interests, it’s often because an idea about their identity—their values, their community, their place in the world—has taken precedence.

Takeaway 2: Policy Is Driven by Beliefs About How the World Works

Policymakers and political groups don’t operate in a vacuum; they act based on their “worldviews,” or their mental models of how the economy and society function. These underlying ideas create the entire framework for political debate and lead to vastly different policy preferences. Think of the great economic debates: laissez-faire vs. planning, free trade vs. protectionism, or Keynesian vs. Hayekian economics. Each position stems from a different core idea about how the world works.

The 2008 global financial crisis is a perfect case study. It’s easy to blame powerful banking interests for the policies that led to the meltdown, and they were certainly a factor. However, their success was enabled by a prevailing set of ideas that favored financial liberalization and self-regulation. The argument that won the day wasn’t that deregulation was good for Wall Street, but that it was good for Main Street—that it was in the public interest.

But this isn’t a one-sided phenomenon. As Rodrik points out, the other side of the debate was also driven by ideas. Many observers argued the crisis was caused by excessive government intervention to support housing markets. This view wasn’t just a cover for other interests; it was grounded in powerful ideas about the social value of homeownership and the need to correct for the financial sector’s inattentiveness to lower-income borrowers. Powerful interests rarely win by nakedly arguing for their own gain; they seek legitimacy by framing their goals within a popular and persuasive idea. This is critical because it tells us that changing policy isn’t just about overpowering an opposing group. It requires challenging the underlying ideas and narratives that give that group’s position its legitimacy in the first place.

Takeaway 3: Political Gridlock Can Be Broken by Creative Policy—Not Just Power Shifts

A common argument in political economy is that entrenched elites often block efficient, growth-oriented policies because they fear losing their political power. If a new policy threatens their position, they will fight it, even if it benefits society as a whole. This creates a state of permanent gridlock where progress is impossible.

Rodrik offers a more optimistic counter-argument, introducing a concept he calls the “political transformation frontier”—the set of maximal economic outcomes elites believe they can achieve without losing power. The standard view assumes this frontier is fixed. But Rodrik argues that new policy ideas can shift the entire frontier outward, creating win-win scenarios that allow for progress without directly threatening elite power. The key is not to overpower the elites, but to reframe the problem with an innovative solution.

China’s “dual-track” reform is the prime example. In the 1970s, liberalizing agriculture would have created huge efficiency gains but destroyed the state’s tax base. Instead of abolishing the old system, Chinese leaders grafted a market system on top of it. Farmers still had to meet state grain quotas at fixed prices, but they were free to sell any surplus on the open market. This creative idea allowed China to gain the benefits of market incentives while protecting the rents and power of the state sector. The Communist Party was strengthened, not weakened.

This principle is a recurring pattern, not a one-off. A similar dynamic played out in Japan after the Meiji restoration. There, elites spurred industrialization but designed it in a way that would “strengthen the centralized government and increasing the entrenchment of bureaucratic elites.” In both cases, a creative idea allowed elites to pursue economic gains not as a threat to their power, but as a means of consolidating it. This takeaway has an optimistic implication: many political problems that seem impossible may be solvable with the right innovative idea.

Conclusion: It’s the Ideas, Stupid

The traditional view of politics as a raw contest of vested interests is compellingly simple, but ultimately incomplete. Interests are not fixed, pre-ordained forces. They are shaped and defined by ideas—ideas about our identity, ideas about how the world works, and ideas about what is possible.

As Rodrik’s work powerfully argues, the failure to see the role of ideas leads to a pessimistic and static view of political change. By putting ideas back at the center of the analysis, we see that political outcomes are not inevitable. The source text concludes with a thought that perfectly captures this shift in perspective:

“What the economist typically treats as immutable self-interest is too often an artifact of ideas about who we are, how the world works, and what actions are available.”

This leaves us with a final, crucial question. If ideas are this powerful, perhaps the most important political question isn’t just ‘who has power?’ but ‘which ideas will define our future?’

Distinguishing luck and skill

Quantifying Luck’s Role in the Success Equation

“… we vastly underestimate the role of luck in what we see happening around us”

This post is inspired by a recent read of Michael Mauboussin’s book “The Success Equation: Untangling Skill and Luck in Business, Sports and Investing”. Mauboussin focuses on the fact that much of what we experience is a combination of skill and luck but we tend to be quite bad at distinguishing the two. It may not unlock the secret to success but, if you want to get better at untangling the contributions that skill and luck play in predicting or managing future outcomes, then this book still has much to offer.

“The argument here is not that you can precisely measure the contributions of skill and luck to any success or failure. But if you take concrete steps toward attempting to measure those relative contributions, you will make better decisions than people who think improperly about those issues or who don’t think about them at all.”

Structure wise, Mauboussin:

  • Starts with the conceptual foundations for thinking about the problem of distinguishing skill and luck,
  • Explores the analytical tools we can use to figure out the extent to which luck contributes to our achievements, successes and failures,
  • Finishes with some concrete suggestions about how to put the conceptual foundations and analytical tools to work in dealing with luck in decisions.

Conceptual foundations

It is always good to start by defining your terms; Mauboussin defines luck and skill as follows:

“Luck is a chance occurrence that affects a person or a group.. [and] can be good or bad [it] is out of one’s control and unpredictable”

Skill is defined as the “ability to use one’s knowledge effectively and readily in execution or performance.”

Applying the process that Mauboussin proposes requires that we first roughly distinguish where a specific activity or prediction fits on the continuum bookended by skill and luck. Mauboussin also clarifies that:

  • Luck and randomness are related but not the same: He distinguishes luck as operating at the level of the individual or small group while randomness operates at the level of the system where more persistent and reliable statistical patterns can be observed.
  • Expertise does not necessarily accumulate with experience: It is often assumed that doing something for a long time is sufficient to be an expert but Mauboussin argues that in activities that depend on skill, real expertise only comes about via deliberate practice based on improving performance in response to feedback on the ways in which the input generates the predicted outcome.

Mauboussin is not necessarily introducing anything new in his analysis of why we tend to bad at distinguishing skill and luck. The fact that people tend to struggle with statistics is well-known. The value for me in this book lies largely in his discussion of the psychological dimension of the problem which he highlights as exerting the most profound influence. The quote below captures an important insight that I wish I understood forty years ago.

“The mechanisms that our minds use to make sense of the world are not well suited to accounting for the relative roles that skill and luck play in the events we see taking shape around us.”

The role of ideas, beliefs and narratives is a recurring theme in Mauboussin’s analysis of the problem of distinguishing skill and luck. Mauboussin notes that people seem to be pre-programmed to want to fit events into a narrative based on cause and effect. The fact that things sometimes just happen for no reason is not a satisfying narrative. We are particularly susceptible to attributing successful outcomes to skill, preferably our own, but we seem to be willing to extend the same presumption to other individuals who have been successful in an endeavour. It is a good story and we love stories so we suppress other explanations and come to see what happened as inevitable.

Some of the evidence we use to create these narratives will be drawn from what happened in specific examples of the activity, while we may also have access to data averaged over a larger sample of similar events. Irrespective, we seem to be predisposed to weigh the specific evidence more heavily in our intuitive judgement than we do the base rate averaged over many events (most likely based on statistics we don’t really understand). That said, statistical evidence can still be “useful” if it “proves” something we already believe; we seem to have an intuitive bias to seek evidence that supports what we believe. Not only do we fail to look for evidence that disproves our narrative, we tend to actively suppress any contrary evidence we encounter.

Analytical tools for navigating the skill luck continuum

We need tools and processes to help manage the tendency for our intuitive judgements to lead us astray and to avoid being misled by arguments that fall into the same trap or, worse, deliberately exploit these known weaknesses in our decision-making process.

One process proposed by Mauboussin for distinguishing skill from luck is to:

  • First form a generic judgement on what the expected accuracy of our prediction is likely to be (i.e. make a judgement on where the activity sits on the skill-luck continuum)
  • Next look at the available empirical or anecdotal evidence, distinguishing between the base rate for this type of activity (if it exists) and any specific evidence to hand
  • Then employ the following rule:
    • if the expected accuracy of the prediction is low (i.e. luck is likely to be a significant factor), you should place most of the weight on the base rate
    • if the expected accuracy is high (i.e. there is evidence that skill plays the prime role in determining the outcome of what you are attempting to predict), you can rely more on the specific case.
  • use the data to test if the activity conforms to your original judgement of how skill and luck combine to generate the outcomes

Figuring out where the activity sits on the skill-luck continuum is the critical first step and Mauboussin offers three methods for undertaking this part of the process: 1) The “Three Question” approach, 2) Simulation and 3) True Score Theory. I will focus here on the first method which involves

  1. First ask if you can easily assign a cause to the effect you are seeking to predict. In some instances the relationship will be relatively stable and linear (and hence relatively easy to predict) whereas the results of other activities are shaped by complex dependencies such as cumulative advantage and social preference. Skill can play a part in both activities but luck is likely to be a more significant factor in the latter group.
  2. Determining the rate of reversion to the mean: Slow reversion is consistent with activities dominated by skill, while rapid reversion comes from luck being the more dominant influence. Note however that complex activities where cumulative advantage and social preference shape the outcome may not have a well-defined mean to revert to. The distribution of outcomes for these activities frequently conform to a power law (i.e. there are lots of small values and relatively few large values).
  3. Is there evidence that expert prediction is useful? When experts have wide disagreement and predict poorly, that is evidence that luck is a prime factor shaping outcomes.

One of the challenges with this process is to figure out how large a sample size you need to determine if there is a reliable relationship between actions and outcome that evidences skill.  Another problem is that a reliable base rate may not always be available. That may be because the data has just not been collected but also because a reliable base rate simply may not even exist.

The absence of a reliable base rate to guide decisions is a feature of activities that do not have simple linear relationships between cause and effect. These activities also tend to fall into Nassim Taleb’s “black swan” domain. The fundamental lesson in this domain of decision making is to be aware of the risks associated with naively applying statistical probability based methods to the problem. Paul Wilmott and David Orrell use the idea of a “zone of validity” to make the same point in “The Money Formula”.

The need to understand power laws and the mechanisms that generate them also stands out in Mauboussin’s discussion of untangling skill and luck.

The presence of a power law depends in part on whether events are dependent on, or independent of, one another. In dependent systems, initial conditions matter and come to matter more and more as time goes on. The final outcomes are (sometimes surprisingly) sensitive to both minor variations in the initial conditions and to the path taken over time. Mauboussin notes that a number of mechanisms are responsible for this phenomenon including preferential attachment, critical points and phase transitions are also crucial.

“In some realms, independence and bell-shaped distributions of luck can explain much of what we see. But in activities such as the entertainment industry, success depends on social interaction. Whenever people can judge the quality of an item by several different criteria and are allowed to influence one another’s choices, luck will play a huge role in determining success or failure.”

“For example, if one song happens to be slightly more popular than another at just the right time, it will tend to become even more popular as people influence one another. Because of that effect, known as cumulative advantage, two songs of equal quality, or skill, will sell in substantially different numbers. …  skill does play a role in success and failure, but it can be overwhelmed by the influence of luck. In the jar model, the range of numbers in the luck jar is vastly greater than the range of numbers in the skill jar.”

“The process of social influence and cumulative advantage frequently generates a distribution that is best described by a power law.”

“The term power law comes from the fact that an exponent (or power) determines the slope of the line. One of the key features of distributions that follow a power law is that there are very few large values and lots of small values. As a result, the idea of an “average” has no meaning.”

Mauboussin’s discussion of power laws does not offer this specific example but the idea that the average is meaningless is also true of loan losses when you are trying to measure expected loss over a full loan loss cycle. What we tend to observe is lots of relatively small values when economic conditions are benign and a few very large losses when the cycle turns down, probably amplified by endogenous factors embedded in bank balance sheets or business models. This has interesting and important implications for the concept of Expected Loss which is a fundamental component of the advanced Internal Rating Based approach to bank capital adequacy measurement.

Mauboussin concludes with a list of ten suggestions for untangling and navigating the divide between luck and skill:

  1. Understand where you are on the luck skill continuum
  2. Assess sample size, significance and swans
  3. Always consider a null hypothesis – is there some evidence that proves that my base  belief is wrong
  4. Think carefully about feedback and rewards; High quality feedback is key to high performance. Where skill is more important, then deliberate practice is essential to improving performance. Where luck plays a strong role, the focus must be on process
  5. Make use of counterfactuals; To maintain an open mind about the future, it is very useful to keep an open mind about the past. History is a narrative of cause and effect but it is useful to reflect on how outcomes might have been different.
  6. Develop aids to guide and improve your skill; On the luck side of the continuum, skill is still relevant but luck makes the outcomes more probabilistic. So the focus must be on good process – especially one that takes account of behavioural biases. In the middle of the spectrum, the procedural is combined with the novel. Checklists can be useful here – especially when decisions must be made under stress. Where skill matters, the key is deliberate practice and being open to feedback
  7. Have a plan for strategic interactions. Where your opponent is more skilful or just stronger, then try to inject more luck into the interaction
  8. Make reversion to the mean work for you; Understand why reversion to the mean happens, to what degree it happens, what exactly the mean is. Note that extreme events are unlikely to be repeated and most importantly, recognise that the rate of reversion to the mean relates to the coefficient of correlation
  9. Develop useful statistics (i.e.stats that are persistent and predictive)
  10. Know your limitations; we can do better at untangling skill and luck but also must recognise how much we don’t know. We must recognise that the realm may change such that old rules don’t apply and there are places where statistics don’t apply

All in all, I found Maubossin’s book very rewarding and can recommend it highly. Hopefully the above post does the book justice. I have also made some more detailed notes on the book here.

Tony