How Progress Ends

1. The Dual Nature of Progress: Exploration vs. Exploitation

Stated simply, centralized bureaucratic management is most advantageous for exploiting low-hanging technological fruit and spearheading technological catch-up, while decentralized systems are better for exploring new technological trajectories, which is the only way to make progress once the technological frontier is reached.

Two paths to progress. Economic development hinges on two distinct approaches: exploration and exploitation. Centralized, bureaucratic systems excel at exploitation, efficiently scaling existing technologies and driving catch-up growth by adopting proven methods. Conversely, decentralized systems, characterized by numerous small-scale experiments, are superior for exploration, pushing the boundaries of knowledge and creating radical new inventions.

Dynamic efficiency requires sacrifice. Achieving dynamic efficiency—technological progress over time—often necessitates a temporary loss of static efficiency. This means sacrificing immediate output or predictable outcomes to invest in uncertain, long-term exploration. For instance, the American Prohibition, intended to boost static efficiency by curbing alcohol-related absenteeism, inadvertently stifled innovation by disrupting social networks where ideas were exchanged.

Optimal governance evolves. The ideal form of economic governance is not static; it must adapt to the prevailing stage of technological development. A system perfectly suited for exploiting established technologies will inevitably become ill-suited for generating breakthrough innovations. Societies must recognize this shift and adjust their institutional frameworks to avoid stagnation.

2. Institutional Inertia: The Enemy of Progress

When this happens, it must either adapt or perish.

Resistance to change. A fundamental barrier to sustained progress is institutional inertia, where powerful incumbents—be they guilds, large corporations, or political elites—resist disruptive innovation. These groups, deeply invested in the existing order, seek to protect their rents and privileges, often at the expense of societal advancement. This dynamic explains why industrialization was delayed in many regions where craft guilds held significant sway.

The "party of innovation." Societies that successfully navigate technological transitions are those whose governments align with the "party of innovation," actively overriding incumbent interests. Britain's Industrial Revolution, for example, thrived not just on secure property rights but on Parliament's willingness to bypass landowners and guilds to facilitate infrastructure projects and mechanization. This contrasts sharply with regions where local authorities sided with those resisting change.

Creative destruction is essential. Progress inherently involves creative destruction, where new technologies and industries displace old ones. When institutions fail to create space for this process, often by protecting established players, innovation stalls. This highlights a recurring theme: the conflict between those whose interests lie with existing economic activities and those who champion new ones.

3. China's Early Centralized Advantage and Later Stagnation

This early technological advantage was intimately related to China’s highly centralized economic and political system.

Centralization's early benefits. Ancient China's highly centralized state bureaucracy, dating back to the Western Zhou dynasty, initially served as a powerful catalyst for technological progress. Innovations like advanced soil mapping, unified writing systems, and large-scale irrigation projects (e.g., the Grand Canal) were state-driven, enhancing administrative governance and supporting a doubling of the population during the Song dynasty. This allowed China to become the most prosperous place on the planet by 1090 A.D.

The "Needham Puzzle." Despite its early lead, China eventually fell behind Europe, leading to the "Needham puzzle." The very institutions that fostered early state-led progress—such as the civil service examination system focused on Confucian ideology—ultimately stifled grassroots innovation and diversity of thought. This system channeled the most talented individuals into bureaucracy, diverting them from scientific or entrepreneurial pursuits.

Autocracy's inherent limits. The Ming and Qing dynasties further illustrate the perils of unchecked autocratic power. When external threats diminished, rulers prioritized consolidating personal authority over technological advancement, leading to policies like the "sea ban" that isolated China. This weakened the state from within, fostered a patronage economy, and suppressed the kind of intellectual and commercial dynamism seen in Europe, contributing to centuries of stagnation.

4. Europe's Fragmented Path to Innovation

The chief virtue of Europe’s competitive state system, in other words, was that no central government could crack down on people with a disruptive idea that might shake up the existing order.

Post-Roman decentralization. Unlike China, Europe's technological backwardness after the fall of the Roman Empire prevented the formation of a strong, centralized bureaucratic state. Instead, it fragmented into hundreds of competing territories, fostering a decentralized political tradition. This environment, combined with the Latin Church's policies undermining kinship networks, led to broader social networks and the emergence of voluntary groups, educational institutions, and autonomous chartered towns.

A "market for ideas." This political fragmentation created a "Republic of Letters," a transnational intellectual community where ideas flowed freely. Scholars and innovators could move between competing states, universities, and cities to escape repression or find patronage for heterodox ideas. The printing press and postal services further accelerated this exchange, fostering a vibrant intellectual bazaar that fueled the Scientific Enlightenment.

Competition as an innovation engine. The competitive state system meant no single authority could impose a uniform orthodoxy, allowing disruptive ideas to flourish. The persecution of Huguenots in France, for example, led to a brain drain that enriched Protestant nations like Prussia with skilled artisans and new technologies. This constant rivalry for talent and ideas was crucial for Europe's eventual technological leap, contrasting sharply with China's unified, often repressive, intellectual landscape.

5. The State's Visible Hand in Catch-Up Growth

If the British Industrial Revolution had been guided by Adam Smith’s invisible hand, Alfred Chandler’s visible hand played a much larger role among latecomers.

Emulating the leader. For latecomer economies, the path to industrialization often involved a more active role for the state, learning from the British example without necessarily adopting its laissez-faire policies. Countries like Prussia, France, and Japan actively studied British technological progress, sending officials on inspection tours and implementing reforms to spur catch-up.

Prussia's "revolution from above." Prussia, in particular, leveraged its strong, meritocratic bureaucracy to drive industrialization. The Stein-Hardenberg Reforms, enacted after defeat by Napoleon, abolished serfdom and guild restrictions, creating occupational freedom and secure property rights. This "revolution from above" allowed the state to override entrenched interests and expedite the adoption of foreign technologies, fostering large-scale enterprises and a world-leading education system.

East Asian state-led development. Japan's Meiji Restoration, sparked by the "Black Ships" incident, similarly involved a comprehensive, state-led effort to acquire Western technology and rebuild a centralized state. The government established institutions like the Institute of Barbarian Books for technical translation and fostered close ties with emerging zaibatsu conglomerates. Post-WWII, Japan and South Korea continued this model, with agencies like MITI and the EPB coordinating technology transfer and industrial development, often through labor repression and export targets.

6. America's Decentralized Engine of Frontier Innovation

The U.S. patent system was second to none, which is why Britain’s patent reform of 1852 was modelled on her economic rival across the Atlantic.

Weak government, strong markets. The United States developed with a weak, decentralized federal government, a legacy of its founding against British centralized power. This limited the state's capacity for direct industrial policy but fostered a vast internal common market with secure private property rights and few impediments to trade. This environment attracted adventurous, self-reliant immigrants, fueling a culture of individualism and entrepreneurship.

Democratized innovation. America's patent system, particularly after the 1836 Patent Act, was designed to be accessible and democratic, with low fees and public access to detailed invention descriptions. This enabled a broad swath of the population, including self-taught individuals from diverse backgrounds, to participate in innovation.

• Samuel Morse (painter) invented the electric telegraph.
• Cyrus McCormick (farmer/blacksmith) developed the mechanical reaper.
• Isaac Singer (actor) invented the popular sewing machine.
  This "democratization of invention" meant that most key innovations of the 19th century came from individuals, not large corporations.

Public-private partnerships. Despite a weak federal state, America overcame its infrastructure deficit through public-private partnerships. The Postal Service Act of 1792 facilitated information flow, and the Pacific Railroad Act of 1862 used land grants and loans to spur railway construction. These initiatives, while not direct state control, created the conditions for a vast, integrated market that rewarded mass production and further fueled innovation.

7. War, Planning, and the Routinization of Innovation

While World War II involved enormous R&D expenditures, the urgency of war heavily favoured the mass production of military technologies and placed a low premium on basic research, building on what had already been done inside universities.

Wartime mobilization. World War II dramatically expanded the bureaucratic capacity of the American state and its industrial enterprises. President Roosevelt's "Arsenal of Democracy" mobilized leading industrial centers, rapidly converting civilian production to armaments. This required unprecedented coordination, often led by private sector executives within government agencies like the War Production Board.

Innovation's mixed impact. While the war effort produced remarkable feats of mass production (e.g., Ford's Willow Run bomber factory) and mission-oriented technological advancements (e.g., radar, the atomic bomb), it also diverted scientific talent towards destructive ends. Patenting plummeted, and basic research was often deprioritized. The popular notion that WWII directly spawned peacetime prosperity overlooks the innovations that were delayed or never materialized due to wartime demands.

Postwar exploitation. The "Golden Age" of postwar growth in the US was largely driven by the exploitation and diffusion of pre-war technological breakthroughs, such as electricity and the internal combustion engine, which delivered their main productivity gains between 1920 and 1970. The war, however, left behind enduring institutional changes, including a larger public sector role in R&D (e.g., NSF) and a routinization of innovation within large corporate labs like Bell Labs, shifting away from the era of the independent inventor.

8. The Computer Revolution: A Triumph of Decentralization

The computer age took off in 1982, when the cover of Time’s Person of the Year issue was dedicated not to any human but to the PC, dubbed the “Machine of the Year.”

From colossus to personal. Early computers like ENIAC were massive, specialized machines, leading to predictions of a tiny market. However, the invention of the microprocessor in 1971 shattered these barriers, enabling the personal computer (PC) and democratizing computing. This, combined with the internet, transformed stand-alone devices into interconnected networks, unleashing a new era of productivity.

Silicon Valley's unique culture. The Computer Revolution was largely an American, and specifically a Silicon Valley, phenomenon. Unlike Boston's Route 128, which relied heavily on military contracts and traditional hierarchies, Silicon Valley fostered a culture of openness, autonomy, and job-hopping.

• Companies like HP and Intel pioneered flat organizational structures.
• Profit-sharing and equity compensation were common.
• Informal collaboration flourished in places like Xerox PARC and the Homebrew Computer Club.
  This decentralized environment, supported by California's ban on non-compete agreements, allowed engineers to leave established firms and launch innovative startups.

Antitrust as a catalyst. America's strong antitrust regime played a crucial role in fostering competition. The threat of antitrust action, particularly the decade-long case against IBM, forced the company to unbundle its hardware and software, creating opportunities for new players like Microsoft. Similarly, the breakup of AT&T in 1984 decentralized the telecommunications sector, preventing a single entity from monopolizing the nascent internet and allowing its commercialization to flourish.

9. The Dictator's Dilemma: Control vs. Innovation

The dictator’s dilemma should not, however, lead one to conclude that state power inevitably retards progress at the technological frontier.

The inherent tension. Autocratic regimes face a fundamental dilemma: how to drive technological progress while maintaining tight political control. Frontier innovation thrives on decentralized exploration, which requires freedom and autonomy—qualities often at odds with a dictator's need for centralized oversight and suppression of dissent.

Soviet stagnation. The Soviet Union's command economy, while effective for mass mobilization and exploiting existing technologies (e.g., heavy industry, military), ultimately stifled innovation. The system's rigid planning, output targets, and lack of market incentives discouraged risk-taking and experimentation. Stalin's purges and the KGB's surveillance further created a climate of fear, making scientists cautious and conformist.

• Innovation was expensive due to political connections.
• Unproductive initiatives persisted due to lack of accountability.
• Censorship hindered scientific exchange and information flow.
  Despite massive R&D spending, the USSR lagged significantly in the Computer Revolution, as its centralized system was ill-equipped for flexible production and rapid technological change.

China's balancing act. Post-Mao China, under Deng Xiaoping, navigated this dilemma through a unique model of "political capitalism." It combined authoritarian political centralization with regional economic decentralization, allowing local experiments and yardstick competition among provinces. This fostered market incentives and attracted foreign investment, driving rapid catch-up growth. However, as China approaches the technological frontier, the tension between Party control and the need for decentralized innovation is intensifying, posing new challenges.

10. The Perils of Modern Concentration: A New Gilded Age

The primary economic conflict, I think, is between people whose interests are with already-established economic activities, and those whose interests are with the emergence of new economic activities.

Incumbents' growing power. In recent decades, both the United States and other liberal democracies have seen a troubling trend: increasing market concentration and the growing political influence of large corporations. This mirrors America's "Gilded Age" of the late 19th century, where industrial titans used their wealth to sway public policy and stifle competition.

Lobbying and regulatory capture. Modern corporations, often larger than national governments in terms of revenue, wield immense political power through lobbying and campaign donations. This influence can lead to regulatory capture, where regulations are designed to benefit established players and create barriers to entry for new, innovative firms.

• Non-compete clauses restrict job-hopping, hindering knowledge flow.
• "Killer acquisitions" allow incumbents to buy and shut down promising rival technologies.
• Weak antitrust enforcement permits unchecked consolidation.
  This stifles creative destruction and reduces business dynamism, as evidenced by fewer startups challenging dominant incumbents.

Erosion of dynamism. The decline in business dynamism is particularly concerning because startups are crucial drivers of innovative breakthroughs. When powerful incumbents successfully lobby to entrench their positions, it blunts the incentives for radical innovation and diverts resources from productive pursuits to rent-seeking activities. This trend contributes to overall economic stagnation, as the economy becomes less adaptable to technological change.

11. AI's Promise and the Risk of "AI-tocracy"

With the help of artificial intelligence or multiple intelligence, our perception of the world will be elevated to a new level. As such, big data will make the market smarter and make it possible to plan and predict market forces so as to allow us to finally achieve a planned economy.

AI's transformative potential. Artificial intelligence, particularly large language models (LLMs), holds immense promise for transforming various sectors, from drug discovery to autonomous vehicles. Proponents like Jack Ma envision AI enabling a "planned economy" by making markets smarter and more predictable, echoing earlier socialist calculation debates.

The "AI-tocracy" risk. However, AI's development is susceptible to centralization, raising concerns about "AI-tocracy." Authoritarian regimes, especially China, are investing heavily in AI for surveillance and political control. This demand can spur innovation in related commercial applications, but it also risks reinforcing autocratic power and stifling broad-based, decentralized innovation.

• China's "Great Firewall" and social credit system demonstrate extensive digital control.
• Government procurement of facial recognition AI has accelerated its development.
• Restrictions on information access and international collaboration hinder scientific progress.
  This approach, while securing control, may lead to a "natural resource curse" of data, locking China into a less promising type of AI development.

Limits of current AI. Despite impressive advancements, current AI models, particularly LLMs, still face fundamental limitations. They excel at statistical associations and memorization but struggle with generalization, logical reasoning, and truly novel situations.

• Susceptible to "hallucinations" and fabricating content.
• Performance drops dramatically on new, unseen problems.
• Lack understanding of human-like concepts (e.g., reciprocal relationships).
  These challenges suggest that AI is not yet a substitute for decentralized human exploration and creativity, which remains crucial for pushing the technological frontier.

12. Progress is Fragile: The Need for Constant Adaptation

The primary economic conflict, I think, is between people whose interests are with already-established economic activities, and those whose interests are with the emergence of new economic activities. This is a conflict that can never be put to rest except by economic stagnation.

The enduring conflict. The history of progress is a continuous struggle between the forces of stability and change. Incumbents, whether guilds, large corporations, or political elites, naturally seek to preserve their established interests, often resisting the disruptive forces of new technologies. Sustained progress requires societies to constantly adapt their institutions to overcome this inherent conflict.

Balancing centralization and decentralization. The optimal institutional framework shifts with the stage of technological development. Centralized systems are effective for exploiting existing technologies and catch-up growth, while decentralized systems are vital for exploration and frontier innovation. The challenge lies in transitioning between these modes without succumbing to the inertia of entrenched interests.

• China's success in high-speed rail demonstrates effective state-led exploitation of foreign technology.
• America's historical dynamism stemmed from its decentralized, competitive environment for innovation.
• Europe's postwar growth relied on coordinated capitalism to absorb American tech, but struggled with the computer age.

Vigilance in liberal democracies. Even in liberal democracies, progress is fragile. The rise of corporate power and lobbying can lead to a "New Gilded Age," where competition is stifled and innovation slows. Protecting competition, fostering open systems, and ensuring that the benefits of progress are widely shared are crucial for maintaining dynamism. Public awareness and collective action are essential to counter powerful interests that seek to entrench the status quo.

The future of AI. The trajectory of AI development will depend on whether societies embrace open, decentralized exploration or succumb to centralized control. While AI offers immense potential, its true benefits will only be realized if institutions allow for continuous experimentation and creative destruction, rather than prioritizing control and the interests of established players. The lessons from history are clear: progress is not automatic; it requires constant vigilance and adaptation.

Last updated: December 17, 2025