Why is it that:

“The need for a pervasive, permanent role of government in directing investment also emerges from more recent studies of economic development. As documented in the research of Alice Amsden, Ha-Joon Chang, Dani Rodrik, and others, the countries that have grown fastest have ignored the advice of the World Bank, IMF, and other advocates of free trade and laissez-faire. Instead, successful development has been based on skillful, continual government involvement in nurturing promising industries, supporting education, research, and infrastructure, and managing international trade. The government’s leading role in development can certainly be done wrong, but it can’t be done without.”

As the current financial meltdown has demonstrated, the market is not an optimal system, especially when left on its own devices.

Frank Ackerman offers a good summary of arguments why this is so:

“Broadly speaking, there are four fundamental flaws in the theory that private greed reliably creates social good. The financial crisis highlights the fourth and least familiar item in the list, involving access to information. But it will be helpful to begin with a brief review of the other flaws.

First, the theoretical defense of market outcomes rests on Pareto optimality, an absurdly narrow definition of social goals. A proposal to raise taxes on the richest five percent and lower taxes on everyone else is not “optimal” by this standard, since it makes only 95 percent of the population, not everyone, better off. Important public policies typically help some people at the expense of others: pollution controls are good for those who value clean air and water, but bad for the profits of major polluters. The invisible hand won’t achieve such non-consensual results; public goods require public choices.

Second, market competition only leads to the right outcomes if everything that matters is a marketable commodity with a meaningful price. Marxists and others have objected to the treatment of labor as a mere commodity; environmentalists have likewise objected to the view of nature as something to buy and sell. This is not a new idea: in the words of the 18th century philosopher Immanuel Kant, some things have a price, or relative worth; other things have a dignity, or intrinsic worth. Respect for the dignity of labor and of nature leads into a realm of rights and absolute standards, not prices and markets. It doesn’t matter how much someone would be willing to pay for the opportunity to engage in slavery, child labor, or the extinction of species; those options are not for sale. Which issues call for absolute standards, and which can safely be left to the market?

This foundational question precedes and defines the legitimate scope of market competition; it cannot be answered from within the apparatus of economics as usual.

Third, the theory of competitive markets and the proof of their optimality rest on the assumption that no enterprise is large enough to wield noticeable power in the marketplace. Adam Smith’s butchers and bakers operated in a relentlessly competitive environment, as do the small producers and consumers of modern general equilibrium theory. In reality, businesses big enough to wield significant power over prices, wages, and production processes can be found throughout the economic landscape.

Big businesses thrive, in part, thanks to economies of scale in technology and work organization: bigger boilers and furnaces are physically more efficient than small ones; assembly lines can make labor more productive than individual craft work; computers are often more productive when they run the same software used by everyone else. Economies of scale are also important in establishing and advertising well-known brands: since no one ever has complete information about the market, as discussed below, there is a value to knowing exactly what to expect when you walk into a McDonald’s or a Starbucks.

Bigness can also be based on unethical, even illegal manipulation of markets to create monopoly or near-monopoly positions. Manipulation constantly reappears because the “rules of the game” create such a powerful incentive to break the rules. The story of the invisible hand, and its formalization in the theory of perfectly competitive markets, offers businesses only the life of the Red Queen in Alice in Wonderland, running faster and faster to stay in the same place. Firms must constantly compete with each other to create better and cheaper products; as soon as they succeed and start to make greater profits, their competitors catch up with them, driving profits back down to the low level that is just enough to keep them all in business. An ambitious, profit-maximizing individual could easily conclude that there is more money to be made by cheating. In the absence of religious or other extra-economic commitments to play by the rules, the strongest incentive created by market competition is the search for an escape from competition, legitimately or otherwise.

Opportunities to cheat are entwined with the fourth flaw in the theory of perfect competition: all participants in the market are assumed to have complete information about products and prices. Adam Smith’s consumers were well-informed through personal experience about what the baker and the butcher were selling; their successors in conventional economic theory are likewise assumed to know the full range of what is for sale on the market, and how much they would benefit from buying each item. In the realm of finance, mortgage crises and speculative bubbles would be impossible if every investor knew the exact worth of every available investment – as, stereotypically, small-town bankers were once thought to know the credit-worthiness of households and businesses in their communities.

The assumption of complete information fails on at least two levels, both relevant to the current crisis: a general issue of the sheer complexity of the market; and a more specific problem involving judgment of rare but costly risks. In general terms, a modern market economy is far too complex for any individual to understand and evaluate everything that is for sale. This limitation has inspired a number of alternative approaches to economics, ranging from Herbert Simon’s early theories of bounded rationality through the more recent work on limited and asymmetric information by Joseph Stiglitz and others. Since no one ever has complete information about what’s available on the market, there is no guarantee that unregulated private markets will reach the ideal outcome. Regulations that improve the flow of information can lead to an overall improvement, protecting the unwary and the uninformed.

When people buy things about which they are poorly informed, markets can work quite perversely. If people trust someone else’s judgment more than their own – as, for instance, many do when first buying a computer – then decisions by a small number of early adopters can create a cascade of followers, picking a winner based on very little information. Windows may not have been the best possible microcomputer operating system, but a small early lead in adoption snowballed into its dominant position today. Investment fads, market bubbles, and fashions of all sorts display the same follow-the-leader dynamics (but without the staying power of Windows).

When people have to make excessively complex decisions, there is no guarantee that they will choose wisely, or pick the option that is in their own best interest. Yet in areas such as health care and retirement savings, individuals are forced to make economic decisions that depend on detailed technical knowledge. The major decisions are infrequent and the cost of error is often high, so that learning by experience is not much help.

The same overwhelming complexity of available choices exists throughout financial markets. The menu of investment options is constantly shifting and expanding; financial innovation, i.e. creating and selling new varieties of securities, is an inexpensive process, requiring little more than a clever idea, a computer programmer, and a lawyer. Such innovation allows banks and other financial institutions to escape from old, regulated markets into new, ill-defined, and unregulated territory, potentially boosting their profits. Even at its best, the pursuit of financial novelty and the accompanying confusion undermines the traditional assumption that buyers always make well-informed choices. At its worst, the process of financial innovation provides ample opportunity to cheat, knowingly selling new types of securities for more than they are worth. Information about the reliability of many potential investments is ostensibly provided by bond rating agencies. One of the minor scandals of the current financial crisis is the fact that the rating agencies are private firms working for the companies they are rating. Naturally, you are more likely to be rehired if you present your clients in the best possible light; indeed, it might not hurt your future prospects to occasionally bend the truth a bit in their favor. The Enron scandal similarly involved accounting firms that wanted to continue working for Enron – and reported that nothing was wrong with the company’s books, at a time when the top executives were engaged in massive fraud.”

Related book: Frank Ackerman and Lisa Heinzerling, Priceless: On Knowing the Price of Everything and the Value of Nothing (The New Press, 2004)

Read the full essay here, to see how these flaws also suggest a particular set of remedies.