[This is the third installment in my serialization of the first three chapters of my book-in-progress, tentatively titled Desktop Regulatory State]

I. The Systematic Stupidity of Hierarchies

The intrusion of power into human relationships creates irrationality and systematic stupidity. As Robert Anton Wilson argued in “Thirteen Choruses for the Divine Marquis,”

A civilization based on authority-and-submission is a civilization without the means of self-correction. Effective communication flows only one way: from master-group to servile-group. Any cyberneticist knows that such a one-way communication channel lacks feedback and cannot behave “intelligently.”

The epitome of authority-and-submission is the Army, and the control-and-communication network of the Army has every defect a cyberneticist’s nightmare could conjure. Its typical patterns of behavior are immortalized in folklore as SNAFU (situation normal—all fucked-up), FUBAR (fucked-up beyond all redemption) and TARFU (things are really fucked-up). In less extreme, but equally nosologic, form these are the typical conditions of any authoritarian group, be it a corporation, a nation, a family, or a whole civilization.

That same theme featured prominently in The Illuminatus! Trilogy, which Wilson coauthored with Robert Shea. “….[I]n a rigid hierarchy, nobody questions orders that seem to come from above, and those at the very top are so isolated from the actual work situation that they never see what is going on below.”

A man with a gun is told only that which people assume will not provoke him to pull the trigger. Since all authority and government are based on force, the master class, with its burden of omniscience, faces the servile class, with its burden of nescience, precisely as a highwayman faces his victim. Communication is possible only between equals. The master class never abstracts enough information from the servile class to know what is actually going on in the world where the actual productivity of society occurs…. The result can only be progressive deterioration among the rulers.

This inability of those in authority to abstract sufficient information from below, and this perception of management by workers as “a highwayman,” result in the hoarding of information by those below and their use of it as a source of rents. The power differential, by creating a zero-sum relationship, renders the pyramid opaque to those at its top.

Radical organization theorist Kenneth Boulding, in similar vein, wrote of the value of “analysis of the way in which organizational structure affects the flow of information,”

hence affects the information input into the decision-maker, hence affects his image of the future and his decisions…. There is a great deal of evidence that almost all organizational structures tend to produce false images in the decision-maker, and that the larger and more authoritarian the organization, the better the chance that its top decision-makers will be operating in purely imaginary worlds.

Or in the pithy phrasing of Robert Theobald: “A person with great power gets no valid information at all.”

In his discussion of m?tis (i.e. distributed, situational and job-related knowledge), James C. Scott draws a connection between it and mutuality—“as opposed to imperative, hierarchical coordination”—and acknowledges his debt to anarchist thinkers like Kropotkin and Proudhon for the insight. M?tis flourishes only in an environment of two-way communication between equals, where the person in contact with the situation—the person actually doing the work—is in a position of equality.

Interestingly, R.A. Wilson had previously noted the same connection between mutuality—bilateral communication between equals—and accurate information—in “Thirteen Choruses.” And he included his own allusion to Proudhon, no less:

Proudhon was a great communication analyst, born 100 years too soon to be understood. His system of voluntary association (anarchy) is based on the simple communication principles that an authoritarian system means one-way communication, or stupidity, and a libertarian system means two-way communication, or rationality.

The essence of authority, as he saw, was Law — that is, fiat — that is, effective communication running one way only. The essence of a libertarian system, as he also saw, was Contract — that is, mutual agreement — that is, effective communication running both ways. (“Redundance of control” is the technical cybernetic phrase.)

To say that a hierarchical organization is systematically stupid is just to say that it is incapable of knowing what it knows, or making effective use of the knowledge of its members; it is less than the sum of its parts. Clay Shirky quotes John Seely Brown and Paul Duguid:
“What if HP knew what HP knows?” They had observed that the sum of the individual minds at HP had much more information than the company had access to, even though it was allowed to direct the efforts of those employees. Brown and Duguid documented ways in which employees do better at sharing information with one another directly than when they go through official channels.

There’s a great scene in the 1985 movie Brazil. Jackbooted thugs from the Ministry of Information’s Information Retrieval Department (i.e., the secret police) have just invaded an apartment by sawing a hole through the floor above and sliding down firemen’s poles—and then arrested the wrong man based on a computer error. In the aftermath, the Ministry of Works shows up to plug the hole:

JILL: There must be some mistake … Mr Buttle’s harmless…

BILL: We don’t make mistakes.

[So saying, he drops the manhole cover, which is faced with same material as the floor, over the hole in the floor. To his surprise it drops neatly through the floor into the flat below.]

CHARLIE: Bloody typical, they’ve gone back to metric without telling us.

That’s the way things work in real life in a hierarchical institution, because it is unable to aggregate the intelligence of its members and bring it to bear effectively on the policy-making process. So policies have a myriad of unintended consequences, and various policies operate at cross-purposes with each other in unanticipated ways. And to top it all off, the transaction costs of getting information to management about the real-world consequences of its policies are prohibitive for the same reason that the transaction costs of aggregating the information required for effective policy-making in the first place were prohibitive.

But no worries. Because the CEO and his chums in the C-Suite don’t live under the effects of their ass-brained policy, and subordinates are afraid to tell them what a clusterfuck they created, the CEO will happily inform the CEOs at other organizations of how wonderfully his new “best practice” worked out. And because these “competing” organizations actually exist in an oligopoly market of cost-plus markup and administered pricing, and share the same pathological institutional cultures, they suffer no real competitive penalty for their bureaucratic irrationality.

A hierarchy is a device for telling naked emperors how great their clothes look….

When you constantly operate on the assumption that you’re going to internalize the effects of your own actions, you have an incentive to anticipate things that could go wrong. And when you make a decision, you continually revise it in response to subsequent experience. Normal, sane human beings—that is, human beings who are in contact with their environments and not insulated from them by hierarchies—are always correcting our own courses of action.

Authority short-circuits this process: it shifts the negative consequences of decisions downward and the benefits upward, so that decision-makers operate based on a distorted cost-benefit calculus; and it blocks negative feedback so that the locus of organizational authority is subject to the functional equivalent of a psychotic break with reality.

When policy isn’t the result of systematic stupidity, it’s an elaborate exercise in shining it on. The primary purpose is to give management plausible deniability, the ability to say “But they knew about our written policy,” when the inevitable shortcuts to compensate for deliberate understaffing and irrational interference result in a public relations disaster. Auschwitz probably had a “written policy” against killing Jews.

The lack of feedback means that most organizations are “successful” at achieving goals that are largely artificial—goals that are defined primarily by the interests of their governing hierarchies, rather than being defined by the ostensible customers or those engaged in directly serving customer needs. On the other hand, organizational frameworks like networks, which are based on two-way feedback between equals, result in a high rate of “failure.” As Clay Shirky puts it, “[t]he bulk of open source projects fail.”

Open source is a profound threat, not because the open source ecosystem is outsucceeding commercial efforts but because it is outfailing them. Because the open source ecosystem, and by extension open social systems generally, rely on peer production, the work on those systems can be considerably more experimental, at considerably less cost, and any firm can afford. Why? The most important reasons are that open systems lower the cost of failure, [and] they do not create biases in favor of predictable but substandard outcomes….

Hierarchical institutions, on the other hand, are almost uniformly successful because everyone’s scared to tell the bosses how stupid their policies are and how shitty their products are. Failure is in fact a byproduct of the process by which success is achieved: most products in the corporate economy are only considered “good enough” because customers are powerless.
The problem, to repeat, is that no matter how intelligent the people staffing a large institution are as individuals, hierarchy makes their intelligence unusable. Given that the institution does not exist as a vehicle for the goals of its members, given that there is no intrinsic connection between their personal motivation and their roles in the organization, and given that the information and agency problems of a hierarchy prevent consequences from being fully internalized by actors, individuals simply cannot be trusted with the discretion to act on their own intelligence or common sense. That’s the whole idea behind standardized work-rules, job descriptions, and all the rest of the Weberian model of bureaucratic rationality: because someone, somewhere might use her initiative in ways that produce results that are detrimental to the interests of the organization, you need a set of rules in place that prevent anyone from doing anything at all.
The need to impose constraints on freedom of action, and to impede individual initiative in directly adopting the most common-sense and lowest-cost solutions to immediate problems, was explained by Paul Goodman:

…the government Peace Corps is many times as expensive as similar less official operations largely because an errant twenty-year-old well-digger might become an International Incident, so one cannot be too careful in selecting him. Convenience of supervision overrides performance. And the more “objective” the better. If the punch card [i.e. computer punch card—this was the mid-1960s] approves, no one is guilty. To bureaucrats, a fatal hallmark of decentralist enterprises is their variety in procedure and persons; how can one know, with a percentage validity, that these methods and persons are right?

The result is a world which is hard to distinguish from such parodies as “The Feds” in Neal Stephenson’s Snow Crash, or Brazil‘s Ministry of Central Services in which one cannot replace a blown fuse without a Form 27-B. The problem of replacing a door catch in the New York public school system, which suggests “Form 27-B” was hardly even a parody, is a good example:

…To remove a door catch that hampers the use of a lavatory requires a long appeal through headquarters, because it is “city property.”….

…An old-fashioned type of hardware is specified for all new buildings, that is kept in production only for the New York school system.

* * *

When the social means are tied up in such complicated organizations, it becomes extraordinarily difficult and sometimes impossible to do a simple thing directly, even though the doing is common sense and would meet with universal approval, as when neither the child, nor the parent, nor the janitor, nor the principal of the school can remove the offending door catch.

A corporate hierarchy interferes with the judgment of what Friedrich Hayek called “people-on-the-spot,” and with the collection of dispersed knowledge of circumstances, in exactly the same way a state does.

Most production jobs involve a fair amount of distributed, job-specific knowledge, and depend on the initiative of workers to improvise, to apply skills in new ways, in the face of events which are either totally unpredictable or cannot be fully anticipated. Rigid hierarchies and rigid work rules only work in a predictable environment. When the environment is unpredictable, the key to success lies with empowerment and autonomy for those in direct contact with the situation.

Hierarchical organizations are—to borrow a wonderful phrase from Martha Feldman and James March—systematically stupid. For all the same Hayekian reasons that make a planned economy unsustainable, no individual is “smart” enough to manage a large, hierarchical organization. Nobody—not Einstein, not John Galt—possesses the qualities to make a bureaucratic hierarchy function rationally. Nobody’s that smart, any more than anybody’s smart enough to run Gosplan efficiently—that’s the whole point. As Matt Yglesias put it,

I think it’s noteworthy that the business class, as a set, has a curious and somewhat incoherent view of capitalism and why it’s a good thing. Indeed, it’s in most respects a backwards view that strongly contrasts with the economic or political science take on why markets work.

The basic business outlook is very focused on the key role of the executive. Good, profitable, growing firms are run by brilliant executives. And the ability of the firm to grow and be profitable is evidence of its executives’ brilliance. This is part of the reason that CEO salaries need to keep escalating—recruiting the best is integral to success. The leaders of large firms become revered figures…. Their success stems from overall brilliance….

The thing about this is that if this were generally true—if the CEOs of the Fortune 500 were brilliant economic seers—then it would really make a lot of sense to implement socialism. Real socialism. Not progressive taxation to finance a mildly redistributive welfare state. But “let’s let Vikram Pandit and Jeff Immelt centrally plan the economy—after all, they’re really brilliant!”

But in the real world, the point of markets isn’t that executives are clever and bureaucrats are dimwitted. The point is that nobody is all that brilliant.

No matter how insightful and resourceful they are, no matter how prudent, as human beings in dealing with actual reality, nevertheless by their very nature hierarchies insulate those at the top from the reality of what’s going on below, and force them to operate in imaginary worlds where all their intelligence becomes useless. No matter how intelligent managers are as individuals, a bureaucratic hierarchy makes their intelligence less usable. Chris Dillow describes it this way:

But why don’t firms improve with practice in the way that individuals’ musical or sporting performance improves? Here are four possible differences:

1. Within firms, there’s no mechanism for translating individuals’ learning, or incremental knowledge, into corporate knowledge. As Hayek said, hierarchies are terrible at using fragmentary, tacit, dispersed knowledge.

2. Job turnover means that job-specific human capital gets lost.

3. Bosses are selected for overconfidence. But overconfidence militates against learning.

4. In companies, the feedback that’s necessary for improvement gets warped by adverse incentives or ego involvement. If I play a phrase or chord badly, my ears tell me to practice it more. But if a company gets some adverse feedback—falling sales, say—no-one has an incentive or desire to say “I screwed up: I‘d better improve.” And formal efforts to generate feedback, such as performance reviews, often backfire.

What I’m saying is what every methodological individualist knows: companies are not individuals writ large. The differences between them can mitigate against learning by doing.

And herein lies the cost of the banking crisis. Because productivity growth comes from entry and exit rather than firms’ learning on the job, anything that retards the entry process—such as a lack of finance—will retard aggregate productivity growth, and hence economic growth.

The only solution is to give discretion to those in direct contact with the situation. As security analyst Bruce Schneier writes in regard to security against attack:

Good security has people in charge. People are resilient. People can improvise. People can be creative. People can develop on-the-spot solutions…. People are the strongest point in a security process. When a security system succeeds in the face of a new or coordinated or devastating attack, it’s usually due to the efforts of people.

The problem with authority relations in a hierarchy is that, given the conflict of interest created by the presence of power, those in authority cannot afford to allow discretion to those in direct contact with the situation. Systematic stupidity results, of necessity, from a situation in which a bureaucratic hierarchy must develop some metric for assessing the skills or work quality of a labor force whose actual work they know nothing about, and whose material interests militate against remedying management’s ignorance. When management doesn’t know (in Paul Goodman’s words) “what a good job of work is,” they are forced to rely on arbitrary metrics.

Most of the constantly rising burden of paperwork exists to give an illusion of transparency and control to a bureaucracy that is out of touch with the actual production process. Every new layer of paperwork is added to address the perceived problem that stuff still isn’t getting done the way management wants, despite the proliferation of paperwork saying everything has being done exactly according to orders. In a hierarchy, managers are forced to see “in a glass darkly” a process which is necessarily opaque to them because they are not directly engaged in it. They are forced to carry out the impossible task of developing accurate metrics for evaluating the behavior of subordinates, based on the self-reporting of people with whom they have a fundamental conflict of interest. All of the paperwork burden that management imposes on workers reflects an attempt to render legible a set of social relationships that by its nature must be opaque and closed to them, because they are outside of it.

Each new form is intended to remedy the heretofore imperfect self-reporting of subordinates. The need for new paperwork is predicated on the assumption that compliance must be verified because those being monitored have a fundamental conflict of interest with those making the policy, and hence cannot be trusted; but at the same time, the paperwork itself relies on their self-reporting as the main source of information. Every time new evidence is presented that this or that task isn’t being performed to management’s satisfaction, or this or that policy isn’t being followed, despite the existing reams of paperwork, management’s response is to design yet another—and equally useless—form.

Weberian work rules result of necessity when performance and quality metrics are not tied to direct feedback from the work process itself. They’re a metric of work for someone who is neither a creator/provider not an end user. And they are necessary—again—because those at the top of the pyramid cannot afford to allow those at the bottom the discretion to use their own common sense. A bureaucracy cannot afford to allow its subordinates such discretion, because someone with the discretion to do things more efficiently will also have the discretion to do something bad. And because the subordinate has a fundamental conflict of interest with the superior, and does not internalize the benefits of applying her intelligence, she cannot be trusted to use her intelligence for the benefit of the organization. In such a zero-sum relationship, any discretion can be abused.