FAIR USE NOTICE

A BEAR MARKET ECONOMICS BLOG


This site may contain copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in an effort to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. we believe this constitutes a ‘fair use’ of any such copyrighted material as provided for in section 107 of the US Copyright Law.

In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit to those who have expressed a prior interest in receiving the included information for research and educational purposes. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml

If you wish to use copyrighted material from this site for purposes of your own that go beyond ‘fair use’, you must obtain permission from the copyright owner.

FAIR USE NOTICE FAIR USE NOTICE: This page may contain copyrighted material the use of which has not been specifically authorized by the copyright owner. This website distributes this material without profit to those who have expressed a prior interest in receiving the included information for scientific, research and educational purposes. We believe this constitutes a fair use of any such copyrighted material as provided for in 17 U.S.C § 107.

Read more at: http://www.etupdates.com/fair-use-notice/#.UpzWQRL3l5M | ET. Updates
FAIR USE NOTICE FAIR USE NOTICE: This page may contain copyrighted material the use of which has not been specifically authorized by the copyright owner. This website distributes this material without profit to those who have expressed a prior interest in receiving the included information for scientific, research and educational purposes. We believe this constitutes a fair use of any such copyrighted material as provided for in 17 U.S.C § 107.

Read more at: http://www.etupdates.com/fair-use-notice/#.UpzWQRL3l5M | ET. Updates

All Blogs licensed under Creative Commons Attribution 3.0

Thursday, August 13, 2015

The End of Rational Economics






Recession

The End of Rational Economics

From the July–August 2009 Issue
In 2008, a massive earthquake reduced the financial world to rubble. Standing in the smoke and ash, Alan Greenspan, the former chairman of the U.S. Federal Reserve once hailed as “the greatest banker who ever lived,” confessed to Congress that he was “shocked” that the markets did not operate according to his lifelong expectations. He had “made a mistake in presuming that the self-interest of organizations, specifically banks and others, was such that they were best capable of protecting their own shareholders.”

We are now paying a terrible price for our unblinking faith in the power of the invisible hand. We’re painfully blinking awake to the falsity of standard economic theory—that human beings are capable of always making rational decisions and that markets and institutions, in the aggregate, are healthily self-regulating. If assumptions about the way things are supposed to work have failed us in the hyperrational world of Wall Street, what damage have they done in other institutions and organizations that are also made up of fallible, less-than-logical people? And where do corporate managers, schooled in rational assumptions but who run messy, often unpredictable businesses, go from here?

We are finally beginning to understand that irrationality is the real invisible hand that drives human decision making. It’s been a painful lesson, but the silver lining may be that companies now see how important it is to safeguard against bad assumptions. Armed with the knowledge that human beings are motivated by cognitive biases of which they are largely unaware (a true invisible hand if there ever was one), businesses can start to better defend against foolishness and waste.

The emerging field of behavioral economics offers a radically different view of how people and organizations operate. In this article I will examine a small set of long-held business assumptions through a behavioral economics lens. In doing so I hope to show not only that companies can do a better job of making their products and services more effective, their customers happier, and their employees more productive but that they can also avoid catastrophic mistakes.

Behavioral Economics 101


Drawing on aspects of both psychology and economics, the operating assumption of behavioral economics is that cognitive biases often prevent people from making rational decisions, despite their best efforts. (If humans were comic book characters, we’d be more closely related to Homer Simpson than to Superman.) Behavioral economics eschews the broad tenets of standard economics, long taught as guiding principles in business schools, and examines the real decisions people make—how much to spend on a cup of coffee, whether or not to save for retirement, deciding whether to cheat and by how much, whether to make healthy choices in diet or sex, and so on. For example, in one study where people were offered a choice of a fancy Lindt truffle for 15 cents and a Hershey’s kiss for a penny, a large majority (73%) chose the truffle. But when we offered the same chocolates for one penny less each—the truffle for 14 cents and the kiss for nothing—only 31% of participants selected it. The word “free,” we discovered, is an immensely strong lure, one that can even turn us away from a better deal and toward the “free” one.

For the past few decades, behavioral economics has been largely considered a fringe discipline—a somewhat estranged little cousin of standard economics. Though practitioners of traditional economics reluctantly admitted that people may behave irrationally from time to time, they have tended to stick to their theoretical guns. They have argued that experiments conducted by behavioral economists and psychologists, albeit interesting, do not undercut rational models because they are carried out under controlled conditions and without the most important regulator of rational behavior: the large, competitive environment of the market. Then, in October 2008, Greenspan made his confession. Belief in the ultimate rationality of humans, organizations, and markets crumbled, and the attendant dangers to business and public policy were fully exposed.

Unlike the FDA, for example, which forces medical practitioners and pharmaceutical companies to test their assumptions before sending treatments into the marketplace, no entity requires business (and also the public sector) to get at the truth of things. Accordingly, it’s up to firms to begin investigating basic beliefs about customers, employees, operations, and policies. When organizations acknowledge and anticipate irrational behavior, they can learn to offset it and avoid damaging results. Let’s take a closer look at a few examples.

The Dark Side of Teamwork


A few years ago, my colleagues and I found that most individuals, operating on their own and given the opportunity, will cheat—but just a little bit, all the while indulging in rationalization that allows them to live with themselves. (See “How Honest People Cheat,” HBR, February 2008.) We also found that the simple act of asking people to think of their ethical foundations—say, the Ten Commandments—or their own moral code before they had the opportunity to cheat eliminated the dishonesty.

Most individuals, operating on their own and given the opportunity, will cheat—but just a little bit.
But what happens when people collaborate? Do autonomous teams make better, more ethical decisions? We decided to find out. In a series of three experiments, we gave participants 20 math problems to solve in five minutes and paid them 50 cents for each correct answer. In our first treatment (the control condition), individual participants were asked to write the number of problems they answered correctly on collection slips and give them to an experimenter, who checked the totals against the problem sheets. In a second treatment, participants shredded their answer sheets without verification and simply submitted their collection slips to the experimenter. Perhaps not surprisingly, we found these participants lied, saying that they’d correctly answered two more questions, on average, than those in the control treatment.

Things got more interesting in the third treatment, where participants worked in pairs and shared the spoils. The results showed that when a person realizes that his or her fudging would benefit other team members by increasing the payout, dishonesty further increased by 25%.

In another setup, we tried to discover whether monitoring and supervision would counteract team cheating. In fact, it did not. Though cheating decreased somewhat, it didn’t disappear. More disturbingly, as the members of our experimental group became better acquainted, the tendency to cheat for the sake of the team increased even more. Other experiments revealed that if one person is clearly seen to be cheating, team members—particularly those who feel connected to the cheater—are likely to depart from their own moral compasses and increase their cheating. It seems that cheating is infectious.

These findings have serious implications for unsupervised collaborative work in organizations. Although work groups can have many social and functional advantages, they may also be more vulnerable to unethical conduct.

The Revenge Motive: When Customers Are Unhappy


Now let’s look at customer behavior, an area that is particularly fraught with irrationality. It’s a rare company that consistently makes its customers happy, though many nobly try. And well they should; too many firms fail to understand the price of customer unhappiness. Indeed, given the right circumstances, most of us are more than happy to seek revenge.

Ayalet Gneezy of the University of San Diego and I set out to discover if even low levels of annoyance would cause people to seek retribution. If so, we could assume that vengeful behavior in the real world of dropped calls, flight cancellations, and credit card penalties would be even greater.
Daniel, an actor who was our hired “agent,” gave participants in a coffee shop several sheets of paper covered with letters and asked them to find matching pairs. Participants were promised $5 each for completing the task. On doing so, each signed a receipt and received a stack of $1 bills. Daniel “mistakenly” overpaid some of them by two, three, or four dollars.

In the “no annoyance” condition, Daniel explained the task and set the participants to it. In the “annoyance” condition, he pretended to answer his cell phone in the midst of giving instructions, talked for 15 seconds with a friend about pizza, hung up the phone, and then continued with the instructions without acknowledging or apologizing for taking the call. We wanted to discover whether the “annoyed” participants would exact revenge by keeping the extra money he gave them.

A mere 14% of those subjected to Daniel’s rude treatment returned the additional money, compared with 45% of those in the other group. The fact that only 45% returned the extra cash was depressing enough, but it was striking that a 15-second phone call vastly decreased the likelihood that the participants would return the cash.

In another version of the experiment, we wanted to find out more about the impulse to punish. Would it make a difference whether Daniel claimed that he was working for someone else? Would participants punish the principal (the researchers behind the study) for the agent’s misbehavior? Our results suggested that if people feel the need to take revenge, they don’t differentiate between the two.

This is bad news for employers. If someone who works for you upsets a customer—even in ways unrelated to the job—you will very likely pay the price. Even the smallest transgression on the part of an employee can ignite the instinct for strong revenge against the employer, regardless of who is at fault.

What can company representatives or individuals do to soothe the instinct for revenge in business or personal exchanges? Apologies work, at least temporarily. In yet another version of our experiment, Daniel apologized for the phone call interruption. We were surprised to find that the show of regret was a perfect remedy. The percentage of people who returned the extra cash was the same in the “apology condition” as in the “no annoyance” condition. As it turns out the word “sorry” completely counteracted the annoyance. (Of course, the effectiveness of an apology is likely to diminish with the frequency of the annoyance.)

Revenge and cheating are only two of the irrational behaviors that underlie employees’ and customers’ decisions.

Revenge and cheating are only two of the irrational behaviors that companies will find underlying their employees’ and customers’ decisions and actions. Recognizing that, what is the way forward?

Experimenting with Behavior


Behavioral economics experiments that get to the bottom of people’s decisions and actions are very different from the kinds of tests companies traditionally use to try out new product ideas and marketing concepts or to discover opportunities. The difference is not in the research methodology itself but in the process of selecting ideas to be tested.




The standard business approach to experiments is similar to an engineering project. It makes strong assumptions about the laws that govern the behavior of the different actors; the only question is how to combine them in a way that makes sense for a particular application. (Companies that gather large amounts of transaction data are well ahead in this area. The casino giant Harrahs, for example, is famous for running experiments using customer data to develop a set of tailored services and offers.) A behavioral economics approach, in contrast, is more like a science project: We search simultaneously for the governing principles and how to implement them. Consider, for example:

Pricing.


I don’t know whether Apple’s executives were conducting a behavioral economics experiment when they introduced the iPhone at a price of $600 and then quickly discounted it to $400, but that move revealed something important about human behavior. By imprinting the price of $600 in people’s minds, Apple was able to make consumers think that $400 was a real bargain. In a standard approach to price setting, the people running Apple’s pricing group might have asked focus groups about various price points for the phone, and based on participants’ feedback, picked the price they thought would maximize profits ($400). But if Apple had set the initial price at $400, consumers would have had no basis for comparison, since they’d never seen such a product before.

Adopting a behavioral economics perspective, Apple might have started by questioning the assumption that people would know how to value the pathbreaking product and so set up various pricing experiments. In this type of test, the goal is not simply to find out the optimal price but also to discover how people arrive at a decision to buy at that price. Companies also need to consider how the introductory price could influence the perception of value for a long time.

Product launches.


Behavioral economists might also look at the roles of habit and trust in consumer choice. Take a manufacturer who is planning to sell a triple-concentrated detergent, on the theory that environmentally conscious consumers would prefer to eliminate waste. Given shoppers’ almost automatic impulse to reach for the same detergent, should the manufacturer package the concentrate in the standard-size bottle and charge more for it? Or should the manufacturer try to break consumers’ force of habit and package the concentrate in a bottle one-third the size of the original? And what about trust? If consumers don’t trust the manufacturer to deliver a more concentrated product, given that the product smells and looks the same as before, will they be willing to pay for it? How can the manufacturer overcome this hurdle?

Customers.


A variety of companies now use a behavioral economics approach to more closely examine customer and employee behavior. For example, one automobile insurer discovered that most people, when filling out forms that ask how many miles they’ve driven in a year, claimed that they drove less than they actually had. Building on the discovery that people are less inclined to cheat after being reminded of their own ethical standards, the company moved the signature line to the top of the form. Applicants who signed the form at the top reported driving an average of 2,700 more miles a year than those who signed at the end.

In another example, the cable giant Comcast began addressing the customer-revenge problem by using Twitter to respond to problems proactively. The director of digital care, Frank Eliason, discovered that by searching for the word “Comcast” (or, sometimes, “Comcrap”), he could locate unhappy customers who were simply venting to themselves and to their friends, and respond to their problems before they became formal complaints. (Other companies, including JetBlue, General Motors, Kodak, Dell, and Whole Foods Market are now also tracking customers’ comments on Twitter.)

Building a Behavioral Economics Capability


Behavioral economics can be seen as depressing; after all, many of our experiments show human beings as incapable of making good decisions. Multiple findings demonstrate that we are emotional, myopic, and easily confused and distracted. Nevertheless, companies that make an investment in behavioral experimentation can radically improve decision making and lessen risk.

Firms interested in experimenting with behavior should understand that the process is time-consuming and delicate. All too often, companies set out to learn something about their customers’ habits only to find that the way they devised their research was invalid and the conclusions incorrect. Smart organizations will develop a behavioral economics capability by hiring qualified experimenters and conducting small trials that build on each other.

Once the understanding of irrationality is embedded in the fabric of the organization, a behavioral economics approach can be applied to virtually every area of the business, from governance and employee relations to marketing and customer service. It is probably most useful in the areas that we know the least about—such as the relationships between compensation and performance, risk and reward, loyalty and consumer habits, and pricing and purchasing behavior. As companies become more willing to question their assumptions, discover something about their stakeholders’ predilections, and share the results of their learning, they will no doubt become a good deal wiser.


A version of this article appeared in the July–August 2009 issue of Harvard Business Review.








In his best-selling book, Predictably Irrational: The Hidden Forces That Shape Our Decisions, Dan Ariely presents often humorous and peculiar research findings that provide new insights into human behavior that can help us make better decisions as individuals, as corporations, and as a society.

How Did Economists Get It So Wrong?




How Did Economists Get It So Wrong?




Published: September 2, 2009



I. MISTAKING BEAUTY FOR TRUTH
It’s hard to believe now, but not long ago economists were congratulating themselves over the success of their field. Those successes — or so they believed — were both theoretical and practical, leading to a golden era for the profession. On the theoretical side, they thought that they had resolved their internal disputes. Thus, in a 2008 paper titled “The State of Macro” (that is, macroeconomics, the study of big-picture issues like recessions), Olivier Blanchard of M.I.T., now the chief economist at theInternational Monetary Fund, declared that “the state of macro is good.” The battles of yesteryear, he said, were over, and there had been a “broad convergence of vision.” And in the real world, economists believed they had things under control: the “central problem of depression-prevention has been solved,” declared Robert Lucas of the University of Chicago in his 2003 presidential address to the American Economic Association. In 2004, Ben Bernanke, a former Princeton professor who is now the chairman of the Federal Reserve Board, celebrated the Great Moderation in economic performance over the previous two decades, which he attributed in part to improved economic policy making.

Last year, everything came apart.

Few economists saw our current crisis coming, but this predictive failure was the least of the field’s problems. More important was the profession’s blindness to the very possibility of catastrophic failures in a market economy. During the golden years, financial economists came to believe that markets were inherently stable — indeed, that stocks and other assets were always priced just right. There was nothing in the prevailing models suggesting the possibility of the kind of collapse that happened last year.

Meanwhile, macroeconomists were divided in their views. But the main division was between those who insisted that free-market economies never go astray and those who believed that economies may stray now and then but that any major deviations from the path of prosperity could and would be corrected by the all-powerful Fed. Neither side was prepared to cope with an economy that went off the rails despite the Fed’s best efforts.

And in the wake of the crisis, the fault lines in the economics profession have yawned wider than ever. Lucas says the Obama administration’s stimulus plans are “schlock economics,” and his Chicago colleague John Cochrane says they’re based on discredited “fairy tales.” In response, Brad DeLong of theUniversity of California, Berkeley, writes of the “intellectual collapse” of the Chicago School, and I myself have written that comments from Chicago economists are the product of a Dark Age of macroeconomics in which hard-won knowledge has been forgotten.

What happened to the economics profession? And where does it go from here?

As I see it, the economics profession went astray because economists, as a group, mistook beauty, clad in impressive-looking mathematics, for truth. Until the Great Depression, most economists clung to a vision of capitalism as a perfect or nearly perfect system. That vision wasn’t sustainable in the face of mass unemployment, but as memories of the Depression faded, economists fell back in love with the old, idealized vision of an economy in which rational individuals interact in perfect markets, this time gussied up with fancy equations. The renewed romance with the idealized market was, to be sure, partly a response to shifting political winds, partly a response to financial incentives. But while sabbaticals at the Hoover Institution and job opportunities on Wall Street are nothing to sneeze at, the central cause of the profession’s failure was the desire for an all-encompassing, intellectually elegant approach that also gave economists a chance to show off their mathematical prowess.

Unfortunately, this romanticized and sanitized vision of the economy led most economists to ignore all the things that can go wrong. They turned a blind eye to the limitations of human rationality that often lead to bubbles and busts; to the problems of institutions that run amok; to the imperfections of markets — especially financial markets — that can cause the economy’s operating system to undergo sudden, unpredictable crashes; and to the dangers created when regulators don’t believe in regulation.
It’s much harder to say where the economics profession goes from here. But what’s almost certain is that economists will have to learn to live with messiness. That is, they will have to acknowledge the importance of irrational and often unpredictable behavior, face up to the often idiosyncratic imperfections of markets and accept that an elegant economic “theory of everything” is a long way off. In practical terms, this will translate into more cautious policy advice — and a reduced willingness to dismantle economic safeguards in the faith that markets will solve all problems.

II. FROM SMITH TO KEYNES AND BACK

The birth of economics as a discipline is usually credited to Adam Smith, who published “The Wealth of Nations” in 1776. Over the next 160 years an extensive body of economic theory was developed, whose central message was: Trust the market. Yes, economists admitted that there were cases in which markets might fail, of which the most important was the case of “externalities” — costs that people impose on others without paying the price, like traffic congestion or pollution. But the basic presumption of “neoclassical” economics (named after the late-19th-century theorists who elaborated on the concepts of their “classical” predecessors) was that we should have faith in the market system.

This faith was, however, shattered by the Great Depression. Actually, even in the face of total collapse some economists insisted that whatever happens in a market economy must be right: “Depressions are not simply evils,” declared Joseph Schumpeter in 1934 — 1934! They are, he added, “forms of something which has to be done.” But many, and eventually most, economists turned to the insights of John Maynard Keynes for both an explanation of what had happened and a solution to future depressions.

Keynes did not, despite what you may have heard, want the government to run the economy. He described his analysis in his 1936 masterwork, “The General Theory of Employment, Interest and Money,” as “moderately conservative in its implications.” He wanted to fix capitalism, not replace it. But he did challenge the notion that free-market economies can function without a minder, expressing particular contempt for financial markets, which he viewed as being dominated by short-term speculation with little regard for fundamentals. And he called for active government intervention — printing more money and, if necessary, spending heavily on public works — to fight unemployment during slumps.

It’s important to understand that Keynes did much more than make bold assertions. “The General Theory” is a work of profound, deep analysis — analysis that persuaded the best young economists of the day. Yet the story of economics over the past half century is, to a large degree, the story of a retreat from Keynesianism and a return to neoclassicism. The neoclassical revival was initially led by Milton Friedman of the University of Chicago, who asserted as early as 1953 that neoclassical economics works well enough as a description of the way the economy actually functions to be “both extremely fruitful and deserving of much confidence.” But what about depressions?

Friedman’s counterattack against Keynes began with the doctrine known as monetarism. Monetarists didn’t disagree in principle with the idea that a market economy needs deliberate stabilization. “We are all Keynesians now,” Friedman once said, although he later claimed he was quoted out of context. Monetarists asserted, however, that a very limited, circumscribed form of government intervention — namely, instructing central banks to keep the nation’s money supply, the sum of cash in circulation and bank deposits, growing on a steady path — is all that’s required to prevent depressions. Famously, Friedman and his collaborator, Anna Schwartz, argued that if the Federal Reserve had done its job properly, the Great Depression would not have happened. Later, Friedman made a compelling case against any deliberate effort by government to push unemployment below its “natural” level (currently thought to be about 4.8 percent in the United States): excessively expansionary policies, he predicted, would lead to a combination of inflation and high unemployment — a prediction that was borne out by the stagflation of the 1970s, which greatly advanced the credibility of the anti-Keynesian movement.

Eventually, however, the anti-Keynesian counterrevolution went far beyond Friedman’s position, which came to seem relatively moderate compared with what his successors were saying. Among financial economists, Keynes’s disparaging vision of financial markets as a “casino” was replaced by “efficient market” theory, which asserted that financial markets always get asset prices right given the available information. Meanwhile, many macroeconomists completely rejected Keynes’s framework for understanding economic slumps. Some returned to the view of Schumpeter and other apologists for the Great Depression, viewing recessions as a good thing, part of the economy’s adjustment to change. And even those not willing to go that far argued that any attempt to fight an economic slump would do more harm than good.

Not all macroeconomists were willing to go down this road: many became self-described New Keynesians, who continued to believe in an active role for the government. Yet even they mostly accepted the notion that investors and consumers are rational and that markets generally get it right.
Of course, there were exceptions to these trends: a few economists challenged the assumption of rational behavior, questioned the belief that financial markets can be trusted and pointed to the long history of financial crises that had devastating economic consequences. But they were swimming against the tide, unable to make much headway against a pervasive and, in retrospect, foolish complacency.

III. PANGLOSSIAN FINANCE

In the 1930s, financial markets, for obvious reasons, didn’t get much respect. Keynes compared them to “those newspaper competitions in which the competitors have to pick out the six prettiest faces from a hundred photographs, the prize being awarded to the competitor whose choice most nearly corresponds to the average preferences of the competitors as a whole; so that each competitor has to pick, not those faces which he himself finds prettiest, but those that he thinks likeliest to catch the fancy of the other competitors.”

And Keynes considered it a very bad idea to let such markets, in which speculators spent their time chasing one another’s tails, dictate important business decisions: “When the capital development of a country becomes a by-product of the activities of a casino, the job is likely to be ill-done.”

By 1970 or so, however, the study of financial markets seemed to have been taken over by Voltaire’s Dr. Pangloss, who insisted that we live in the best of all possible worlds. Discussion of investor irrationality, of bubbles, of destructive speculation had virtually disappeared from academic discourse. The field was dominated by the “efficient-market hypothesis,” promulgated by Eugene Fama of the University of Chicago, which claims that financial markets price assets precisely at their intrinsic worth given all publicly available information. (The price of a company’s stock, for example, always accurately reflects the company’s value given the information available on the company’s earnings, its business prospects and so on.) And by the 1980s, finance economists, notably Michael Jensen of the Harvard Business School, were arguing that because financial markets always get prices right, the best thing corporate chieftains can do, not just for themselves but for the sake of the economy, is to maximize their stock prices. In other words, finance economists believed that we should put the capital development of the nation in the hands of what Keynes had called a “casino.”

It’s hard to argue that this transformation in the profession was driven by events. True, the memory of 1929 was gradually receding, but there continued to be bull markets, with widespread tales of speculative excess, followed by bear markets. In 1973-4, for example, stocks lost 48 percent of their value. And the 1987 stock crash, in which the Dow plunged nearly 23 percent in a day for no clear reason, should have raised at least a few doubts about market rationality.

These events, however, which Keynes would have considered evidence of the unreliability of markets, did little to blunt the force of a beautiful idea. The theoretical model that finance economists developed by assuming that every investor rationally balances risk against reward — the so-called Capital Asset Pricing Model, or CAPM (pronounced cap-em) — is wonderfully elegant. And if you accept its premises it’s also extremely useful. CAPM not only tells you how to choose your portfolio — even more important from the financial industry’s point of view, it tells you how to put a price on financial derivatives, claims on claims. The elegance and apparent usefulness of the new theory led to a string of Nobel prizes for its creators, and many of the theory’s adepts also received more mundane rewards: Armed with their new models and formidable math skills — the more arcane uses of CAPM require physicist-level computations — mild-mannered business-school professors could and did become Wall Street rocket scientists, earning Wall Street paychecks.

To be fair, finance theorists didn’t accept the efficient-market hypothesis merely because it was elegant, convenient and lucrative. They also produced a great deal of statistical evidence, which at first seemed strongly supportive. But this evidence was of an oddly limited form. Finance economists rarely asked the seemingly obvious (though not easily answered) question of whether asset prices made sense given real-world fundamentals like earnings. Instead, they asked only whether asset prices made sense given other asset prices. Larry Summers, now the top economic adviser in the Obama administration, once mocked finance professors with a parable about “ketchup economists” who “have shown that two-quart bottles of ketchup invariably sell for exactly twice as much as one-quart bottles of ketchup,” and conclude from this that the ketchup market is perfectly efficient.

But neither this mockery nor more polite critiques from economists like Robert Shiller of Yale had much effect. Finance theorists continued to believe that their models were essentially right, and so did many people making real-world decisions. Not least among these was Alan Greenspan, who was then the Fed chairman and a long-time supporter of financial deregulation whose rejection of calls to rein in subprime lending or address the ever-inflating housing bubble rested in large part on the belief that modern financial economics had everything under control. There was a telling moment in 2005, at a conference held to honor Greenspan’s tenure at the Fed. One brave attendee, Raghuram Rajan (of the University of Chicago, surprisingly), presented a paper warning that the financial system was taking on potentially dangerous levels of risk. He was mocked by almost all present — including, by the way, Larry Summers, who dismissed his warnings as “misguided.”

By October of last year, however, Greenspan was admitting that he was in a state of “shocked disbelief,” because “the whole intellectual edifice” had “collapsed.” Since this collapse of the intellectual edifice was also a collapse of real-world markets, the result was a severe recession — the worst, by many measures, since the Great Depression. What should policy makers do? Unfortunately, macroeconomics, which should have been providing clear guidance about how to address the slumping economy, was in its own state of disarray.

IV. THE TROUBLE WITH MACRO

“We have involved ourselves in a colossal muddle, having blundered in the control of a delicate machine, the working of which we do not understand. The result is that our possibilities of wealth may run to waste for a time — perhaps for a long time.” So wrote John Maynard Keynes in an essay titled “The Great Slump of 1930,” in which he tried to explain the catastrophe then overtaking the world. And the world’s possibilities of wealth did indeed run to waste for a long time; it took World War II to bring the Great Depression to a definitive end.

Why was Keynes’s diagnosis of the Great Depression as a “colossal muddle” so compelling at first? And why did economics, circa 1975, divide into opposing camps over the value of Keynes’s views?
I like to explain the essence of Keynesian economics with a true story that also serves as a parable, a small-scale version of the messes that can afflict entire economies. Consider the travails of the Capitol Hill Baby-Sitting Co-op.

This co-op, whose problems were recounted in a 1977 article in The Journal of Money, Credit and Banking, was an association of about 150 young couples who agreed to help one another by baby-sitting for one another’s children when parents wanted a night out. To ensure that every couple did its fair share of baby-sitting, the co-op introduced a form of scrip: coupons made out of heavy pieces of paper, each entitling the bearer to one half-hour of sitting time. Initially, members received 20 coupons on joining and were required to return the same amount on departing the group.

Unfortunately, it turned out that the co-op’s members, on average, wanted to hold a reserve of more than 20 coupons, perhaps, in case they should want to go out several times in a row. As a result, relatively few people wanted to spend their scrip and go out, while many wanted to baby-sit so they could add to their hoard. But since baby-sitting opportunities arise only when someone goes out for the night, this meant that baby-sitting jobs were hard to find, which made members of the co-op even more reluctant to go out, making baby-sitting jobs even scarcer. . . .

In short, the co-op fell into a recession.

O.K., what do you think of this story? Don’t dismiss it as silly and trivial: economists have used small-scale examples to shed light on big questions ever since Adam Smith saw the roots of economic progress in a pin factory, and they’re right to do so. The question is whether this particular example, in which a recession is a problem of inadequate demand — there isn’t enough demand for baby-sitting to provide jobs for everyone who wants one — gets at the essence of what happens in a recession.

Forty years ago most economists would have agreed with this interpretation. But since then macroeconomics has divided into two great factions: “saltwater” economists (mainly in coastal U.S. universities), who have a more or less Keynesian vision of what recessions are all about; and “freshwater” economists (mainly at inland schools), who consider that vision nonsense.

Freshwater economists are, essentially, neoclassical purists. They believe that all worthwhile economic analysis starts from the premise that people are rational and markets work, a premise violated by the story of the baby-sitting co-op. As they see it, a general lack of sufficient demand isn’t possible, because prices always move to match supply with demand. If people want more baby-sitting coupons, the value of those coupons will rise, so that they’re worth, say, 40 minutes of baby-sitting rather than half an hour — or, equivalently, the cost of an hours’ baby-sitting would fall from 2 coupons to 1.5. And that would solve the problem: the purchasing power of the coupons in circulation would have risen, so that people would feel no need to hoard more, and there would be no recession.

But don’t recessions look like periods in which there just isn’t enough demand to employ everyone willing to work? Appearances can be deceiving, say the freshwater theorists. Sound economics, in their view, says that overall failures of demand can’t happen — and that means that they don’t. Keynesian economics has been “proved false,” Cochrane, of the University of Chicago, says.

Yet recessions do happen. Why? In the 1970s the leading freshwater macroeconomist, the Nobel laureate Robert Lucas, argued that recessions were caused by temporary confusion: workers and companies had trouble distinguishing overall changes in the level of prices because of inflation ordeflation from changes in their own particular business situation. And Lucas warned that any attempt to fight the business cycle would be counterproductive: activist policies, he argued, would just add to the confusion.

By the 1980s, however, even this severely limited acceptance of the idea that recessions are bad things had been rejected by many freshwater economists. Instead, the new leaders of the movement, especially Edward Prescott, who was then at the University of Minnesota (you can see where the freshwater moniker comes from), argued that price fluctuations and changes in demand actually had nothing to do with the business cycle. Rather, the business cycle reflects fluctuations in the rate of technological progress, which are amplified by the rational response of workers, who voluntarily work more when the environment is favorable and less when it’s unfavorable. Unemployment is a deliberate decision by workers to take time off.

Put baldly like that, this theory sounds foolish — was the Great Depression really the Great Vacation? And to be honest, I think it really is silly. But the basic premise of Prescott’s “real business cycle” theory was embedded in ingeniously constructed mathematical models, which were mapped onto real data using sophisticated statistical techniques, and the theory came to dominate the teaching of macroeconomics in many university departments. In 2004, reflecting the theory’s influence, Prescott shared a Nobel with Finn Kydland of Carnegie Mellon University.

Meanwhile, saltwater economists balked. Where the freshwater economists were purists, saltwater economists were pragmatists. While economists like N. Gregory Mankiw at Harvard, Olivier Blanchard at M.I.T. and David Romer at the University of California, Berkeley, acknowledged that it was hard to reconcile a Keynesian demand-side view of recessions with neoclassical theory, they found the evidence that recessions are, in fact, demand-driven too compelling to reject. So they were willing to deviate from the assumption of perfect markets or perfect rationality, or both, adding enough imperfections to accommodate a more or less Keynesian view of recessions. And in the saltwater view, active policy to fight recessions remained desirable.

But the self-described New Keynesian economists weren’t immune to the charms of rational individuals and perfect markets. They tried to keep their deviations from neoclassical orthodoxy as limited as possible. This meant that there was no room in the prevailing models for such things as bubbles and banking-system collapse. The fact that such things continued to happen in the real world — there was a terrible financial and macroeconomic crisis in much of Asia in 1997-8 and a depression-level slump in Argentina in 2002 — wasn’t reflected in the mainstream of New Keynesian thinking.

Even so, you might have thought that the differing worldviews of freshwater and saltwater economists would have put them constantly at loggerheads over economic policy. Somewhat surprisingly, however, between around 1985 and 2007 the disputes between freshwater and saltwater economists were mainly about theory, not action. The reason, I believe, is that New Keynesians, unlike the original Keynesians, didn’t think fiscal policy — changes in government spending or taxes — was needed to fight recessions. They believed that monetary policy, administered by the technocrats at the Fed, could provide whatever remedies the economy needed. At a 90th birthday celebration for Milton Friedman, Ben Bernanke, formerly a more or less New Keynesian professor at Princeton, and by then a member of the Fed’s governing board, declared of the Great Depression: “You’re right. We did it. We’re very sorry. But thanks to you, it won’t happen again.” The clear message was that all you need to avoid depressions is a smarter Fed.

And as long as macroeconomic policy was left in the hands of the maestro Greenspan, without Keynesian-type stimulus programs, freshwater economists found little to complain about. (They didn’t believe that monetary policy did any good, but they didn’t believe it did any harm, either.)

It would take a crisis to reveal both how little common ground there was and how Panglossian even New Keynesian economics had become.

V. NOBODY COULD HAVE PREDICTED . . .

In recent, rueful economics discussions, an all-purpose punch line has become “nobody could have predicted. . . .” It’s what you say with regard to disasters that could have been predicted, should have been predicted and actually were predicted by a few economists who were scoffed at for their pains.
Take, for example, the precipitous rise and fall of housing prices. Some economists, notably Robert Shiller, did identify the bubble and warn of painful consequences if it were to burst. Yet key policy makers failed to see the obvious. In 2004, Alan Greenspan dismissed talk of a housing bubble: “a national severe price distortion,” he declared, was “most unlikely.” Home-price increases, Ben Bernanke said in 2005, “largely reflect strong economic fundamentals.”

How did they miss the bubble? To be fair, interest rates were unusually low, possibly explaining part of the price rise. It may be that Greenspan and Bernanke also wanted to celebrate the Fed’s success in pulling the economy out of the 2001 recession; conceding that much of that success rested on the creation of a monstrous bubble would have placed a damper on the festivities.

But there was something else going on: a general belief that bubbles just don’t happen. What’s striking, when you reread Greenspan’s assurances, is that they weren’t based on evidence — they were based on the a priori assertion that there simply can’t be a bubble in housing. And the finance theorists were even more adamant on this point. In a 2007 interview, Eugene Fama, the father of the efficient-market hypothesis, declared that “the word ‘bubble’ drives me nuts,” and went on to explain why we can trust the housing market: “Housing markets are less liquid, but people are very careful when they buy houses. It’s typically the biggest investment they’re going to make, so they look around very carefully and they compare prices. The bidding process is very detailed.”

Indeed, home buyers generally do carefully compare prices — that is, they compare the price of their potential purchase with the prices of other houses. But this says nothing about whether the overall price of houses is justified. It’s ketchup economics, again: because a two-quart bottle of ketchup costs twice as much as a one-quart bottle, finance theorists declare that the price of ketchup must be right.

In short, the belief in efficient financial markets blinded many if not most economists to the emergence of the biggest financial bubble in history. And efficient-market theory also played a significant role in inflating that bubble in the first place.

Now that the undiagnosed bubble has burst, the true riskiness of supposedly safe assets has been revealed and the financial system has demonstrated its fragility. U.S. households have seen $13 trillion in wealth evaporate. More than six million jobs have been lost, and the unemployment rate appears headed for its highest level since 1940. So what guidance does modern economics have to offer in our current predicament? And should we trust it?

VI. THE STIMULUS SQUABBLE

Between 1985 and 2007 a false peace settled over the field of macroeconomics. There hadn’t been any real convergence of views between the saltwater and freshwater factions. But these were the years of the Great Moderation — an extended period during which inflation was subdued and recessions were relatively mild. Saltwater economists believed that the Federal Reserve had everything under control. Fresh­water economists didn’t think the Fed’s actions were actually beneficial, but they were willing to let matters lie.
But the crisis ended the phony peace. Suddenly the narrow, technocratic policies both sides were willing to accept were no longer sufficient — and the need for a broader policy response brought the old conflicts out into the open, fiercer than ever.

Why weren’t those narrow, technocratic policies sufficient? The answer, in a word, is zero.

During a normal recession, the Fed responds by buying Treasury bills — short-term government debt — from banks. This drives interest rates on government debt down; investors seeking a higher rate of return move into other assets, driving other interest rates down as well; and normally these lower interest rates eventually lead to an economic bounceback. The Fed dealt with the recession that began in 1990 by driving short-term interest rates from 9 percent down to 3 percent. It dealt with the recession that began in 2001 by driving rates from 6.5 percent to 1 percent. And it tried to deal with the current recession by driving rates down from 5.25 percent to zero.

But zero, it turned out, isn’t low enough to end this recession. And the Fed can’t push rates below zero, since at near-zero rates investors simply hoard cash rather than lending it out. So by late 2008, with interest rates basically at what macroeconomists call the “zero lower bound” even as the recession continued to deepen, conventional monetary policy had lost all traction.

Now what? This is the second time America has been up against the zero lower bound, the previous occasion being the Great Depression. And it was precisely the observation that there’s a lower bound to interest rates that led Keynes to advocate higher government spending: when monetary policy is ineffective and the private sector can’t be persuaded to spend more, the public sector must take its place in supporting the economy. Fiscal stimulus is the Keynesian answer to the kind of depression-type economic situation we’re currently in.

Such Keynesian thinking underlies the Obama administration’s economic policies — and the freshwater economists are furious. For 25 or so years they tolerated the Fed’s efforts to manage the economy, but a full-blown Keynesian resurgence was something entirely different. Back in 1980, Lucas, of the University of Chicago, wrote that Keynesian economics was so ludicrous that “at research seminars, people don’t take Keynesian theorizing seriously anymore; the audience starts to whisper and giggle to one another.” Admitting that Keynes was largely right, after all, would be too humiliating a comedown.

And so Chicago’s Cochrane, outraged at the idea that government spending could mitigate the latest recession, declared: “It’s not part of what anybody has taught graduate students since the 1960s. They [Keynesian ideas] are fairy tales that have been proved false. It is very comforting in times of stress to go back to the fairy tales we heard as children, but it doesn’t make them less false.” (It’s a mark of how deep the division between saltwater and freshwater runs that Cochrane doesn’t believe that “anybody” teaches ideas that are, in fact, taught in places like Princeton, M.I.T. and Harvard.)

Meanwhile, saltwater economists, who had comforted themselves with the belief that the great divide in macroeconomics was narrowing, were shocked to realize that freshwater economists hadn’t been listening at all. Freshwater economists who inveighed against the stimulus didn’t sound like scholars who had weighed Keynesian arguments and found them wanting. Rather, they sounded like people who had no idea what Keynesian economics was about, who were resurrecting pre-1930 fallacies in the belief that they were saying something new and profound.

And it wasn’t just Keynes whose ideas seemed to have been forgotten. As Brad DeLong of the University of California, Berkeley, has pointed out in his laments about the Chicago school’s “intellectual collapse,” the school’s current stance amounts to a wholesale rejection of Milton Friedman’s ideas, as well. Friedman believed that Fed policy rather than changes in government spending should be used to stabilize the economy, but he never asserted that an increase in government spending cannot, under any circumstances, increase employment. In fact, rereading Friedman’s 1970 summary of his ideas, “A Theoretical Framework for Monetary Analysis,” what’s striking is how Keynesian it seems.

And Friedman certainly never bought into the idea that mass unemployment represents a voluntary reduction in work effort or the idea that recessions are actually good for the economy. Yet the current generation of freshwater economists has been making both arguments. Thus Chicago’s Casey Mulligan suggests that unemployment is so high because many workers are choosing not to take jobs: “Employees face financial incentives that encourage them not to work . . . decreased employment is explained more by reductions in the supply of labor (the willingness of people to work) and less by the demand for labor (the number of workers that employers need to hire).” Mulligan has suggested, in particular, that workers are choosing to remain unemployed because that improves their odds of receiving mortgage relief. And Cochrane declares that high unemployment is actually good: “We should have a recession. People who spend their lives pounding nails in Nevada need something else to do.”

Personally, I think this is crazy. Why should it take mass unemployment across the whole nation to get carpenters to move out of Nevada? Can anyone seriously claim that we’ve lost 6.7 million jobs because fewer Americans want to work? But it was inevitable that freshwater economists would find themselves trapped in this cul-de-sac: if you start from the assumption that people are perfectly rational and markets are perfectly efficient, you have to conclude that unemployment is voluntary and recessions are desirable.
Yet if the crisis has pushed freshwater economists into absurdity, it has also created a lot of soul-searching among saltwater economists. Their framework, unlike that of the Chicago School, both allows for the possibility of involuntary unemployment and considers it a bad thing. But the New Keynesian models that have come to dominate teaching and research assume that people are perfectly rational and financial markets are perfectly efficient. To get anything like the current slump into their models, New Keynesians are forced to introduce some kind of fudge factor that for reasons unspecified temporarily depresses private spending. (I’ve done exactly that in some of my own work.) And if the analysis of where we are now rests on this fudge factor, how much confidence can we have in the models’ predictions about where we are going?

The state of macro, in short, is not good. So where does the profession go from here?

VII. FLAWS AND FRICTIONS

Economics, as a field, got in trouble because economists were seduced by the vision of a perfect, frictionless market system. If the profession is to redeem itself, it will have to reconcile itself to a less alluring vision — that of a market economy that has many virtues but that is also shot through with flaws and frictions. The good news is that we don’t have to start from scratch. Even during the heyday of perfect-market economics, there was a lot of work done on the ways in which the real economy deviated from the theoretical ideal. What’s probably going to happen now — in fact, it’s already happening — is that flaws-and-frictions economics will move from the periphery of economic analysis to its center.
There’s already a fairly well developed example of the kind of economics I have in mind: the school of thought known as behavioral finance. Practitioners of this approach emphasize two things. First, many real-world investors bear little resemblance to the cool calculators of efficient-market theory: they’re all too subject to herd behavior, to bouts of irrational exuberance and unwarranted panic. Second, even those who try to base their decisions on cool calculation often find that they can’t, that problems of trust, credibility and limited collateral force them to run with the herd.

On the first point: even during the heyday of the efficient-market hypothesis, it seemed obvious that many real-world investors aren’t as rational as the prevailing models assumed. Larry Summers once began a paper on finance by declaring: “THERE ARE IDIOTS. Look around.” But what kind of idiots (the preferred term in the academic literature, actually, is “noise traders”) are we talking about? Behavioral finance, drawing on the broader movement known as behavioral economics, tries to answer that question by relating the apparent irrationality of investors to known biases in human cognition, like the tendency to care more about small losses than small gains or the tendency to extrapolate too readily from small samples (e.g., assuming that because home prices rose in the past few years, they’ll keep on rising).
Until the crisis, efficient-market advocates like Eugene Fama dismissed the evidence produced on behalf of behavioral finance as a collection of “curiosity items” of no real importance. That’s a much harder position to maintain now that the collapse of a vast bubble — a bubble correctly diagnosed by behavioral economists like Robert Shiller of Yale, who related it to past episodes of “irrational exuberance” — has brought the world economy to its knees.

On the second point: suppose that there are, indeed, idiots. How much do they matter? Not much, argued Milton Friedman in an influential 1953 paper: smart investors will make money by buying when the idiots sell and selling when they buy and will stabilize markets in the process. But the second strand of behavioral finance says that Friedman was wrong, that financial markets are sometimes highly unstable, and right now that view seems hard to reject.

Probably the most influential paper in this vein was a 1997 publication by Andrei Shleifer of Harvard and Robert Vishny of Chicago, which amounted to a formalization of the old line that “the market can stay irrational longer than you can stay solvent.” As they pointed out, arbitrageurs — the people who are supposed to buy low and sell high — need capital to do their jobs. And a severe plunge in asset prices, even if it makes no sense in terms of fundamentals, tends to deplete that capital. As a result, the smart money is forced out of the market, and prices may go into a downward spiral.

The spread of the current financial crisis seemed almost like an object lesson in the perils of financial instability. And the general ideas underlying models of financial instability have proved highly relevant to economic policy: a focus on the depleted capital of financial institutions helped guide policy actions taken after the fall of Lehman, and it looks (cross your fingers) as if these actions successfully headed off an even bigger financial collapse.

Meanwhile, what about macroeconomics? Recent events have pretty decisively refuted the idea that recessions are an optimal response to fluctuations in the rate of technological progress; a more or less Keynesian view is the only plausible game in town. Yet standard New Keynesian models left no room for a crisis like the one we’re having, because those models generally accepted the efficient-market view of the financial sector.

There were some exceptions. One line of work, pioneered by none other than Ben Bernanke working with Mark Gertler of New York University, emphasized the way the lack of sufficient collateral can hinder the ability of businesses to raise funds and pursue investment opportunities. A related line of work, largely established by my Princeton colleague Nobuhiro Kiyotaki and John Moore of the London School of Economics, argued that prices of assets such as real estate can suffer self-reinforcing plunges that in turn depress the economy as a whole. But until now the impact of dysfunctional finance hasn’t been at the core even of Keynesian economics. Clearly, that has to change.

VIII. RE-EMBRACING KEYNES

So here’s what I think economists have to do. First, they have to face up to the inconvenient reality that financial markets fall far short of perfection, that they are subject to extraordinary delusions and the madness of crowds. Second, they have to admit — and this will be very hard for the people who giggled and whispered over Keynes — that Keynesian economics remains the best framework we have for making sense of recessions and depressions. Third, they’ll have to do their best to incorporate the realities of finance into macroeconomics.

Many economists will find these changes deeply disturbing. It will be a long time, if ever, before the new, more realistic approaches to finance and macroeconomics offer the same kind of clarity, completeness and sheer beauty that characterizes the full neoclassical approach. To some economists that will be a reason to cling to neoclassicism, despite its utter failure to make sense of the greatest economic crisis in three generations. This seems, however, like a good time to recall the words of H. L. Mencken: “There is always an easy solution to every human problem — neat, plausible and wrong.”

When it comes to the all-too-human problem of recessions and depressions, economists need to abandon the neat but wrong solution of assuming that everyone is rational and markets work perfectly. The vision that emerges as the profession rethinks its foundations may not be all that clear; it certainly won’t be neat; but we can hope that it will have the virtue of being at least partly right.

Paul Krugman is a Times Op-Ed columnist and winner of the 2008 Nobel Memorial Prize in Economic Science. His latest book is “The Return of Depression Economics and the Crisis of 2008.”

This article has been revised to reflect the following correction:

Correction: September 6, 2009

Because of an editing error, an article on Page 36 this weekend about the failure of economists to anticipate the latest recession misquotes the economist John Maynard Keynes, who compared the financial markets of the 1930s to newspaper beauty contests in which readers tried to correctly pick all six eventual winners. Keynes noted that a competitor did not have to pick “those faces which he himself finds prettiest, but those that he thinks likeliest to catch the fancy of the other competitors.” He did not say, “nor even those that he thinks likeliest to catch the fancy of other competitors.”