Science is about discarding models, not making them

As someone who is preparing to apply for jobs as a professional economist, I can't help feeling down when I see Paul Krugman write something like this:

I’ve never liked the notion of talking about economic “science”...Still, when I was younger I firmly believed that economics was a field that progressed over time, that every generation knew more than the generation before. 

The question now is whether that’s still true...I think you can actually make the case that in important ways the profession knew more in 1971 than it does now... 

[M]any economists aren’t even trying to get at the truth. When I look at a lot of what prominent economists have been writing in response to the ongoing economic crisis, I see no sign of intellectual discomfort, no sense that a disaster their models made no allowance for is troubling them; I see only blithe invention of stories to rationalize the disaster... 

And all this makes me wonder what kind of an enterprise I’ve devoted my life to.

Paul Krugman is one of my personal heroes; it was the brilliance of his theories that lured me to the profession in the first place. So it makes me pretty sad to hear him questioning the value of the whole enterprise.

But I can't really disagree with his characterization. There's a reason you keep seeing these "is economics a science?" posts popping up around the blogosphere. The crisis made a lot of people question the value of economics, in its current form, as a useful tool for understanding and affecting the world. But when many economists seemed not to experience similar doubts, it raised the question of whether the profession approaches the world in a scientific way.

About 400 years ago, see, humans discovered a new way of understanding and interacting with the world, which seemed to work a lot better than what we had been doing before. The new approach involved things like doing experiments, writing down models, and observing the natural world. But what exactly was so great about this approach, which we called "science"? What were the key features of this new approach that made it so much more effective? If we have the models without the experiments, or vice versa, will we still get the same results?

I believe that we - some of us, anyway - have had a good answer to this question for a long time. The answer is that science works when it has a way to discard ideas that do not work. People have phrased this answer in different ways. Francis Bacon called it eliminative induction. Karl Popper called it "falsifiability." Richard Feynman called it "doubt." But it's the same idea. You actively look for what doesn't work, and throw it out.

Economists, I think, do not always see this as the essence of science. For example, here is Scott Sumner, in March 2011:

I’ve always thought that the debate over whether economics is a science is actually a debate about the meaning of the term ’science.’  If science is when people build models to better understand the world around us, then economics is a science.

And here is Stephen Williamson:

I looked up a definition of "science," and came up with this:

The intellectual and practical activity encompassing the systematic study of the structure and behavior of the physical and natural world through observation and experiment. 

So, as economists we are systematic, we study the structure of the economy and the behavior of individuals in it, and we observe and experiment. Thus, apparently, we are a science.

Now, far be it from me to argue with the Oxford English Dictionary. But I contend that the real question here is substantive, not semantic. Whether or not an object is an "airplane," the important question is whether it flies. And whether or not economics is a "science," the important question is whether or not it discards its bad ideas. 

Economics has several ways of discarding bad ideas. The most common is to use empirical data to reject models. One particularly effective way of doing this is to use "natural experiments". A second method is to use experiments, which can be laboratory experiments or field experiments. 

So, procedurally, economics is a science.

But there is a difference between what a field can do and what it does do. A field is not just a set of techniques, it is also an established culture that chooses when and how to use those techniques. It is in this sense that economics does not always live up to its potential as a science. 

What I am not talking about are models that are that non-testable. Economics does make some of these, but that's fine. These models may be interesting as pure mathematics, or they may be of use in doing future science, even though they themselves are not science. I don't lose any sleep over these.

What bothers me is that the economics profession, as a culture, does not always insist that the testable models be tested and discarded. When enough economists ignore the facts and keep believing in models that can and should be discarded, then even though economics is procedurally a science, culturally it is not behaving like one. Obviously, I am referring to most of the popular macro models in use before the crisis.

So I disagree very strongly with Scott Sumner. The heart of science is not building models, but discarding them. Economics as a discipline has the tools to prune its tree of models, but economics as a culture has not been using these tools harshly enough. And so we've ended up as a sort of massive supermarket of models, where we have a model (or twenty models) for any conceivable phenomenon, but little way of knowing which model we should use in which situation. Economists who protest that "we had plenty of models that described financial crises" can't answer the question of why nobody was paying attention to those models before the crisis slapped us in the face.

This, I believe, is why the public has become incredibly skeptical of our profession. We let ourselves get away with too much. When we saw data that flatly contradicted our favorite theories, we just shrugged and kept on believing. And many of us seem still to be doing this, even now. And that is why these philosophy-of-science blog posts keep coming and coming. As well they should.


(Note: Most of what I'm complaining about here, like much of what I write on this blog, really goes only for macro. Micro, which is just a lot easier to test, is generally much better about being "scientific." But the difficulty of testing macro models doesn't let the field off the hook...)