Winner of the New Statesman SPERI Prize in Political Economy 2016

Wednesday, 12 April 2017

Economics is an inexact science

When I wrote about why the BBC should treat a clear consensus in economics the same way as it now treated climate science, I got a number of comments about why economics is not a science. A common theme was that economics couldn’t prove theories ‘beyond doubt’ the same way as the hard sciences could. A more sophisticated version of this complaint is that most economic theories cannot be disproved in the same way that Popper thought scientific theories could be disproved.

All this ignores a key feature of any social science, which is their inexact nature. Instead we have accumulations of evidence that confirm the applicability of some theories and reject the applicability of others. Economists’ views about what models are applicable change as this evidence accumulates.

A good example involves the minimum wage, as Noah Smith suggests. The basic economic model suggested even a modest minimum wage should significantly reduce employment, but economists discovered that the evidence did not show this. As this evidence accumulated, alternative theories and models (monopsony and search) were thought to be more relevant. It is this response to evidence that makes economics a science.

Jo Michell writes “The scientific method of forming a hypothesis and then testing that hypothesis against reality can never be the final arbiter of knowledge, as it can in the physical sciences.” He is right that no single experiment or regression can kill a theory, but wrong that the accumulation of evidence is not the final arbiter, because no other arbiter is available. He links to a post by Noah Smith which talks about the failures of forecasting. But as that post makes clear, this is not about data rejecting models, but the inability of models to predict the future. We would never dream of condemning medics because they cannot predict the exact time of our death, still less suggest that this failure indicates they are not doing science.

Of course economics involves cases where economists appear too reluctant to give up their favoured models. You can find similar stories in the hard sciences. There will be more such stories in economics because the inexact nature of economics makes it easier to discount any single piece of evidence. What I cannot understand is what leads someone like Russ Roberts to argue against the use of evidence, and instead that “economics is primarily a way of organizing one’s thinking”. Astrology is also a way of organising one’s thinking, but it fails because evidence does not back it up.

That comparison is slightly unfair, because while the theory behind astrology is obviously implausible, the basic principles of microeconomics are not. In a class on economic methodology I once drew a huge tree that showed how most of economics could be derived from principles of rational choice. But go beyond the basics, and add in complications involving information and transactions costs (to name but two) and you very quickly derive competing models. There is no single model that comes from thinking like an economist, so for that reason alone we need data to tell us which models are more applicable.

So thinking like an economist does not tell me at what point raising the minimum wage will reduce employment. But why would anyone want to keep their models from being proved relevant or otherwise by data? The only reason I can think of is that some models give answers that are ideologically convenient. Of course allowing data to establish the relevance of some models over others does not make economics ideology proof. For example people can always select the one study that suggests that fiscal policy does not influence output and ignore the hundreds that show otherwise. That is why the accumulation of evidence, which includes its replicability, is so important. If you think economics has problems in that respect, have a look at psychology.

This is why economists views about the long term impact of Brexit should be treated as knowledge rather than just an opinion. Here knowledge is shorthand for the accumulation of evidence consistent with plausible theory. Sometimes the theories are common sense, like making trade more difficult will reduce trade. Estimates of the size of trade reduction based on evidence are uncertain, but they are better than estimates based on wishful thinking. Empirical gravity equations consistently show that geography still matters a lot in determining how much is traded. Finally there is clear evidence that trade is positively associated with productivity growth. To say that all this has no more worth than some politicians opinion is ultimately to degrade evidence and the science which interprets it.


  1. "But as that post makes clear, this is not about data rejecting models, but the inability of models to predict the future. We would never dream of condemning medics because they cannot predict the exact time of our death, still less suggest that this failure indicates they are not doing science."

    Using reductio ad absurdum in trying to prove economics is a hard science is rather counter productive.

    Medicine is about both diagnosis and prediction, with the former necessarily following the latter. Without hard evidence of medical outcomes (both with and without treatments), medicine would certainly not be a science.

    That economic models are both inevitably backward looking and abstracted away from reality to the extent necessary to make them fit whatever theory is being advanced is but one of many reasons why economics can't be and shouldn't be regarded as a hard science.

    Indeed, Economics as a whole's predilection for creating arcane models that inevitably break down when applied to new data is hampering it and stopping it from offering something far more important than telling us what exact % one factor changes when another factor itself is changed by x%.

    Namely, that economics should be about teaching lessons and imparting knowledge that is broadly true, not necessarily universally.

    Austerity being deleterious to an economy is broadly true, though not always. The fact that economists fought bitterly over economic models trying ever more desperately to fit data to a model that proved their opinion was counter productive and harmful to tens (if not hundreds) of millions of people as economics was disregarded as having anything of value to offer.

    To teach more, economics needs to teach less.

    1. Economists need lose the term "law." There are no "laws of economics," or any other social science, that are comparable the laws of nature discovered in the natural science, owing to the differences in subject matter. There are no "laws of history," and economics is historical — regardless of how much formalists would like to believe otherwise.

      There may be regularities that observable in economics, but they do not rise to the level of universality that is characteristic of laws of nature. Economists should stop implying they they do. It is dishonest, and it is bringing considerable criticism down on the profession for apparent failures.

      These failures appear because of the way economist approach their discipline. They set narrow scope conditions that are instead advertised as law-based. Then they they have to resort to "exogenous factors" when things go wrong. This is not doing science. It is either confused, or else ideological persuasion based on rhetoric rather than reasoning and evidence.

      My advice is to declare your scope conditions and then don't give the impression that you are able to provide answers for anything beyond the scope of the modeling assumptions. Of course, this limits what economists can claim because restrictive assumptions narrow the scope of the model.

      Those who like to pontificate about "laws" don't like that. Now they are on the receiving end of not only criticism but also derision because of the exaggerated expectations about prediction and certainty they cultured in their audience that imploded in the GFC, for instance, and their prescription afterward have shown that they don't know how to fix thing they were involved in breaking.

      So when you hear "economic laws," think BS.

    2. I don't think Simon's trying to claim that economics is a hard science. Although both hard and soft sciences (a colloquial distinction sayeth Wikipedia) would, I think, be seeking to establish and confirm something as known...

    3. Hamtaro. Economists are making the diagnosis/prediction thing all the time. That is why we have central banks that react to inflation by raising rates. That is very different from the unconditional macro forecasts that people are familiar with.

      Anon. You will be glad to hear that 'Law' is not a term used very much in economics nowadays - I do not think I ever use it in teaching.

  2. "Economists views about the long term impact of Brexit should be treated as knowledge rather than just an opinion. Here knowledge is shorthand for the accumulation of evidence consistent with plausible theory."

    We have a word for a conclusion which is reached based upon the accumulation of evidence consistent with plausible theory. That word is "opinion". Lawyers draft opinions. Opinions are not statements of fact. They say "here are the facts, the best account of the law and justice is X, the result should be Y". That is the same endeavour you're also engaged in.

    Opinions are not just beliefs based upon no evidence at all. Some opinions are, of course, more justifiable than others, but they are not facts out there in the world.

    The problem you face cannot be overcome by your linguistic sleight of hand. Of course rational people should give much greater weight to a consensus of opinion amongst economists than to, say, the musings of Michael Gove. That is because they are based on more knowledge and a more plausible theory. But they are still opinions, for all that.

    1. I think that's a misuse of the legal sense of 'opinion'. Although I do feel like there's something funny about that statement.

    2. "We have a word for a conclusion which is reached based upon the accumulation of evidence consistent with plausible theory. That word is "opinion"."

      SpinningHugo, can you clarify at what point, if ever, you consider 'opinion' becoming knowledge? If I run a physics experiment many times and frequently reach the same conclusion, I consider that knowledge. Substitute economics for physics in the previous sentence and the same applies.

    3. SH: There is no reason why an individual's opinions need involve either evidence or a theory, so if you like knowledge is opinions derived in a certain way. If you can think of another way of getting at the distinction between the economic consensus and a politician's view please let me know. As far as the BBC is concerned, they are both equally valid.

    4. In the English language there is such reason. You see this in law where if you say "my opinion is X" that is taken as implying you have some fact to back it up.

      Some opinions are worth more than others. They aren't categorically different.

      Some people are in the business of just reporting facts out there in the world. We call them journalists.

      Economists aren't mere journalists.

  3. Popper's thesis mises a lot of features relevant in the practice of scientific activity by proposing a rather positivist view. It has long been left aside in the philosophy of science.

    On the other hand, I will grant to Popper that his underlying intuition about how failures of meaningful attempts at explaining empirical facts is instructive.

    My personal views have recently evolved and, now, I would say along with Box that all models are false, but that some are useful. I would go as far as saying that some of our habits in applied econometrics are wrongheaded -- type I and type II errors make no sense if we acknowledge no model is the "true" model. Some statisticians even joke about chi square types of tests are tests of sample size (if all models are wrong, enough evidence demonstrably will lead to rejection). Here, the value of science, economics included, is largely derived from its success in helping us achieve certain goals.

    In this light, economics is very good and no less scientific than physics. It is less accurate in many ways, but we do make sensible hypotheses, seek to apply them rather systematically to sets of data and try to find out when and how it missfits. Eventually some type of adjustments seem more appropriate than others.

    None of this particularly shelters neoclassical inspired models from all forms of criticism. In our defense, however, to any trained expert, they happen to be extremely convenient ways to formalize extremely rich dynamics. And, yes, we can, and sometimes do make adjustments to account for psychological effects we might believe to be important. It's just NOT an obvious improvement in all cases (because we try to represent the behavior of hundreds and thousands of people).

  4. I am so fed up of people decrying economics and not a science. Speaking as a scientist and a medical doctor, I agree that the analogy with medicine is a helpful one.

    I have not heard scientists disparage economics in this way. I hear it from people who simply use different tactics to discount science when the don't like what it says.

    I have tried to educate myself in economics as an interested amateur over the past decade because of its relevance to society at large. I would argue that economics has suffered more than many disciplines from ideological 'infection' if you will. Whereby proponents of a particular view discount evidence they don't like in order to push their interpretation. This happens in all disciplines; however scientific we might try or claim to be, we are all vulnerable to confirmational bias. Science is ever a human endeavour. It's a messy process whereby we're always working through the evidence for a better view of the truth. Economics is no different.

    The problem with the way people discount economics is that it means good evidence of what will work is ignored when it comes to public policy. And we have seen the huge costs of that in the UK over the past 7 years.

    Prior to the EU referendum I tried to explain to a friend of mine that I was as confident about the economic costs of leaving as I was of most things in medicine. Because I have read widely and understand the arguments put forward by experts who are prepared to explain their reasoning. He pointed me to 'Economists for Brexit.' I suggested this group was the equivalent of the anti-vaccine movement. With the same zeal and ignorance of facts.

    There is one other way in which medicine and economics are similar: we both face a deliberately inaccurate press who will pursue particular agendas. Most medical journalists are deeply ignorant of what they write about. It is a constant fact for most doctors that we are faced with patients scared by the latest horror story. Most recently it was E.coli is a superbug, but our media does an appalling job of informing the public.


    1. From an economist to a medical doctor: thank you

    2. If your medical treatments are analogous to Economics, I hope you're not my doctor!

  5. It's not so much that "some models give answers that are ideologically convenient"; though this is often part of it. It's more often the case that some models give answers that provide governing politicians with a rationale, however tenuous, to advance policies that favour the powerful and influential rent-extracting special interests which have suborned these politicians (or regulators).

    A perfect example is the privatisation and continuing regulation and competition policy oversight of the utility industries. Apart from a few unpersuadable left-wingers, it was viewed by most economists as a “jolly good thing”. It applied established economic theory and practice and allowed for the development of theory and practice – as well as providing lucrative employment and consulting opportunities for many economists. It also provided ample opportunities for rent extraction by investors and market participants.

    However, occasionally a point is reached when the rent extraction at the expense of a majority of households and small businesses becomes too obvious and egregious and there is a clamour that "something must be done". This happened in September 2013 when Ed Miliband proposed a freeze of electricity and gas prices so that the market could be "reset". This panicked the then government which eventually towards mid-2014 was compelled to “allow” an investigation of the energy market by the Competition and Markets Authority (CMA). The investigation ran for two and a half years until December 2016. A key finding was that up to 70% of households were not sufficiently engaged in the market to switch to lower priced offers and, as a result, were being ripped off mercilessly by the energy suppliers. The only recognised economist in the CMA Group appointed to conduct the investigation, Martin Cave, argued for a temporary regulated price cap to protect these consumers. He was outvoted and issued a minority dissent and the Group settled on a price cap only for consumers with pre-payment meters with the remaining “sticky” or “disengaged” consumers being put on a database organised by the energy regulator, Ofgem, so that they can be bombarded with junk mail from the energy suppliers.

    Not surprisingly, this rag-bag of ineffectual remedies has not secured any significant degree of popular support and the Government is considering some form of “relative price cap” restraining the gap between suppliers’ highest and lowest tariff offers. And, equally unsurprisingly, the “Father of RPI-X Regulation”, Stephen Littlechild, and a number of other former regulators, are expressing their total opposition to any form of price cap drawing on their economics model.

    It will be interesting to see how this conflict of economic models will unfold in the context of political expediency. It is also interesting to note that there is a far more effective economic remedy to this problem, based on a statutory collective buyer supplying these “sticky” consumers with the energy suppliers participating in an auction to supply the collective buyer. But this “collective action” approach, which has a long and well-deserved pedigree, will not be considered because it would minimise the extraction of economic rent and is now considered an ideological anathema – and the Labour party is too stupid to advance it.

    This for me is a perfect example of how the discipline of economics is subverted with either the deliberate or unwitting acquiescence of its practitioners.

    1. Excellent piece. It well illustrates the nub of the problem. Have you published this elsewhere? You should push it about, as it is important stuff.

  6. From an MMT perspective these are so silly:

    What’s even more amusing is that it isn’t even required in the Bastardised Keynesian model. If the government announces a big spending program that is supposed to alter expectations and the central bank function responds by pulling its one and only interest rate lever to make space for the spending and ensuring the multiplier impact is 1.

  7. How about the following as a plausible and qualitative difference between the hard sciences and economics? In physics, it would be considered risible to reject relativity and revert to Newtonian physics, even though a great deal of evidence, especially that which concerns lower velocities etc, would fit in with both relativity and Newtonian models. But in economics, it was possible to reject a twentieth century paradigm, Keynsianism, and revert to classical economics, to all intents and purposes (at least, to many lay people, that's how it looks) because of the political and ideological biases of star professors in the subject. Like it or not, that makes physics look like a science and economics like a cult. However unfair this seems (and it is very unfair to economists you read their theory from the facts rather than the other way around), that is now perhaps the common perception of economists. To get to where physics is, wouldn't it be necessary for economists to more or less denounce members of their profession who, for example, insist that austerity is the correct policy at the zero lower bound, when there is a Himalaya mountain range of evidence saying otherwise? If not, then it continues to look like a cult full of ideologues with neo-liberal axes to grind, doesn't it?

    1. I do not recognise what you say about Keynesian economics. As for denouncing, plenty of that goes on in blogs. What further action would you ask for?

  8. Redefining economics
    Comment on Simon Wren-Lewis on ‘Economics is an inexact science’

    Science is well-defined: “Research is in fact a continuous discussion of the consistency of theories: formal consistency insofar as the discussion relates to the logical cohesion of what is asserted in joint theories; material consistency insofar as the agreement of observations with theories is concerned.” (Klant)

    Science is cumulative. Only certain knowledge can be admitted to the corpus of science because nothing can be built upon uncertain knowledge or mere opinion. And here is the crux of the so-called social sciences: “By having a vague theory it is possible to get either result. … It is usually said when this is pointed out, ‘When you are dealing with psychological matters things can’t be defined so precisely’. Yes, but then you cannot claim to know anything about it.” (Feynman)

    This is why there is no growth of knowledge in the so-called social sciences: “Indeed, Alexander Rosenberg maintains that there has been no progress in developing laws of human behavior for the last twenty-five hundred years.” (Hausman)

    Manifest failure in turn is the main reason why the so-called social sciences stubbornly try to soften scientific standards wherever they can, or, as Blaug put it, to play tennis with the net down. This is what the talk of economics as ‘inexact and separate science’ amounts to.

    Since 200+ years economists bridge the chasm between scientific appearance and proto-scientific reality with excuses: “Economics is not a Science with a capital S. It lacks the experimental method as a way of testing hypotheses. … There are always differences of opinion at the cutting edge of a science, … But they last longer in economics . . . and there are reasons for that. As already mentioned, rival theories cannot be put to an experimental test. All there is to observe is history, and history does not conduct experiments: too many things are always happening at once. The inferences that can be made from history are always uncertain, always disputable, … You can’t even count on a long and undisturbed run of history, because the ‘laws’ of behavior change and evolve. ” (Solow)

    Economists have to redefine their subject matter. To explain individual and social behavior is NOT their business but the task of psychology, sociology, political science, social philosophy, history, anthropology, biology, Darwinism/evolution theory, etcetera.

    To explain how the actual economy works is the proper task of economics. Economists have failed at this task. After more than 200 years they have not even figured out what profit is, that is, they do not understand the pivotal phenomenon of their subject matter.

    For deeper methodological reasons, the so-called social sciences cannot rise above the level storytelling. And this is what Walrasianism, Keynesianism, Marxianism, and Austrianism is. Neither approach satisfies the non-negotiable criteria of science, i.e., material and formal consistency.

    Economists face this option: to continue with storytelling and to be expelled from the sciences or to restart economics as a system science. This means in concrete terms to move from false behavioral microfoundations and false Keynesian macrofoundations to objective/structural/behavior-free/consistent macrofoundations. This paradigm shift yields exact and testable systemic laws.*

    Economists have until this day not understood how science works: “When the premises are certain, true, and primary, and the conclusion formally follows from them, this is demonstration, and produces scientific knowledge of a thing.” (Aristotle). This means that economics cannot be built upon NONENTITIES like constrained optimization, rational expectation, or equilibrium. After more than 140 years of stagnation at the proto-scientific level patience with economists’ incompetence and excuses wears thin.

    Egmont Kakarot-Handtke

    * See cross-references Paradigm shift

  9. You're all in denial... economics is a religious enterprise... with two sects (like Protestant & Catholic or Sunni & Shia, like Orthodox & non-Orthodox Jews... etc).

    Don't take offense but the reason economics remains an non-science is this reason.

    It's really very simple.... economics is people doing stuff with currency as a measure of their values at any given moment in time and circumstance. People have different value & and belief systems (econ no different in this regard) so you end up with two sides of every economic "theory" or position... (saltwater/freshwater as one example). And because of this there can never be economic consensus and thus each side uses their own "rules", bias's, "proofs", etc. and thus there's no discipline applied overall.

    If one postulates that science's discipline is to a) discover truths by b) using a commonly agreed set of methods, then economics isn't beyond the capability of becoming a science. But that it doesn't speaks to economic actors having other objectives.

    Look inward for why economic actors have different objectives.

  10. "while the theory behind astrology is obviously implausible, the basic principles of MICROeconomics are not." Fantastic slight of hand, Mainly MACRO; very impressive!

  11. "... I once drew a huge tree that showed how most of economics could be derived from principles of rational choice." But people are much better at rationalizing that at being rational. Basically, we are social (aka herd) animals, and tend to adjust our thinking and our preferences to fit in.

    Thinking about it from a biological point of view, it would be easy to construct a model based on the idea that rationality is adaptive, so that we could expect people to evolve to be rational. The problem is the abundant evidence that people tend not to behave rationally. The most economically important decisions that most of us make are about choosing a trade or profession, getting married, and having children. Among the people you know well, how many were rational about those decisions?

    1. That is why we have behavioral economics. Rationality works as much as it does in part because it is a way of avoiding being exploited.

  12. Yes, the interesting thing about this objection is just how difficult it is for people to articulate exactly what they have in mind as 'science'.

    Of course inquiry into how economies work is science, its just wildly more difficult than many other sciences.
    Though even this claim is difficult as science lives on the frontier, research by definition takes place on the edge between what we know and what we still are learning. And peering at the 'hard' sciences reveals readily enough just how much there is to learn in any area of 'science', presumably the frontier is there because its difficult to push forward.

    All the evidence suggests we are making solid advances in our knowledge regarding economics over generations.
    All the current discussion is focused just across the border to that wide expanse of the unknown, but under exploration, territory of an extra-ordinarily challenging topic.

    Just because the topic is difficult, doesn't make it not a science.

  13. On the topic, I didn't see a lot of people discussing Ricardo Reis's new article (paper?) "Is something really wrong with macroeconomics?" It's a pretty entertaining read and a light of hope.

  14. One thing always strikes me is how often economists seem to follow stylized thinking in the face of evidence. For example, this very article claims that making trade harder hampers trade, but this ignores the US illegal drug market in which making trade harder raised prices which created incentives and provided resources to increase and encourage the drug trade. I think Britain made a mistake with Brexit, but there are good arguments that Brexit might make London even more valuable as an entrepot since it would be governed under different law from that on the continent.

    This always reminds me of an AI project back in the 1980s that tried to help teach naval technicians how to start up a major ship engine from cold iron. Adding cold water to the mix might cool things down or heat things up depending on details of the system state. Interestingly, the system was based on a simulation, and that simulation was known to be reliable as it was used in the engine design and validated repeatedly in practice.

    The problem comes in when making policy. There is way too much certainty on the part of various parties, often in the face of no evidence whatever.

  15. I think Russ is right to doubt the usefulness of macro econometrics. The ability to predict the macro future has more to do with logic and knowing facts about the world. What statistical test can tell me new technologies?

    1. I think this partly reflects that academic macro is rather peculiar at the moment in excluding a lot of evidence. Hopefully that will change.

    2. And there's no way the profession is doing Michael Burry style analysis to predict the next financial crisis. How can we build a discipline that honestly tries to look out for the next financial crisis in real life?

  16. Economics is like the weather, you only know whether it will rain later in the day is after the event.

    What we do know is that when a political system is built around an economic framework we are locked into a downward spiral such as Neo-Liberalism today.

    When we build an economy that is based on peoples needs, we can regulate it to go as fast or as slow as is desired.

    1. Unless you are going to decide what peoples needs are, you will have to use economics when you build that economy.

  17. Of course there are economic laws. Economics has become an arcane art because it ignores the laws of distribution: the law of rent; the law of wages, &c.

  18. I agree with the description of economics as an inexact science. However, I don't agree with your remarks on Brexit.

    Economists know that making trade with EU countries more difficult, which Brexit undoubtedly will, will have a negative effect on British incomes. However, the EU is not only a free trade area but also (and arguably more importantly) a political union, producing a large amount of legislation each year.

    For example, banking regulation is basically completely defined at the EU level, with national discretion restricted to a couple of unimportant parameters and risk-weights.

    That is, Brexit will also have an effect on the laws and regulations under which the UK economy is going to operate. Will this legislative effect of Brexit be positive or negative for British incomes? Nobody can know for sure.

    However, EU legislation certainly has room for improvement (to put it politely) and, in general, smaller political entities tend to be governed better than larger ones. Hence, the legislative effect on British incomes may well be positive. Furthermore, the legislative effect of Brexit might not only be positive but also strong enough to more than offset the negative effect of making trade with EU countries more difficult.

    In short, the net effect of Brexit on British incomes may very well be positive.

  19. Einstein (1934) wrote:

    "Pure logical thinking can give us no knowledge whatsoever of the world of experience; all knowledge about reality begins with experience and terminates in it.
    Conclusions obtained by purely rational processes are, so far as Reality is concerned, entirely empty."

    Economics is not science because its theories are contradicted by facts.

  20. "The basic economic model suggested even a modest minimum wage should significantly reduce employment, but economists discovered that the evidence did not show this."

    I know that the response of one faction in economics is to deny that "evidence did not show this." I have no idea whether that faction is considered crackpot or a respectable faction in economics.

    But how about applying the basic economic model correctly: using a realistic marginal benefit function, that a local franchise or facility might use. Namely:

    f(n) = $50 if n < 20
    f(n) = 0 if n > 20
    f(20) = some number between 0 and $50

    Clearly, the only possible effect of the minimum wage is on the 20th employee -- and only a small fraction of the time. If f(20) < (say) $10 then the 20th employee was not hired. If f(20) > (say) $20 then the 20th employee remains hired. In between, the minimum wage increase means the 20th employee is laid off -- the only time the minimum wage increase affects employment.

    This automatically reduces the theoretical affect of a minimum wage to a bare minimum.

    The minimum wage increases demand for goods, resulting in increased inflation, increased employment to meet the demand, or (most likely) a combination.


Unfortunately because of spam with embedded links (which then flag up warnings about the whole site on some browsers), I have to personally moderate all comments. As a result, your comment may not appear for some time.