Criticisms of econometrics

There have been many criticisms of econometrics' usefulness as a discipline and perceived widespread methodological shortcomings in econometric modelling practices.

At one level these criticisms are based on the fact that economic structure and behaviour is subject to change and accurately observed relationships in the past may not continue into the future. As Robert Lucas famously observed this is particularly true where economic policy changes are implemented based on econometric models that fail to account for the possibility of patterns changing as a deliberate response to the policy.

The heterodox Austrian School of economics entirely rejects the validity of econometrics as a method of predicting human behaviour, preferring to use the deductive method. Some, but by no means all Post Keynesian economists dislike the use of econometric methods, perhaps influenced by Keynes' own aversion to "arid mathematical formalism" in economics.^[1] Joan Robinson was offered the role of vice president of the Econometric Society but declined on the basis she wouldn't want to be part of the editorial committee of a journal she "could not read".^[2]

Difficulties in model specification

Like other forms of statistical analysis, badly specified econometric models may show a spurious correlation where two variables are correlated but causally unrelated. Economist Ronald Coase is widely reported to have said "if you torture the data long enough it will confess".^[3] McCloskey argues that in published econometric work, economists tend to rely excessively on statistical techniques and often fail to use economic reasoning for including or excluding variables.^[4]

Economic variables are not readily isolated for experimental testing, but Edward Leamer argues that there is no essential difference between econometric analysis and randomized trials or controlled trials provided judicious use of statistical techniques eliminates the effects of collinearity between the variables.^[5]

Economists are often faced with a high number of often highly collinear potential explanatory variables, leaving researcher bias to play an important role in their selection. Leamer argues that economists can mitigate this by running statistical tests with different specified models and discarding any inferences which prove to be "fragile", concluding that "professionals ... properly withhold belief until an inference can be shown to be adequately insensitive to the choice of assumptions".^[6]

Lucas critique

Robert Lucas criticised the use of overly simplistic econometric models of the macroeconomy to predict the implications of economic policy, arguing that the structural relationships observed in historical models break down if decision makers adjust their preferences to reflect policy changes. Lucas argued that policy conclusions drawn from contemporary large-scale macroeconometric models were invalid as economic actors would change their expectations of the future and adjust their behaviour accordingly.

Lucas argued a good macroeconometric model should incorporate microfoundations to model the effects of policy change, with equations representing economic representative agents responding to economic changes based on rational expectations of the future; implying their pattern of behaviour might be quite different if economic policy changed.

Modern complex econometric models tend to be designed with the Lucas critique and rational expectations in mind, but Robert Solow argued that some of these modern dynamic stochastic general equilibrium models were no better as the assumptions they made about economic behaviour at the micro level were "generally phony".^[7]

Other mainstream critiques

Looking primarily at macroeconomics, Lawrence Summers has criticized econometric formalism, arguing that "the empirical facts of which we are most confident and which provide the most secure basis for theory are those that require the least sophisticated statistical analysis to perceive." He looks at two highly praised macroeconometric studies (Hansen & Singleton (1982, 1983), and Bernanke (1986)), and argues that while both make brilliant use of econometric methods, both papers do not really prove anything that future theory can build on. Noting that in the natural sciences, "investigators rush to check out the validity of claims made by rival laboratories and then build on them," Summers points out that this rarely happen in economics, which to him is a result of the fact that "the results [of econometric studies] are rarely an important input to theory creation or the evolution of professional opinion more generally." To Summers:^[8]

Successful empirical research has been characterized by attempts to gauge the strength of associations rather than to estimate structural parameters, verbal characterizations of how causal relations might operate rather than explicit mathematical models, and the skillful use of carefully chosen natural experiments rather than sophisticated statistical technique to achieve identification.

Austrian School critique

The current-day Austrian School of economics typically rejects econometrics, stating that historical mathematical data used to make econometric models represents past behaviour which may change in future and is ineffective at isolating causal relationships. In this they continue their belief that mathematics and statistical methods are mostly unsuited for the study of social sciences.^[9]

See also

• Econometrics
• Methodology of econometrics
• Macroeconomic model
• History of macroeconomic thought

Notes

1. Gerrard, Bill (2002) "The role of econometrics in a radical methodology publisher" in: Sheila C. Dow; John Hillard (Ed.) Post Keynesian Econometrics,Microeconomics and the Theory of the Firm, Edward Elgar. ISBN 1858985846 p.113
2. "In praise of … Joan Robinson" The Guardian, Friday 9 July 2010
3. Gordon Tullock, "A Comment on Daniel Klein's 'A Plea to Economists Who Favor Liberty'", Eastern Economic Journal, Spring 2001, note 2 (Text: "As Ronald Coase says, 'if you torture the data long enough it will confess'." Note: "I have heard him say this several times. So far as I know he has never published it.")
4. McCloskey, D.N. (May 1985). "The Loss Function has been mislaid: the Rhetoric of Significance Tests" (PDF). American Economic Review 75 (2): 201–205.
5. Leamer, Edward (March 1983). "Let's Take the Con out of Econometrics". American Economic Review 73 (1): 34.
6. Leamer, Edward (March 1983). "Let's Take the Con out of Econometrics". American Economic Review 73 (1): 43.
7. Solow, R. (2010) "Building a Science of Economics for the Real World", Prepared Statement of Robert Solow, Professor Emeritus, MIT, to the House Committee on Science and Technology, Subcommittee on Investigations and Oversight: July 20, 2010
8. Summers, Lawrence (June 1991). "The Scientific Illusion in Empirical Macroeconomics". Scandinavian Journal of Economics 93 (2): 129–148. JSTOR 3440321.
9. Garrison, Roger - in The Meaning of Ludwig von Mises: Contributions is Economics, Sociology, Epistemology, and Political Philosophy, ed. Herbener, pp. 102-117. "Mises and His Methods"