FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:   || 2 | 3 |

«Significance redux Stephen T. Ziliaka,∗, Deirdre N. McCloskeyb,1 aSchool of Policy Studies, College of Arts and Sciences, Roosevelt University, ...»

-- [ Page 1 ] --

The Journal of Socio-Economics 33 (2004) 665–675

Significance redux

Stephen T. Ziliaka,∗, Deirdre N. McCloskeyb,1

aSchool of Policy Studies, College of Arts and Sciences, Roosevelt University, Chicago,

430 S. Michigan Avenue, Chicago, IL 60605, USA

b College of Arts and Science (m/c 198), University of Illinois at Chicago, 601 South Morgan,

Chicago, IL 60607-7104, USA


Leading theorists and econometricians agree with our two main points: first, that economic sig-

nificance usually has nothing to do with statistical significance and, second, that a supermajority of economists do not explore economic significance in their research. The agreement from Arrow to Zellner on the two main points should by itself change research practice. This paper replies to our critics, showing again that economic significance is what science and citizens want and need.

© 2004 Elsevier Inc. All rights reserved.

JEL CODES: C12; C10; B23; A20 Keywords: Hypothesis testing; Statistical significance; Economic significance Science depends on entrepreneurship, and we thank Morris Altman for his. The sympo- sium he has sparked can be important for the future of economics, in showing that the best economists and econometricians seek, after all, economic significance. Kenneth Arrow as early as 1959 dismissed mechanical tests of statistical significance in his search for eco- nomic significance. It took courage: Arrow’s teacher, the amazing Harold Hotelling, had been one of Fisher’s sharpest disciples. Now Arrow is joined by Clive Granger, Graham ∗ Corresponding author.

E-mail addresses: sziliak@roosevelt.edu (S.T. Ziliak), deirdre2@uic.edu (D.N. McCloskey).

1 For comments early and late we thank Ted Anderson, Danny Boston, Robert Chirinko, Ronald Coase, Stephen Cullenberg, Marc Gaudry, John Harvey, David Hendry, Stefan Hersh, Robert Higgs, Jack Hirshleifer, Daniel Klein, June Lapidus, David Ruccio, Jeff Simonoff, Gary Solon, John Smutniak, Diana Strassman, Andrew Trigg, and Jim Ziliak.

1053-5357/$ – see front matter © 2004 Elsevier Inc. All rights reserved.

doi:10.1016/j.socec.2004.09.038 666 S.T. Ziliak, D.N. McCloskey / The Journal of Socio-Economics 33 (2004) 665–675 Elliott, Joel Horowitz, Ed Leamer, Tony O’Brien, Erik Thorbecke, Jeffrey Wooldridge, and Arnold Zellner (Arnold is in our minds a Zeus in the matter of economic significance, with Ed Leamer as his Mercury). Their examples inspire hope. And our hope is strengthened learning as we do here from colleagues in cognate fields that mechanical tests have been criticized by their best, for decades. It’s time to stop the nonsense and get serious about significance in economics.

Why has it taken until now for economists to catch on? In his own paper Morris Altman makes a good case for path dependence. People have believed that mechanical testing for statistical significance is all right because, after all, it’s been around for so long, something one might say, too, of the labor theory of value, or protectionism, or belief in s´ ances with e the dear departed. As Altman observes, even in psychology, where since the Significance Test Controversy of the early 1970s there has been widespread understanding of the issue by sophisticates, little has changed. Fidler et al. conclude here, too: “psychology has produced a mass of literature criticizing null-hypothesis statistical testing over the past five decades... but there has been little improvement.... Even editorial policy and (admittedly half-hearted) interventions by the American Psychological Association have failed to inspire any substantial change.” Capraro and Capraro (2004), cited in Altman, found that in psychology the number of pages in texts and guidebooks recommending the mechanical use of statistical significance was orders of magnitude larger than the number of pages warning that after all effect size is always the chief scientific issue. Our papers show the same to be true for a supermajority of econometrics texts, from the advanced Handbook of Econometrics through Arthur Goldberger’s latest down to the simplest of introductory textbooks. Students get misled from the beginning. Few see a problem. And even fewer break away.

But the present forum may be the beginning of the end for a silly and unscientific custom in economics. We associate ourselves with the remark by the psychologist W.W.

Rozeboom in 1997, quoted by Bruce Thompson here (Rozeboom has been making the point since 1960): “Null-hypothesis significance testing is surely the most bone-headedly misguided procedure ever institutionalized in the rote training of science students.... It is a sociology-of-science wonderment that this statistical practice has remained so unresponsive to criticism” (Rozeboom, in B. Thompson, p. 335). Precisely.

1. How to deal with random error

To unblock the journal referees and editors and break out of what Altman calls “a steadystate low-level equilibrium” we propose asking major economists and econometricians to state publicly their support for the following propositions: (1) Economists should prefer condence intervals to other methods of reporting sampling variance. (2) Sampling variance is sometimes interesting, but is not the same thing as scientific importance. (3) Economic significance is the chief scientific issue in economics; an arbitrary level of sampling significance is no substitute for it. (4) Fit is not a good all-purpose measure of scientific validity, and should be deemphasized in favor of inquiry into other measures of importance. Every editor of every major journal will be asked. We think that on reflection most economists and econometricians will agree with these propositions.

S.T. Ziliak, D.N. McCloskey / The Journal of Socio-Economics 33 (2004) 665–675 667 Scores of the best statistical investigators in psychology, sociology, and statistics itself have been making such points for a long time, longer even than McCloskey has, who came by her insights honestly, stealing them fair and square 20 years ago from pioneers like Denton Morrison and Ramon Henkel in sociology and Paul Meehl and David Bakan in psychology and Kenneth Arrow in economics (Arrow, 1959; McCloskey and Ziliak, 1996).

Ziliak first learned about the difference between economic and statistical significance in the late 1980s, when he purchased for his job at the State of Indiana Department of Employment and Training Services an elementary book by the two Wonnacott brothers, one an economist, the other a statistician (Wonnacott and Wonnacott, 1982, p. 160). But he met puzzling resistance when he argued to the chief in his division of labor market statistics how it was a shame that rates of unemployment among black urban teenagers in Indiana were not being published, and were therefore not being discussed openly and scientifically, merely because their small sample sizes did not attain conventional levels of statistical significance.

Good fit modulo the present “sample” is nice, even “neat.” But there is no reason to make fit the criterion of model selection. As Arnold Zellner points out in his comments, sometimes of course the fit measured by R2 is perfect because the investigator has regressed US national income precisely on itself. Y fits Y, L fits L, K fits K. “Fit” in a wider scientific sense, which cannot be brought solely and conveniently under the lamppost of sampling theory, is more to the point. How well for example does the model (or parameter estimate) fit phenomena elsewhere? Are there entirely different sorts of evidence—experimental, historical, anecdotal, narrative, and formal—that tend to confirm it? Does it accord with careful introspections about ourselves? What could be lost if policymakers or citizens act as if the hypothesis were true? So we remain skeptical that some simple and equally mechanical refinement of statistical significance will work. Some of the advanced proposals miss the main point, that fit is not the same thing as importance.

2. Precision is nice but oomph is the bomb

The kind of decision-making we advocate can be illustrated thus. Suppose you want to help your mother lose weight, and are considering two diet pills of identical price and side effects. The one, named “Oomph,” will on average take off 10 pounds, but it is rather uncertain in its effects, at plus or minus 5 pounds. Not bad. Alternatively, the pill “Precision” will take off only 3 pounds on average, but it’s less of a roll of the dice: Precision brings a probable error of plus or minus 1 pound. How nice.

The signal-to-noise ratio of diet pill Oomph is 2:1, that for Precision 3:1. Which pill for Mother? “Well,” say some of our scientific colleagues, “the one with the highest signal-tonoise ratio is Precision. So, of course: hurrah for Precision.” Wrong, of course; wrong, that is, for Mother’s weight-management program and wrong for the distressingly numerous victims of scientists in the misled fields from medicine to management. Such scientists decide whether something is important or not, whether it has an effect, as they say, by looking not at its oomph but at how precisely it is estimated. But the pill Oomph promises to shed 5 or 15 pounds. The much less effective Precision will shed less than 4 pounds.

Common sense recommends Oomph. The burden of this symposium is: let’s get back to common sense—to oomph—in science.

668 S.T. Ziliak, D.N. McCloskey / The Journal of Socio-Economics 33 (2004) 665–675

The crucial thing to grasp in the comments gathered here is this: every one of the commentators agrees with our two main points:

1. that economic significance usually has nothing to do with statistical significance, and

2. that a supermajority of economists do not explore economic significance in their research.

The agreement from Arrow to Zellner on the main points should by itself change research practice. Moreover, the tiny objections the critics raise against us, though significant as sociology of science, in no way undermine the consensus. Economic significance, substantive significance, is the body, not statistical significance unadorned. We all here agree.

3. Some reasons statistical significance does not select models

Graham Elliott and Clive Granger agree with our point, but want for some reason to characterize it as “literary” and not “deep.” Perhaps it arises from their mistaken belief that if sample means and so forth are somewhere provided in a paper, then “the economic significance can be determined.” Set aside that, as they admit, in many cases the papers do not provide the data to get beyond a statement that a certain coefficient is or is not “significant.” Our main point is not this stylistic one. It is that “significance” itself is something that needs to be argued out in the context of the scientific or policy issue and cannot be determined on statistical grounds alone. Our point is not to repeat a matter of style, literary matters, or superficialities of presentation. The economic significance cannot “be determined” by simply better reporting on conventional statistical tests. The mistake of Elliott and Granger shows in their claim that what would be at issue in cases of bad reporting is the “statistical comprehension skills” of the reader. No. It is the economic comprehension skills that matter for economic science: that is our main point. We cannot hand science over to a table of Student’s t.

We have learned recently, by the way, that “Student” himself—William Sealy Gosset—did not rely on Student’s t in his own work. To the world’s gain Gosset’s job and passion was to instead learn scientifically how to brew the best Guinness he could brew at the best price the market could bear (see for example E.S. Pearson, ‘Student’: A Statistical Biography of William Sealy Gosset [Oxford: Clarendon Press, 1990, pp. 20, 30–31]).

Student used his t-tables a teensy bit; but Student gave his scientific time and consideration to proportions of yeast and mash, mixing ingredients over time for a maximum oomph in Guinness, as you’d want and expect. R.A. Fisher begged Student for his tables of t to publish in Fisher’s now hugely damaging Statistical Methods for Research Workers. Yet Fisher—himself a decent farmer—did not as we have shown believe he needed to emulate Student’s care for magnitudes of ingredient effect, and focused instead on t.

Often we focus on how to interpret the parameters of a specific model. Elliott and Granger agree with us but then focus their critical energies on a defense of mechanically computed statistical significance to separate theory A and theory B (we believe they mean “model” A and model B, though their comments equivocate.) We are not persuaded.

Pages:   || 2 | 3 |

Similar works:

«NORTHERN MIDLANDS COUNCIL AREA– COMMUNITY/HOUSEHOLD FOOD ACCESS PROFILE Introduction & disclaimer – This profile has been developed by the Heart Foundation Tasmania as part of the Healthy Food Access Tasmania Project. The information highlighted in this profile was gathered in 2014/15 and it provides an overview for the local government area. For any questions or additional information please contact the Heart Foundation. Why does access to healthy food matter? Limited or poor access to...»

«The Sprawl: Alpha Version hamish@ardens.org The Sprawl Alpha Version An Apocalypse World Hack by Hamish Cameron. Contact details at the end. “The sky above the port was the color of television, tuned to a dead channel.” Neuromancer, William Gibson. The Sprawl is a game of mission-based action in a gritty neon-and-chrome Cyberpunk future. You are the extended assets of vast multinational corporations, operating in the criminal underground, and performing the tasks that those multinationals...»

«University of Warwick institutional repository: http://go.warwick.ac.uk/wrap A Thesis Submitted for the Degree of PhD at the University of Warwick http://go.warwick.ac.uk/wrap/35765 This thesis is made available online and is protected by original copyright. Please scroll down to view the document itself. Please refer to the repository record for this item for information to help you to cite it. Our policy information is available from the repository home page.ThE DYNAMICS OF SW)P S1EWARD...»

«Education Policy in the Republic of Korea: Building Block or Stumbling Block? Jisoon Lee School of Economics Seoul National University Copyright © 2002 The International Bank for Reconstruction and Development/The World Bank 1818 H Street, N.W. Washington, D.C. 20433, U.S.A. The World Bank enjoys copyright under protocol 2 of the Universal Copyright Convention. This material may nonetheless be copied for research, educational, or scholarly purposes only in the member countries of The World...»

«How Chinese Innovation and Capital Market Liberalisation are Changing the Global Investment Landscape China is at the heart of the global economy and of investment markets. As the second largest economy in the world and the largest trading nation, China is the primary source of demand for many global companies. China is also home to the second largest capital market in the world behind the US. The lack of integration of these capital markets is a global economic anomaly. The...»

«Journal of Property Investment & Finance Emerald Article: A vision for valuation Barry Gilbertson, Duncan Preston Article information: To cite this document: Barry Gilbertson, Duncan Preston, (2005),A vision for valuation, Journal of Property Investment & Finance, Vol. 23 Iss: 2 pp. 123 140 Permanent link to this document: http://dx.doi.org/10.1108/14635780510699998 Downloaded on: 20-06-2012 References: This document contains references to 27 other documents Citations: This document has been...»

«Copyright Notice The material contained in this article is protected by U.S. Copyright and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. 7 page(s) will be printed. Back Record: 3 Why family businesses are best.; By: Brokaw, L.; Murphy, A., Inc., Mar92, Vol. 14 Issue 3, p72, 7p, 4c, 1bw Database: Academic Search Premier WHY...»

«SECURITIES AND EXCHANGE COMMISSION 17 CFR PARTS 210 and 240 [RELEASE NOS. 33-8829; 34-56203; File No. S7-24-06] RIN 3235-AJ58 Definition of the Term Significant Deficiency AGENCY: Securities and Exchange Commission. ACTION: Final rule. SUMMARY: We are defining the term “significant deficiency” for purposes of the Commission’s rules implementing Section 302 and Section 404 of the Sarbanes-Oxley Act of 2002. EFFECTIVE DATE: September 10, 2007. FOR FURTHER INFORMATION CONTACT: N. Sean...»

«Automatic Exchange of Information WHAT IT IS, HOW IT WORKS, BENEFITS, WHAT REMAINS TO BE DONE Automatic Exchange of Information WHAT IT IS, HOW IT WORKS, BENEFITS, WHAT REMAINS TO BE DONE ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT The OECD is a unique forum where governments work together to address the economic, social and environmental challenges of globalisation. The OECD is also at the forefront of efforts to understand and to help governments respond to new developments and...»

«INFORMS | Vijay Mehrotra interviews Fred Hillier, November 2, 2015 VIJAY I'm Vijay Mehrotra, professor of business analytics and information systems at the University MEHROTRA: of San Francisco. And I have the pleasure and privilege of having a chance to interview my dissertation advisor, Professor Frederick S. Hillier of Stanford University. Fred, I want to take you back to your days in high school. If I think about that, kind of 1950 to 1954, what was it like in those post World War II days,...»

«Permit Process for Signs What is a sign? Signs are deemed to be a name, identification, description, display, illustration, or character which is affixed to, or represented directly or indirectly upon a building, structure, or piece of land and which directs attention to an object, product, place, activity, person, institution, organization, or business. What is an awning? Awnings are a rooflike covering extending over or in front of a door or window as a shelter with signage included. What is...»

«Vijay Mehrotra Department of Finance and Quantitative Analytics School of Business and Professional Studies University of San Francisco 415-422-2257 / vmehrotra@usfca.edu EDUCATION Stanford University, Stanford, CA • PhD, Operations Research, 1992 Advisor: Professor Frederick S. Hillier Dissertation: “An Approximation Procedure for General Closed Multiclass Queueing Networks” • MS, Operations Research, 1989 Focus on Optimization, Stochastic Methods, Policy, and Applications Key Papers:...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.