«Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? Erik Brynjolfsson, MIT and NBER Lorin Hitt, University of ...»
Strength in Numbers:
How Does Data-Driven Decisionmaking Affect Firm Performance?
Erik Brynjolfsson, MIT and NBER
Lorin Hitt, University of Pennsylvania
Heekyung Kim, MIT
We examine whether performance is higher in firms that emphasize decisionmaking based on data
and business analytics (which we term a data-driven decisionmaking approach or DDD). Using detailed
survey data on the business practices and information technology investments of 179 large publicly traded firms, we find that firms that adopt DDD have output and productivity that is 5-6% higher than what would be expected given their other investments and information technology usage. Using instrumental variables methods, we find evidence that these effects do not appear to be due to reverse causality.
Furthermore, the relationship between DDD and performance also appears in other performance measures such as asset utilization, return on equity and market value. Our results provide some of the first large scale data on the direct connection between data-driven decisionmaking and firm performance.
Acknowledgements: We thank Andrew McAfee, Roger Robert, Johnson Sikes and participants at the Workshop for Information Systems and Economics and participants at the 9th Annual Industrial Organization Conference for useful comments and the MIT Center for Digital Business for generous financial support.
1. Introduction How do firms make a better decision? Today, organizational judgment is in the midst of a fundamental change - from a reliance on a leader’s “gut instinct” to increasingly data-based analytics. At the same time, we have been witnessing a data revolution; firms gather extremely detailed data and propagate knowledge from their consumers, suppliers, alliance partners, and competitors. In particular, since 1993, most large companies have invested in large enterprise resource planning (ERP), Supply Chain Management (SCM), Customer Relationship Management (CRM) and similar enterprise information technology (Aral et al., 2006; McAfee, 2002). These systems collect terabytes of detailed data on operations, suppliers, customers and other aspects of the businesses, increasing the amount of data by 10-fold to 1000-fold. Mobile phones, automobiles, factory automation systems and other devices are routinely instrumented to generate streams of data on their activities, making possible an emerging field of “reality mining” to analyze this information (Pentland and Pentland, 2008). Manufacturers and retailers use RFID tags to deliver terabits of data on inventories and supplier interactions and then feed this information into analytical models to optimize and reinvent their business processes. Similarly, clickstream data and keyword searches collected from websites generate a plethora of data, making visible interactions and patterns that previously could only be guessed at.
According to economic theory, as information becomes more fine-grained and current, decisionmakers should optimally put more weight on it and the overall quality of decision should improve on average, changing from intuitive management to more numbers-driven decision-making. As a Microsoft researcher memorably put it, objective, fine-grained data are now replacing HiPPOs (Highest Paid Person’s Opinions) as the basis for decision-making at more and more companies (Kohavi et al., 2009). Managers conduct active experiments with their new business ideas and base their decisions on scientifically valid data. It is common for companies to purchase a “business intelligence” module to try to make use of the flood of data that they now have on their operations. From banks such as PNC, Toronto-Dominion, and Wells Fargo to retailers such as CKE Restaurants, Famous Footwear, Food Lion, Sears, and Subway to online firms such as Amazon, eBay, and Google, firms test many business ideas through a randomized test before launch, called as “information-based strategy” (Davenport, 2009). The innovative process in an online business is now being transformed by the information-based strategy.
While there is a great deal of anecdotal evidence of firms’ using data to gain a competitive edge in the business press and popular books (See e.g. Davenport and Harris, 2007; Ayres, 2008; Loveman, 2003), there has been virtually no systematic data analysis of the productivity effects of data-driven decisionmaking (or DDD) using statistical methods. We seek to address this gap by examining in detail business practices and the information technology investments of 179 publicly traded large firms in the US. We find that DDD can explain a 5-6% increase in their output and productivity, beyond what can be explained by traditional inputs and IT usage. DDD is also associated with significantly higher profitability and market value. While these correlations are consistent with the case evidence, as well as economic theory, econometrics alone cannot rule out the possibilities of reverse causality or omitted variables bias. However, our basic findings remain robust when we use instrumental variables and explore a number of alternative variables that might explain our results. To the best of our knowledge, this is the first study to report a large scale econometric analysis of the relationship between DDD and firm performances.
2. Theory and Literature
Information technology and productivity Modern theories of the value of information typically begin with the seminal work of Blackwell (Blackwell, 1953). Blackwell’s theorem states that decisionmakers observe signals correlated to the state of nature prior to their choice of action and hence “update” the probability distribution before the “optimal” action is chosen. In this framework the value of information (in terms of expected utility) is always positive. Since the mid 90s, many researchers have reported that IT investments indeed increase productivity (e.g. Brynjolfsson and Hitt, 1993; Oliner and Sichel, 2000; Jorgensen and Stiroh, 2000;
Kohli and Devaraj, 2003; Aral, Brynjolfsson and Wu, 2006; Barua et al., 1995). While a clear positive relationship between performance and IT investments has been convincingly demonstrated, there is a great deal of individual variation in firms’ success with information technology. A variety of factors including the type of IT, the usage, management practices, and organizational structure, as well as the industry environments affect the variation (e.g. Bresnahan et al., 2002; Devaraj and Kohli, 2003). Our study is aligned with the stream of this literature, showing that, after controlling for IT use as well as other traditional inputs and industry, DDD appears to differentiate high-performing firms from others.
Some case studies and a limited analysis on the relationship between DDD and firm performance have been featured in the business press and popular books. For example, Loveman (2003), the CEO of Harrah’s Entertainment, states that use of database and decision-science-based analytical tools was the key to his firm’s success. Davenport and Harris (2007) have listed many firms in a variety of industries that gained competitive advantage through use of data and analytical tools for decisionmaking: Procter & Gamble in consumer products, Capital One in financial services, JCPenny in retail, to name a few. They also show a high correlation between higher levels of analytics use and robust 5-year compound annual growth rate from their survey of 32 organizations. A more recent study (Lavalle et al., 2010) has reported that organizations using business information and analytics to differentiate themselves within their industry are twice as likely to be top performers as lower performers. Our study advances the understanding about the relationship between DDD and firm performances by applying a standard econometric method to survey and financial data on publicly traded large 179 firms. In addition, using instrumental variables, we address a potential problem of reverse causality and omission variable bias and show a causal relationship between DDD and firm performance.
Information technology, profitability, and organizational learning Our study is also related to the literature of organizational learning. Knowledge properties affect the rate of knowledge creation, accumulation, retention, and transfer. Research on the nature of knowledge characterizes knowledge with the degree to which it is codified or tacit (Polany, 1966; Nelson and Winter, 1982a). Codified knowledge refers to knowledge that is transmittable in formal, symbolic language, while tacit knowledge is hard to articulate and acquired through experience (Polany, 1966). Other researchers characterize non-transferrable information as “sticky”( Szulanski, 2000; Von Hippel, 1994).
Numerous researchers have pointed out that tacit knowledge is more difficult to transfer than codified knowledge (e.g., Zander and Kogut, 1995; Nadler et al., 2003), because the transfer of tacit knowledge requires proximity, interpersonal interaction, mentorship, apprenticeship, and/or repeated practice over a period of time (Davenport and Prusak, 2000; Hansen, 1999; Szulanski, 1996; Spender, 1996).
An important research question in the aspect of knowledge transfer is whether the transfer of knowledge occurs only within an organization or across the organization and how firms keep knowledge transfer within their firm boundaries while deterring imitation by competitors. Researchers report that technology adoption in a form of codified knowledge tends to equalize organizational performances (Edmondson et al., 2003). Consistent with this research, Carr (Carr, 2003) put forth that “IT doesn’t matter” because IT became a commodity that any firm can buy and use. On the other hand, other researchers report that differences in firm performances tend to increase in IT-intensive industries (Brynjolfsson et al., 2006) and firms take a strategy to balance transferring knowledge internally with keeping the knowledge in a form that it is hard for other organizations to imitate (Rivkin, 2001). Many researchers in the literature of IT and productivity also report that differences in organizational structures and human capital would deter imitation of business processes across firm boundary while replication of business processes within firm boundary becomes rapid in a codified platform (or IT enterprise platform) (e.g.. McAfee and Brynjolfsson, 2008). It is still unclear whether codified knowledge can be a competitive advantage for a firm or enable imitation by competitors and lose the competitive advantage.
We examine the impact of DDD on profitability as well as productivity. While the productivity measures the production of outputs for a given quantity of inputs and DDD, the business profitability indicates whether firms can gain competitive advantage and earn higher profits by employing DDD than they would have earned otherwise. Extending previous research on the impact of IT investment (e.g., Hitt and Brynjolfsson, 1996; Aral, Brynjolfsson and Wu, 2006) on profitability, we examine the impact of DDD on profitability after controlling for IT use. Similarly to the impact of DDD on productivity, the impact of DDD on profitability was also estimated by using instrumental methods in order to address a potential problem of omission variable bias and reverse causality.
Information technology and market value The final performance metric we examined is the total market value of the firm. Accounting measures such as return on assets, return on equity, and return on sales have some weaknesses in capturing firm performance: 1) they typically only reflect past information and are not forward looking;
2) they are not adjusted for risk; 3) they are distorted by temporary disequilibrium effects, tax laws, and accounting conventions; 4) they do not capture the value of intangible assets; 5) they are insensitive to time lags necessary for realizing the potential of organizational change. Financial market-based measures can be a useful alternative to these accounting measures. In particular, variants on Tobin’s q ratio, defined as the ratio of the stock market valuation of a firm to its measured book value, has been used as measure of business performance (Chen and Lee, 1995), intangible assets (Hall, 1993;Hirschey, 1982), technological assets (Griliches, 1981), brand equity (Simon and Sullivan, 1993), and other characteristics of firms.
In the context of IT-investments, market value has been used to estimate the value of intangible assets such as organizational capital, often associated with IT assets (e.g. Brynjolfsson et al., 2002;Saunders and Brynjolfsson, 2010). The underlying principle is that the total value of financial claims on the firm should be equal to the sum of the firm’s assets (Baily et al., 1981;Hall et al., 2000;Hall, 2001). Therefore, the value of intangible assets can be estimated by subtracting the value of other tangible inputs from the sum of financial claims. Other researchers used Tobin’s q to examine the effects of information technology on firm performance (Bharadwaj et al., 1999). Related work found that e-commerce announcements (Subramani and Walden, 2001) and Internet channel addition (Geyskens et al., 2002) were correlated with changes in market value.
We build in particular on the intangible assets literature and model the value of financial claims against the firm, MV, as the sum of each of its n assets, A.
= What the above model formulates is that the market value of a firm is simply equal to the current stock of its capital assets when all assets can be documented and no adjustment costs are incurred in making them fully productive. For example, Google is valued at approximately $190 billion but the company lists $40 billion in total assets on its balance sheet. The difference, $150 billion, can be interpreted as the sum of its intangible assets. We consider three classes of intangibles assets in this paper: those associated with IT, with advertising, and with R&D. For IT-related intangibles, we use data on the number of IT employees.
The firm-specific human and organization capital of these employees, especially in knowledge-intensive firms, could be an important asset of their firm and may be reflected in the valuation of the firm. The value associated with IT-employee would thus reflect the value of IT-intangibles of the firm.
The second model is to introduce DDD in the basic model as an interaction term with the other
asset as the following (Brynjolfsson, Hitt and Yang, 2002):