FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:   || 2 |

«Prepared for the U.S. Department of Energy under Contract DE-AC05-76RL01830 CCSI Risk Estimation: An Application of Expert Elicitation DW Engel AC ...»

-- [ Page 1 ] --


Prepared for the U.S. Department of Energy

under Contract DE-AC05-76RL01830

CCSI Risk Estimation: An Application

of Expert Elicitation

DW Engel

AC Dalton

October 2012


This report was prepared as an account of work sponsored by an agency of the

United States Government. Neither the United States Government nor any agency

thereof, nor Battelle Memorial Institute, nor any of their employees, makes any

warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or Battelle Memorial Institute. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.


operated by BATTELLE for the


under Contract DE-AC05-76RL01830 Printed in the United States of America Available to DOE and DOE contractors from the Office of Scientific and Technical Information, P.O. Box 62, Oak Ridge, TN 37831-0062;

ph: (865) 576-8401 fax: (865) 576-5728 email: reports@adonis.osti.gov Available to the public from the National Technical Information Service, U.S. Department of Commerce, 5285 Port Royal Rd., Springfield, VA 22161 ph: (800) 553-6847 fax: (703) 605-6900 email: orders@ntis.fedworld.gov online ordering: http://www.ntis.gov/ordering.htm This document was printed on recycled paper.

(9/2003) PNNL-21785 CCSI Risk Estimation: An Application of Expert Elicitation DW Engel AC Dalton October 2012 Prepared for the U.S. Department of Energy under Contract DE-AC05-76RL01830 Pacific Northwest National Laboratory Richland, Washington 99352 Contents

1.0 Background

2.0 Expert Elicitation Background and Significance

3.0 CCSI Element 7 Expert Elicitation Approach

4.0 Status Report and Next Steps

5.0 Reference

Figures Figure 1 CCSI Decision-Making Framework

Figure 2 CCSI Risk Factor Expert Elicitation Approach

–  –  –

Another focal point of the expert elicitation literature intersects with psychological research on heuristics and biases, and explores the effects of human cognitive patterns on elicitation. Heuristics refer to the effort-reduction methods employed by decision makers to alleviate judgmental burdens, and judgment biases refer to prejudicial mental behavior that undermines rational decision making (Arnott 1998; Shah and Oppenheimer, 2008; Simon 1990; Tversky and Kahneman 1974). Prominent classes of heuristics include representativeness (assessments based on similarity between events), availability (assessments based on one’s ability to recall past events), and adjustment and anchoring (assessments bounded by initial judgment) (Tversky and Kahneman 1974). Remus and Kottemann (1986) grouped heuristics based on their relevance to data presentation and information processing (Remus and Kottemann 1986);

Hogarth (1987) categorized heuristics based on the four stages in decision-making ranging from “acquisition, processing, output, to action” (Arnott 1998, p. 28; see Arnott 1998 for a comprehensive review). In probabilistic risk elicitation, the most widely observed effects of judgmental biases are systematic under- or over-estimation and overconfidence (Mosleh et al. 1988). Approaches to reducing systematic bias and overconfidence include calibration training (Lichtenstein et al. 1982) and the encouragement of considering both affirmative and contradicting evidence in experts’ decision making (Spetzler and von Holstein 1975). Moreover, improving the formulation of decision problems may prove useful for enhancing the quality of elicitation. For example, problem decomposition, the idea of breaking a decision problem into multiple smaller decision problems, allows experts to assess probabilities for each component of the decision question, which is then synthesized into an overall probability estimate.

Additionally, transforming a probability elicitation question into a less statistically esoteric format, such as a betting game or pairwise comparison of decision options, may be more appropriate for experts with limited statistical knowledge.

Another key component in the research on expert elicitation documents techniques of aggregating input from multiple experts (Clemen and Winkler 1990; Winkler 1981). Generally, combination can be implemented behaviorally and mathematically. In the absence of expert interaction, an opinion pool can help aggregate assessments. Elicitors can assign different weights to different experts, especially according to the accuracy of their assessments from training/calibration sessions. For a detailed discussion on mathematical combination methods, see Clemen and Winkler (1999). If interaction is permitted among experts, the elicitor can distribute the assessments among the experts to achieve a group consensus. An example of this approach is the Delphi technique, which seeks to derive a consensus through iterative deliberation among all the experts. Note, however, the interactive group consensus method may be influenced by dominant personalities in the group. In terms of performance, some evidence suggests that mathematical aggregation outperforms behavioral aggregation (Cooke and Goossens 2004; Clemen and Winkler 1999; Hora 2009) while others suggest the determination of aggregation approach should be dictated by the decision tasks (DeWispelare et al. 1995).

Expert elicitation has been applied to a broad range of domains, ranging from environmental protection (Salvi and Gaston 2004), infrastructure vulnerability (Vrijling et al. 2004), accident consequence modeling (Cooke and Goossens 2000), nuclear waste regulations (DeWispelare et al. 1995), medical diagnosis (Lau and Leung 1999) to carbon capture and storage technology (Chan et al. 2011). Regardless of the variations in elicitation approaches and the application domains, for expert elicitation to maintain its scientific rigor as a consensus methodology, it is imperative that the elicitation methods, processes, and tools are transparent and free of biases; the selection of expertise is based on relevant expertise to ensure proper domain coverage while minimizing bias; and the analysis approach and elicitation results are repeatable (Cooke and Goossens 2000).

CCSI Element 7 Expert Elicitation Approach 3.0 The fundamental purpose of CCSI expert elicitation activities is to enable and facilitate the sharing of relevant, diverse, and highly specialized knowledge to collectively inform the risks associated with the development and deployment of carbon capture technologies. Engagement of the CCSI risk analysis team with experts for risk elicitation is critical. First, carbon capture technology is relatively new and highly complex with a host of potentially latent risk factors that might compromise the success of the technology if left unaddressed. Thus, leveraging the diverse expertise on the project team can help identity and address these latent risk factors in addition to the risks captured by the financial risk, technical risk, and technical maturity assessments. Furthermore, given the focus on simulation and modeling, some system features might not be modeled due to their fine granularity. These under-specified system characteristics might present potentially significant risks, thus demanding a finer-grained risk analysis of the modeling assumptions, parameters, operational risks, and those characteristics that cannot be captured by formal modeling but nonetheless have operational impact. Additionally, the engagement of experts and stakeholders through elicitation early in the technology development process provides an excellent opportunity for continuous and iterative risk communication to help them better understand and address risks as the technology development unfolds, resulting in greater risk reduction and technology acceptance by the stakeholders.

The CCSI Element 7 is developing an elicitation approach that combines both qualitative risk rankings and quantitative risk assessments to capture the under-specified and not-yet-modeled risk factors in carbon capture technologies. Figure 2 provides a step-by-step illustration of the elicitation activities.

Figure 2 CCSI Risk Factor Expert Elicitation Approach The elicitation plan will build on the previous solid sorbent CO2 capture technology risk elicitation framework (see Appendix ). This framework identified five main risk drivers, including (1) knowledge maturity and development capacity, (2) uncertainties related to financial/regulatory/public opinion, (3) technical performance in testing and deployment, (4) costs and efficiency, and (5) compatibility between simulation tools and physical systems. An extensive list of potential risk factors (totaling 98) was organized into three major risk dimensions: technical risk, economic risk, and regulatory/public risk. Each risk category was further operationalized by multiple sub risk factor categories as shown in the following


–  –  –

Using this risk factor list, the CCSI Element 7 team will organize a qualitative risk factor identification exercise with a panel of experts representing all the task teams within the initiative as well as from industry partners. This exercise will accomplish two objectives: (1) to eliminate irrelevant risk factors or risk categories; and (2) to incorporate relevant factors/risk categories that are missing from the list. With a streamlined list of potential risk factors, the experts will participate in a risk ranking elicitation where they will be asked to rank each risk factor on a numeric scale. The risk rankings will produce relevance weights for each risk factor, and help sharpen the focus of the follow-up elicitation effort. Once the priority risk factors are identified based on rankings, the elicitation team will develop contextualized risk statements for these factors and these statements will be pooled to produce a new CCSI risk factor expert elicitation instrument to be used in a systematic quantitative risk assessment. Risk is characterized by the severity of potentially adverse events and the likelihood of the occurrences of these events. To that end, we will distribute the instrument and ask expert respondents to provide a likelihood assessment and related magnitude assessment for each risk statement on a numerical scale (for an example instrument, see Hunton and Williams 2008). Mathematical aggregation algorithms will be employed to produce a combined probability distribution for each risk factor (Clemen and Winkler 1999). Next, the Element 7 team will create future risk scenarios using the identified risk factors to forecast failure rates over varying time horizons. For example, two programmatic performance criteria, a minimum 90% carbon capture efficiency rate; and a 30% cost increase ceiling for carbon capture technology in long-term operations, can be used in risk scenario development to compare the current and the desired level of technical capabilities. Again, experts will each provide a probabilistic risk forecast for these scenarios and their input will be aggregated. The resulting probability distributions from the scenarios as well as from the elicitation questionnaire will be integrated into the overall risk analysis framework to improve the quality of the risk assessment of carbon capture technology as a whole. A final step in this risk assessment effort requires the development of a targeted risk mitigation strategic plan to address the risks and reduce their adverse effects on CCSI program outcomes. It is important to note that the CCSI expert elicitation effort will be iterative in nature, and will closely follow the progress in carbon capture technology simulation and testing.

Status Report and Next Steps 4.0 The success of expert elicitation critically depends on the active collaboration across CCSI task teams and the support from the external stakeholders. To enhance the collaboration within CCSI for the expert elicitation, effort is under way to engage scientists from other Element teams to help modify and refine the technology maturity levels (TRL) questionnaire. The CCSI E7 Team has created a wiki-based elicitation and model development tool to enable novel, probabilistic technology maturity assessment by CCSI project teams (E1, E2, E3, and E97) as well as external experts. This collaborative effort will pave the way for more elaborate and involved elicitation activities described in this report. With respect to engagement with external experts, the Element 7 team conducted a site visit to Eastman Chemical Company in August 2012. This visit offered an invaluable learning opportunity to understand how risk analysis is routinely performed in business organizations. It also signals the establishment of a solid CCSI-industry partnership crucial for expert elicitation tasks in the near future. It is anticipated that in the coming program year, greater attention will be invested in the elicitation-driven risk assessment to compliment the financial and technical risk analyses and technology maturity risk modeling that are the focus of the present program year. If successful, the expert-driven risk elicitation will contribute to a better understanding of the potential vulnerabilities and risks in developing, testing, deploying, and commercializing carbon capture technologies, and provide mitigation strategies to reduce risk and uncertainty, and help the technology reach the stage of commercialization more reliability and rapidly.

Reference 5.0 Arnott, D. 1998. A Taxonomy of Decision Biases. Unpublished Technical Report. Monash University School of Information Management & Systems.

Pages:   || 2 |

Similar works:

«PROSPECTUS ALFM Euro Bond Fund 17/F, BPI Head Office Building, Ayala Avenue corner Paseo de Roxas, Makati City 1226 Tel No. (02) 845-5424 (An Open-end investment company organized under Philippine Laws) An Offer of up to the Number of Authorized Shares of ALFM Euro Bond Fund at an Offer Price of Net Asset Value per Share on the date of subscription ALFM EURO BOND FUND Number of Authorized Shares 80,000 Minimum Initial Investment EUR 500.00 PAR value PhP 10,000.00 Securities will be traded over...»

«Broker-Dealer Litigation 2013 Annual Survey Editors Steven M. Malina Scott E. Rahn GREENBERG TRAURIG, LLP Chicago, Illinois Los Angeles, California © March 2014 This Survey identifies significant court decisions and administrative proceedings before the SEC and FINRA, which involve issues relevant to broker-dealers and investment banking firms during the year 2013. Although the Survey does not purport to necessarily be comprehensive, it is intended to include all notable decisions and...»

«            ...»

«GUÍA DEL ESTADO DE ILLINOIS SOBRE RESCATE FINANCIERO Y BANCARROTA Escrita y Publicada con fines educacionales por RICHARD FONFRIAS, J. D. Abogado especializado en Rescate Financiero y Bancarrota Solución a sus problemas financieros. Protección a su paz mental. Fundador y Socio-director de FONFRIAS LAW GROUP, LLC  Defensa en demandas judiciales de acreedores.  Defensa en demandas de tarjetas de crédito.  Defensa en ejecuciones bancarias de su casa.  Reducción de deudas y...»

«AELE Home Page — Publications Menu — Seminar Information Cite as: 2011 (7) AELE Mo. L. J. 201 ISSN 1935-0007 Employment Law Section July 2011 Pre-employment Polygraph Examinations of Public Safety Applicants Contents Part 1 (This Issue) • Introduction • Validity of Polygraph Examinations • General Laws on Employment Polygraph Testing • Questions With a Discriminatory Impact • Polygraphs and Disability Discrimination • Resources and References Part 2 (August Issue) • Screening...»

«ENFORCING EQUAL EMPLOYMENT OPPORTUNITIES IN CHINA XUN ZENG I. INTRODUCTION: MR. JIANG TAO'S STORY' In December 2001, Jiang Tao, a law student at Sichuan University, read an advertisement, which indicated that the Chengdu Branch of People's Bank of China (PBOC), was hiring new staff members. The advertisement contained a minimum height requirement: male candidates were required to be at least 168cm (5ft 6in) tall and females 155cm (5ft 3in).3 Such height requirements are very common in China;...»

«CESTAT RULING (SERVICE TAX) 2015-TIOL-2334-CESTAT-MUM M/s Dai Ichikarkaria Ltd Vs CCE (Dated: October 20, 2015) ST Department is not supposed to know each and every declaration made outside the Central Excise law ST Registration Certificate, ST Returns and Audit Report did not reveal that CENVAT Credit is being utilized to pay service tax on the renting of property located at Mumbai No basis for setting aside demand on basis of limitation ROM dismissed: CESTAT [para 2] Also see analysis of the...»

«Licence Applications must be returned to: Road Transport Operator Licensing Unit Department of Transport, Tourism and Sport Clonfert House, Bride Street, Loughrea, Co. Galway A GUIDE TO ROAD PASSENGER TRANSPORT OPERATOR LICENSING Please read this guide before completing the Licence Application Form This is a guide only and as such does not attempt to cover every issue in relation to your application for a Road Passenger Transport Operator Licence. It does not constitute a legal interpretation...»


«16th Annual National Prosecutors’ Conference 10 October 2015, Dublin Castle Conference Centre Provocation; still a graveyard for trial Judges 1 Brendan Grehan SC, Law Library Introduction 1. A series of recent appellate decisions overturning murder convictions have once again underlined provocation’s pole position as the greatest pitfall for judges directing juries in murder trials. The Court of Appeal has even gone so far in one judgment as to suggest that when it arises, a trial judge...»

«BASIS OF FINANCIAL ARITHMETIC FROM THE VIEWPOINT OF THE UTILITY THEORY Krzysztof Piasecki1 The main goal of this paper is presentation a modern axiomatic approach to financial arithmetic. At the first, the axiomatic financial arithmetic theory was proposed by Peccati who has introduced the axiomatic definition of the future value. This theory has been extensively developed in past years. Proposed approach to financial arithmetic is based on the financial flow utility concept. This utility...»

«PLAINTIFF DUE PROCESS RIGHTS IN ASSERTIONS OF PERSONAL JURISDICTION R. D. REES* Personaljurisdiction proceedingsformally focus on the defendant's liberty interest in avoiding the reach of an overextending court. In this Note, R. D. Rees argues that such an approach may fail to provide the plaintiff due process. The laws of various jurisdictionsconvert a single set of underlying facts into distinct causes of action, and the Supreme Court understands these statutory programs to create property...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.