The Role of Expertise in Risky Engineering Decisions

Sun, 17 Nov 2019 09:00:00 -0400

Engineering design decisions increasingly entail making risky decisions in complex situations. Here, we investigate the role of expertise in engineering problem solving. Specifically, we aim to answer the question of how engineering experts’ risk judgments affect design decisions, disentangling two concepts, expertise and knowledge. We draw upon Fuzzy Trace Theory (FTT), which makes specific predictions about how experts’ risk judgment drive design decisions. According to FTT, when making decisions, people rely on a continuum of mental representations, ranging from precise and quantitative verbatim representations of risk information to qualitative, categorical gist representations capturing the bottom line meaning of that information in context. The theory posits that individuals rely more on gist representations than verbatim representations – the so-called fuzzy processing preference. Furthermore, expertise enables individuals to retrieve the proper gist; consequently, this preference is more prevalent among experts, as a result of their developmental advancements in subject matter. To test FTT’s predictions in an engineering design decision context, we compared two samples: 1) 41 experts at NASA, and 2) 233 nonexperts recruited via Amazon Mechanical Turk (MTurk). Using a survey, we measured subjects’ risk judgments for a hypothetical mission design scenario that involved safety risk. Subjects in the MTurk sample were randomly assigned into three groups: a control group, a verbatim group that received technical verbatim information, and a gist+verbatim group that received the same information along with its bottom line meaning expressed as a gist. We applied Exploratory Factor Analysis (EFA) to our survey results, yielding three separate gists: categorical schedule risk, categorical safety risk, and categorical cost risk for the mission. Compared to the MTurk sample, NASA experts were more likely to agree that there was ‘some risk’ associated with the categorical cost and categorical schedule gists, whereas the MTurk sample judged the cost and schedule risks of the same scenario as ‘essentially nil’ (m=0.9, p<.001 for categorical cost; and m=0.76, p<.001 for categorical schedule cost risks). Within the MTurk sample, the we did not observe a significant difference between the verbatim group and the control group (m=0.27, p=ns for cost and m= 0.18, p=ns; for schedule risks). However, the gist+verbatim group’s risk perceptions became statistically indistinguishable from those of NASA experts (m=0.31, p=ns), highlighting the distinction between verbatim knowledge and gist. We also analyzed the relation between gist-based thinking and risky choice for the mission design. We next conducted a logistic regression analysis to examine the effects of the gist and verbatim factors on risky choice. We found categorical schedule (z=3.54, p<0.001) and categorical cost (z=-3.31, p<0.001) gist factors were significantly associated with risky choice; however, we did not find a significant association between the factor capturing verbatim representation and risky choice (z=1.35, p=ns for verbatim factor). NASA experts and the MTurk sample differed the most along the factor capturing categorical cost risk, which was the strongest predictor of risky choice. Next, within the NASA sample, we examined demographic factors most likely to predict categorical cost gist using a multi-way ANOVA. Surprisingly, the subject’s ability to recall a major failure significantly predicted categorical cost gist (F(1,31)= 7.77, p<0.001) whereas years in the aerospace industry did not (F(1,31)=0, p< 0.001). These results are consistent with the development of expertise based on feedback from the environment rather than simply number of years on the job. Overall, our results support FTT’s predictions regarding distinct gist and verbatim representations in an engineering context. Furthermore, as predicted, gist is more strongly associated with risky choice than is verbatim. Finally, our results suggest that expertise is informed by feedback from the environment, and therefore insightful.

What is the Gist of the Fourfold Pattern of Risk?

Fri, 15 Nov 2019 13:30:00 -0400

Prospect Theory (PT) predicts a “Fourfold Pattern of Risk Attitudes:” risk seeking for low-probability losses/high-probability gains, and risk aversion for high-probability losses/low-probability gains. Previously, subjects primarily exhibited this pattern for willingness-to-accept tasks. Per Fuzzy Trace Theory (FTT), subjects make binary choices using “some” vs. “none” gists, relying on verbatim representations only when necessary. FTT expects risk-aversion for gains and risk-seeking for losses for non-negligible probabilities, that truncating complements removing categorical contrasts attenuates this pattern, and that reliance on verbatim representations varies with metacognition. We tested these hypotheses by administering binary risky choices online, manipulating framing (gain vs. loss) and probability (low vs. high), holding gamble rewards constant. Gists were measured as “some” or “none”. We found risk-aversion for gains and risk-seeking for losses regardless of probability. Effect sizes varied as expected with individual differences, gists, and truncation. Contrary to PT, the fourfold pattern seems limited to verbatim comparisons.

Presentation at the TMP Consortium

Thu, 13 Jun 2019 10:50:00 -0400

How Do NASA Engineers Perceive MMOD Risks? A Fuzzy Trace Approach

Sat, 30 Jun 2018 00:00:00 +0000

Improving Expert Decision Making Processes on High Impact Low Probability Events

Mon, 27 Jun 2016 00:00:00 +0000

Decision-Making and Systems Architecture Laboratory

The Role of Expertise in Risky Engineering Decisions

What is the Gist of the Fourfold Pattern of Risk?

Presentation at the TMP Consortium

How Do NASA Engineers Perceive MMOD Risks? A Fuzzy Trace Approach

Improving Expert Decision Making Processes on High Impact Low Probability Events