The Role of Expertise in Risky Engineering Decisions

Engineering design decisions increasingly entail making risky decisions in complex situations. Here, we investigate the role of expertise in engineering problem solving. Specifically, we aim to answer the question of how engineering experts’ risk judgments affect design decisions, disentangling two concepts, expertise and knowledge. We draw upon Fuzzy Trace Theory (FTT), which makes specific predictions about how experts’ risk judgment drive design decisions. According to FTT, when making decisions, people rely on a continuum of mental representations, ranging from precise and quantitative verbatim representations of risk information to qualitative, categorical gist representations capturing the bottom line meaning of that information in context. The theory posits that individuals rely more on gist representations than verbatim representations – the so-called fuzzy processing preference. Furthermore, expertise enables individuals to retrieve the proper gist; consequently, this preference is more prevalent among experts, as a result of their developmental advancements in subject matter. To test FTT’s predictions in an engineering design decision context, we compared two samples: 1) 41 experts at NASA, and 2) 233 nonexperts recruited via Amazon Mechanical Turk (MTurk). Using a survey, we measured subjects’ risk judgments for a hypothetical mission design scenario that involved safety risk. Subjects in the MTurk sample were randomly assigned into three groups: a control group, a verbatim group that received technical verbatim information, and a gist+verbatim group that received the same information along with its bottom line meaning expressed as a gist. We applied Exploratory Factor Analysis (EFA) to our survey results, yielding three separate gists: categorical schedule risk, categorical safety risk, and categorical cost risk for the mission. Compared to the MTurk sample, NASA experts were more likely to agree that there was ‘some risk’ associated with the categorical cost and categorical schedule gists, whereas the MTurk sample judged the cost and schedule risks of the same scenario as ‘essentially nil’ (m=0.9, p<.001 for categorical cost; and m=0.76, p<.001 for categorical schedule cost risks). Within the MTurk sample, the we did not observe a significant difference between the verbatim group and the control group (m=0.27, p=ns for cost and m= 0.18, p=ns; for schedule risks). However, the gist+verbatim group’s risk perceptions became statistically indistinguishable from those of NASA experts (m=0.31, p=ns), highlighting the distinction between verbatim knowledge and gist. We also analyzed the relation between gist-based thinking and risky choice for the mission design. We next conducted a logistic regression analysis to examine the effects of the gist and verbatim factors on risky choice. We found categorical schedule (z=3.54, p<0.001) and categorical cost (z=-3.31, p<0.001) gist factors were significantly associated with risky choice; however, we did not find a significant association between the factor capturing verbatim representation and risky choice (z=1.35, p=ns for verbatim factor). NASA experts and the MTurk sample differed the most along the factor capturing categorical cost risk, which was the strongest predictor of risky choice. Next, within the NASA sample, we examined demographic factors most likely to predict categorical cost gist using a multi-way ANOVA. Surprisingly, the subject’s ability to recall a major failure significantly predicted categorical cost gist (F(1,31)= 7.77, p<0.001) whereas years in the aerospace industry did not (F(1,31)=0, p< 0.001). These results are consistent with the development of expertise based on feedback from the environment rather than simply number of years on the job. Overall, our results support FTT’s predictions regarding distinct gist and verbatim representations in an engineering context. Furthermore, as predicted, gist is more strongly associated with risky choice than is verbatim. Finally, our results suggest that expertise is informed by feedback from the environment, and therefore insightful.