Recent & Upcoming Talks on Decision-Making and Systems Architecture Laboratory

The Role of Expertise in Risky Engineering Decisions

Sun, 17 Nov 2019 09:00:00 -0400

Engineering design decisions increasingly entail making risky decisions in complex situations. Here, we investigate the role of expertise in engineering problem solving. Specifically, we aim to answer the question of how engineering experts’ risk judgments affect design decisions, disentangling two concepts, expertise and knowledge. We draw upon Fuzzy Trace Theory (FTT), which makes specific predictions about how experts’ risk judgment drive design decisions. According to FTT, when making decisions, people rely on a continuum of mental representations, ranging from precise and quantitative verbatim representations of risk information to qualitative, categorical gist representations capturing the bottom line meaning of that information in context. The theory posits that individuals rely more on gist representations than verbatim representations – the so-called fuzzy processing preference. Furthermore, expertise enables individuals to retrieve the proper gist; consequently, this preference is more prevalent among experts, as a result of their developmental advancements in subject matter. To test FTT’s predictions in an engineering design decision context, we compared two samples: 1) 41 experts at NASA, and 2) 233 nonexperts recruited via Amazon Mechanical Turk (MTurk). Using a survey, we measured subjects’ risk judgments for a hypothetical mission design scenario that involved safety risk. Subjects in the MTurk sample were randomly assigned into three groups: a control group, a verbatim group that received technical verbatim information, and a gist+verbatim group that received the same information along with its bottom line meaning expressed as a gist. We applied Exploratory Factor Analysis (EFA) to our survey results, yielding three separate gists: categorical schedule risk, categorical safety risk, and categorical cost risk for the mission. Compared to the MTurk sample, NASA experts were more likely to agree that there was ‘some risk’ associated with the categorical cost and categorical schedule gists, whereas the MTurk sample judged the cost and schedule risks of the same scenario as ‘essentially nil’ (m=0.9, p<.001 for categorical cost; and m=0.76, p<.001 for categorical schedule cost risks). Within the MTurk sample, the we did not observe a significant difference between the verbatim group and the control group (m=0.27, p=ns for cost and m= 0.18, p=ns; for schedule risks). However, the gist+verbatim group’s risk perceptions became statistically indistinguishable from those of NASA experts (m=0.31, p=ns), highlighting the distinction between verbatim knowledge and gist. We also analyzed the relation between gist-based thinking and risky choice for the mission design. We next conducted a logistic regression analysis to examine the effects of the gist and verbatim factors on risky choice. We found categorical schedule (z=3.54, p<0.001) and categorical cost (z=-3.31, p<0.001) gist factors were significantly associated with risky choice; however, we did not find a significant association between the factor capturing verbatim representation and risky choice (z=1.35, p=ns for verbatim factor). NASA experts and the MTurk sample differed the most along the factor capturing categorical cost risk, which was the strongest predictor of risky choice. Next, within the NASA sample, we examined demographic factors most likely to predict categorical cost gist using a multi-way ANOVA. Surprisingly, the subject’s ability to recall a major failure significantly predicted categorical cost gist (F(1,31)= 7.77, p<0.001) whereas years in the aerospace industry did not (F(1,31)=0, p< 0.001). These results are consistent with the development of expertise based on feedback from the environment rather than simply number of years on the job. Overall, our results support FTT’s predictions regarding distinct gist and verbatim representations in an engineering context. Furthermore, as predicted, gist is more strongly associated with risky choice than is verbatim. Finally, our results suggest that expertise is informed by feedback from the environment, and therefore insightful.

What is the Gist of the Fourfold Pattern of Risk?

Fri, 15 Nov 2019 13:30:00 -0400

Prospect Theory (PT) predicts a “Fourfold Pattern of Risk Attitudes:” risk seeking for low-probability losses/high-probability gains, and risk aversion for high-probability losses/low-probability gains. Previously, subjects primarily exhibited this pattern for willingness-to-accept tasks. Per Fuzzy Trace Theory (FTT), subjects make binary choices using “some” vs. “none” gists, relying on verbatim representations only when necessary. FTT expects risk-aversion for gains and risk-seeking for losses for non-negligible probabilities, that truncating complements removing categorical contrasts attenuates this pattern, and that reliance on verbatim representations varies with metacognition. We tested these hypotheses by administering binary risky choices online, manipulating framing (gain vs. loss) and probability (low vs. high), holding gamble rewards constant. Gists were measured as “some” or “none”. We found risk-aversion for gains and risk-seeking for losses regardless of probability. Effect sizes varied as expected with individual differences, gists, and truncation. Contrary to PT, the fourfold pattern seems limited to verbatim comparisons.

Invited Speaker at the London School of Hygeine and Tropical Medicine

Wed, 13 Nov 2019 10:45:00 +0000

Dr. Broniatowski gave an invited talk titled “Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate” at the London School of Hygiene and Tropical Medicine’s Advancing Research on Social Media and Vaccine Confidence meeting.

Invited Panelist at the 5th Annual Pandemic Policy Summit

Mon, 11 Nov 2019 11:00:00 -0500

Dr. Broniatowski gave an invited panel presentation at the Fifth Annual Pandemic Policy Summit: Pandemic & Biosecurity Threats in the 21st Century at the The Bush School of Government & Public Service at Texas A&M University.

Invited Panelist at APHA 2019

Tue, 05 Nov 2019 15:00:00 -0400

This presentation discusses how malicious actors on social media use health communication to carry out several hidden agendas. Among these are state-sponsored “trolls” that use vaccination as a wedge issue to promote discord.

Machine Learning Classifiers for Socio-Demographics of Social Media Users: Limitations and Possibilities

Tue, 05 Nov 2019 13:00:00 -0400

Background: Social media analyses of health behaviors, such as vaccination, have shown significant promise for surveillance. However, existing research, both quantitative and qualitiative, suggests that health behaviors varies significantly with socio-demographic factors, particularly race, ethnicity, and gender. Often, these factors are not explicitly disclosed on social media platforms and must instead be inferred, raising the possibility of methodological bias. This session examines existing tools and methodologies used to infer socio-demographics of social media users. Methods: We survey several socio-demographic classifiers that work with Twitter and Reddit data. These classifiers use features including users’ language patterns, follower behaviors, and choice of names. These classifiers predict labels including users’ gender, race and ethnicity, or filter out social media accounts run by organizations.

Results: We explain how the data for these classifiers is collected, how the classification models are trained, and how they could be applied to public health research. We in particular discuss the limitations that these classifiers have, including possible methodological bias introduced by the challenges of large-scale data collection of social media users’ demographic information.

Discussion: Health behaviors vary with socio-demographic factors, which are challenging to measure on social media platforms. Machine learning classification of socio-demographics is possible, but requires interdisciplinary considerations.

Translating Trust in Vaccines from Surveys to Twitter

Tue, 05 Nov 2019 13:00:00 -0400

background: Trust is considered an important factor in vaccine-related decisions. Traditional surveys measure this with thoughtful and rigorous research, but that can be slow and expensive compared to social media analyses. Social media research currently lags behind surveys in terms of rigor because, among several reasons, it requires measuring via inference instead of self-report. There is an opportunity to innovate to leverage strengths of both data sources. objective: This session will exemplify how to combine social media and survey data when characterizing online discussion on trust in vaccines to leverage strengths of both data sources while mitigating weaknesses.

methods: We build a factor analysis linking social media messages with traditional survey questions conveying constructs related to trust in vaccines. Specifically, we recruit respondents to both answer survey questions and indicate agreement with (and whether they would share) tweets related to trust in vaccines. We assemble a set of scaled tweets that convey constructs in relevant survey research, such as trust in government or vaccine uptake. We perform a factor analysis to identify significant relationships between survey answers and agreement with tweets. These significant factors provide a means to translating from Twitter to surveys. We randomly sample Twitter data, apply this translation, and statistically characterize Twitter discussion as it relates to traditional survey results.

results: We will show significant relationships between constructs on surveys and constructs on Twitter, and we will show how these relationships can be used to characterize social media discussions on trust in vaccines.

discussion: This type of analysis can be used to quantify associations and relationships across different data sources like social media and surveys as constructs are conveyed. Relating more free-form social media to traditional surveys whose set of responses is more bounded would facilitate interpretation along important constructs related to trust in vaccines.

Invited Panelist at UCLA's Institute for Digital Research and Education Symposium

Fri, 01 Nov 2019 11:00:00 -0700

Dr. Broniatowski gave an invited panel talk titled “Addressing the Vaccine Crisis: The Digital World, Big Data, and Public Health” at UCLA’s Institute for Digital Research and Education’s symposium. His talk covered the ways in which state-sponsored and profit-seeking entities use health communication about vaccines on social media to accomplish a variety of malicious tasks, including promoting discord, spreading malware, and spamming. He covered the different types of malicious actors on Twitter and the specific ways these are used to achieve the goals identified above, and some promising theoretical approaches to combating them.

Invited Speaker at the National Academies of Sciences

Fri, 04 Oct 2019 10:00:00 -0400

Dr. Broniatowski gave an invited talk to the National Academies Health and Medicine Division Committee on October 4. His talk was titled “Malicious Misinformation and Threats to Public Health.”

Invited Panelist at the CSIS Conference on Global Immunization

Sat, 28 Sep 2019 12:00:00 -0400

Dr. David Broniatowski was an invited panelist at the Center for Strategic and International Studies Panel on Restoring Trust in Vaccines at the CSIS Conference on Global Immunization. The panel was held September 28.

Invited Speaker at OBSSR Methodology Seminar

Fri, 09 Aug 2019 14:45:00 -0400

SBP-BRIMS Disinformation Panelist

Fri, 12 Jul 2019 10:30:00 -0400

Dr. Broniatowski presented an invited panel presentation on disinformation from a public health perspective at the 2019 SBP-BRIMS conference

Hot Topics Presentation to CDC

Thu, 13 Jun 2019 12:00:00 -0400

Presentation at the TMP Consortium

Thu, 13 Jun 2019 10:50:00 -0400

Invited Panelist at Health Executive Leadership Summit

Tue, 04 Jun 2019 13:00:00 -0400

Invited Panelist at eMerge Americas 2019

Mon, 29 Apr 2019 14:55:00 -0400

Several SEAS faculty were invited to represent GW at the 2019 eMerge Americas conference, held April 29-30 in Miami, FL. eMerge Americas is the premier technology event that links Latin America to the rest of the world. Research conducted by Drs. Michael Keidar (MAE), Tian Lan (ECE) and Guru Venkataramani (ECE), James Hahn (CS), Danmeng Shuai (CEE), and Zhenyu Li (BME) was showcased in an interactive booth exhibit at the conference. The GW Today article “GW Researchers to Display Their Technologies at eMerge Americas” outlined the GW research presented at the conference. The event also included panel discussions. Dr. David Broniatowski (EMSE) was an invited member of the April 29 panel “The Promise & Peril of Digital Technology”.

Invited SMA speaker session

Thu, 18 Apr 2019 10:00:00 -0400

SMA hosted a speaker session presented by Dr. David Broniatowski (George Washington University) as a part of its SMA General Speaker Series. During his presentation, Dr. Broniatowski discussed intelligence analysis, stressed the importance of informing decision makers, and provided policy recommendations in his presentation. He stated that providing decision makers with more details (verbatim) is not enough; communicators should express the gist of an analysis product, that is, its meaning in context. Dr. Broniatowksi proceeded to discuss Fuzzy-Trace Theory, which explains why analysis products can be prone to misunderstanding, with dire consequences. He explained that policy decisions are more informed by gist representation as opposed to verbatim details; therefore, experts should represent the gist of the analysis and not just the verbatim—which could be misunderstood. Dr. Broniatowski then stated that gists must be elicited from experts, and these gists may differ if they are informed by different sources of expertise (including “lay” cultural expertise). If experts differ, an overarching gist must be extracted to for a more thorough understanding. Furthermore, he explained that a more precise analysis will be more compelling if it is linked to a categorical gist. To conclude, Dr. Broniatowski stated that the IC should support its good communicators through further development and by training them to become skilled gist communicators.

Invited Panelist at Annual Conference of Vaccinology Research

Fri, 05 Apr 2019 08:30:00 -0400

So You Want to be a Scholar?

Tue, 02 Apr 2019 10:30:00 -0400

Dr. Broniatowski spoke to PhD students from systems engineering programs around the country at the SEANET Doctoral Consortium associated with the Conference on Systems Engineering Research. His talk was titled “So You Want to Be a Scholar” and covered aspects of his PhD journey.

ASPPH 2019

Fri, 22 Mar 2019 16:00:00 -0400