David A. Broniatowski on Decision-Making and Systems Architecture Laboratory

The Role of Expertise in Risky Engineering Decisions

Sun, 17 Nov 2019 09:00:00 -0400

Engineering design decisions increasingly entail making risky decisions in complex situations. Here, we investigate the role of expertise in engineering problem solving. Specifically, we aim to answer the question of how engineering experts’ risk judgments affect design decisions, disentangling two concepts, expertise and knowledge. We draw upon Fuzzy Trace Theory (FTT), which makes specific predictions about how experts’ risk judgment drive design decisions. According to FTT, when making decisions, people rely on a continuum of mental representations, ranging from precise and quantitative verbatim representations of risk information to qualitative, categorical gist representations capturing the bottom line meaning of that information in context. The theory posits that individuals rely more on gist representations than verbatim representations – the so-called fuzzy processing preference. Furthermore, expertise enables individuals to retrieve the proper gist; consequently, this preference is more prevalent among experts, as a result of their developmental advancements in subject matter. To test FTT’s predictions in an engineering design decision context, we compared two samples: 1) 41 experts at NASA, and 2) 233 nonexperts recruited via Amazon Mechanical Turk (MTurk). Using a survey, we measured subjects’ risk judgments for a hypothetical mission design scenario that involved safety risk. Subjects in the MTurk sample were randomly assigned into three groups: a control group, a verbatim group that received technical verbatim information, and a gist+verbatim group that received the same information along with its bottom line meaning expressed as a gist. We applied Exploratory Factor Analysis (EFA) to our survey results, yielding three separate gists: categorical schedule risk, categorical safety risk, and categorical cost risk for the mission. Compared to the MTurk sample, NASA experts were more likely to agree that there was ‘some risk’ associated with the categorical cost and categorical schedule gists, whereas the MTurk sample judged the cost and schedule risks of the same scenario as ‘essentially nil’ (m=0.9, p<.001 for categorical cost; and m=0.76, p<.001 for categorical schedule cost risks). Within the MTurk sample, the we did not observe a significant difference between the verbatim group and the control group (m=0.27, p=ns for cost and m= 0.18, p=ns; for schedule risks). However, the gist+verbatim group’s risk perceptions became statistically indistinguishable from those of NASA experts (m=0.31, p=ns), highlighting the distinction between verbatim knowledge and gist. We also analyzed the relation between gist-based thinking and risky choice for the mission design. We next conducted a logistic regression analysis to examine the effects of the gist and verbatim factors on risky choice. We found categorical schedule (z=3.54, p<0.001) and categorical cost (z=-3.31, p<0.001) gist factors were significantly associated with risky choice; however, we did not find a significant association between the factor capturing verbatim representation and risky choice (z=1.35, p=ns for verbatim factor). NASA experts and the MTurk sample differed the most along the factor capturing categorical cost risk, which was the strongest predictor of risky choice. Next, within the NASA sample, we examined demographic factors most likely to predict categorical cost gist using a multi-way ANOVA. Surprisingly, the subject’s ability to recall a major failure significantly predicted categorical cost gist (F(1,31)= 7.77, p<0.001) whereas years in the aerospace industry did not (F(1,31)=0, p< 0.001). These results are consistent with the development of expertise based on feedback from the environment rather than simply number of years on the job. Overall, our results support FTT’s predictions regarding distinct gist and verbatim representations in an engineering context. Furthermore, as predicted, gist is more strongly associated with risky choice than is verbatim. Finally, our results suggest that expertise is informed by feedback from the environment, and therefore insightful.

What is the Gist of the Fourfold Pattern of Risk?

Fri, 15 Nov 2019 13:30:00 -0400

Prospect Theory (PT) predicts a “Fourfold Pattern of Risk Attitudes:” risk seeking for low-probability losses/high-probability gains, and risk aversion for high-probability losses/low-probability gains. Previously, subjects primarily exhibited this pattern for willingness-to-accept tasks. Per Fuzzy Trace Theory (FTT), subjects make binary choices using “some” vs. “none” gists, relying on verbatim representations only when necessary. FTT expects risk-aversion for gains and risk-seeking for losses for non-negligible probabilities, that truncating complements removing categorical contrasts attenuates this pattern, and that reliance on verbatim representations varies with metacognition. We tested these hypotheses by administering binary risky choices online, manipulating framing (gain vs. loss) and probability (low vs. high), holding gamble rewards constant. Gists were measured as “some” or “none”. We found risk-aversion for gains and risk-seeking for losses regardless of probability. Effect sizes varied as expected with individual differences, gists, and truncation. Contrary to PT, the fourfold pattern seems limited to verbatim comparisons.

Vaccine-related advertising in the Facebook Ad Archive

Wed, 13 Nov 2019 00:00:00 +0000

Machine Learning Classifiers for Socio-Demographics of Social Media Users: Limitations and Possibilities

Tue, 05 Nov 2019 13:00:00 -0400

Background: Social media analyses of health behaviors, such as vaccination, have shown significant promise for surveillance. However, existing research, both quantitative and qualitiative, suggests that health behaviors varies significantly with socio-demographic factors, particularly race, ethnicity, and gender. Often, these factors are not explicitly disclosed on social media platforms and must instead be inferred, raising the possibility of methodological bias. This session examines existing tools and methodologies used to infer socio-demographics of social media users. Methods: We survey several socio-demographic classifiers that work with Twitter and Reddit data. These classifiers use features including users’ language patterns, follower behaviors, and choice of names. These classifiers predict labels including users’ gender, race and ethnicity, or filter out social media accounts run by organizations.

Results: We explain how the data for these classifiers is collected, how the classification models are trained, and how they could be applied to public health research. We in particular discuss the limitations that these classifiers have, including possible methodological bias introduced by the challenges of large-scale data collection of social media users’ demographic information.

Discussion: Health behaviors vary with socio-demographic factors, which are challenging to measure on social media platforms. Machine learning classification of socio-demographics is possible, but requires interdisciplinary considerations.

Translating Trust in Vaccines from Surveys to Twitter

Tue, 05 Nov 2019 13:00:00 -0400

background: Trust is considered an important factor in vaccine-related decisions. Traditional surveys measure this with thoughtful and rigorous research, but that can be slow and expensive compared to social media analyses. Social media research currently lags behind surveys in terms of rigor because, among several reasons, it requires measuring via inference instead of self-report. There is an opportunity to innovate to leverage strengths of both data sources. objective: This session will exemplify how to combine social media and survey data when characterizing online discussion on trust in vaccines to leverage strengths of both data sources while mitigating weaknesses.

methods: We build a factor analysis linking social media messages with traditional survey questions conveying constructs related to trust in vaccines. Specifically, we recruit respondents to both answer survey questions and indicate agreement with (and whether they would share) tweets related to trust in vaccines. We assemble a set of scaled tweets that convey constructs in relevant survey research, such as trust in government or vaccine uptake. We perform a factor analysis to identify significant relationships between survey answers and agreement with tweets. These significant factors provide a means to translating from Twitter to surveys. We randomly sample Twitter data, apply this translation, and statistically characterize Twitter discussion as it relates to traditional survey results.

results: We will show significant relationships between constructs on surveys and constructs on Twitter, and we will show how these relationships can be used to characterize social media discussions on trust in vaccines.

discussion: This type of analysis can be used to quantify associations and relationships across different data sources like social media and surveys as constructs are conveyed. Relating more free-form social media to traditional surveys whose set of responses is more bounded would facilitate interpretation along important constructs related to trust in vaccines.

Identifying Nuances in Fake News vs. Satire: Using Semantic and Linguistic Cues

Mon, 04 Nov 2019 00:00:00 +0000

Health Wars and Beyond: The Rapidly Expanding and Efficient Network Insurgency Interlinking Local and Global Online Crowds of Distrust

Fri, 04 Oct 2019 00:00:00 +0000

Government Role in Regulating Vaccine Misinformation on Social Media Platforms

Thu, 05 Sep 2019 00:00:00 +0000

Elites and foreign actors among the alt-right: The Gab social media platform

Sun, 01 Sep 2019 00:00:00 +0000

To illuminate and motivate: a fuzzy-trace model of the spread of information online

Mon, 12 Aug 2019 00:00:00 +0000

Selective stimulation of human intrinsic laryngeal muscles: Analysis in a mathematical three‐dimensional space

Tue, 23 Jul 2019 00:00:00 +0000

Does Causal Coherence Predict Online Spread of Social Media?

Mon, 01 Jul 2019 00:00:00 +0000

Identifying Burns in Medical Device Reports

Mon, 01 Jul 2019 00:00:00 +0000

Malicious Actors on Twitter: A Guide for Public Health Researchers

Wed, 01 May 2019 00:00:00 +0000

Characterizing Trends in Human Papillomavirus Vaccine Discourse on Reddit (2007-2015): An Observational Study

Fri, 01 Mar 2019 00:00:00 +0000

Communicating Meaning in the Intelligence Enterprise

Fri, 01 Mar 2019 00:00:00 +0000

Can online self-reports assist in real-time identification of influenza vaccination uptake? A cross-sectional study of influenza vaccine-related tweets in the USA, 2013–2017

Tue, 01 Jan 2019 00:00:00 +0000

Characterizing System Architectures Using Network Data

Tue, 01 Jan 2019 00:00:00 +0000

A formal model of fuzzy-trace theory: Variations on framing effects and the Allais Paradox.

Mon, 01 Oct 2018 00:00:00 +0000

Weaponized Health Communication: Twitter Bots and Russian Trolls Amplify the Vaccine Debate

Mon, 01 Oct 2018 00:00:00 +0000

Understanding Controversial Discussions in Twitter: Women's Driving in Saudi Arabia

Fri, 13 Jul 2018 00:00:00 +0000

Analyzing Perceived Risk of Flu Vaccine on Twitter

Thu, 12 Jul 2018 00:00:00 +0000

Detecting and Characterizing Bot-Like Behavior on Twitter

Wed, 11 Jul 2018 00:00:00 +0000

Gab: The Alt-Right Social Media Platform

Sun, 01 Jul 2018 00:00:00 +0000

Patients’ and Clinicians’ Perceptions of Antibiotic Prescribing for Upper Respiratory Infections in the Acute Care Setting

Sun, 01 Jul 2018 00:00:00 +0000

What makes a rumor popular? A case study of Cryptocurrency Rumors on Reddit and Twitter

Sun, 01 Jul 2018 00:00:00 +0000

How Do NASA Engineers Perceive MMOD Risks? A Fuzzy Trace Approach

Sat, 30 Jun 2018 00:00:00 +0000

Measuring Public Perception of Engineered Systems: Opportunities and Challenges of Public Opinion in Vaccination

Fri, 01 Jun 2018 00:00:00 +0000

Building the tower without climbing it: Progress in engineering systems

Tue, 01 May 2018 00:00:00 +0000

Cultural Worldviews on an Aerospace Standards Committee: A Preliminary Analysis

Mon, 01 Jan 2018 00:00:00 +0000

Do design decisions depend on “dictators”?

Mon, 01 Jan 2018 00:00:00 +0000

The Flexibility of Generic Architectures: Lessons from the Human Nervous System

Mon, 01 Jan 2018 00:00:00 +0000

Monitoring Real-time Spatial Public Health Discussions in the Context of Vaccine Hesitancy

Sat, 04 Nov 2017 00:00:00 +0000

Recognizing Images of Eating Disorders in Social Media

Sat, 04 Nov 2017 00:00:00 +0000

Assessing causal claims about complex engineered systems with quantitative data: internal, external, and construct validity

Wed, 01 Nov 2017 00:00:00 +0000

Characterizing HPV Vaccine Discourse on Reddit

Wed, 01 Nov 2017 00:00:00 +0000

Categorical Risk Perception Drives Variability in Antibiotic Prescribing in the Emergency Department: A Mixed Methods Observational Study

Sun, 01 Oct 2017 00:00:00 +0000

How Does Twitter User Behavior Vary Across Demographic Groups?

Tue, 01 Aug 2017 00:00:00 +0000

The Emergence and Collapse of Knowledge Boundaries

Tue, 01 Aug 2017 00:00:00 +0000

Vaccine opponents' use of Twitter during the 2016 US presidential election: Implications for practice and policy

Tue, 01 Aug 2017 00:00:00 +0000

Effects of demography and social groups on opinion dynamics

Fri, 07 Jul 2017 00:00:00 +0000

Measuring Perceived Causal Relationships Between Narrative Events with a Crowdsourcing Application on Mturk

Sat, 01 Jul 2017 00:00:00 +0000

Flexibility Due to Abstraction and Decomposition

Mon, 01 May 2017 00:00:00 +0000

Combating Misinformation and Disinformation Online: The "Battle of the Narrative"

Wed, 01 Feb 2017 00:00:00 +0000

Examining Patterns of Influenza Vaccination in Social Media

Wed, 01 Feb 2017 00:00:00 +0000

Impact of Transit Ridership and Weather Conditions on Flu Propagation: Exploratory Analysis

Sun, 08 Jan 2017 00:00:00 +0000

Decoupling of the Minority PhD Talent Pool and Assistant Professor Hiring in Medical School Basic Science Departments in the US

Thu, 17 Nov 2016 00:00:00 +0000

Decoupling of the minority PhD talent pool and assistant professor hiring in medical school basic science departments in the US

Tue, 01 Nov 2016 00:00:00 +0000

Measuring Flexibility, Descriptive Complexity, and Rework Potential in Generic System Architectures

Thu, 01 Sep 2016 00:00:00 +0000

Process Modeling of Qualitative Decision Under Uncertainty

Wed, 10 Aug 2016 00:00:00 +0000

Designing a Crowdsourcing Tool to Measure Perceived Causal Relationships Between Narrative Events

Thu, 30 Jun 2016 00:00:00 +0000

Modeling Influenza by Modulating Flu Awareness

Sun, 12 Jun 2016 00:00:00 +0000

Effective vaccine communication during the disneyland measles outbreak

Wed, 01 Jun 2016 00:00:00 +0000

Flexibility due to Decomposition and Abstraction in a Simulated Intermodal Transportation Network

Wed, 01 Jun 2016 00:00:00 +0000

Zika vaccine misconceptions: A social media analysis

Wed, 01 Jun 2016 00:00:00 +0000

Flexibility Multipliers

Sat, 21 May 2016 00:00:00 +0000

Not All Networks are Equal: Empirical Analysis of Flexibility and Controllability in Software Systems

Sat, 21 May 2016 00:00:00 +0000

Understanding Vaccine Refusal: Why We Need Social Media Now

Fri, 01 Apr 2016 00:00:00 +0000

Understanding Vaccine Refusal: Why we need social media now

Fri, 01 Apr 2016 00:00:00 +0000

Towards Real-Time Measurement of Public Epidemic Awareness: Monitoring Influenza Awareness through Twitter

Tue, 01 Mar 2016 00:00:00 +0000

Formalizing Risky Choice with a Logistic Model of Fuzzy Trace Theory

Wed, 22 Jul 2015 00:00:00 +0000

Does Systems Architecture Drive Risk Perception?

Mon, 01 Jun 2015 00:00:00 +0000

Expert Teams and Shared Knowledge Networks: Review and Future Directions

Mon, 01 Jun 2015 00:00:00 +0000

Using Social Media to Perform Local Influenza Surveillance in an Inner City Hospital: A Retrospective Observational Study

Fri, 29 May 2015 00:00:00 +0000

Worldwide Influenza Surveillance through Twitter

Sun, 25 Jan 2015 00:00:00 +0000

Germs Are Germs, and Why Not Take a Risk? Patients’ Expectations for Prescribing Antibiotics in an Inner-City Emergency Department

Thu, 01 Jan 2015 00:00:00 +0000

Germs Are Germs, and Why Not Take a Risk? Patients’ Expectations for Prescribing Antibiotics in an Inner-City Emergency Department

Thu, 01 Jan 2015 00:00:00 +0000

Using Social Media to Perform Local Influenza Surveillance in an Inner-City Hospital: A Retrospective Observational Study

Thu, 01 Jan 2015 00:00:00 +0000

Twitter Improves Influenza Forecasting

Tue, 28 Oct 2014 00:00:00 +0000

Twitter Improves Influenza Forecasting

Wed, 01 Oct 2014 00:00:00 +0000

A Mathematical Formalization of Fuzzy Trace Theory

Thu, 24 Jul 2014 00:00:00 +0000

Twitter: Big data opportunities

Tue, 01 Jul 2014 00:00:00 +0000

Flexibility, Complexity, and Controllability in Large Scale Systems

Tue, 10 Jun 2014 00:00:00 +0000

Requirements Upflow: Using Requirement Documents to Uncover the Latent Architecture of Engineered Systems

Tue, 10 Jun 2014 00:00:00 +0000

HealthTweets. org: A Platform for Public Health Surveillance using Twitter

Wed, 01 Jan 2014 00:00:00 +0000

Incorporating Interpretation into Risky Decision-Making

Wed, 01 Jan 2014 00:00:00 +0000

National and Local Influenza Surveillance Through Twitter: An Analysis of the 2012-2013 Influenza Epidemic

Mon, 09 Dec 2013 00:00:00 +0000

National and Local Influenza Surveillance through Twitter: An Analysis of the 2012-2013 Influenza Epidemic

Sun, 01 Dec 2013 00:00:00 +0000

Does Seating Location Impact Voting Behavior on Food and Drug Administration Advisory Committees?

Tue, 01 Oct 2013 00:00:00 +0000

Does Seating Location Impact Voting Behavior on Food and Drug Administration Advisory Committees?

Sun, 01 Sep 2013 00:00:00 +0000

Gist and Verbatim in Narrative Memory

Tue, 01 Jan 2013 00:00:00 +0000

Analogies Between Complex Systems and Phases of Matter

Wed, 01 Aug 2012 00:00:00 +0000

Studying Group Behaviors: A tutorial on text and network analysis methods

Thu, 01 Mar 2012 00:00:00 +0000

Studying Group Behaviors: A Tutorial on Text and Network Analysis Methods

Thu, 16 Feb 2012 00:00:00 +0000

Extracting social values and group identities from social media text data

Sun, 01 Jan 2012 00:00:00 +0000

Towards a Computational Analysis of Status and Leadership Styles on FDA Panels

Sat, 01 Jan 2011 00:00:00 +0000

Analysis of social dynamics on FDA panels using social networks extracted from meeting transcripts

Sat, 21 Aug 2010 00:00:00 +0000

A Method for Analysis of Expert Committee Decision-Making Applied to FDA Medical Device Panels

Tue, 01 Jun 2010 00:00:00 +0000

Computational Analysis of Committee Decision-Making

Thu, 20 May 2010 00:00:00 +0000

Decision-Making by Technical Expert Committees for Engineering Systems

Tue, 16 Jun 2009 00:00:00 +0000

Bayesian Analysis of Decision Making in Technical Expert Committees

Mon, 20 Apr 2009 00:00:00 +0000

Quantitative analysis of group decision making for complex engineered systems

Mon, 23 Mar 2009 00:00:00 +0000

Generating status hierarchies from meeting transcripts using the author-topic model

Thu, 01 Jan 2009 00:00:00 +0000

A Framework for Evaluating International Cooperation in Space Exploration

Sat, 01 Nov 2008 00:00:00 +0000

A framework for evaluating international cooperation in space exploration

Sat, 01 Nov 2008 00:00:00 +0000

Articulating the Space Exploration Policy–Technology Feedback Cycle

Mon, 01 Sep 2008 00:00:00 +0000

Articulating the space exploration policy–technology feedback cycle

Mon, 01 Sep 2008 00:00:00 +0000

The Political Sustainability of Space Exploration

Fri, 01 Aug 2008 00:00:00 +0000

The political sustainability of space exploration

Fri, 01 Aug 2008 00:00:00 +0000

The influence of institutional background on the approval of engineered systems

Fri, 04 Apr 2008 00:00:00 +0000

System of systems architecture: The case of space situational awareness

Mon, 01 Jan 2007 00:00:00 +0000

Decision-Making in the Political and Technical Environments

Sun, 01 Jan 2006 00:00:00 +0000

Balancing the Needs for Space Research and National Security in the ITAR

Mon, 01 Aug 2005 00:00:00 +0000

An international approach to lunar exploration in preparation for Mars

Sat, 01 Jan 2005 00:00:00 +0000

Paradigm Shift in Design for NASA’s Space Exploration Initiative: Results from MIT’s Spring 2004 Study

Sat, 01 Jan 2005 00:00:00 +0000

A Deployable Instrument Package for Paleontological Research

Mon, 01 Jan 2001 00:00:00 +0000