Evaluation of a
Social Science Simulation by Uncertainty Analysis and Comparison The
military missions referred to as irregular warfare, which include counterinsurgency,
counterterrorism, and stability operations, share the essential tenet
of influence over populations. Within the defense analysis community,
a new emphasis on such influence has been realized simultaneously with
the emergence of the field of computational social science. This intersection
of urgent requirement and nascent capability has resulted in new simulation
models being rushed into service. In some instances these complex simulations
are employed without empirical knowledge of their behaviors.
This thesis examines
one such model, Nexus Schema Learner and its performance, over a large
number of replications. Several input parameters of the Nexus model
are explored through experiments. The propagation of uncertainty in
these parameters to the output of the model is examined. The results
of Nexus are also compared to a very simple model of a single social
theory. The research provides perspectives, techniques, and measures
important to the practical application of Nexus. Influential parameters
and associated broad categories of behavior are identified. These behaviors
are also compared to the simple model. The findings provide an opportunity
to consider empirical results of the simulation against its intended
use and the social theories it is purports to represent. Additional
areas for conceptual, technical, and analytical development are outlined,
pursuit of which will make future simulation studies using Nexus Schema
Learner more effective and efficient. Some of these results provide
compelling lessons for developers and potential users of all types of
computational social science models.
