Concurrent Programming CAN be Introduced into the Lower-Level Undergraduate Curriculum

Michael B. Feldman and Bruce D. Bachus

mfeldman@seas.gwu.edu

A later version of this paper appeared in the Second Annual Conference on Intergrating Technology into Computer Science Education, Uppsala, Sweden, June 1997.

Abstract

There have been selected cases of satisfactorily teaching undergraduates the topic of concurrent programming. However, these reported experiences did not validate nor invalidate the feasibility claim as they did not provide statistical evidence and validity threat acknowledgments. Prior to this study, there have not been any controlled experiments to address the feasibility of introducing concurrent programming into the curriculum. Unfortunately, in curriculum development, it is seldom that we make decisions based on experimentation. We conducted a controlled experiment to determine whether introducing concurrent programming into the novice or lower-level undergraduate curriculum is possible. The experimental hypotheses examined concurrent programming from a concepts perspective and a problem-solving perspective. We conducted a detailed analysis of both internal and external validity threats to assess not only the validity, but the generalizability of the findings. The findings provide scientific evidence that introducing concurrent programming into the novice or lower-level undergraduate curriculum is possible.

1 Introduction

ACM Curriculum 91 recommended introducing distributed and parallel programming constructs into the undergraduate curriculum Unfortunately, ACM Curriculum 91 did not specify when, within the confines of the undergraduate curriculum, we should introduce these concurrent/parallel constructs.

Teaching concurrent programming is not prevalent in the undergraduate curriculums as one may have suspected given the proclaimed endorsement of parallel and distributed processing found in Curriculum 91. A review of the literature[1,2] provides only a few examples of experiences in introducing concurrent programming into the undergraduate curriculum. These reported experiences revealed that if concurrent programming was introduced into the undergraduate curriculum, then it was more likely to be introduced at the upper-level. Second, there do not appear to be very many institutions reporting experiences in teaching concurrency. Furthermore, as a result of an extensive literature search, we have not found any attempts to conduct an experiment which answers the question of whether or not concurrent programming can or could be introduced into the novice or lower-level undergraduate curriculum. With the belief that experimentation is vitally important to curriculum development, we decided to fill this void by conducting our experiment.

2 Novice Programmer Definition

The literature is inconsistent in defining the term novice or beginner programmer. Our experiment dealt specifically with novice programmers -- those which we would typically enroll in lower-level undergraduate curriculum. The related curriculum for the definition of a novice programmer would be the lower-level undergraduate curriculum .. We define the term novice (or beginner) as a subject who

1. has completed no more than two semesters of formal programming course work during the past two years and is unable to write a trivial program* (one consisting of 200-300 lines of code) in more than one high level programming language, or

2. is unable to write a trivial program (one consisting of 200-300 lines of code) in more than two high level programming languages, or

3. would logically take a CS1- or CS2-level programming course during the next semester as part of the subject's normal academic matriculation.

3 Basic Research Question

The basic research question is Can we introduce concurrent programming into the novice or lower-level undergraduate curriculum? The question of can relates to pedagogical issues concerning possibilities and feasibilities whereas the question of should relates to implementation issues such as trade off decisions, logistic support decisions and so forth. Once we have satisfactorily answered the can question, follow-on efforts may then address the should aspects of teaching concurrent programming.

4 Hypotheses

Question 1: Are the concepts inherent in concurrent programming too difficult for the novice programmer?

Hypothesis 1: Students who have received concurrency instruction will have significantly higher exam scores on a test which measures concurrency concept knowledge than those students who have not had concurrency instruction.

Question 2: Is the act of problem solving , that is, algorithm development and subsequent coding, enhanced if we have exposed the novice programmer to the concurrent programming paradigm?

Hypothesis 2: Students who have received concurrency instruction will have significantly higher exam scores on a test which requires the student to design a solution to a problem and subsequently write code which implements the design, than those students who have not had concurrency instruction.

5 Pilot Tests

Someone once said, When in doubt, pilot it out. Accordingly, we conducted two pilot tests which preceded our actual experiment-- one at Ft. Gordon, Georgia and another at The George Washington University (GWU).

The purpose of the Ft. Gordon pilot test was to compare the final exam scores with the exams of all the other previous classes. The question being raised was whether or not the concurrency instruction would have a negative effect on the existing in-place curriculum. The result of this pilot test was that the test scores of the experimental group were within the range of means taken from all previous classes. In other words, the pilot test provided evidence that introducing concurrency topics would not have a negative effect on the existing novice programmer curriculum. Additionally, we obtained feedback from the subjects regarding the method of teaching and the content presented.

The primary value obtained from the GWU pilot test came from an analysis of the audiovisual recordings which we took during the class instruction and the course critique forms which highlighted areas which were troublesome to the students. The audiovisual recordings with the course critique forms caused further adjustments to both the course material, course structure, and testing instruments.

6 The Treatment -- A Mini-Course in Concurrency

The treatment given in the experiment consisted of eight class sessions of which the first and last sessions were the pretest and posttest respectively. Each class period was 2 1/2 to 3 hours in duration. The concepts taught in the mini- course were based on a survey of the various concurrency textbooks[1]. This survey revealed a common set of concurrency topics which included nondeterminism, process creation and activation, synchronization, process termination, process scheduling, and interprocess communication.

There were four homework assignments and one class project. The purpose of having homework assignments and the class project was to provide the student an opportunity to apply the concurrency concepts to related programming problems. Recall that we oriented the hypotheses on both concurrency concept knowledge and also programming design and coding. At the start of each class session, homework problem solutions were discussed followed by a lecture which emphasized a particular concurrency concept. To help reinforce the understanding of concepts, both the instructor and students participated in role playing.

For example, by using an automobile plant, various students could play the role of automobile assembler, painter, inspector and dispatcher while other students could play the role of raw materials being processed through the assembly line. Parking lots helped explain the use of buffers during interprocess communication while the creation of additional hired hands or processes demonstrated process creation and activation.

Another example of using role playing involved having several students, while acting like independent processes, simultaneously attempt to write numbers at the same location on the blackboard. After the pushing and shoving ended, the results were an undecipherable mess. The students understood what was meant by a shared resource, a critical section, and process scheduling.

7 Instructional Materials

Instructional Materials included the following:

A. Language and System Support for Concurrent Programming, Curriculum Module CM-25. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, Apr. 1990. Author: Professor Michael B. Feldman.

B. Concepts of Concurrent Programming, Curriculum Module CM-24, Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, Apr. 1990. Author: Dr. David W. Bustard.

C. Class Notes by Professor Feldman -- miscellaneous class notes taken from an advanced graduate course (CS358 Concurrency and Parallelism in Programming Languages) which reviewed both concurrency issues and the Ada Tasking Model.

8 Language Selection for the Mini-Course

The language selected for use during the mini-course was Ada. One of the reasons for choosing Ada was that it has built-in concurrency constructs via its Ada Tasking Model. Other languages (primarily Pascal and C++) were considered, however, only by way of language extensions (Concurrent Pascal and Concurrent C++) could concurrency be supported. Another reason which supports Ada is the availability of free compilers and free tutorials which can easily be executed on PC, MAC, and Unix platforms.

9 Subjects

There were two experimental groups and one control group. All of the subjects were volunteers. There were 7 subjects in experimental group 1, and 15 subjects from experimental group 2. The experimental group subjects responded to a solicitation for participation in the experiment which was broadcasted via E-Mail to all students in the School of Engineering and Applied Science (SEAS) at GWU. There were 20 subjects from the control group which came from students who at the time were enrolled in CS172 (Introduction to Design of Computers) and CS182 (Digital Computer Architecture).

A correlation analysis was conducted to identify significant correlations between GPA, number of programming languages known, number of programming languages learned in college, years in college, gender, and prior knowledge of Ada; and both the pretest and gain scores. The results failed to show a consistent significant covariant between any of the demographic data and test scores.

10 Pretest/Posttest

The pretest and posttest were identical tests consisting of a concepts section and a problem solving section. The concepts section consisted of 16 short answer questions worth three points each. The problem solving section consisted of eight questions which had a design part and a coding part. The design part and coding part were each worth 3 points each.

There was only one grader. We developed rubrics to provide consistent grading for both the concepts section as well as the problem solving section. The grader was unable to determine the owner of the exam nor was he able to determine if the particular exam was a pretest or a posttest.

11 Results

The dependent variables were the differences in test scores between the pretest and posttest for the Concepts Section of the tests and the differences between the pretest and posttest for the Problem Solving Section of the tests. The independent variable was simply whether or not the subject had been exposed to the mini-course. The results of our statistical analysis are:

1. At the p=.0000001 level, our experiment supports hypothesis 1, that is, subjects who have received concurrency instruction will score significantly higher on an exam which measures concurrency concept knowledge than those who have not received concurrency instruction.

2. At the p = 0.001 level, our experiment supports hypothesis 2, that is, subjects who have received concurrency instruction will score significantly higher on an exam which measures concurrency concept knowledge than those who have not received concurrency instruction.

(Note: p=.001 means that the conclusion drawn would have occurred by chance alone regardless of the mini-course treatment, in 1 case out of 1000.)

12 Validity Threats

By "validity threats" we mean factors that may challenge the validity of the experimental results. Internal validity threats are those which may prevent us from concluding that the treatment (the mini-course) actually caused an effect on the test scores. The internal validity threats which were controlled included the following: History, Maturation, Testing, Instrumentation, Statistical Regression, Mortality, and Selection of Subjects.

External validity threats are those which may prevent us from generalizing to other experimental conditions or to other populations. The external validity assessment included the following: Experimentally accessible population versus target population, Interaction of treatment effects and subject characteristics, Describing the independent variable, Describing and measuring the dependent variable, Multiple-treatment interference, Interaction of history and treatment effects, Interaction of time of measurement and treatment effects, Pretest and posttest sensitization, Hawthorne effect, Novelty and disruption effect, and the Experimental or Rosenthal effect.

13 Conclusion

The results of this experiment prove that concurrent programming can be introduced into the novice or lower-level undergraduate curriculum. The acceptance of this experimentally-based finding means that there is evidence to counter the claim that concurrency is a more advanced topic. Courses which may require adjustments include programming courses which could become oriented on concurrency constructs during the early stages, data structures which can be viewed as not simply residing on one processor but distributed over a network, and architecture, hardware, and database courses which may introduce parallel configurations at a much earlier stage in the curriculum.

It is clear that future computing will be heavily based on the concurrent programming paradigm, and that we, as educators, have an obligation to educate our students in preparation for the challenges of the future. Hopefully, we have provided you with justifiable ammunition to objectively begin making necessary curriculum development changes. So now, whats your excuse for not teaching your lower-level undergraduates the vital topic of concurrent programming? As a final note, future decisions related to Computer Science curriculum including what language should be used for AP exams, should depend on experimental evidence and not merely opinions or experience.

REFERENCES

1 Bachus, Bruce D., Determining the Feasibility of Introducing Concurrent Programming into the Lower-Level Curriculum via a Controlled Experiment DSc Dissertation, The George Washington University, 1996.

2 Miller, Russ, "The Status of Parallel Processing Education, Special Report, Computer, Vol. 27, No. 8, Aug 94, pp. 40-43.

Media, McGraw-Hill