CSCI 3411 - Operating Systems

Posted on August 1, 2020 by Gabriel Parmer

This course covers the fundamental concepts of operating systems, focusing on resource management and abstraction. This includes OS structure, processes and thread management, communication with peripherals (I/O), synchronization, deadlocks, memory management, Virtual Machines, cloud infrastructures, and abstractions for cloud computation. The workload for this class is heavy and programming intensive.

Instructional Team Contact info

For general questions about homeworks and course work that would be useful to the broader class, please use Piazza.

Fall 2020 Class Time/Location:

Your Immediate TODO

  1. Register for the class on Piazza. I’m opening this early, over the summer so that you can ask any questions you may have in preparation for the class.
  2. On Piazza, find the zoom location for the first class.
  3. See the post on Piazza that includes a link to the class survey. You must fill that out before the first Thursday of the class. You cannot receive any grades until you fill this out.
  4. You should also find a link to the first homework (HW0) in that survey that you can retrieve using github classroom (which requires you to make/have a github account).
  5. Please read about installing the class’ infrastructure: Ubuntu 18 LTS and VS Code (see the class resources for guidance) and #tech-setup on discord if you have any issues.

Course Overview

Learning Outcomes

Objectives - In completing this class, students will…

Structure - This class is broken into two main activities: lectures and lab.

Direct Instruction and Independent Learning

Each semester, you’re expected to spend at least:

Course Prerequisites and Student Responsibilities

Prerequisites:

Responsibilities - Students must

Professor and Instructional Staff Responsibilities

You should depend on the instructional staff (including Gabe) for the class to

These can be remembered as “we’re here to make it clear what you should do, what you should know, and what to do if you don’t feel like you know what you’re supposed to do or know”. If at any point, you have concerns about any of these, of if we’re dropping the ball on any of them, please let us know and we’ll do better. There are trade-offs made in the class (for example, homeworks are not formally specified as I don’t want them to be 50 pages each), but you always encouraged to ask questions and clarify.

Above all, it is important that this class provides an environment for learning and self improvement. Gabe takes this very seriously, so if it is not adequately serving that purpose, don’t hesitate to let him know why.

If you don’t feel comfortable directly telling Gabe where the class could be doing something better, please don’t hesitate to use the anonymous class feedback form shared on Piazza that is viewable only by Gabe.

OS in the Age of Covid-19

Despite any changes to the instruction and organization of this course for Covid-19, this class has a number of goals. The class will

  1. maintain high standards and ensure that all students will meet the learning objectives and be competitive and competent computer scientists in the area of Operating Systems,
  2. maintain a high degree of engagement from students, including both in-person (where possible), and online, and
  3. maintain a high degree of engagement of the instructional staff to provide a consistent and available class experience.

Many changes are necessary to the class to enable a strong online class experience. It goes without saying that the class will be quite different this semester. However, consistent with the first goal of the class, we have redesigned such that your will learn all the necessary material, just the same. This version of the class will be different, and not worse. Indeed, when classes go back to normal in future years, we plan on adopting multiple of these changes.

A Better OS Class

Some important aspects of the class this semester are an improvement over previous years. The goal this semester is to make the class material even more accessible, and the delivery more successful. We aim to do this by:

What to expect

Regularly scheduled classes will be synchronously attended by everyone. Classes will, by default, use Zoom, and we’ll make significant use of group rooms on Discord to do group work.

The class will have four instructional aspects:

  1. Lectures - These will be asynchronous. Most if not all lectures will be recorded, and provided on youtube and linked from Piazza. Previous semester instances of the class focus on a dynamic, discussion-oriented in-person class. To maintain the same level of feedback and discussion, the asynchronous lectures will be accompanied with questions (via google forms). Answering the lecture questions (via google forms), and providing questions is mandatory, and factors into your grade.
    The questions are due 2 hours before the corresponding class starts.

    The first part of each class will start on zoom, and will focus on Gabe providing feedback on your answers and questions. This is meant to be a discussion, so please use the “raise hand” and related features to interact. These zoom sessions will be recorded and privately shared with everyone in the class via google drive. Completion of all questions will count toward class participation, as will participating in class.

  2. Discussion and exercises - The synchronous instruction during the normal class time should be attended by everyone, remotely. This time will be devoted to two purposes:

    1. to discuss lecture material guided by the form answers to lecture questions and by a Q&A, and
    2. based on group exercises, usually based around programming or system exploration.

    For the latter, teams will coordinate in team-specific rooms on Discord.

  3. Labs - Labs provide an intermediary between the ideas discussed in class, and the practical programming of the homeworks. They are required and enable you to complete the homeworks and project. Without completing the lab work, you should not expect to complete the homeworks nor project. They will be conducted via Discord.

  4. Homeworks and Project - Most of the rest of the class is in place to enable you to approach and successfully complete the homeworks and project. The plan is for most homeworks and projects to be team-based as I want everyone to maintain tight contact within the class throughout the semester.

Your role!

Provide feedback. Many aspects of the class are new and experimental, thus they might require modifications throughout the course of the class. Because of this, all students must take the contents of the section below on “Professor and Instructional Staff Responsibilities” seriously: if something isn’t working for you, assume that the class needs a tweak, and contact the professor, or provide anonymous feedback. Gabe will take this feedback seriously.

Community and online social contract. As an increased part of the class is online, it is necessary that everyone conduct themselves respectfully and productively online. Please see the class’ “Online Social Contract”. The class will have a zero-tolerance approach toward disrespectful conduct, or harassment.

Take responsibility for your own education. Much of the work for this class will be more group-oriented than in normal instances of the class. If you burden your partner with most of the work, you are setting yourself up for failure. If you take on your partner’s responsibilities, you are setting them up for failure. The homeworks build on each other. The benefit of this is that by the end of the class you’ll laugh at how easy some of the earlier homeworks are as you’ve massively leveled up. The challenge is that if you don’t put in a genuine effort on each homework, you will find later homeworks increasingly intractable. If you don’t put effort in for each homework, you’ll create dis-proportionally more work for yourself in later homeworks. If you cannot carry your own weight, you will not be able to achieve the learning objectives for the class.

Also remember that your team members will change throughout the semester, and they happen to be the network you’ll use to help get a job. Not impressing them hurts your future opportunities.

Class Technologies

How you comport yourself online in the class just adhere to our online social contract. For the class we’ll use the following technologies for the specified purposes:

Please see the discussion about Academic Integrity below. A rule of thumb is that you should not post code on any discussion medium with reach beyond your group. We do not use blackboard.

Absences

If you need to be absent, please send me an email to let me know that you can’t make it, and please let me know why. If you know you’re going to be absent (religious holiday), try and let me know in advance, otherwise, let me know as soon as you can. I’ll follow up and help you plan your involvement in the class (e.g. group projects). If you’re sick, keep me updated on how you’re feeling if you can.

Office Hours

HOWTO office hours. Office hours will be conducted on Discord. Please join the #office-hours-organization channel. We’ll use that channel to

  1. organize students into groups that are interested in the same topics,
  2. jump into voice chat/video sharing channels with students interested in the same topic.

When you join the organization channel, please tersely describe what you’d like help with. Check to see (in the immediate history on the channel) if anyone else is interested in a similar topic. Also check who else is already in the channel (they “go” before you). The “priority” student is the student that entered the channel first. The instructor will finish with the first student(s), and will re-join #office-hours-organization and say “Who’s next? #office-hours-x” at which point the priority student can join the Q&A in the #office-hours-x channel (for some integer x).

Discuss with the students awaiting help if your question “clusters with” other student’s questions. The goal is to determine if you should all join the Q&A together. Note that you can never join a Q&A together if doing so would violate academic integrity (i.e., if you want to discuss your code with screen share). You can join the Q&A along with the priority student if there is a shared question. If they wish to then ask a private question, you can exit the Q&A, reentering the organization channel with the same priority you had before you entered the Q&A.

Preparing your code for Office Hours. If you want an instructor to look at code then you must adhere to style conventions. See style conventions below. At the most basic level, you must

If you don’t demonstrate to us that you actively tried to understand your own code by making it as easy to understand as possible, then it isn’t reasonable for us to dive into your code. Note that you really should take all of these steps when developing, debugging, and testing your own code, regardless. In the “real world” it is expected that you generate code that adheres to these guidelines.

Experiments

Part of keeping a course “fresh” and successful is trying new things, and keeping the things that work. Please keep in mind that I might not have fully ironed out the kinks in each of these, so your feedback is valuable. Given this, the experiments I’m running this year include:

Online class. As you hopefully well-know, we’re online this semester. Many of the other experiments follow from this.

Async lectures + form-based feedback. Given the remote format, the discussion-based lectures that make the class work would be too challenging. Thus, we’re moving the lectures into recorded segments along with a set of forms you must fill out to answer lecture questions and ask questions. These questions and answers will be discussed in synchronous lectures, thus focusing that time on interaction.

Linux-based group exercises. We’ll add more of a practical component to the class with in-class, group-based, Linux programming and exploration. This will add yet another pragmatic component to augment the ideas conveyed in lectures.

Pervasive group projects. In the real-world, most work is done in groups. We’re going to do most homeworks and the project in groups. The expands the use of teams in the class as all homeworks were previously performed individually. This is not a new concept at large, but is in this class.

Mandated git usage. As most homeworks will be group-based, we’re going to treat them as much closer to software development in a company. This means that you’ll need to plan out who does what work, and as you “tick off” the features, you mark them as accomplished, linking to the corresponding set of commits. As part of this, you’ll need to assign tasks to different team members. You must commit your initial breakdown and assignment of tasks within the first 3 days after a homework has been assigned. The details of the protocol for how you must use github is pending.

Course Schedule

Topic xv6 Book Chapters/Additional Reading1 Silberschatz Chapters2
OS Fundamentals: Hardware and Structure 0, 1, 3 and this post 1, 2
Processes, IPC, Threads, Isolation this post 3, 4
Scheduling and Critical Sections 5, 4 5, 6, 19
Mutexes, Condition Vars, Deadlock N/A 6, 7
Memory Management and Allocation N/A 8
Memory Protection 2 9
File Systems and Storage 6 10, 11, 12, 13
Security N/A 15

You can find the lectures on Piazza after they are delivered. By default, I’ll provide these after they are fully covered in class. I’ve found that people read ahead which can harm the learning processes (as quite a bit of information is not on the slides) as it removes the problem solving processes. If this policy hinders your learning process, talk to me and we can discuss options. For your reference, all of the lecture’s pdfs are on the following topics:

Course Material

Required reading. You must read through the material at the repo for class resources. This includes some intuition as to how to develop C, how to think about debugging and testing, and how to think about generating good code.

Strongly encouraged (but optional) Text. This book is useful to help you understand the concepts that might not be clear from the lectures.

xv6 code documentation. If you have trouble understanding the xv6 source code, or want to better understand it, I recommend the following resources:

You can fork the xv6 source on github. (This is the same code that is the foundation of most of the homeworks.) When you do, test that you can run it with make, and make qemu.

In the end, you have to get comfortable walking through the code, and understanding it. Using a code indexing system will make this easier. ggtags or ctags for emacs does great, but there are corresponding technologies for nearly all editors.

If you’re having trouble with C, here is a list of references:

It is your responsibility to quickly get up to speed in C, so please use these resources.

Assignment Submission

All of your homeworks for this class will be submitted via github. The last commit that you make to the repo will be graded, and the lateness penalties of that commit (see “Late Policy” below) will apply. Please do not make pushes to your repo after the version you want graded. Please also not that it is necessary to issue a git pull up to the github repo for submission. Local commits to your own repo do not update github, thus cannot be counted as submission.

All assignments are submitted on github by doing a push of your commits before the deadline. There are a huge number of errors that one can make, so I highly suggest that you do the following procedure. When you submit an assignment, I suggest that you follow this procedure:

  1. git status to make sure that you have git added all of the intended files.
  2. git diff and git diff --stat to make sure that you’re committing what you think you are. Of note, check to see if you’re committing any printfs that are intended for debugging, not for program output.
  3. git commit and git push to upload your code to github.
  4. Go to the associated webpage for the repository, and make sure that the most recent commit is present in your repository.
  5. Use git clone ... to get a fresh version of the repository, build it, and run your tests. Students who omit this step get bad grades for very avoidable reasons. Always make sure that your submission matches what you think you’re submitting.

Git Usage on Group Projects

We will use your git commit logs and diffs, along with partner feedback to assess work done in groups. We don’t expect the work to be exactly equal between team members, but we do expect each group member to pull their own weight. Thus, we expect you to follow some simple rules when using git:

  1. Don’t rebase. Don’t rebase your commits to clean up your commit history. I wanted to see your entire commit history, thus avoid squashing commits. If you don’t know what this is, then you likely don’t need to worry.
  2. Commit with your GWU email. Set your git email address to your GWU email address. Failure to do this will mean that we see you doing no commits!
  3. When pair programming, swap who’s driving, and commit often. If you program together in your group, that’s fine. However, you need to swap back and forth regularly. Set a timer.
  4. Notate commits that don’t compile. If you make a commit when the code doesn’t compile, include DOES NOT COMPILE in the commit message body. Many commits made when pair programming will use this.

Your commit logs will not feed directly into your grade. However, they will play a role in how we determine groups, and they might lead to meetings to discuss your contributions.

Grading

Grades will be assigned with the following proportions:

Homeworks include smaller programming assignment, written responses, and a long final programming project. Each of these components can involve individual interviews about your code and design.

Participation will be graded not only on attendance, but also on interactivity and involvement during class and lab. This will include the quality of your questions and comments on the class forms.

If you cannot make it to any aspect of the class, or cannot meet any deadline, please let me know as soon as possible.

Homework Grade Modifications:

The following modifies apply to your grade on all homeworks (but not the project).

Your grade can be minimum 0%, but your maximum grade can be higher than 100% (if there is extra credit). There is one exception:

I cannot emphasize this point enough: The only way people have failed this class is by not trying or by cheating.

Late Policy:

The only acceptable excuses for late submissions are medical, family, and related issues. Generally unacceptable excuses include:

You get 0 credit for late work that is unexcused. However, once the grades are released, you can resubmit your code within seven days, rounded up to the closest midnight. For example, if the grades are released at 2:30pm on the 8th, resubmissions are due at midnight on the 15th (i.e. \(8 + 7 = 15\)). You will get 0 credit for submissions after this date.

We will grade the last commit before the resubmission deadline, and you will receive 50% of the credit on top of the original grade, rounded up to the nearest percent. For example, if you got a 43% on the initial deadline, and resubmit and get a 68%, your final grade (\(g\)) will be \(g = \lceil 43 + \frac{(68 - 43)}{2} \rceil = 56\).

Do not come to rely on the resubmission process. Homeworks are due every week or every other week, so it is very harmful to get behind on them, or to add more work to your schedule. Thus, our late policy is quite strict to discourage procrastination.

Please note that in terms of academic honesty, you cannot share your code with your peers even after it is due.

Testing your implementation:

When you are required to test your code, please note that our test cases will best every edge case in the implementation. If you do not write code to test the edge-cases of the specifications, then you will lose credit. This is the largest reason why students get lower grades than they believe they deserve. If you believe that an error was made in grading, then please explicitly address why you believe your test cases are sufficient to test all cases. See the discussion about testing and debugging below.

Style and Code Craftsmanship:

Any homework graded by demo (i.e. the project) must adhere to style guidelines, and you must curate your code. It must be simple, readable, and clean. See this post For some thoughts on how to make your code more readable, see the Composite Style Guide.

Academic Honesty

Just as you can do a google search for code online, it is trivial for us to do the same. We have caught numerous people cheating in the past in this way. If you feel pressured about an assignment, please come see me instead of cheating.

You are not allowed to collaborate on the homeworks and the lab assignments. The group projects require collaboration within each group, but no collaboration between teams is permitted. Please refer to the academic integrity policy linked from the course web page. This policy will be strictly enforced. If you’re having significant trouble with an assignment, please contact me. If you have not signed the academic integrity statement for homework 0, you will not receive a passing grade for the class.

Summary: I want to be very clear:

In the past, the only way that people have failed this class is to either not try (not do homeworks), or to cheat. There is almost no reason to cheat. Come to me, and we can discuss where you’re at, and what you have to do to make it through this difficult class!

Justification: Cheating hurts your ability to compete for CS jobs. Everyone came to college to learn. You learn by doing work to reinforce class ideas. Understanding ideas from class is nowhere near as effective at learning as applying those ideas in an implementation. Thus, you need to do your own implementation to be a competitive CS student in the real-world!

Cheating hurts your peers. Courses that are graded on a curve have the side-effect that if someone inflates their grade not by hard work, but by cheating, your good grade (should the Professor not catch you) will hurt the grade of all of your colleagues that put the work in.

Cheating also hurts the quality of the OS class in the long term, thus hurts all future generations of GW CS undergrads. If you cheat, especially using previous year student’s work, to the point where the Professor believes that they must develop new assignments every year, then those assignments will not be as strong. They will not meet learning objectives in a course like this. All future CS students will be less prepared for competing for the best jobs.

Please see additional information about university policies.

University Policies

Please find details about university policies regarding

  1. Find the book online↩

  2. Silberschatz chapters are specified with respect to the 9th edition↩