disco-running-commodity-operating-systems-on-scalable-multiprocessors.md

Disco: Running Commodity Operating Systems on Scalable Multiprocessors

One-line Summary

Disco uses virtual machines to run multiple commodity operating systems on large-scale shared-memory multiprocessors. Disco VMM hides NUMA-ness from non-NUMA aware OSes, requires low effort to implement, and introduces moderate overhead due to virtualization.

Paper Structure Outline

1. Introduction
2. Problem Description
3. A Return to Virtual Machine Monitors
   1. Challenges Facing Virtual Machines
4. Disco: A Virtual Machine Monitor
   1. Disco's Interface
   2. Implementation of Disco
      1. Virtual CPUs
      2. Virtual Physical Memory
      3. NUMA Memory Management
      4. Virtual I/O Devices
      5. Copy-on-write Disks
      6. Virtual Network Interface
   3. Running Commodity Operating Systems
      1. Necessary Changes for MIPS Architecture
      2. Device Drivers
      3. Changes to the HAL
      4. Other Changes to IRIX
   4. SPLASHOS: A Specialized Operating System
5. Experimental Results
   1. Experimental Setup and Workloads
   2. Execution Overheads
   3. Memory Overheads
   4. Scalability
   5. Dynamic Page Migration and Replication
6. Related Work
   1. System Software for Scalable Shared Memory Machines
   2. Virtual Machine Monitors
   3. Other System Software Structuring Techniques
   4. ccNUMA Memory Management
7. Conclusions

Background & Motivation

The motivation is to enable existing commodity operating systems to handle Non-Uniform Memory Access (NUMA) architectures. Instead of modifying existing operating systems to run on scalable shared-memory multiprocessors, an additional layer (VM monitor) is inserted between the hardware and the OS.

Cache-coherent Non-Uniform Memory Architecture (cc-NUMA) makes hardware scalable, while SMP ensures the same performance to all memory from everywhere. Both ensure correctness, though.

Design and Implementation

The advantages of using virtual machines in the context of this work are:

• The Disco layer understands the NUMA architecture
• It's a portability layer
• Monitors are smaller and easier to understand & trust than operating systems
• Allows to run different OSes concurrently (almost unmodified)

The drawbacks of using virtual machines are:

• Overhead: cost of virtualizing
  • Time: VMM (Disco) acts as an emulator. Most instructions can just run, but privileged instructions + TLB instructions must be trapped & emulated
  • Space: Multiple copies (OS code & each OS's file cache) waste memory
• Resource management: Lack of information to make good policy decisions
  • Lost information about what is being used
    • CPU - idle thread
    • Memory - pages on the free list
• Communication and Sharing problems:
  • Hard to communicate between standalone VMs
  • Most OSes require exclusive access to disks

Evaluation

This paper started off VMWare (which was founded by authors of Disco in 1998 and successfully commercialized this work) and revived virtual machines for the next 20 years. Now VMs are commodities, and every cloud provider and virtually every enterprise uses VMs today.

New Vocabulary

• IRIX: A variety of UNIX System V with BSD extensions.
• NUMA: What is NUMA?

Links

• Paper PDF
• Paper review notes from CS 443 @ Northwestern by Joseph Paris
• Discussion panel from CS 736 @ UW-Madison
• Lecture slides from CS 262a @ Berkeley by Prof. Ion Stoica and Ali Ghodsi

{% file src="../../.gitbook/assets/21-Disco-Instructor-Notes Combined with Questions.pdf" %} Prof. Andrea's notes on Disco {% endfile %}