diff --git a/week10/slides.md b/week10/slides.md index 3fccc95..4d7f794 100644 --- a/week10/slides.md +++ b/week10/slides.md @@ -23,183 +23,695 @@ paginate: true
+## Concurrency + +**Problem** of handling many tasks at once + + ## Parallelism -- Work on multiple tasks at the same time -- Utilizes multiple processors/cores +**Solution** of working on multiple tasks at the same time -
-
+* **Key difference:** Parallelism utilizes **multiple** workers -## Concurrency +--- -- Manage multiple tasks, but only do one thing at a time. -- Better utilizes a single processor/core -
-
-
+# Workers + +Parallelism divides tasks among workers. -* These terms are used (and abused) interchangably +* In hardwareland, we call these workers **processors** and **cores**. + +* In softwareland, + * "Processors" => Processes + * "Cores" => Threads + +* **Key difference:** Parallelism utilizes **multiple** processors/cores + * Some concurrency models don't! + + + -
--- + # Parallelism vs. Concurrency + --- -# Parallelism vs. Concurrency + +# Parallelism vs. Concurrency: Examples
## Parallelism - +* 8 cores, 8 threads +* 4 threads load the webpage, 4 threads update the progress bar
-## Concurrency +## Alternate Concurrency Model + +* 1 core, 1 thread +* When blocked on loading the webpage, update the progress bar + + +
+
+ + +--- + + +# Today +- Multithreading +- Interthread Communication + - Shared Memory + - Data Races + - Synchronization, Atomics + - Message Passing + - `Send` and `Sync` + + +--- + + +# Multithreading + +For this lecture only, +* Our workers are threads +* **Thread:** "stream of instructions" + + + + +--- + + +# Multithreading + +Suppose we're painting an image to the screen. + +* Divide image into eight regions + +* One region per thread + +* Easy! "Embarassingly parallel" + + + +--- + + +# Motivating Communication + +Say our image is more complex. + +* We're painting circles +* Circles overlap +* The _order_ we paint circles affects their color + + + + +--- + + +# Motivating Communication + +Now threads need to talk to each other! + +* For each pixel + * How many circles have been drawn? + * Do _not_ paint this pixel until previous circles are done + + +--- + + +# Motivating Communication + +**Problem:** How do threads communicate? + +**Solution:** +* Approach 1: Shared Memory +* Approach 2: Message Passing + + + + +--- + + +# Approach 1: Shared Memory + +For each pixel, +* Create shared variable `x` +* Increment `x` when thread touches pixel + +```c +static int x = 0; +``` + +Now threads know +* How many circles have been drawn? + + +--- + + +# Shared Memory: Data Races + +Are we done? + +Not quite... + +* Shared memory is ingredient for **data races** +* Let's illustrate + + + +--- + + +# Shared Memory: Data Races + +First, we have a shared variable `x`. + +```c +static int x = 0; +``` + + + + +--- + + +# Shared Memory: Data Races + +First, we have a shared variable `x`. + +`x` must satisfy a property to be correct. + +```c +// x is # of times *any* thread has called `update_x` +static int x = 0; +``` + + +--- + + +# Shared Memory: Data Races + +Second, `x` becomes incorrect mid-update. + +```c +// x is # of times *any* thread has called `update_x` +static int x = 0; + +static void update_x(void) { + int temp = x; // <- x is INCORRECT + temp += 1; // <- x is INCORRECT + x = temp; // <- x is CORRECT +} +``` + + +--- + + +# Shared Memory: Data Races + +Third, when multiple threads update at once... + +```c +// x is # of times *any* thread has called `update_x` +static int x = 0; + +static void update_x(void) { + int temp = x; // <- x is INCORRECT + temp += 1; // <- x is INCORRECT + x = temp; // <- x is CORRECT +} +// +for (int i = 0; i < 20; ++i) { + create_thread(update_x); +} +``` + + +--- + + +# Shared Memory: Data Races + +Third, when multiple threads update at once...they interleave! + + +| Thread 1 | Thread 2 | +|---------------|---------------| +| temp = x | | +| | temp = x | +| temp += 1 | | +| | temp += 1 | +| x = temp | | +| | x = temp | + + + + + +--- - + +# Shared Memory: Data Races + +We want `x = 2`, but we get `x = 1`! + + +| Thread 1 | Thread 2 | +|---------------|---------------| +| Read temp = 0 | | +| | Read temp = 0 | +| Set temp = 1 | | +| | Set temp = 1 | +| Set x = 1 | | +| | Set x = 1 | + + +--- + + +# Shared Memory: Data Races + +Want bolded operations to be **atomic**. + + + + +
+ +
+ +**Not Atomic** + +| Thread 1 | Thread 2 | +|---------------|---------------| +| **temp = x** | | +| | temp = x | +| **temp += 1** | | +| | temp += 1 | +| x = temp | | +| | x = temp | + +
+
+ +**Atomic** + +| Thread 1 | Thread 2 | +|---------------|---------------| +| **temp = x** | | +| **temp += 1** | | +| | temp = x | +| | temp += 1 | +| x = temp | | +| | x = temp |
+ +--- + + +# Fixing a Data Race + +We must eliminate one: +1. `x` is shared +2. `x` becomes incorrect mid-update +3. Unsynchronized updates + + + +--- + + +# Fixing a Data Race + +**Approach 1: Synchronization** + +Take turns! No "cutting in" mid-update. + +1. `x` is shared +2. `x` becomes incorrect mid-update +3. ~~Unsynchronized updates~~ + + +--- + + +# Synchronization + +We need to establish *mutual exclusion*, so that threads don't interfere with each other. +* Mutual exclusion means "Only one thread can do something at a time" +* A common tool for this is a mutex lock + + + +--- + + +# Sharing Resources With Mutual Exclusion + +```c +static int x = 0; +static mtx_t x_lock; + +static void thread(void) { + mtx_lock(&x_lock); + int temp = x; + temp += 1; + x = temp; + mtx_unlock(&x_lock); +} +// +``` +- Only one thread can hold the mutex lock at a time + +- This provides *mutual exclusion*--only one thread may access `x` at the same time. + + +--- + + +# Fixing a Data Race + +**Approach 2: Atomics** + +One airtight update! Cannot be "incorrect" mid-update. + +1. `x` is shared +2. ~~`x` becomes incorrect mid-update~~ +3. Unsynchronized updates + + +--- + + +# Atomics + +Rust provides atomic primitive types, like `AtomicBool`, `AtomicI8`, `AtomicIsize`, etc. +* Safe to share between threads (implementing `Sync`), providing ways to access the values atomically from any thread +* 100% lock free, using bespoke assembly instructions +* Highly performant, but very difficult to use +* Requires an understanding of *memory ordering*—one of the most difficult topics in computer systems +* We won't cover it further in this course, but the API is largely 1:1 with the C++20 atomics. + + + +--- + + +# Fixing a Data Race + +**Approach 3: No Shared Memory** + +If we eliminate shared memory, + +1. ~~`x` is shared~~ +2. `x` becomes incorrect mid-update +3. Unsynchronized updates + + +--- + + +# Fixing a Data Race + +**Approach 3: No Shared Memory** + +If we eliminate shared memory, race is trivially gone. + +1. ~~`x` is shared~~ +2. ~~`x` becomes incorrect mid-update~~ +3. ~~Unsynchronized updates~~ + + +--- + + +# Message Passing + +**Problem:** How do threads communicate? + +**Solution:** +- Approach 1: Shared Memory +* Approach 2: Message Passing + * Eliminates shared memory + + +--- + +# Message Passing + +* Threads communicate via channels + + + + +--- + + +# Approach 2: Message Passing + +Previously, shared memory solution was + +> For each pixel, +> - Create shared variable `x` +> - Increment `x` when thread touches pixel + + +--- + + +# Approach 2: Message Passing + +In message passing, we eliminate shared memory, + +For each pixel, +- Create a counter `x` for each thread +> - Increment `x` when thread touches pixel + + +--- + + +# Approach 2: Message Passing + +We broadcast updates to other threads via **message passing**. + +For each pixel, +- Create a counter `x` for each thread +- Increment `x` when thread touches pixel + * Send message to other threads + * So they update their copies of `x` + + +--- + + +# Message Passing Example + +```rust +let (tx, rx) = mpsc::channel(); +``` +* Channels have two halves, a transmitter and a receiver +* Connor writes "Review the ZFOD PR" on a rubber duck and it floats down the river (transmitter) + * Ben finds the duck downstream, and reads the message (receiver) +* Note that communication is one-way here +* Note also that each channel can only transmit/receive one type + * e.g. `Sender`, `Receiver` can't transmit integers + --- -# Parallelism vs. Concurrency (Examples) -
-
-## Parallelism +# Message Passing Example +```rust +let (tx, rx) = mpsc::channel(); -- Have one processor work on loading the webpage, while another updates the progress bar -* Often used to divide tasks into smaller units that can run at the same time - * e.g. Processing 100x100px regions of an image on each core - * "Divide and conquer" -
-
+thread::spawn(move || { // Take ownership of `tx` + let val = String::from("review the ZFOD PR!"); + tx.send(val).unwrap(); // Send val through the transmitter +}); -## Concurrency +let received = rx.recv().unwrap(); // receive val through the receiver +println!("I am too busy to {}!", received); +``` +* Note that, after we send `val`, we no longer have ownership of it! -* As we load a webpage, take a break sometimes to update the loading progress bar -* Often used to do other things while we wait for blocking I/O operations - * e.g. Running garbage collection while we wait for a response over the network +--- -
-
+# Message Passing in Rust +We can also use receivers as iterators! ---- +```rust +let (tx, rx) = mpsc::channel(); +thread::spawn(move || { // Take ownership of `tx` + let val = String::from("review the ZFOD PR!"); + tx.send(val).unwrap(); // Send val through the transmitter + tx.send("buy Connor lunch".into()).unwrap(); +}); -# Today: Parallelism -- Threads -- Synchronization -- Message Passing -- `Send` and `Sync` -- More Synchronization +for msg in rx { + println!("I am too busy to {}!", msg); +} +``` +* Wait, what does `mpsc` stand for? --- -# Terminology: Threads -* Dangerously overloaded term—can mean one of many things -* For this lecture, we define it as a "stream of instructions" -* In Rust, language threads are 1:1 with OS threads -* **Key point:** Threads share the same resources +# `mpsc` ⟹ Multiple Producer, Single Consumer ---- +This means we can `clone` the transmitter end of the channel, and have *multiple producers*. -# Sharing Resources +```rust +let (tx, rx) = mpsc::channel(); -```c -static int x = 0; +let tx1 = tx.clone(); +thread::spawn(move || { // owns tx1 + tx1.send("yo".into()).unwrap(); + thread::sleep(Duration::from_secs(1)); +}); -static void thread(void) { - int temp = x; - temp += 1; - x = temp; -} -// -for (int i = 0; i < 20; ++i) { - create_thread(thread); // helper function not shown +thread::spawn(move || { // owns tx + tx.send("hello".into()).unwrap(); + thread::sleep(Duration::from_secs(1)); +}); + +for received in rx { + println!("Got: {}", received); } ``` -* What is the value of `x` after we join on all 20 threads? - * What is the next slide's title going to be? - - --- -# Race Conditions -When multiple threads have access to the same data, things get complicated... -* Specifically, this is about *data races* +# `Send` and `Sync` --- -# The Bad Slide +# `Send` and `Sync` -| Thread 1 | Thread 2 | -|---------------|---------------| -| temp = x (temp = 0) | | -| | temp = x (temp = 0) | -| temp += 1 (temp = 0 + 1) | | -| | temp += 1 (temp = 0 + 1) | -| x = temp (x = 1) | | -| | x = temp (x = 1) | +Everything we have gone over so far is a *standard library* feature. The language itself provides two marker traits to enforce safety when dealing with multiple threads, `Send` and `Sync`. -* Uh oh... - + --- -# Synchronization +# `Send` vs. `Sync` -To make sure instructions happen in a reasonable order, we need to establish *mutual exclusion*, so that threads don't interfere with each other. -* Mutual exclusion means "Only one thread can do something at a time" -* A common tool for this is a mutex lock +## `Send` - ---- +* Indicates that the type is safe to *send* between threads. +* `Rc` does not implement this trait, because it is not thread safe. -# Sharing Resources With Mutual Exclusion +## `Sync` -```c -static int x = 0; -static mtx_t x_lock; +* Indicates that the type implementing `Send` can be referenced from multiple threads +* For example, `RefCell` from last lecture implements `Send` but not `Sync` +* `Rc` does not implement `Sync` either -static void thread(void) { - mtx_lock(&x_lock); - int temp = x; - temp += 1; - x = temp; - mtx_unlock(&x_lock); -} -// -``` -- Only one thread can hold the mutex lock at a time + -- This provides *mutual exclusion*--only one thread may access `x` at the same time. + +--- + +# Using `Send` and `Sync` +* It is generally rare that you would implement these traits yourself + * Structs containing all `Send`/`Sync` types automatically derive `Send`/`Sync` + * Explicitly implementing either one requires using `unsafe` +* This would be an example of a trait you might want to *unimplement* + * e.g. If you are doing something with `unsafe` that is not thread-safe + * `impl !Send for CoolType {}` + + --- + # Threads in Rust +Some implementation gotcha's + --- # Threads in Rust @@ -505,142 +1017,9 @@ println!("Final value of x: {}", *x.lock().unwrap()); * `x` is 20, *every time*. * And it is illegal for it to be anything else in safe Rust. ---- - -# Parallelism Checkpoint -Up until now, we have been talking about parallelism with *shared state*. Let's shift gears and talk about *message passing*. - ---- - -# Message Passing - -Rather than sharing state between threads, an increasingly popular approach to safe concurrency is message passing. -* In this approach, threads communicate with each other through channels -* Golang famously utilizes this approach - - ---- - -# Message Passing Example - -```rust -let (tx, rx) = mpsc::channel(); -``` -* Channels have two halves, a transmitter and a receiver -* Connor writes "Review the ZFOD PR" on a rubber duck and it floats down the river (transmitter) - * Ben finds the duck downstream, and reads the message (receiver) -* Note that communication is one-way here -* Note also that each channel can only transmit/receive one type - * e.g. `Sender`, `Receiver` can't transmit integers - ---- - -# Message Passing Example - -```rust -let (tx, rx) = mpsc::channel(); - -thread::spawn(move || { // Take ownership of `tx` - let val = String::from("review the ZFOD PR!"); - tx.send(val).unwrap(); // Send val through the transmitter -}); - -let received = rx.recv().unwrap(); // receive val through the receiver -println!("I am too busy to {}!", received); -``` -* Note that, after we send `val`, we no longer have ownership of it! - ---- - -# Message Passing Example -We can also use receivers as iterators! - -```rust -let (tx, rx) = mpsc::channel(); - -thread::spawn(move || { // Take ownership of `tx` - let val = String::from("review the ZFOD PR!"); - tx.send(val).unwrap(); // Send val through the transmitter - tx.send("buy Connor lunch".into()).unwrap(); -}); - -for msg in rx { - println!("I am too busy to {}!", msg); -} -``` -* Wait, what does `mpsc` stand for? - ---- - -# `mpsc` ⟹ Multiple Producer, Single Consumer - -This means we can `clone` the transmitter end of the channel, and have *multiple producers*. - -```rust -let (tx, rx) = mpsc::channel(); - -let tx1 = tx.clone(); -thread::spawn(move || { // owns tx1 - tx1.send("yo".into()).unwrap(); - thread::sleep(Duration::from_secs(1)); -}); - -thread::spawn(move || { // owns tx - tx.send("hello".into()).unwrap(); - thread::sleep(Duration::from_secs(1)); -}); - -for received in rx { - println!("Got: {}", received); -} -``` - ---- - -# `Send` and `Sync` - ---- - -# `Send` and `Sync` - -Everything we have gone over so far is a *standard library* feature. The language itself provides two marker traits to enforce safety when dealing with multiple threads, `Send` and `Sync`. - - - - ---- - -# `Send` vs. `Sync` - - -## `Send` - -* Indicates that the type is safe to *send* between threads. -* `Rc` does not implement this trait, because it is not thread safe. - - -## `Sync` - -* Indicates that the type implementing `Send` can be referenced from multiple threads -* For example, `RefCell` from last lecture implements `Send` but not `Sync` -* `Rc` does not implement `Sync` either - - - --- -# Using `Send` and `Sync` -* It is generally rare that you would implement these traits yourself - * Structs containing all `Send`/`Sync` types automatically derive `Send`/`Sync` - * Explicitly implementing either one requires using `unsafe` -* This would be an example of a trait you might want to *unimplement* - * e.g. If you are doing something with `unsafe` that is not thread-safe - * `impl !Send for CoolType {}` - - - ---- # More Shared State Primitives @@ -692,20 +1071,6 @@ thread::spawn(move || { --- -# One more thing... - ---- - -# `std::sync::atomic` - -Rust provides atomic primitive types, like `AtomicBool`, `AtomicI8`, `AtomicIsize`, etc. -* Safe to share between threads (implementing `Sync`), providing ways to access the values atomically from any thread -* 100% lock free, using bespoke assembly instructions -* Highly performant, but very difficult to use -* Requires an understanding of *memory ordering*—one of the most difficult topics in computer systems -* We won't cover it further in this course, but the API is largely 1:1 with the C++20 atomics. - ---- # Review: "Fearless Concurrency" @@ -715,6 +1080,7 @@ What we have gone over today is referred to as "fearless concurrency" in the rus * Subjectively, this may be the single best reason to use this language * Both parallelism and concurrency, as introduced in this lecture, benefit from these guarantees + --- # Next Lecture: Concurrency