How To Scale Your Model #2
Replies: 3 comments 1 reply
-
|
There's a comment box at the bottom of each section you can use to ask questions. We can't reply to every comment but we'll try to address as many as we can. We'd like to keep improving this book! Please ask questions or point out if we've made a mistake! Enjoy! |
Beta Was this translation helpful? Give feedback.
-
|
Just wanted to thank you and the team for writing this! I'm trying to self-study this material and also write a blog on ML Acceleration from first principles and this has been extremely helpful to go through. Very rare to find resources (especially on JAX/TPUs!) like this! 😃 |
Beta Was this translation helpful? Give feedback.
-
|
In the Matrix multiplication paragraph of the Roofline paragraph, it is not explicitly mentioned why the matmul take 2BDF flops - it's because of multiply-accumulate right? |
Beta Was this translation helpful? Give feedback.
-
|
Yes exactly, https://jax-ml.github.io/scaling-book/transformers/ has more details on this. I'll update the footnote too. |
Beta Was this translation helpful? Give feedback.
-
Discussion for the introduction of "How To Scale Your Model"
Beta Was this translation helpful? Give feedback.
All reactions