Comprehensive Rust 🦀 translation of the book's code!

[nerdai]

Hey all,

First off, would like to say this is probably near the top, if not at the top, of my favourite DL books to read. I personally appreciated the careful explanations in every aspect of building an LLM -- I learned a lot in reading this book.

Second, I'm just about finished translating nearly all of the book's code into Rust using the Candle ML/DL framework. So, I thought it'd be a good time to share here in case there are others that might be interested in following this fantastic text with Rust "equivalent" code. Link to the repo and the docs are provided below. Contributions are welcome 🤗

Cheers!

• github repo
• docs (NOTE: I will publish a new version of the docs probably by end of this week that will include all of the ch07 translations, which are already merged in the main branch of the github repo.)

[rasbt]

Glad you enjoyed the book. And wow, your repo is an amazing project! Haven't dabbled with Rust, yet, but now there's one more reason to get into it 😊. Thanks for sharing!!

[nerdai]

💯 on your note dev vs. prod

So the rust translation is also able to run on cuda-enabled devices! (Candle also supports metal backend, but I haven't tested out Metal in my project).

# with cuda
cargo run --features cuda example 05.07

# without cuda
cargo run example 05.07

If you try and run with the cuda feature flag and you don't have a cuda device, then it should default to CPU. Similar to PyTorch, Candle has this device mapping logic Device::cuda_if_available().

[rasbt]

Oh that's great!

Btw regarding Metal, it works pretty well now as of a few months ago, but it you'll still see some numerical discrepancies vs CUDA or CPU. This becomes especially apparent during training runs where small rounding/precision differences add up over time. During inference, there's a non-noticeable difference though.

[nerdai]

Oh interesting! Thanks for this information. In that case, I should probably test these translations on Metal.

[rasbt]

I must say it got better in recent months though. And it mostly only affected training where numerical differences add up during backprop.

But yeah, here are some relevant discussions/observations from the Manning reader forum (you may have to be logged in to see the full reader comments; if you got the book from Amazon, there should be instructions on the first page on how to connect the book to the Manning account):

• https://livebook.manning.com/forum?product=raschka&comment=572340
• https://livebook.manning.com/forum?comment=572996
• https://livebook.manning.com/forum?comment=572762
• https://livebook.manning.com/forum?comment=573286

[nerdai]

Oh this is great! I'll give this a look. Thanks

[raravena80]

Related FWIW. Huggingface's TGI is written in Rust: https://huggingface.co/docs/text-generation-inference/en/index

[rasbt]

Whoa, I had no idea! Thanks for sharing!