Working with GPUs and Cuda

[sdspieg]

Could somebody please just post instructions for how to get spaCy to work with CUDA on Ubuntu or Windows? CUDA is now at version 12.1. Is it really true that we have to go back to 11.6 to get this to work with pytorch? And should we then do this in a separate virtual environment where we then set the CUDA_HOME environment variable to the path of the older CUDA version? WIll that interfere with the (more up-to-date) host version of CUDA? Are there any other tips or tricks to take maximal advantage of these new GPU beasts? It would be really great for non-experts like myself to have somebody who knows more about this to walk us through this. Thanks!

[adrianeboyd]

We realize that installing CUDA and PyTorch can be frustrating, but spacy doesn't have any unusual project-specific requirements. The main things to get installed and working are CUDA, CuPy, and optionally PyTorch for transformer models.

Each project provides better up-to-date install guides than we would be able to maintain:

• CUDA: https://docs.nvidia.com/cuda/cuda-quick-start-guide/index.html
• CuPy: https://docs.cupy.dev/en/stable/install.html
• PyTorch: https://pytorch.org/get-started/locally/

Right now I do think it is easier to stick with CUDA 11.x. CUDA 11.8 should be fine with the current PyTorch/CuPy releases. If you're looking for the CUDA 11.8 links on the nvidia website:

• https://docs.nvidia.com/cuda/archive/11.8.0/
• https://developer.nvidia.com/cuda-11-8-0-download-archive

Our general install advice related to installing transformers + PyTorch: https://spacy.io/usage/embeddings-transformers/#transformers-installation. You typically only need to set CUDA_PATH if you have more than one version of CUDA installed in parallel or CUDA is not in a standard location in your path.