Setup Guide: Multi GPU Training of Neural Networks

PMLS-Caffe also supports multi-GPU training of neural networks on one machine. If you want to use this feature, make sure you have successfully installed PMLS-Caffe by following our installation guide and you also have prior knowledge about how to start a training instance under PMLS-Caffe by reading our setup guide for distributed training.

Multi-GPU Training

To enable multiple-GPU training, one need to specify the GPU device IDs in the starting script. For example, suppose you are going to train GoogleNet using 2 machines, each of which has two GPUs with device ID 0 and 1, in total 4 GPUs.

1. First set the machine IPs and ports in the localserver.

2. Then specify device = [0, 1] in examples/googlenet/run_local.py, or if you prefer bash script, specify device IDs as device="0,1" and set num_app_threads=2 in example/googlenet/train_googlent.sh.

3. Start the script.

The log will show both GPUs are enabled for training in every machine.