Skip to content

Latest commit

 

History

History
324 lines (207 loc) · 9.15 KB

install.rst

File metadata and controls

324 lines (207 loc) · 9.15 KB

Installing DLR

Contents

Installing Pre-built DLR Wheels for Your Device

DLR has been built and tested against many devices. You can install DLR with the corresponding S3 link via

pip install  link-to-matching-wheel

Please see [Releases](https://github.com/neo-ai/neo-ai-dlr/releases) to download DLR wheels for each DLR release.

Building DLR from source

Building DLR consists of two steps:

  1. Build the shared library from C++ code (libdlr.so for Linux, libdlr.dylib for macOS, and dlr.dll for Windows).
  2. Then install the Python package dlr.

Note

Use of Git submodules

DLR uses Git submodules to manage dependencies. So when you clone the repo, remember to specify --recursive option:

git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr

Building on Linux

Requirements

Ensure that all necessary software packages are installed: GCC (or Clang), CMake, and Python. For example, in Ubuntu, you can run

sudo apt-get update
sudo apt-get install -y python3 python3-distutils build-essential cmake curl ca-certificates
curl https://bootstrap.pypa.io/pip/3.6/get-pip.py -o /tmp/get-pip.py
sudo python3 /tmp/get-pip.py
rm /tmp/get-pip.py
sudo pip3 install -U pip setuptools wheel

Building for CPU

First, clone the repository.

git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr

Create a subdirectory build:

mkdir build
cd build

Invoke CMake to generate a Makefile and then run GNU Make to compile:

cmake ..
make -j4         # Use 4 cores to compile sources in parallel

Once the compilation is completed, install the Python package by running setup.py:

cd ../python
python3 setup.py install --user

Building for NVIDIA GPU on Jetson Devices

By default, DLR will be built with CPU support only. To enable support for NVIDIA GPUs, enable CUDA, CUDNN, and TensorRT by calling CMake with these extra options.

DLR requires CMake 3.13 or greater. First, we will build CMake from source.

sudo apt-get install libssl-dev
wget https://github.com/Kitware/CMake/releases/download/v3.17.2/cmake-3.17.2.tar.gz
tar xvf cmake-3.17.2.tar.gz
cd cmake-3.17.2
./bootstrap
make -j4
sudo make install

Now, build DLR.

git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr
mkdir build
cd build
cmake .. -DUSE_CUDA=ON -DUSE_CUDNN=ON -DUSE_TENSORRT=ON
make -j4
cd ../python
python3 setup.py install --user

Building for NVIDIA GPU (Cloud or Desktop)

By default, DLR will be built with CPU support only. To enable support for NVIDIA GPUs, enable CUDA, CUDNN, and TensorRT by calling CMake with these extra options.

If you do not have a system install of TensorRT, first download the relevant .tar.gz file from https://developer.nvidia.com/nvidia-tensorrt-download Please follow instructions from https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-tar to install TensorRT. Now, provide the extracted .tar.gz folder path to -DUSE_TENSORRT when configuring cmake.

If you have a system install of TensorRT via Deb or RPM package, you can instead use -DUSE_TENSORRT=ON which will find the install directory automatically.

git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr
mkdir build
cd build
cmake .. -DUSE_CUDA=ON -DUSE_CUDNN=ON -DUSE_TENSORRT=/path/to/TensorRT/
make -j4
cd ../python
python3 setup.py install --user

See Additional Options for TensorRT Optimized Models to learn how to enable FP16 precision and more for your Neo optimized models which use TensorRT.

Building for OpenCL Devices

Similarly, to enable support for OpenCL devices, run CMake with -DUSE_OPENCL=ON:

git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr
mkdir build
cd build
cmake .. -DUSE_OPENCL=ON
make -j4
cd ../python
python3 setup.py install --user

Building on macOS

Install CMake from Homebrew:

brew update
brew install cmake
git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr
mkdir build
cd build
cmake ..
make -j4

NVIDIA GPUs are not supported for macOS target.

Once the compilation is completed, install the Python package by running setup.py:

cd ../python
python3 setup.py install --user --prefix=''

Building on Windows

DLR requires Visual Studio 2017 as well as CMake.

In the DLR directory, first run CMake to generate a Visual Studio project:

git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr
mkdir build
cd build
cmake .. -G"Visual Studio 15 2017 Win64"

If CMake run was successful, you should be able to find the solution file dlr.sln. Open it with Visual Studio. To build, choose Build Solution on the Build menu.

NVIDIA GPUs are not yet supported for Windows target.

Once the compilation is completed, install the Python package by running setup.py:

cd ../python
python3 setup.py install --user

Building for Android on ARM

Android build requires Android NDK. We utilize the android.toolchain.cmake file in NDK package to configure the crosscompiler

Also required is NDK standlone toolchain. Follow the instructions to generate necessary build-essential tools.

Once done with above steps, invoke cmake with following commands to build Android shared lib:

git clone --recursive https://github.com/neo-ai/neo-ai-dlr
cd neo-ai-dlr
mkdir build
cd build
cmake .. -DANDROID_BUILD=ON \
  -DNDK_ROOT=/path/to/your/ndk/folder \
  -DCMAKE_TOOLCHAIN_FILE=/path/to/your/ndk/folder/build/cmake/android.toolchain.cmake \
  -DANDROID_PLATFORM=android-21

make -j4

ANDROID_PLATFORM should correspond to minSdkVersion of your project. If ANDROID_PLATFORM is not set it will default to android-21.

For arm64 targets, add

-DANDROID_ABI=arm64-v8a

to cmake flags.

Building for Android Archive (AAR) file

Install Android Studio.

cd aar
# create file local.properties
# put line containing path to Android/sdk
# sdk.dir=/Users/root/Library/Android/sdk

# Run gradle build
./gradlew assembleRelease

# dlr-release.aar file will be under dlr/build/outputs/aar/ folder
ls -lah dlr/build/outputs/aar/dlr-release.aar

Building DLR with TensorFlow C library

We can use DLR to run Tensorflow 2.x saved models (including TensorRT converted models) via TensorFlow C library.

TensorFlow C library can be downloaded from tensorflow.org or built from source.

TensorFlow C/C++ headers can be obtained by building the following Tensorflow target.

bazel build --config=opt tensorflow:install_headers

# The headers will be saved to bazel-bin/tensorflow/include

Another way to get TensorFlow C/C++ headers is to install Tensorflow python package using pip. The headers can be found inside the installation folder, e.g.

sudo pip3 install tensorflow==<tf_version>

# The headers can be found in /usr/local/lib/python3.x/dist-packages/tensorflow/include

To build DLR with TensorFlow C library:

# Build DLR with libtensorflow
cmake .. \
  -DWITH_TENSORFLOW2_LIB=<path to libtensorflow lib folder> \
  -DTENSORFLOW2_INCLUDE=<tensorflow include folder>
make -j8

# Test DLR with libtensorflow
export LD_LIBRARY_PATH=<path to libtensorflow lib folder>
./dlr_tensorflow2_test

Building DLR with Hexagon support

To build DLR with Hexagon compiled models support use flag -DWITH_HEXAGON=1

cmake .. -DWITH_HEXAGON=1
./dlr_hexagon_test

Validation After Build (Linux Only)

cd tests/python/integration/
python load_and_run_tvm_model.py
python load_and_run_treelite_model.py