Skip to content

POSIX port

Felipe Aburaya edited this page Dec 31, 2017 · 11 revisions

The project has been ported to POSIX and every release is tested in linux. The following development environment is being used:

Clang was chosen for being a modern and fast toolchain, which is already fully compliant with C++11 and can compile everything MSVC 2013 can (and more). Previous releases of 3FD made use of libc++ as well, but now libstdc++ is default and you should be aware of this because that means you cannot reuse your past builds of POCO and Boost with 3FD from now on. You can set up this development environment by running the shell script setup.sh, or you can do it manually yourself following the steps below.

1. Install the toolchain

Install the packages for Clang version 3.8 or later (the port is being tested with version 3.8, but version 3.3 is likely to be enough, because it is already C++11 fully compliant).

2. Install CMake

The easiest way is to use CMake as make system. You can install CMake in ubuntu running the command

sudo apt install cmake

3. Build POCO ++ Libraries

Build the POCO C++ libraries using Clang and libc++ too. You will need only Foundation, XML, Utils and Data/ODBC (built as static libraries). The remaining modules can be omitted. For that you can follow these instructions. You can learn more about the building process of POCO here, but it suffices to say that you will need ODBC and OpenSSL

sudo apt install unixodbc unixodbc-dev openssl libssl-dev

then you will run

./configure --omit=Data/MySQL,MongoDB,PageCompiler --config=Linux-clang --static --no-tests --no-samples --prefix=/opt/poco-1.7.8p3
make -s -j4
sudo make install

Notice that you should create and use a directory like /opt/poco-1.7.8p3, which is the location where the framework projects expect to find the libraries for linking. The exact location is hardcoded into the build scripts and make files.

4. Build Boost C++ Libraries

Download the source for Boost C++ libraries v1.6x and, once again, build it with clang. First you will build and install Boost.build. In order to get the libraries (static and with shared link to runtime) with debug symbols, you need to run

./tools/build/b2 -j 2 variant=debug link=static threading=multi toolset=clang cxxflags="-std=c++11" runtime-link=shared --layout=tagged

Finally, in order to get the same libraries in release mode, you will run

./tools/build/b2 -j 2 variant=release link=static threading=multi toolset=clang cxxflags="-std=c++11" runtime-link=shared --layout=tagged

At the end you can manually copy the compiled libraries and headers to another location, such as /opt/boost-1.65.1, which is the location where the framework projects expect to find the libraries for linking. The exact location is hardcoded into the make files.

5. Build googletest framework

You can skip this step if you do not plan to run 3FD unit & integration tests.

No need to download googletest because it is already included in 3FD source code. But you are going to need CMake. Again, clang need to be your toolchain, so when you run cmake, make sure to pass the following arguments:

-DCMAKE_CXX_COMPILER="clang++" -DCMAKE_CXX_FLAGS="-stdlib=libc++"

In fact, such options are already applied by configure.sh as described in the next step.

6. Prepare to make integration tests work

This is optional, but you might want to run the integration tests in order to test how compliant to the framework requirements your environment is. The modules that require special attention are the OpenCL wrappers and MSSQL broker helpers.

6.1. OpenCL

I test the OpenCL wrappers in my Linux VM, so the OpenCL tests are already adapted to use the CPU as the accelerator device. You can change it to GPU if you are running your POSIX in real hardware with OpenCL enabled drivers. My testing environment has AMD hardware, so I have installed the AMD APP SDK, which should be enough to make things work. However, those things rarely install without glitches. In my case, the installation scripts always fail just after registering the OpenCL device vendor in /etc/OpenCL/vendors. Then I need to go into the AMD APP SDK installation directory (usually /opt/AMDAPPSDK-X.Y) and

  1. Merge the directory lib/sdk into lib (overwriting the link libOpenCL.so)
  2. Add this lib path to /etc/ld.so.conf.d/amdapp_x86_64.conf (for x64)
  3. Run ldconfig (with -v parameter to check if everything is okay)
  4. Test the setup running bin/clinfo that comes with the AMD APP SDK

Notice that even when you set everything right, some tests might not pass. In my Linux environment, for example, the CPU device is not capable of enqueuing an asynchronous buffer mapping with out-of-order execution. Those things can happen even on Windows. For example, I have an Intel Core i5 machine whose integrated GPU cannot perform most of the buffer mapping operations. I guess these are short-comings of the OpenCL implementations of those vendors. (Normally, AMD hardware devices are the most compliant with Khronos OpenCL standard.)

6.2. Broker

The helpers for Microsoft SQL Server Service Broker rely on an ODBC connection to the SQL Server instance. For that to work, you need to make sure that

  • The SQL Server instance allows connections via TCP (take note of the port, which is 1433 by default)
  • The firewall allows your Linux box to connect to such TCP port
  • You have created the database and login for tests, which can be done by running the script CreateMsSqlSvcBrokerDatabase.sql
  • You have installed the Microsoft ODBC driver for SQL Server
  • The connection string is right in IntegrationTests.3fd.config, which means there you need to have the version of the driver you installed and the network address of you SQL Server instance (including the TCP port)

ATTENTION!

Make sure that the MSSQL ODBC driver is correctly set for all ODBC driver managers. For example, I have 3 driver managers in my Linux box:

  1. iODBC installed via apt package manager
  2. unixODBC installed via apt package manager (this version has the dynamically linked libraries)
  3. unixODBC built from source (this version has the statically linked libraries)

My POCO C++ libraries were built using unixODBC static libraries in order to reduce dependency on 3rd party stuff, and the installation of such driver manager is in /usr/local, which uses odbcinst.ini in /usr/local/etc/ instead of /etc. After setup of OpenCL environment (which is completely unrelated), I saw the ODBC connections failing because they could no longer find the driver. I fixed that appending the content of /opt/microsoft/msodbcsql/etc/odbcinst.ini to /usr/local/etc/odbcinst.ini. Voilá, ODBC was working again ;)

7. Build 3FD

Edit the shell script configure.sh so as to set the correct location for your builds of POCO and Boost, and only then you can run build.sh:

~/projects/3fd $ source ./configure.sh
...
~/projects/3fd $ ./build.sh

Notice that the script configure.sh invokes make clean and regenerates the makefile from CMakeLists.txt.

8. Link your project to 3FD

Once the framework is compiled, you are going to set up your solution to link to the framework just like you do for any other static library. Then there are just a couple of things to set:

  • In your enviroment, define the macros ENABLE_3FD_CST and ENABLE_3FD_ERR_IMPL_DETAILS (recommended, so as to enable the stack tracing feature in release mode, plus exception detailed info) and NDEBUG in release mode;

  • Provide a XML configuration file for the framework instance used by you application. This file must have the same name of the application executable (including the extension) plus ".3fd.config". A template for this file is provided in the source.