Skip to content

Installation

Jump to bottom
Konstantinos Papadopoulos edited this page Mar 1, 2017 · 89 revisions

NSTAT Installation

The complete NSTAT architecture is depicted in the Figure below.

In princple, NSTAT is a multinode application software consisting of the NSTAT node itself, the SouthBound emulator node, the NorthBound emulator node and the SDN Controller node. For creating the NSTAT testing environment, once can either build all individual nodes following a step by step guide, or either refer to the automated process as this described in the nstat-cperf repository.

Nodes deployment

Essential tools

In order to deploy the required nodes of NSTAT the following tools are required

  • docker: docker installation instructions on linux machines can be found here
  • docker-compose: docker-compose installation instructions on linux machines can be found here

After installing the aforementioned tools, there are two deployment options:

  • deploy Docker images locally, using the provided Dockerfiles.
  • use the prebuild Docker images.

Before continuing with the deployment section, the user should be added to the docker group

sudo usermod -a -G docker $(whoami)

Deploy Docker images locally

  • Clone NSTAT repository

    git clone https://github.com/intracom-telecom-sdn/nstat.git

  • cd into the deploy/docker folder and select whether you want docker image with proxy or no_proxy settings.

  • Inside the corresponded folder you will find a file with the name Dockerfile. From this path execute the command

    docker compose -t base_nstat_image .

    in order to create the base NSTAT image, necessary for the containers deployment of each NSTAT node.

    In case you use the proxy Dockerfile before you run the command to build the Docker image, edit the Dockerfile to change the proxy settings according to your needs

    ENV http_proxy="http://<your_proxy>:<your_port>/"
ENV https_proxy="http://<your_proxy>:<your_port>/"

RUN echo 'Acquire::http::Proxy "http://<your_proxy>:<your_port>/";' | tee -a /etc/apt/apt.conf
RUN echo 'Acquire::https::Proxy "http://<your_proxy>:<your_port>/";'| tee -a /etc/apt/apt.conf

RUN echo "http_proxy=http://<your_proxy>:<your_port>/" | tee -a /etc/environment
RUN echo "https_proxy=http://<your_proxy>:<your_port>/" | tee -a /etc/environment
RUN echo "no_proxy=127.0.0.1,localhost,<ip_addresses_of_containers>" | tee -a /etc/environment
RUN echo "HTTP_PROXY=http://<your_proxy>:<your_port>" | tee -a /etc/environment
RUN echo "HTTPS_PROXY=http://<your_proxy>:<your_port>/" | tee -a /etc/environment
RUN echo "NO_PROXY=127.0.0.1,localhost,<ip_addresses_of_containers>" | tee -a /etc/environment

    Once the image is built, you can verify its existence using the following command

    docker images

    the output of this command should look like

    REPOSITORY         TAG                   IMAGE ID          CREATED           SIZE

<repo_name>       base_nstat_image      c2c3152907b5      4 minutes ago     275.1 MB

  • Create on the same folder path the docker-compose.yml. Edit the file and paste the following content

    version: '2'

services:
  nstat:
    image: <repo_name>:base_nstat_image
    privileged: true
    container_name: nstat
    hostname: nstat-node
    networks:
      nstat_net:
        ipv4_address: 10.0.1.10

  controller:
    image: <repo_name>:base_nstat_image
    privileged: true
    container_name: controller
    hostname: ctrl-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.11

  mtcbench:
    image: <repo_name>:base_nstat_image
    privileged: true
    container_name: mtcbench
    hostname: mtcbench-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.12

  nbgen:
    image: <repo_name>:base_nstat_image
    privileged: true
    container_name: nbgen
    hostname: nbgen-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.12

  mn01:
      image: <repo_name>:base_nstat_image
      privileged: true
      container_name: mn-01
      hostname: mn01-node
      depends_on:
        - nstat
      networks:
        nstat_net:
          ipv4_address: 10.0.1.13

  mn02:
    image: <repo_name>:base_nstat_image
    privileged: true
    container_name: mn-02
    hostname: mn02-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.14

networks:
  nstat_net:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 10.0.1.0/24
        gateway: 10.0.1.1

    The above sample of docker-compose.yml, is suitable for small scale testing. If you want to start a bigger topology of switches, using [Multinet]() as a SouthBound emulator, you should follow the steps described in the next section Required actions on the host sidebefore any other action described in this section. Additionally you should add the following lines in the end of all nodes created in thedocker-compose.yml` file

        ulimits:
    nofile:
      soft: 1000000
      hard: 1000000

    so for instance the nstat node will be look like the following sample in the docker-compose.yml

    nstat:
  image: intracom/nstat:proxy
  privileged: true
  container_name: nstat
  hostname: nstat-node
  networks:
    nstat_net:
      ipv4_address: 10.0.1.10
  ulimits:
    nofile:
      soft: 1000000
      hard: 1000000

    To increase the scale of a Multinet network add at most 600 switches per multinet worker and increase the number of Multinet nodes in the docker-compose.yml file. For more details see the corresponded tests description that uses multinet as a SouthBound emulator.

  • Run the command

    docker-compose up -d

    in order to start the containers and topologies described in the docker-compse.yml file

    Created containers can be verified using the command

    docker ps -a

    The output of this command should look like the following sample

    CONTAINER ID      IMAGE                                   COMMAND                  CREATED              STATUS             PORTS    NAMES
 4c05473bb7c8     <repo_name>:base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   controller
 72e4572878e2     <repo_name>:base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   mn-01
 13d191e6ef75     <repo_name>:base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   mn-02
 60db64735a26     <repo_name>:base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   nstat
 2b7ca294d1b4     <repo_name>:base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   nbgen
 72e4572878e2     <repo_name>:base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   mtcbench

Use prebuild Docker images from Docker Hub

There are prebuild images that can be pulled directly from Docker Hub. In this way we bypass the build steps of the image. You can see those images on the corresponded Docker Hub repository

In order to use those images to create the Docker containers for NSTAT nodes

  • Create the docker-compose.yml files that describes the nodes and paste the following context

    version: '2'

services:
  nstat:
    image: intracom/nstat:proxy
    privileged: true
    container_name: nstat
    hostname: nstat-node
    networks:
      nstat_net:
        ipv4_address: 10.0.1.10

  controller:
    image: intracom/nstat-sdn-controllers:proxy
    privileged: true
    container_name: controller
    hostname: ctrl-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.11

  mtcbench:
    image: intracom/mtcbench:proxy
    privileged: true
    container_name: mtcbench
    hostname: mtcbench-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.12

  nbgen:
    image: intracom/nstat-nb-generator:proxy
    privileged: true
    container_name: nbgen
    hostname: nbgen-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.12

  mn01:
      image: intracom/multinet:proxy
      privileged: true
      container_name: mn-01
      hostname: mn01-node
      depends_on:
        - nstat
      networks:
        nstat_net:
          ipv4_address: 10.0.1.13

  mn02:
    image: intracom/multinet:proxy
    privileged: true
    container_name: mn-02
    hostname: mn02-node
    depends_on:
      - nstat
    networks:
      nstat_net:
        ipv4_address: 10.0.1.14

networks:
  nstat_net:
    driver: bridge
    ipam:
      driver: default
      config:
      - subnet: 10.0.1.0/24
        gateway: 10.0.1.1

    if you want to deploy the no_proxy node containers just remove the :proxy suffix from the image: sections in the above file context

    The above sample of docker-compose.yml, is suitable for small scale testing. If you want to start a bigger topology of switches, using multinet SouthBound emulator, you should follow the steps described in the next section Required actions on the host sidebefore any other action described in this section. Additionally you should add the following lines in the end of all nodes created in thedocker-compose.yml` file

        ulimits:
    nofile:
      soft: 1000000
      hard: 1000000

    so for instance the nstat node will be look like the following sample in the docker-compose.yml

    nstat:
  image: intracom/nstat:proxy
  privileged: true
  container_name: nstat
  hostname: nstat-node
  networks:
    nstat_net:
      ipv4_address: 10.0.1.10
  ulimits:
    nofile:
      soft: 1000000
      hard: 1000000

    To increase the scale of a Multinet network add at most 600 switches per multinet worker and increase the number of Multinet nodes in the docker-compose.yml file. For more details see the corresponded tests description that uses multinet as a SouthBound emulator.

  • Run the command

    docker-compose up -d

    in order to start the containers and topologies described in the docker-compse.yml file. The images should be pulled automatically from Docker Hub.

    Created containers can be verified using the command

    docker ps -a

    The output of this command should look like the following sample

    CONTAINER ID      IMAGE                       COMMAND                  CREATED              STATUS             PORTS    NAMES
 4c05473bb7c8     base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   controller
 72e4572878e2     base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   mn-01
 13d191e6ef75     base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   mn-02
 60db64735a26     base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   nstat
 2b7ca294d1b4     base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   nbgen
 72e4572878e2     base_nstat_image           "/usr/sbin/sshd -D"   About a minute ago     Up About a minute  22/tcp   mtcbench

Required actions on the host side

In general there is no need to make any changes on the host machine, unless you want to generate a large scale topology, using Multinet SouthBound emulator. If the scale of the topology is more than 2000 switches (that number is not a standard and depends on the system the test is running), then you must change some limits of the operating system before you deploy any Docker containers, described in the previous section.

In order to change those limits do the following steps

  • Edit file /etc/sysctl.conf and paste in the end the following lines

    kernel.pty.max = 60000
fs.file-max = 26233259
net.core.wmem_max = 12582912
net.core.rmem_max = 12582912
net.core.rmem_default = 12582912
net.core.wmem_default = 12582912

  • Edit /etc/security/limits.conf and paste before the line #End of file the following lines

    * hard nofile 1000000
* soft nofile 1000000

  • Reboot the system so the changes will take effect

Repositories of NSTAT nodes

For each NSTAT node a separate repository has been created

Intro

Stress Tests

Emulators

Monitoring tools

  • OpenFlow monitoring tools

Design

Releases

ODL stress tests performance reports

Sample Performance Results

Clone this wiki locally