Skip to content

Latest commit

 

History

History
203 lines (155 loc) · 3.45 KB

README.md

File metadata and controls

203 lines (155 loc) · 3.45 KB

Deprecated

The whole GKVS project was deprecated, please use https://github.com/shvid/datafire

gkvs-cpp

Server Node written on C++, keep it as legacy code. The actual server node will be in GoLang.

Generic Key-Value Storage

Universal API to access data in multiple key-value databases (proxy and state-less)

Supported Databases

  • Redis
  • Aerospike

Coming databases

  • MongoDB
  • FoundationDB

Vision

  • Zero-config
  • Server written on C/C++, client on Java
  • Protocol on gRPC/Protobuf
  • Schema-less, all data in MessagePack (aka binary JSON)
  • Automatic load-balancer could work in front of GKVS nodes (must support gRPC protocol)
  • Supports SSL

API Design

  • Key is byte array or crypto hash, NOT NULL
  • Value is byte array in MessagePack, NOT NULL

Basic Operations:

  • get - gets value (with/only metadata) by key. If not found return null.
  • put - puts value by key
  • compareAndSet - puts value by key only if version match
  • remove - removes value by key

Bulk Operations:

  • multiGet - gets multi entries in one request

Stream Operations:

  • getAll - gets values as stream
  • putAll - puts values as stream
  • removeAll - removes values as stream
  • scan - query key-values as stream

Errors

All errors are pass-through to client.

Dependencies

Required libraries:

  • Protobuf 3.5.1
  • GRPC++ 1.2.0
  • Aerospike Client C
  • Hiredis
  • LuaJIT 2.0.5
  • Msgpack 3.0 and upper
  • LibZ
  • OpenSSL

Quick Start

  • Install Build Essential
# Ubuntu
sudo apt-get install build-essential cmake pkg-config autoconf automake libtool
sudo apt-get install clang libc++-dev g++ libc6-dev ncurses-dev

# MacOs
sudo xcode-select --install
brew install autoconf automake libtool shtool cmake
  • Install OpenSSL
# Ubuntu
sudo apt-get install libssl-dev

# MacOs
brew install openssl
  • Install LibEvent (needed for Aerospike Client)
# Ubuntu
sudo apt-get install libevent-dev

# MacOs
brew install libevent
  • Install gflags (needed for GRPC++ and GKVS)
# Ubuntu
sudo apt-get install libgflags-dev

# MacOs
brew install gflags
  • Install LuaJit (needed by GKVS)
# Ubuntu
git clone http://luajit.org/git/luajit-2.0.git
cd luajit-2.0
git checkout tags/v2.0.5
make
sudo make install

# MacOs
brew install luajit
  • Install golang (needed to build GRPC)
# Ubuntu
sudo apt-get install golang

# MacOs
brew install golang
  • Checkout GKVS and sub-modules
git clone https://github.com/gkvs/gkvs
cd gkvs
git submodule update --init --recursive
  • Install SubModules for MacOs
brew install hiredis
brew install protobuf grpc glog
  • Build SubModules Manually (recommended)

# Build MsgPack
cd modules/msgpack-c
mkdir build
cd build
cmake ..
make
# optionally - sudo make install
cd ../../..

# Build gRPC
cd modules/grpc
mkdir build
cd build
cmake ..
make
cd ../../..

# Build hiredis
cd modules/hiredis
make
# optionally - sudo make install
cd ../..

# Build Aerospike
cd modules/aerospike-client-c
make EVENT_LIB=libevent
cd ../..

# Build GKVS
cd gkvs
mkdir build
cd build
cmake ..
make

Run

GKVS reads LUA script on startup and initialize itself

# Must have local Redis running
gkvs ../example/redis1-lua.conf

# Must have local Aerospike running
gkvs ../example/as1-lua.conf

Configure

GKVS supports config scripts written in LUA language This is an example gkvs-lua.conf


add_cluster("redis1", "redis", { host = "127.0.0.1", port = 6379 } );

add_table("test", "redis1", { ttl = 100 } );

add_view("TEST", "redis1", "test");