NLP Microservice

The goal of this project is to provide a gRPC server for resource-heavy NLP tasks—for instance, computing vectors/embeddings for words or sentences. By using protobuf internally, our NLP server provides native and strongly typed interfaces for many programming languages. There are multiple advantages that arise from outsourcing such computations to such a server:

• If multiple apps rely on NLP, the underlying models (which are usually quite large) only need to be loaded once into the main memory.
• All programming languages supported by gRPC get easy access to state-of-the-art NLP architectures (e.g., transformers).
• The logic is consolidated at a central place, drastically decreasing the maintenance effort required.

In addition to the server, we also provide a client containing convenience functions. This makes it easier for python applications to interact with the gRPC server. We will discuss the client at the end of this README.

Installation and Setup

We are using nix and poetry to manage the dependencies and provide a ready-to-use Docker image.

Docker (recommended)

The container caches the downloaded models, so you should not pass --rm to docker run.

docker run ghcr.io/recap-utr/nlp-service:latest "0.0.0.0:50100"

Nix (advanced)

nix run github:recap-utr/nlp-service -- "127.0.0.1:50100"
# or after cloning this repository
nix develop -c poetry run python -m nlp_service "127.0.0.1:50100"

Poetry (advanced)

# The server dependencies are optional, thus they have to be installed explicitly.
poetry install --extras all
# To run the server, you need to specify the address it should listen on.
# In this example, it should liston on port 5678 on localhost.
poetry run python -m nlp_service "127.0.0.1:50100"

General Usage

Once the server is running, you are free to call any of the functions defined in the underlying protobuf file. The corresponding documentation is located at the Buf Schema Registry. Please note: The examples here use the Python programming language, but are also directly applicable to any other language supported by gRPC.

import grpc
from arg_services.nlp.v1 import nlp_pb2, nlp_pb2_grpc

# First of all, we are creating a channel (i.e., establish a connection to our server)
channel = grpc.insecure_channel("127.0.0.1:5678")

# The channel can now be used to create the actual client (allowing us to call all available functions)
client = nlp_pb2_grpc.NlpServiceStub(channel)

# Now the time has come to prepare our actual function call.
# We will start by creating a very simple NlpConfig with the default spacy model.
# FOr details about the parameters, please have a look at the next section.
config = nlp_pb2.NlpConfig(
  language="en",
  spacy_model="en_core_web_lg",
)

# Next, we will build a request to query vectors from our server.
request = nlp_pb2.VectorsRequest(
  # The first parameter is a list of strings that shall be embedded by our server.
  texts=["What a great tutorial!", "I will definitely recommend this to my friends."],
  # Now we need to specify which embeddings have to be computed. In this example, we create one vector for each text
  embedding_levels=[nlp_pb2.EmbeddingLevel.EMBEDDING_LEVEL_DOCUMENT],
  # The only thing missing now is the spacy configuration we created in the previous step.
  config=config
)

# Having created the request, we can now send it to the server and retrieve the corresponding response.
response = client.Vectors(request)

# Due to technical constraints, we cannot directly transfer numpy arrays, thus we convert our response.
vectors = [np.array(entry.document.vector) for entry in response.vectors]

Advanced Usage

A central piece for all available function is the NlpConfig message, allowing you to create even complex embedding models easily. In addition to its documentation, we will in the following present some examples to demonstrate the possibilities you have.

from arg_services.nlp.v1 import nlp_pb2

# In the example above, we already introduced a quite basic config:
config = nlp_pb2.NlpConfig(
  # You have to provide a language for every config: https://spacy.io/usage/models#languages
  language="en",
  # Also, you need to specify the model that spacy should load: https://spacy.io/models/en
  spacy_model="en_core_web_lg",
)

# A central feature of our library is the possibility to combine multiple embedding models, potentially capturing more contextual information.
config = nlp_pb2.NlpConfig(
  language="en",
  # This parameter expects a list of models. If you pass more than one, the respective vectors are **concatenated** to each other
  # (e.g., two 300-dimensional embeddings will result in a 600-dimensional one).
  # This approach is based on https://arxiv.org/abs/1803.01400
  embedding_models=[
    nlp_pb2.EmbeddingModel(
      # First select the type of model you would like to use (e.g., SBERT/Sentence Transformers).
      model_type=nlp_pb2.EmbeddingType.EMBEDDING_TYPE_SENTENCE_TRANSFORMERS,
      # Then select the actual model.
      # Any of those specified on the website (https://www.sbert.net/docs/pretrained_models.html) are allowed.
      model_name="all-mpnet-base-v2"
    ),
    nlp_pb2.EmbeddingModel(
      # It is also possible to use a standard spacy model
      model_type=nlp_pb2.EmbeddingType.EMBEDDING_TYPE_SPACY,
      model_name="en_core_web_lg",
      # Since we have selected a word embedding (i.e., it cannot directly encode sentences), the token vectors need to be aggregated somehow.
      # The default strategy is to use the arithmetic mean, but you are free to use other strategies (e.g., the geometric mean).
      pooling_type=nlp_pb2.Pooling.POOLING_GMEAN
    ),
    nlp_pb2.EmbeddingModel(
      model_type=nlp_pb2.EmbeddingType.EMBEDDING_TYPE_SPACY,
      model_name="en_core_web_lg",
      # Alternatively, it is also possible to use the generalized mean / power mean.
      # In this example, the selected pmean corresponds to the geometic mean (thus this embedding is identical to the previous one).
      # This approach is based on https://arxiv.org/abs/1803.01400
      pmean=0
    )
  ]
  # This setting is now optional and only needed if you need spacy features (e.g., POS tagging) besides embeddings.
  # spacy_model="en_core_web_lg",
)

# If computing the similarity between strings, you get one additional parameter.
config = nlp_pb2.NlpConfig(
  language="en",
  # To keep the example simple, we will now only use a single spacy model instead of the more powerful embedding models.
  # However, it is of course possible to use them here as well.
  spacy_model="en_core_web_lg",
  # If not specified, we will always use the cosine similarity when comparing two strings.
  # As indicated in a recent paper (https://arxiv.org/abs/1904.13264), you may achieve better results with alternative approaches like DynaMax Jaccard.
  # Please note that this particular method ignores your selected pooling method due to the fact that even plain word embeddings are not pooled at all.
  similarity_method=nlp_pb2.SimilarityMethod.SIMILARITY_METHOD_DYNAMAX_JACCARD
)

# It is also possible to determine a similarity score without the use of embeddings.
config = nlp_pb2.NlpConfig(
  language="en",
  spacy_model="en_core_web_sm",
  # Traditional metric (Jaccard similarity and Levenshtein edit distance) are also available
  similarity_method=nlp_pb2.SimilarityMethod.SIMILARITY_METHOD_EDIT
)