From 1f7cdc4896fb0221a46ea58947d7e6ea27aae39c Mon Sep 17 00:00:00 2001 From: William Hiver Date: Fri, 9 Mar 2018 21:04:30 +0100 Subject: [PATCH] Update feature list and add perfs tips --- README.md | 33 +++++++++++++++++++-------------- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 85409c9..88b83a4 100644 --- a/README.md +++ b/README.md @@ -5,20 +5,12 @@ An Apache NiFi processor to encode and decode data using Google Protocol Buffers As this processor is a work-in-progress, the features already implemented and tested are checked. -### Main functionalities -- [x] Graphical configuration interface in Apache NiFi -- [x] Encode a JSON payload to Protocol Buffers -- [x] Decode a Protocol Buffers payload to JSON -- [x] Support dependencies in proto files (see [below](#usage)) -- [ ] Allow to encode/decode from/to XML, JSON and flowfile properties - -### Protobuf schema specification -- [x] Support compiled schema at a specified user location on the disk -- [ ] Support compiled schema embedded in the flowfile -- [x] Support embedding raw .proto files in the flowfile - -### Processor packaging -- [x] Provide a ready-to-use Docker image of Apache NiFi with the NiFi Protobuf Processor +- Encode/decode Protocol Buffer messages from/to JSON format +- Read a compiled schema file (`.desc`) from disk +- Use directly a raw `.proto` schema file, from disk or directly embedded in a property +- Can handle embedded `.proto` files at processor level (as a processor property) or directly in a flowfile property +- Support dependencies in proto files (see [below](#usage)) +- Provide a ready-to-use Docker image of Apache NiFi with the NiFi Protobuf Processor ## Installation @@ -80,6 +72,19 @@ However, if you cannot compile your `.proto` file, you can set it directly as a For now, the only structured format the processors can process is the JSON. In the future, there should be more formats available (XML and flowfile properties are expected). +### Performances + +By design, this processor cannot use precompiled code to handle messages (otherwise you would have already generated them) +and wouldn't be here. So this processor is using the runtime part of the Protobuf library, which dynamically parses the files, +given a compiled schema (`.desc`). + +For convenience, the processor also allows you to provide a raw `.proto` file but, to be used, it must be compiled, so this +is what the processor does before anything else. To avoid multiple compilation when not needed, the result file is cached, +and if you specified the schema in the processor configuration (and not in the flowfile properties), it will be directly +reused for each operation, and it will even avoid reading the schema from the disk. + +So, if you can, *specify the schema at the processor level to get the best performances*. + ## Contributing This project is Free as in Freedom, so feel free to contribute by posting bug report or pull requests!