a prometheus based monitoring (#4)

* Implement prometheus_endpoint

* Add Grafana + some dashboards

* move code to lib, implement falkor_process that manage and monitor the falkor process lifecycle

* Adapt init_falkor and run_falkor to use falkor_process

* Monitor the queries that are queue and executing by falkordb

* Add more metrics

* a bit of refactor

* add metrics for the number of nodes and relationship, add cli command to prepare queries (unimplemented)

* Update to latest falkor, update dashboard.json

* Add missing main

* MPSC  (#6)

* mpsc implementation -- seems that it is slower

* Fix the slowness with the mpsc

* protected-mode no

* replace falkor_process.rs with more resilient implementation, add more panels to dashboard

* clippy

* remove dead code, change project layout, fix waiting request metric

* stop redis before copy the dump file
display progress for every 10 messages instead of 1000

* Add parking lot deadlock detection

* Add send trace to tokio console

* Give names to tokio spawns

* Display message every 1000 queries

* Add message when the consumer of the messages get None

* add race when sending data to server

* Update docker compose to work on linux

* remove san from cli, add histogram and dashboards, remove timeout from calls to falkor client

* Fix code review
Use ro_exec on read query (should work with local falkordb-rs from branch https://github.com/FalkorDB/falkordb-rs/tree/fix_ro_query_builder

* Usage falkordb 0.1.10 from crate.io

* Clippy, GitHub actions, remove unused dependency

* Update nodes and edges instead of creating new ones

* simulate scheduler and workers as in https://www.scylladb.com/2021/04/22/on-coordinated-omission/

* Compute interval in nonos as to support higher values than 1000 msg per second

* Use the new scheduler
Add generate-queries step (see readme)
change init to load (see readme)

* add the simulate flag to run command, see readme

* Make neo4j work with the scheduler on multiple clients.
Add dashboards for neo4j.
Add write_ratio when generating the input

* update dependencies

* update dashboard

* update dashboard

* update for linux

* generate docker-compose and co from script, according to the os

* update git ignore

* bind on all network interfaces

* Add node exported on linux

* Add node info full to the dashboards

* Add CPU metrics to falkor

* Clippy

* refactor out run_metrics_reporter and use it to report neo4j memory and cpu

* Update neo4j dashboard

* Clippy

.github/workflows/ci.yml

@@ -29,7 +29,10 @@ jobs:
         run: rustup update ${{ matrix.toolchain }} && rustup default ${{ matrix.toolchain }}
 
       - name: Build
-        run: cargo build --verbose
+        run: cargo build --verbose --all-targets --all-features
+
+      - name: Run Clippy
+        run: cargo clippy --package benchmark --all-targets --all-features -- -D warnings
 
       - name: Test
         run: cargo test --verbose

.gitignore

@@ -14,4 +14,14 @@ falkor-results.md
 falkor-metrics*.json
 neo4j-metrics*.json
 html
+
+falkordb.log
 .DS_Store
+dump.rdb
+index.html
+queries.json
+docker-compose.yml
+grafana-datasources.yml
+prometheus.yml
+.DS_Store
+