feat(ccmodel-hepheastus): add Hepheastus plugin

[RafaelAPB]

Implementation of the paper Hepheastus https://ieeexplore.ieee.org/document/10363680

Authored-by: Bruno Mateus brumat315@gmail.com
Co-authored-by: Rafael Belchior rafael.belchior@tecnico.ulisboa.pt
Co-authored-by: Peter Somogyvari peter.somogyvari@accenture.com
Signed-off-by: Rafael Belchior rafael.belchior@tecnico.ulisboa.pt

fix(ccmodel-hepheastus): incorporate PR feedback

Authored-by: Bruno Mateus brumat315@gmail.com
Signed-off-by: Rafael Belchior rafael.belchior@tecnico.ulisboa.pt
Signed-off-by: Peter Somogyvari peter.somogyvari@accenture.com

Pull Request Requirements

• Rebased onto upstream/main branch and squashed into single commit to help maintainers review it more efficient and to avoid spaghetti git commit graphs that obfuscate which commit did exactly what change, when and, why.
• Have git sign off at the end of commit message to avoid being marked red. You can add -s flag when using git commit command. You may refer to this link for more information.
• Follow the Commit Linting specification. You may refer to this link for more information.

Character Limit

• Pull Request Title and Commit Subject must not exceed 72 characters (including spaces and special characters).
• Commit Message per line must not exceed 80 characters (including spaces and special characters).

A Must Read for Beginners
For rebasing and squashing, here's a must read guide for beginners.

[petermetz]

@RafaelAPB I left some suggestions in the code, but it's fine by me if you'd prefer to address those later or never. I want to make sure I'm not slowing down the development.

[brunoffmateus]

@petermetz @RafaelAPB I also added a quick message about the python dependency in the prerequisites of README.md. If you don't find it necessary I can remove it.
Otherwise I think everything is done in this commit: e1626c4. You should be able to cherry-pick and squash it.

[petermetz]

@brunoffmateus Dependencies should always be documented as explicitly as possible (better yet via automation that fails with a helpful error message). So please do not remove it. It's great that you did what you did. :-)

[petermetz]

@brunoffmateus LGTM but please fix the linter

[RafaelAPB]

@petermetz i believe your concerns are addressed. Can we use the squash and merge functionality (there are 2 PRs), or should they be squashed?

[dpulls]

🎉 All dependencies have been resolved !

[petermetz]

@petermetz i believe your concerns are addressed. Can we use the squash and merge functionality (there are 2 PRs), or should they be squashed?

@RafaelAPB @brunoffmateus Yes, they were, thank you! I'd still recommend using the "manual" method of rebasing onto main interactively and squash them together that way just to be sure. The github GUI can have negative & unintended side effects sometimes.

[outSH]

Looks awesome, thank you for this contribution! I've left a few comments / questions :)

[petermetz]

@outSH: I’m not aware of it. Good to create one in my opinion. It would be nice to have a `custom-check` for it verifying a lot of these fields as well (but I understand that’s a bigger piece of work then just doing a one-off search and replace)

[petermetz]

@RafaelAPB This one is ready for merge, but before we do please squash the commits together and rebase onto upstream/main

[RafaelAPB]

@outSH and @petermetz thanks a lot for the reviews.

[petermetz]

@outSH and @petermetz thanks a lot for the reviews.

@RafaelAPB My pleasure! :-)

[petermetz]

@RafaelAPB Took a quick look and found a few low hanging fruit fixes (easy to spot bugs that were breaking the build). I'm fixing them but I'll also post some comments here for awareness:

The root tsconfig.json file that ties all the packages together for the typescript compiler was missing the hephaestus plugin's reference so the plugin code was not part of the compilation when you did a yarn tsc from the project root.

The indirect way this manifested as an issue was that the build was seemingly good (it compiled successfully) but despite that when you tried to run one of the testcases of the package, Jest would crash with compilation errors within the test case itself.

Adding this snippet below to the root tsconfig.json file fixed the root cause but then it highlighted the other issues that were otherwise hidden until now, more on those later.

 {
    "path": "./packages/cactus-plugin-ccmodel-hephaestus/tsconfig.json"
  },

[petermetz]

@RafaelAPB Afterwards, I moved my attention to the compilation issues (see screenshots for full picture).

1. It seems like the IRunTransactionV1Exchange type is inconsistently named, because in the besu connector it is called IRunTransactionV1Exchange but in the ethereum connector it is missing the I (probably my fault for missing it in one of my reviews that the I was missing, sorry about that)
2. The IRunTxReqWithTxId from the fabric connector was being imported as RunTxReqWithTxId, again I remember asking for changes on the interfaces so that they are prefixed with I and this is most likely what caused the issues down the line in the dependent PRs so sorry about the extra overhead here as well.
3. The third issue was a more boring one, the versions of the package were set to 2.0.0 but since we've released 2.1.0 and so this again caused a bunch of trouble with the build, but that's also fixed now.
4. So after applying all of those fixes and me trying to run one of the test cases ( yarn test:jest:all 'packages/cactus-plugin-ccmodel-hephaestus/src/test/typescript/integration/besu-ethereum-asset-transfer.test.ts') it still fails unfortunately with some python dependency issues which are either because I haven't read the documentation on how to set them up or because they are missing in some other way. The logs for that crash are below just in case.
5. I know that these python dependencies are required right now with good reason, but for the longer term, if possible, it would be nice to keep it NodeJS only just so that it's simpler to set up the development environment (though I have given up on trying to achieve this goal, it shouldn't stop from us for at least trying to reach it and pick the low hanging fruit wherever possible). So again, not a change request, just a note for ourselves for an ideal future when we have infinite time and resources to tidy everything up... :-)

 Command failed: python3 /home/peter/a/cacti-upstream/packages/cactus-plugin-ccmodel-hephaestus/src/main/python/create_model.py /home/peter/a/cacti-upstream/packages/cactus-plugin-ccmodel-hephaestus/src/test/ccLogs/json/hephaestus_log_1734293360776.json
    Traceback (most recent call last):
      File "/home/peter/a/cacti-upstream/packages/cactus-plugin-ccmodel-hephaestus/src/main/python/create_model.py", line 6, in <module>
        import pm4py
    ModuleNotFoundError: No module named 'pm4py'

       9 |   try {
      10 |     const startTime = new Date();
    > 11 |     const serializedCCModel = execSync(command).toString("utf-8");
         |                                       ^
      12 |     const finalTime = new Date();
      13 |     console.log(
      14 |       `CREATE-MODEL-PM4PY:${finalTime.getTime() - startTime.getTime()}`,

[petermetz]

@RafaelAPB Finally, I'm going to force push now onto your branch with the fixes mentioned above which I'm hoping won't be too confusing and are steps in the right direction. If the CI has the correct setup steps for the python dependencies then it should actually be able to run the tests and have them succeed and if that is the case then I'm good to merge this right after that. If the CI ends up having issues with the tests too then I'll ask that you tweak the CI job to have the python dependencies set up as needed just so that we can see for now at least on the CI passing the tests and I'll make some effort in the future to set up the python dependencies on my dev machine(s) as well.

[RafaelAPB]

@petermetz, I appreciate the help and your time fixing the issues. I take all responsibility for overlooking those issues.

Regarding the Python dependencies, that's a good point. This does not work without that specific dependency, but there is no JS/TS alternative (to the best of my knowledge). I propose that future developments for this plugin can migrate to a JS solution, or we remove it if there is no broader interest in it.
I will check the CI.

[RafaelAPB]

@petermetz seems we do not have the CI configured for this specific plugin. I'd say I add to the documentation the need for this dependency and (optionally) add the CI configuration for running this package tests (however I'd rather not before it is updated/worked upon). What is your opinion?

[RafaelAPB]

@petermetz I've noticed you already added it. Thanks!

[petermetz]

seems we do not have the CI configured for this specific plugin

@RafaelAPB Oh, /facepalm, sorry I didn't even look. We should definitely add a new issue to track adding those CI steps.

I'd say I add to the documentation the need for this dependency and (optionally) add the CI configuration for running this package tests (however I'd rather not before it is updated/worked upon). What is your opinion?

@RafaelAPB I'm OK with that schedule, as long as we track the steps to be done via issues (which guarantees that we don't forget/lose track/falls through the cracks/etc.)

[petermetz]

@petermetz, I appreciate the help and your time fixing the issues. I take all responsibility for overlooking those issues.

Regarding the Python dependencies, that's a good point. This does not work without that specific dependency, but there is no JS/TS alternative (to the best of my knowledge). I propose that future developments for this plugin can migrate to a JS solution, or we remove it if there is no broader interest in it. I will check the CI.

@RafaelAPB I would recommend two ideas for consideration (no idea which one is better or if any of them would work well in practice for now)

1. Use Docker at runtime to run a container containing the python deps for process mining. This might feel like a "from the pan to the fire" type of solution but I think depending on Docker is better than depending on Python because through Docker we get at least the stable environment setup question solved (there won't be any python version issues, we don't have to teach people how to install and configure multiple python environments, solve those dreaded module version constraint hells, etc). The other upside would be that if we depend on Docker, later we can use that as a stepping stone to depend on other non-NodeJS modules as well (Rust/Go for example) and we wouldn't have to worry about environment setup either also due to containerization.
2. Second idea: Try to use something like this which appears to be a direct javascript port of the python library we are using currently (which implies nothing about whether it's maintained as well, bugs, etc.) https://github.com/pm4js/pm4js-core

[RafaelAPB]

Thanks for your suggestions @petermetz. We will commit to implement 1. (or to remove the package altogether because as is it breaks at least one of the principles of the project)