Testing: Issues with pyproject.toml and jsonschema

[gcmoreira]

For the GitHub tests, we initially used requirements-testing.txt to manage the minimal installation of Python modules specifically for testing. With the introduction of the latest changes in pyproject.toml, as part of Update pyproject.toml and remove requirements and setup files an issue was introduced.

This occurs when the jsonschema module was added to the test target indirectly through the dev dependency.

dev = [
    "volatility3[full,cloud]",
    "jsonschema>=4.23.0,<5",
...
]
test = [
    "volatility3[dev]",
...
]

With this module present, as shown here and here, Volatility3 will perform JSON schema validation of each ISF. This validation introduces substantial overhead, significantly impacting the execution time of each test.

To validate the ISF JSON schema should not be part of the GitHub tests, and it's better this module to be excluded during that process. We need to find a way to decouple jsonschema from the test target.

[ikelos]

That's entirely doable, but just before we do that I'd like to look into making jsonschema much, much quicker than it is at the moment by compiling the schema at the start. Gimme a few days to fix that up and get it in and then let's revisit this please...

[ikelos]

Can you check if this was enough of a speedup please? If not, we could maybe still move to fastjsonschema, but I'll need to implement special checking to relax JSON timestamp formats, since a lot of our existing ones (and previously created files, such as all the windows one and potentially those made by dwarf2json) fail due to not including a timezone at the end... 5:S

Fastjsonschema also doesn't really tell you where the failure occurred so is less useful for users and devs alike. I'd probably sooner investigate another method of speed-up, such as a configuration option to disable validation when requested (not just when jsonschema isn't available).