Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Brain Image Synthesis Latent Diffusion from GenerativeModels to model-zoo #722

Merged
merged 44 commits into from
Feb 6, 2025

Conversation

garciadias
Copy link
Contributor

Fixes # No issue assigned

Description

This is part of the effort on deprecating GenerativeModels and having all models available in the model-zoo instead.

Status

Ready

Please ensure all the checkboxes:

  • Codeformat tests passed locally by running ./runtests.sh --codeformat.
  • In-line docstrings updated.
  • Update version and changelog in metadata.json if changing an existing bundle.
  • Please ensure the naming rules in config files meet our requirements (please refer to: CONTRIBUTING.md).
  • Ensure versions of packages such as monai, pytorch and numpy are correct in metadata.json.
  • Descriptions should be consistent with the content, such as eval_metrics of the provided weights and TorchScript modules.
  • Files larger than 25MB are excluded and replaced by providing download links in large_file.yml.
  • Avoid using path that contains personal information within config files (such as use /home/your_name/ for "bundle_root").

Virginia and others added 28 commits December 9, 2024 16:04
…s repo.

Signed-off-by: Virginia <virginia.fernandez@kcl.ac.uk>
… longer necessary. On inference, modification of the entry point from save_jpg to run to allow for run to be called alone, and truncation of loading statements which are too long. Modification of the README to include command example changes.
For metadata, adjusted the versions, setting current to Initial Release. I also removed nibabel from the optional package requirements as this CXR produces JPG and does not deal with nifti files.
…erify_bundle to pass the check for model.pt, since the requirement for two models (autoencoder and diffusion_model) makes sense for them to keep their specific names.

Modification of inference.json to add dummy attributes to pass the ConfigWorkflow check.
Modification of large_files.yml so that models are .pt and not .pth.
…erify_bundle to pass the check for model.pt, since the requirement for two models (autoencoder and diffusion_model) makes sense for them to keep their specific names.

Modification of inference.json to add dummy attributes to pass the ConfigWorkflow check.
Modification of large_files.yml so that models are .pt and not .pth.
Modification of diffusion_model.pt name to model.pt to go through the verify_bundle Python function.
@virginiafdez
Copy link
Contributor

@KumoLiu @ericspod Could you review this PR?

@KumoLiu KumoLiu requested a review from ericspod January 17, 2025 15:20
@yiheng-wang-nv
Copy link
Collaborator

/build

Copy link
Collaborator

@KumoLiu KumoLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution!
Leave several comments inline.

@garciadias
Copy link
Contributor Author

@yiheng-wang-nv

/build

?

@KumoLiu
Copy link
Collaborator

KumoLiu commented Jan 24, 2025

/build

@KumoLiu
Copy link
Collaborator

KumoLiu commented Jan 24, 2025

FYI @garciadias, error from blossom.

[2025-01-24T15:26:37.325Z] 2025-01-24 15:26:37,145 - INFO - Downloaded: models/brain_image_synthesis_latent_diffusion_model/models/model.pt
[2025-01-24T15:26:41.468Z] 2025-01-24 15:26:40,698 - INFO - Verified 'model.pt', md5: 21c3047556fb671caf0556f1cce6ef22.
[2025-01-24T15:26:41.469Z] directory is verified correctly.
[2025-01-24T15:26:41.469Z] version and changelog are verified correctly.
[2025-01-24T15:26:41.469Z] 2025-01-24 15:26:40,699 - INFO - --- input summary of monai.bundle.scripts.verify_metadata ---
[2025-01-24T15:26:41.469Z] 2025-01-24 15:26:40,699 - INFO - > meta_file: 'models/brain_image_synthesis_latent_diffusion_model/configs/metadata.json'
[2025-01-24T15:26:41.469Z] 2025-01-24 15:26:40,699 - INFO - > filepath: 'models/brain_image_synthesis_latent_diffusion_model/eval/schema.json'
[2025-01-24T15:26:41.469Z] 2025-01-24 15:26:40,699 - INFO - ---
[2025-01-24T15:26:41.469Z] 
[2025-01-24T15:26:41.469Z] 
[2025-01-24T15:26:41.469Z] 2025-01-24 15:26:40,699 - INFO - Expected md5 is None, skip md5 check for file models/brain_image_synthesis_latent_diffusion_model/eval/schema.json.
[2025-01-24T15:26:41.469Z] 2025-01-24 15:26:40,699 - INFO - File exists: models/brain_image_synthesis_latent_diffusion_model/eval/schema.json, skipped downloading.
[2025-01-24T15:26:41.469Z] Traceback (most recent call last):
[2025-01-24T15:26:41.469Z]   File "/usr/local/lib/python3.10/dist-packages/monai/bundle/scripts.py", line 1097, in verify_metadata
[2025-01-24T15:26:41.469Z]     validate(instance=metadata, schema=schema, **_args)
[2025-01-24T15:26:41.469Z]   File "/usr/local/lib/python3.10/dist-packages/jsonschema/validators.py", line 1312, in validate
[2025-01-24T15:26:41.469Z]     raise error
[2025-01-24T15:26:41.469Z] jsonschema.exceptions.ValidationError: 'format' is a required property
[2025-01-24T15:26:41.469Z] 
[2025-01-24T15:26:41.469Z] Failed validating 'required' in schema['properties']['network_data_format']['properties']['inputs']['properties']['image']:
[2025-01-24T15:26:41.469Z]     {'properties': {'channel_def': {'type': 'object'},
[2025-01-24T15:26:41.469Z]                     'dtype': {'type': 'string'},
[2025-01-24T15:26:41.469Z]                     'format': {'type': 'string'},
[2025-01-24T15:26:41.469Z]                     'is_patch_data': {'type': 'boolean'},
[2025-01-24T15:26:41.469Z]                     'num_channels': {'type': 'integer'},
[2025-01-24T15:26:41.469Z]                     'spatial_shape': {'type': 'array'},
[2025-01-24T15:26:41.469Z]                     'type': {'type': 'string'},
[2025-01-24T15:26:41.469Z]                     'value_range': {'items': {'type': 'number'},
[2025-01-24T15:26:41.469Z]                                     'type': 'array'}},
[2025-01-24T15:26:41.469Z]      'required': ['type',
[2025-01-24T15:26:41.469Z]                   'format',
[2025-01-24T15:26:41.469Z]                   'num_channels',
[2025-01-24T15:26:41.469Z]                   'spatial_shape',
[2025-01-24T15:26:41.469Z]                   'dtype',
[2025-01-24T15:26:41.469Z]                   'value_range'],
[2025-01-24T15:26:41.469Z]      'type': 'object'}
[2025-01-24T15:26:41.469Z] 
[2025-01-24T15:26:41.469Z] On instance['network_data_format']['inputs']['image']:
[2025-01-24T15:26:41.469Z]     {'channel_def': {'0': 'Gender',
[2025-01-24T15:26:41.469Z]                      '1': 'Age',
[2025-01-24T15:26:41.469Z]                      '2': 'Ventricular volume',
[2025-01-24T15:26:41.469Z]                      '3': 'Brain volume'},
[2025-01-24T15:26:41.469Z]      'dtype': 'float32',
[2025-01-24T15:26:41.469Z]      'is_patch_data': False,
[2025-01-24T15:26:41.469Z]      'num_channels': 1,
[2025-01-24T15:26:41.469Z]      'type': 'tabular',
[2025-01-24T15:26:41.469Z]      'value_range': [0, 1]}
[2025-01-24T15:26:41.469Z] 
[2025-01-24T15:26:41.469Z] The above exception was the direct cause of the following exception:
[2025-01-24T15:26:41.469Z] 
[2025-01-24T15:26:41.469Z] Traceback (most recent call last):
[2025-01-24T15:26:41.469Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 361, in <module>
[2025-01-24T15:26:41.469Z]     verify(bundle, models_path, mode)
[2025-01-24T15:26:41.469Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 323, in verify
[2025-01-24T15:26:41.469Z]     verify_metadata_format(bundle_path)
[2025-01-24T15:26:41.469Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 185, in verify_metadata_format
[2025-01-24T15:26:41.469Z]     verify_metadata(
[2025-01-24T15:26:41.469Z]   File "/usr/local/lib/python3.10/dist-packages/monai/bundle/scripts.py", line 1100, in verify_metadata
[2025-01-24T15:26:41.469Z]     raise ValueError(
[2025-01-24T15:26:41.469Z] ValueError: 'format' is a required property
[2025-01-24T15:26:41.469Z] 
[2025-01-24T15:26:41.469Z] Failed validating against schema `[https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json`.](https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/meta_schema_20220324.json%60.)

@yiheng-wang-nv
Copy link
Collaborator

/build

@yiheng-wang-nv
Copy link
Collaborator

Hi @garciadias , "/build" is used to trigger the CI pipeline, this PR looks good to me, and if all tests passed, I will merge it. Thanks!

@garciadias
Copy link
Contributor Author

Hi @garciadias , "/build" is used to trigger the CI pipeline, this PR looks good to me, and if all tests passed, I will merge it. Thanks!

Hi @yiheng-wang-nv, thank you for explaining and reviewing this. I believe I fixed all the issues pointed out in the CI checks.

I noticed the runtests.sh --autofix was not looking into my test file, so this is why some of the errors went through. I am not sure if that is the expected behaviour or if it's an issue with the configurations. With a quick look at the pyproject.toml and the runtests.sh files, I was unable to identify what may be causing it. I can look at it more thoroughly if you confirm this to be an issue.

Many thanks!

garciadias and others added 4 commits January 27, 2025 13:58
…tadata.json

Co-authored-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>
Signed-off-by: Rafael Garcia-Dias <rafaelagd@gmail.com>
Signed-off-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>
Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk>
@KumoLiu
Copy link
Collaborator

KumoLiu commented Feb 6, 2025

/build

@KumoLiu
Copy link
Collaborator

KumoLiu commented Feb 6, 2025

[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,384 - INFO - --- input summary of monai.bundle.scripts.verify_metadata ---
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,384 - INFO - > meta_file: 'models/brain_image_synthesis_latent_diffusion_model/configs/metadata.json'
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,384 - INFO - > filepath: 'models/brain_image_synthesis_latent_diffusion_model/eval/schema.json'
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,384 - INFO - ---
[2025-02-06T12:06:34.027Z] 
[2025-02-06T12:06:34.027Z] 
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,385 - INFO - Expected md5 is None, skip md5 check for file models/brain_image_synthesis_latent_diffusion_model/eval/schema.json.
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,385 - INFO - File exists: models/brain_image_synthesis_latent_diffusion_model/eval/schema.json, skipped downloading.
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,405 - INFO - metadata is verified with no error.
[2025-02-06T12:06:34.027Z] metadata format is verified correctly.
[2025-02-06T12:06:34.027Z] properties are verified correctly.
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,405 - INFO - --- input summary of monai.bundle.scripts.verify_net_in_out ---
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,405 - INFO - > net_id: 'network_def'
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,405 - INFO - > meta_file: 'models/brain_image_synthesis_latent_diffusion_model/configs/metadata.json'
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,405 - INFO - > config_file: 'models/brain_image_synthesis_latent_diffusion_model/configs/inference.json'
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,405 - INFO - > bundle_root: 'models/brain_image_synthesis_latent_diffusion_model'
[2025-02-06T12:06:34.027Z] 2025-02-06 12:06:33,405 - INFO - ---
[2025-02-06T12:06:34.027Z] 
[2025-02-06T12:06:34.027Z] 
[2025-02-06T12:06:34.953Z] Traceback (most recent call last):
[2025-02-06T12:06:34.953Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 363, in <module>
[2025-02-06T12:06:34.953Z]     verify(bundle, models_path, mode)
[2025-02-06T12:06:34.953Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 344, in verify
[2025-02-06T12:06:34.953Z]     verify_data_shape(bundle_path, net_id, config_file)
[2025-02-06T12:06:34.953Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/ci/verify_bundle.py", line 198, in verify_data_shape
[2025-02-06T12:06:34.953Z]     verify_net_in_out(
[2025-02-06T12:06:34.953Z]   File "/home/jenkins/agent/workspace/MONAI-zoo-premerge/model_zoo_venv/lib/python3.10/site-packages/monai/bundle/scripts.py", line 1251, in verify_net_in_out
[2025-02-06T12:06:34.953Z]     if output.shape[1] != output_channels:
[2025-02-06T12:06:34.953Z] AttributeError: 'tuple' object has no attribute 'shape'
script returned exit code 1

Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk>
@KumoLiu
Copy link
Collaborator

KumoLiu commented Feb 6, 2025

/build

Signed-off-by: Eric Kerfoot <eric.kerfoot@kcl.ac.uk>
@ericspod ericspod merged commit 9c70853 into Project-MONAI:dev Feb 6, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants