diff --git a/README.md b/README.md index 44fd35a..e73d747 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,41 @@ -# deterministic-ml +# Deterministic ML Models execution using Python frameworks  [![Continuous Integration](https://github.com/backend-developers-ltd/deterministic-ml/workflows/Continuous%20Integration/badge.svg)](https://github.com/backend-developers-ltd/deterministic-ml/actions?query=workflow%3A%22Continuous+Integration%22) [![License](https://img.shields.io/pypi/l/deterministic_ml.svg?label=License)](https://pypi.python.org/pypi/deterministic_ml) [![python versions](https://img.shields.io/pypi/pyversions/deterministic_ml.svg?label=python%20versions)](https://pypi.python.org/pypi/deterministic_ml) [![PyPI version](https://img.shields.io/pypi/v/deterministic_ml.svg?label=PyPI%20version)](https://pypi.python.org/pypi/deterministic_ml) +This project is two-part: +* documentation that describes how to ensure deterministic execution of ML models across different frameworks +* a Python package that provides utilities that help to ensure deterministic execution of ML models across different frameworks and versions + +Currently supported frameworks and inference engines: CUDA-based, PyTorch, vLLM. + +The goal is to be able to reproduce exactly the same results on another machine using the same software. +This means, finding a balance between performance and hardware restrictions without compromising reproduciblity. +I.e. if limiting to a single GPU model and vRAM size is required to achieve reproducibility, then it is also acceptable solution, especially if otherwise it would require "dumbing down" other cards just to achieve the same results. + +## Experiment results so far + +Through [Integration testing](docs/MANUAL_INTEGRATION_TESTING.md) we can see that the output of the model can be achieved in a deterministic way. + +Here is the summary of the results for vLLM running llama3 model: +* each card GPU model (combined with its vRAM configuration) has a different output, but is consistent across runs +* the output is consistent across different CUDA versions (more testing is needed here, only small range was tested) +* GPU interface (SXM4, PCIe) does not affect the output +* A100 80GB and A100X 80GB produce the same output +* 2x A100 40GB do not produce the same output as 1x A100 80GB +* driver versions 535.129.03 and 555.58.02 produce the same output + +To learn more about this particular example, please refer to the [Integration testing](docs/MANUAL_INTEGRATION_TESTING.md) documentation and the [tests/integration/experiments/vllm_llama_3_70b_instruct_awq](tests/integration/experiments/vllm_llama_3_70b_instruct_awq) experiment code. + ## Usage > [!IMPORTANT] > This package uses [ApiVer](#versioning), make sure to import `deterministic_ml.v1`. +``` +pip install deterministic_ml[vllm] # pick the right extra for your use case, e.g. [vllm] or [torch] +``` + + ## Versioning This package uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). @@ -38,6 +67,6 @@ If you wish to install dependencies into `.venv` so your IDE can pick them up, y pdm install --dev ``` -### Release process +## Contributing -Run `nox -s make_release -- X.Y.Z` where `X.Y.Z` is the version you're releasing and follow the printed instructions. +Contributions are welcome, especially ones that add to [docs/IMPROVING_CONSITENCY.md](docs/IMPROVING_CONSITENCY.md) docs expanding the list of recommendations for improving the consistency of inference results when using various python frameworks. diff --git a/docs/COMPUTE_PROVIDERS.MD b/docs/COMPUTE_PROVIDERS.MD new file mode 100644 index 0000000..9c27959 --- /dev/null +++ b/docs/COMPUTE_PROVIDERS.MD @@ -0,0 +1,15 @@ +# GPU Compute Providers + +## vast.ai + +For short experiments, vast.ai is a cost-effective solution as it is billed per minute and not per hour. +Please note you also pay for setup time. + +Vast.ai comes with limitation of not being able to run docker containers inside rented machines. +Vast.ai allows you to run your own docker image if it is uploaded to public docker registry (or credentials docker registry password are provided), but this avenue was not explored. + + +## paperspace + +Paperspace is billed per hour, and tends to have a higher cost than vast.ai. +The performance seemed to vary a lot, but the root cause was never identified. diff --git a/docs/IMPROVING_CONSITENCY.md b/docs/IMPROVING_CONSITENCY.md new file mode 100644 index 0000000..8dc8431 --- /dev/null +++ b/docs/IMPROVING_CONSITENCY.md @@ -0,0 +1,80 @@ +# Improving Consistency of Inference + +This document describes the steps to improve the consistency of inference results, from the MUST-HAVE requirements to potential improvements. + +If you a case, where recommendation, especially a "thought to be safe" configuration, does not provide consistent results, please file an Issue report alongside the steps to reproduce the issue. + + +## Recommendations + +* [ ] Have test cases that verify the consistency of inference results. + * Recommendation: + * Before every release, check if the known input-output pairs are still the same as in previous release. It is very important for known output to be as long as possible, operation errors are cumulative the long output is much more likely showcase inconsistencies. +* [ ] All software dependencies need to be locked to a specific version. + * Recommendation: + * Python dependencies: These are the ones that will be most updated. Use [Rye](https://rye.astral.sh/) or [pip-tools](https://github.com/jazzband/pip-tools). + * Binary dependencies: Use doker images and tags to lock the version of the final image. Treat each tag as a reseed of the inference process. + * Use same driver version (Please note, that so far we have yet to document a case where driver version affected inference results) +* [ ] Set seed for all random number generators used in the inference process. + * Recommendation: + * For single-threaded apps, use `deterministic_ml.v1.set_seed()` to set the seed for all known random number generator process wide. + * Whenever initializing a new random generator, explicitly set the seed in deterministic manner. + * For multi-threaded or async applications ensure that random generators are isolated per thread or task. +* [ ] Disable auto-optimization or JIT compilation in the inference process. + * Recommendation: + * Use `deterministic_ml.v1.disable_auto_optimization()` to disable auto-optimization or JIT compilation process wide. +* [ ] Use the same kind of hardware for all inference runs. + * Recommendation: + * Use the same GPU chip model and vRAM size for are inference runs. Hardware interface (PCIe, SXM, etc.) does not seem to affect the results, but 2xA100 40G do not return the same results as 1xA100 80G. + * When testing new, but similar hardware to check if the results are consistent with previously known platform, maximize pseudo-randomization of the inference process (e.g. by setting high temperature and low top-p values). + +## Framework Specific Recommendations + +### PyTorch + +See https://pytorch.org/docs/stable/notes/randomness.html . +The https://docs.nvidia.com/cuda/cublas/index.html#results-reproducibility also apply. + +### vLLM + +vLLM is PyTorch based, so the same constraints apply. +However, vLLM has much narrower scope, hence other than general recommendations from [MUST-HAVE](#must-have) section, only following is required: +* make sure to use exactly the same parameters for the model initialization + * `enforce_eager=True` +* to get the same output for the same input, use the exactly same `SamplingParams` with explicitly set `seed` parameter + + +```python +model = vllm.LLM( + model=model_name, + enforce_eager=True, # Ensure eager mode is enabled +) + + +sampling_params = vllm.SamplingParams( + max_tokens=4096, + # temperature=1000, # High value encourages pseudo-randomization + # top_p=0.1, # Low value encourages pseudo-randomization + seed=42, +) + +response = model.generate(requests, sampling_params) + +``` + + +## Unconfirmed advice + +### Consistent results across different CUDA hardware + +It should be theoretically possible to get consistent results across different hardware, but even if limited to CUDA-capatible GPUs, it will be at the cost of performance. + +* [ ] Use `torch.backends.cudnn.deterministic = True` and `torch.backends.cudnn.benchmark = False` to ensure that the results are consistent across different CUDA hardware. + * Recommendation: + * Use `deterministic_ml.v1.set_seed()` to set the seed for all known random number generator process wide. + * Use `deterministic_ml.v1.disable_auto_optimization()` to disable auto-optimization or JIT compilation process wide. + * Use `torch.backends.cudnn.deterministic = True` and `torch.backends.cudnn.benchmark = False` to ensure that the results are consistent across different CUDA hardware. + * Use the same GPU chip model and vRAM size for are inference runs. Hardware interface (PCIe, SXM, etc.) does not seem to affect the results, but 2xA100 40G do not return the same results as 1xA100 80G. + + +See [TO_BE_INVESTIGATED.md](TO_BE_INVESTIGATED.md) for more potential improvements. diff --git a/docs/MANUAL_INTEGRATION_TESTING.md b/docs/MANUAL_INTEGRATION_TESTING.md new file mode 100644 index 0000000..26c9f54 --- /dev/null +++ b/docs/MANUAL_INTEGRATION_TESTING.md @@ -0,0 +1,73 @@ +# Manual integration testing + +This document describes how we test whenever we are able to achieve deterministic results for particular experiment scenarios against tested hardware and software configurations. + +1) For each hardware configuration: + 1) Create target machine \[manual\] + 2) Run the target experiment scenario on the target machine. + `./run.py vllm_llama_3_70b_instruct_awq -n machine_name username@host -p ssh_port` + 3) Destroy the target machine \[manual\] +2) Analyze the results \[manual\] + +## Initial local setup + +```bash +pdm install -G test +``` + +## Choosing experiment scenario + +Either use already defined scenario in [tests/integration/experiments](../tests/integration/experiments), e.g. +`vllm_llama_3_70b_instruct_awq` or create a new one. + +Each scenario may define target machine environment initial setup steps, including: +* `setup.sh` - Shell script executed on the target machine, for example, installing binary dependencies. +* `requirements.txt` - Python packages installed on the target machine in dedicated python virtual environment. + +And must include: +* `__main__.py` - Main experiment script, which is executed on the target machine taking as first argument the output directory to which `output.yaml` file should be saved. + +## Running the experiment scenario + + +### Run experiment against N target machines + +Repeat following for a number of target machines. +You may want to even mix some of configurations, e.g. different GPU models, different CUDA versions, etc. to get a better understanding what influences the determinism of output. + +### Create target machine + +You can use service cheap GPU machines like ones provided by [vast.ai](https://vast.ai/), [paperspace](https://www.paperspace.com/) etc. +Please note that for one-off experiment, services like vast.ai are more cost-effective since they are billed per minute and not per hour. +See [Compute Providers document](COMPUTE_PROVIDERS.md) for more information. + +Example machine configuration: vast.io, on-demand, 1x NVIDIA A100 80GB, 100GB disk with Ubuntu-based template +CUDA drivers installed. + +### Run the experiment scenario + +In [tests/integration/experiments](../tests/integration/experiments) directory, run + +```bash +./run.py vllm_llama_3_70b_instruct_awq -c target_comment username@host -p ssh_port +``` + +### Destroy the target machine + +Destroy the target machine to avoid unnecessary costs. + + +## Analyzing the results + +Results are stored in [`results` directory](../tests/integration/results). +They are grouped by experiment scenario, then target machine name and timestamp. + +Each result contains: +* `experiment.log` - Experiment log output. +* `output.yaml` - Experiment output in YAML format. This is the most important file to analyze. +* `sysinfo.yaml` - System information of the target machine, used to cluster results by hardware configuration. + +You can use `./analyze.py` script to analyze the results. + +```bash +./analyze.py vllm_llama_3_70b_instruct_awq +``` diff --git a/docs/TO_BE_INVESTIGATED.md b/docs/TO_BE_INVESTIGATED.md new file mode 100644 index 0000000..45c599e --- /dev/null +++ b/docs/TO_BE_INVESTIGATED.md @@ -0,0 +1,6 @@ +# To be investigated + +## Disabling "fast math" optimizations + +To check: +* as this is a compiler flag - what Python-related software is affected by it? \ No newline at end of file diff --git a/pdm.lock b/pdm.lock index ae5619a..2bd77d0 100644 --- a/pdm.lock +++ b/pdm.lock @@ -5,7 +5,106 @@ groups = ["default", "lint", "release", "test"] strategy = ["cross_platform", "inherit_metadata"] lock_version = "4.4.2" -content_hash = "sha256:206e06a728b8f3ddd1c4fbc34f39cb1b0fbe268d1ef5a2fecc0275a2790a4bd2" +content_hash = "sha256:c2c2d152882bea9a7b65227babd94f5847d682d2c2616ad0ad29f7c13863940f" + +[[package]] +name = "bcrypt" +version = "4.2.0" +requires_python = ">=3.7" +summary = "Modern password hashing for your software and your servers" +groups = ["test"] +files = [ + {file = "bcrypt-4.2.0-cp37-abi3-macosx_10_12_universal2.whl", hash = "sha256:096a15d26ed6ce37a14c1ac1e48119660f21b24cba457f160a4b830f3fe6b5cb"}, + {file = "bcrypt-4.2.0-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c02d944ca89d9b1922ceb8a46460dd17df1ba37ab66feac4870f6862a1533c00"}, + {file = "bcrypt-4.2.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:1d84cf6d877918620b687b8fd1bf7781d11e8a0998f576c7aa939776b512b98d"}, + {file = "bcrypt-4.2.0-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:1bb429fedbe0249465cdd85a58e8376f31bb315e484f16e68ca4c786dcc04291"}, + {file = "bcrypt-4.2.0-cp37-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:655ea221910bcac76ea08aaa76df427ef8625f92e55a8ee44fbf7753dbabb328"}, + {file = "bcrypt-4.2.0-cp37-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:1ee38e858bf5d0287c39b7a1fc59eec64bbf880c7d504d3a06a96c16e14058e7"}, + {file = "bcrypt-4.2.0-cp37-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:0da52759f7f30e83f1e30a888d9163a81353ef224d82dc58eb5bb52efcabc399"}, + {file = "bcrypt-4.2.0-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:3698393a1b1f1fd5714524193849d0c6d524d33523acca37cd28f02899285060"}, + {file = "bcrypt-4.2.0-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:762a2c5fb35f89606a9fde5e51392dad0cd1ab7ae64149a8b935fe8d79dd5ed7"}, + {file = "bcrypt-4.2.0-cp37-abi3-win32.whl", hash = "sha256:5a1e8aa9b28ae28020a3ac4b053117fb51c57a010b9f969603ed885f23841458"}, + {file = "bcrypt-4.2.0-cp37-abi3-win_amd64.whl", hash = "sha256:8f6ede91359e5df88d1f5c1ef47428a4420136f3ce97763e31b86dd8280fbdf5"}, + {file = "bcrypt-4.2.0-cp39-abi3-macosx_10_12_universal2.whl", hash = "sha256:c52aac18ea1f4a4f65963ea4f9530c306b56ccd0c6f8c8da0c06976e34a6e841"}, + {file = "bcrypt-4.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3bbbfb2734f0e4f37c5136130405332640a1e46e6b23e000eeff2ba8d005da68"}, + {file = "bcrypt-4.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3413bd60460f76097ee2e0a493ccebe4a7601918219c02f503984f0a7ee0aebe"}, + {file = "bcrypt-4.2.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:8d7bb9c42801035e61c109c345a28ed7e84426ae4865511eb82e913df18f58c2"}, + {file = "bcrypt-4.2.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:3d3a6d28cb2305b43feac298774b997e372e56c7c7afd90a12b3dc49b189151c"}, + {file = "bcrypt-4.2.0-cp39-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:9c1c4ad86351339c5f320ca372dfba6cb6beb25e8efc659bedd918d921956bae"}, + {file = "bcrypt-4.2.0-cp39-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:27fe0f57bb5573104b5a6de5e4153c60814c711b29364c10a75a54bb6d7ff48d"}, + {file = "bcrypt-4.2.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:8ac68872c82f1add6a20bd489870c71b00ebacd2e9134a8aa3f98a0052ab4b0e"}, + {file = "bcrypt-4.2.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:cb2a8ec2bc07d3553ccebf0746bbf3d19426d1c6d1adbd4fa48925f66af7b9e8"}, + {file = "bcrypt-4.2.0-cp39-abi3-win32.whl", hash = "sha256:77800b7147c9dc905db1cba26abe31e504d8247ac73580b4aa179f98e6608f34"}, + {file = "bcrypt-4.2.0-cp39-abi3-win_amd64.whl", hash = "sha256:61ed14326ee023917ecd093ee6ef422a72f3aec6f07e21ea5f10622b735538a9"}, + {file = "bcrypt-4.2.0-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:39e1d30c7233cfc54f5c3f2c825156fe044efdd3e0b9d309512cc514a263ec2a"}, + {file = "bcrypt-4.2.0-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:f4f4acf526fcd1c34e7ce851147deedd4e26e6402369304220250598b26448db"}, + {file = "bcrypt-4.2.0-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:1ff39b78a52cf03fdf902635e4c81e544714861ba3f0efc56558979dd4f09170"}, + {file = "bcrypt-4.2.0-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:373db9abe198e8e2c70d12b479464e0d5092cc122b20ec504097b5f2297ed184"}, + {file = "bcrypt-4.2.0.tar.gz", hash = "sha256:cf69eaf5185fd58f268f805b505ce31f9b9fc2d64b376642164e9244540c1221"}, +] + +[[package]] +name = "build" +version = "1.2.1" +requires_python = ">=3.8" +summary = "A simple, correct Python build frontend" +groups = ["test"] +dependencies = [ + "colorama; os_name == \"nt\"", + "packaging>=19.1", + "pyproject-hooks", +] +files = [ + {file = "build-1.2.1-py3-none-any.whl", hash = "sha256:75e10f767a433d9a86e50d83f418e83efc18ede923ee5ff7df93b6cb0306c5d4"}, + {file = "build-1.2.1.tar.gz", hash = "sha256:526263f4870c26f26c433545579475377b2b7588b6f1eac76a001e873ae3e19d"}, +] + +[[package]] +name = "cffi" +version = "1.17.0" +requires_python = ">=3.8" +summary = "Foreign Function Interface for Python calling C code." +groups = ["lint", "test"] +dependencies = [ + "pycparser", +] +files = [ + {file = "cffi-1.17.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:c5d97162c196ce54af6700949ddf9409e9833ef1003b4741c2b39ef46f1d9720"}, + {file = "cffi-1.17.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:5ba5c243f4004c750836f81606a9fcb7841f8874ad8f3bf204ff5e56332b72b9"}, + {file = "cffi-1.17.0-cp311-cp311-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:bb9333f58fc3a2296fb1d54576138d4cf5d496a2cc118422bd77835e6ae0b9cb"}, + {file = "cffi-1.17.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:435a22d00ec7d7ea533db494da8581b05977f9c37338c80bc86314bec2619424"}, + {file = "cffi-1.17.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d1df34588123fcc88c872f5acb6f74ae59e9d182a2707097f9e28275ec26a12d"}, + {file = "cffi-1.17.0-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:df8bb0010fdd0a743b7542589223a2816bdde4d94bb5ad67884348fa2c1c67e8"}, + {file = "cffi-1.17.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a8b5b9712783415695663bd463990e2f00c6750562e6ad1d28e072a611c5f2a6"}, + {file = "cffi-1.17.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:ffef8fd58a36fb5f1196919638f73dd3ae0db1a878982b27a9a5a176ede4ba91"}, + {file = "cffi-1.17.0-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:4e67d26532bfd8b7f7c05d5a766d6f437b362c1bf203a3a5ce3593a645e870b8"}, + {file = "cffi-1.17.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:45f7cd36186db767d803b1473b3c659d57a23b5fa491ad83c6d40f2af58e4dbb"}, + {file = "cffi-1.17.0-cp311-cp311-win32.whl", hash = "sha256:a9015f5b8af1bb6837a3fcb0cdf3b874fe3385ff6274e8b7925d81ccaec3c5c9"}, + {file = "cffi-1.17.0-cp311-cp311-win_amd64.whl", hash = "sha256:b50aaac7d05c2c26dfd50c3321199f019ba76bb650e346a6ef3616306eed67b0"}, + {file = "cffi-1.17.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:aec510255ce690d240f7cb23d7114f6b351c733a74c279a84def763660a2c3bc"}, + {file = "cffi-1.17.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:2770bb0d5e3cc0e31e7318db06efcbcdb7b31bcb1a70086d3177692a02256f59"}, + {file = "cffi-1.17.0-cp312-cp312-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:db9a30ec064129d605d0f1aedc93e00894b9334ec74ba9c6bdd08147434b33eb"}, + {file = "cffi-1.17.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a47eef975d2b8b721775a0fa286f50eab535b9d56c70a6e62842134cf7841195"}, + {file = "cffi-1.17.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f3e0992f23bbb0be00a921eae5363329253c3b86287db27092461c887b791e5e"}, + {file = "cffi-1.17.0-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6107e445faf057c118d5050560695e46d272e5301feffda3c41849641222a828"}, + {file = "cffi-1.17.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eb862356ee9391dc5a0b3cbc00f416b48c1b9a52d252d898e5b7696a5f9fe150"}, + {file = "cffi-1.17.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:c1c13185b90bbd3f8b5963cd8ce7ad4ff441924c31e23c975cb150e27c2bf67a"}, + {file = "cffi-1.17.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:17c6d6d3260c7f2d94f657e6872591fe8733872a86ed1345bda872cfc8c74885"}, + {file = "cffi-1.17.0-cp312-cp312-win32.whl", hash = "sha256:c3b8bd3133cd50f6b637bb4322822c94c5ce4bf0d724ed5ae70afce62187c492"}, + {file = "cffi-1.17.0-cp312-cp312-win_amd64.whl", hash = "sha256:dca802c8db0720ce1c49cce1149ff7b06e91ba15fa84b1d59144fef1a1bc7ac2"}, + {file = "cffi-1.17.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:6ce01337d23884b21c03869d2f68c5523d43174d4fc405490eb0091057943118"}, + {file = "cffi-1.17.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:cab2eba3830bf4f6d91e2d6718e0e1c14a2f5ad1af68a89d24ace0c6b17cced7"}, + {file = "cffi-1.17.0-cp313-cp313-manylinux_2_12_i686.manylinux2010_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:14b9cbc8f7ac98a739558eb86fabc283d4d564dafed50216e7f7ee62d0d25377"}, + {file = "cffi-1.17.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b00e7bcd71caa0282cbe3c90966f738e2db91e64092a877c3ff7f19a1628fdcb"}, + {file = "cffi-1.17.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:41f4915e09218744d8bae14759f983e466ab69b178de38066f7579892ff2a555"}, + {file = "cffi-1.17.0-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e4760a68cab57bfaa628938e9c2971137e05ce48e762a9cb53b76c9b569f1204"}, + {file = "cffi-1.17.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:011aff3524d578a9412c8b3cfaa50f2c0bd78e03eb7af7aa5e0df59b158efb2f"}, + {file = "cffi-1.17.0-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:a003ac9edc22d99ae1286b0875c460351f4e101f8c9d9d2576e78d7e048f64e0"}, + {file = "cffi-1.17.0-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:ef9528915df81b8f4c7612b19b8628214c65c9b7f74db2e34a646a0a2a0da2d4"}, + {file = "cffi-1.17.0-cp313-cp313-win32.whl", hash = "sha256:70d2aa9fb00cf52034feac4b913181a6e10356019b18ef89bc7c12a283bf5f5a"}, + {file = "cffi-1.17.0-cp313-cp313-win_amd64.whl", hash = "sha256:b7b6ea9e36d32582cda3465f54c4b454f62f23cb083ebc7a94e2ca6ef011c3a7"}, + {file = "cffi-1.17.0.tar.gz", hash = "sha256:f3157624b7558b914cb039fd1af735e5e8049a87c817cc215109ad1c8779df76"}, +] [[package]] name = "click" @@ -53,12 +152,51 @@ version = "0.4.6" requires_python = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7" summary = "Cross-platform colored terminal text." groups = ["release", "test"] -marker = "sys_platform == \"win32\" or platform_system == \"Windows\"" +marker = "os_name == \"nt\" or sys_platform == \"win32\" or platform_system == \"Windows\"" files = [ {file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"}, {file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"}, ] +[[package]] +name = "cryptography" +version = "43.0.0" +requires_python = ">=3.7" +summary = "cryptography is a package which provides cryptographic recipes and primitives to Python developers." +groups = ["lint", "test"] +dependencies = [ + "cffi>=1.12; platform_python_implementation != \"PyPy\"", +] +files = [ + {file = "cryptography-43.0.0-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:64c3f16e2a4fc51c0d06af28441881f98c5d91009b8caaff40cf3548089e9c74"}, + {file = "cryptography-43.0.0-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3dcdedae5c7710b9f97ac6bba7e1052b95c7083c9d0e9df96e02a1932e777895"}, + {file = "cryptography-43.0.0-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3d9a1eca329405219b605fac09ecfc09ac09e595d6def650a437523fcd08dd22"}, + {file = "cryptography-43.0.0-cp37-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:ea9e57f8ea880eeea38ab5abf9fbe39f923544d7884228ec67d666abd60f5a47"}, + {file = "cryptography-43.0.0-cp37-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:9a8d6802e0825767476f62aafed40532bd435e8a5f7d23bd8b4f5fd04cc80ecf"}, + {file = "cryptography-43.0.0-cp37-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:cc70b4b581f28d0a254d006f26949245e3657d40d8857066c2ae22a61222ef55"}, + {file = "cryptography-43.0.0-cp37-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:4a997df8c1c2aae1e1e5ac49c2e4f610ad037fc5a3aadc7b64e39dea42249431"}, + {file = "cryptography-43.0.0-cp37-abi3-win32.whl", hash = "sha256:6e2b11c55d260d03a8cf29ac9b5e0608d35f08077d8c087be96287f43af3ccdc"}, + {file = "cryptography-43.0.0-cp37-abi3-win_amd64.whl", hash = "sha256:31e44a986ceccec3d0498e16f3d27b2ee5fdf69ce2ab89b52eaad1d2f33d8778"}, + {file = "cryptography-43.0.0-cp39-abi3-macosx_10_9_universal2.whl", hash = "sha256:7b3f5fe74a5ca32d4d0f302ffe6680fcc5c28f8ef0dc0ae8f40c0f3a1b4fca66"}, + {file = "cryptography-43.0.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ac1955ce000cb29ab40def14fd1bbfa7af2017cca696ee696925615cafd0dce5"}, + {file = "cryptography-43.0.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:299d3da8e00b7e2b54bb02ef58d73cd5f55fb31f33ebbf33bd00d9aa6807df7e"}, + {file = "cryptography-43.0.0-cp39-abi3-manylinux_2_28_aarch64.whl", hash = "sha256:ee0c405832ade84d4de74b9029bedb7b31200600fa524d218fc29bfa371e97f5"}, + {file = "cryptography-43.0.0-cp39-abi3-manylinux_2_28_x86_64.whl", hash = "sha256:cb013933d4c127349b3948aa8aaf2f12c0353ad0eccd715ca789c8a0f671646f"}, + {file = "cryptography-43.0.0-cp39-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:fdcb265de28585de5b859ae13e3846a8e805268a823a12a4da2597f1f5afc9f0"}, + {file = "cryptography-43.0.0-cp39-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:2905ccf93a8a2a416f3ec01b1a7911c3fe4073ef35640e7ee5296754e30b762b"}, + {file = "cryptography-43.0.0-cp39-abi3-win32.whl", hash = "sha256:47ca71115e545954e6c1d207dd13461ab81f4eccfcb1345eac874828b5e3eaaf"}, + {file = "cryptography-43.0.0-cp39-abi3-win_amd64.whl", hash = "sha256:0663585d02f76929792470451a5ba64424acc3cd5227b03921dab0e2f27b1709"}, + {file = "cryptography-43.0.0-pp310-pypy310_pp73-macosx_10_9_x86_64.whl", hash = "sha256:2c6d112bf61c5ef44042c253e4859b3cbbb50df2f78fa8fae6747a7814484a70"}, + {file = "cryptography-43.0.0-pp310-pypy310_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:844b6d608374e7d08f4f6e6f9f7b951f9256db41421917dfb2d003dde4cd6b66"}, + {file = "cryptography-43.0.0-pp310-pypy310_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:51956cf8730665e2bdf8ddb8da0056f699c1a5715648c1b0144670c1ba00b48f"}, + {file = "cryptography-43.0.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:aae4d918f6b180a8ab8bf6511a419473d107df4dbb4225c7b48c5c9602c38c7f"}, + {file = "cryptography-43.0.0-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:232ce02943a579095a339ac4b390fbbe97f5b5d5d107f8a08260ea2768be8cc2"}, + {file = "cryptography-43.0.0-pp39-pypy39_pp73-manylinux_2_28_aarch64.whl", hash = "sha256:5bcb8a5620008a8034d39bce21dc3e23735dfdb6a33a06974739bfa04f853947"}, + {file = "cryptography-43.0.0-pp39-pypy39_pp73-manylinux_2_28_x86_64.whl", hash = "sha256:08a24a7070b2b6804c1940ff0f910ff728932a9d0e80e7814234269f9d46d069"}, + {file = "cryptography-43.0.0-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:e9c5266c432a1e23738d178e51c2c7a5e2ddf790f248be939448c0ba2021f9d1"}, + {file = "cryptography-43.0.0.tar.gz", hash = "sha256:b88075ada2d51aa9f18283532c9f60e72170041bba88d7f37e49cbb10275299e"}, +] + [[package]] name = "execnet" version = "2.1.1" @@ -70,6 +208,17 @@ files = [ {file = "execnet-2.1.1.tar.gz", hash = "sha256:5189b52c6121c24feae288166ab41b32549c7e2348652736540b9e6e7d4e72e3"}, ] +[[package]] +name = "filelock" +version = "3.15.4" +requires_python = ">=3.8" +summary = "A platform independent file lock." +groups = ["test"] +files = [ + {file = "filelock-3.15.4-py3-none-any.whl", hash = "sha256:6ca1fffae96225dab4c6eaf1c4f4f28cd2568d3ec2a44e15a08520504de468e7"}, + {file = "filelock-3.15.4.tar.gz", hash = "sha256:2207938cbc1844345cb01a5a95524dae30f0ce089eba5b00378295a17e3e90cb"}, +] + [[package]] name = "freezegun" version = "1.5.1" @@ -84,6 +233,17 @@ files = [ {file = "freezegun-1.5.1.tar.gz", hash = "sha256:b29dedfcda6d5e8e083ce71b2b542753ad48cfec44037b3fc79702e2980a89e9"}, ] +[[package]] +name = "fsspec" +version = "2024.6.1" +requires_python = ">=3.8" +summary = "File-system specification" +groups = ["test"] +files = [ + {file = "fsspec-2024.6.1-py3-none-any.whl", hash = "sha256:3cb443f8bcd2efb31295a5b9fdb02aee81d8452c80d28f97a6d0959e6cee101e"}, + {file = "fsspec-2024.6.1.tar.gz", hash = "sha256:fad7d7e209dd4c1208e3bbfda706620e0da5142bebbd9c384afb95b07e798e49"}, +] + [[package]] name = "iniconfig" version = "2.0.0" @@ -100,7 +260,7 @@ name = "jinja2" version = "3.1.4" requires_python = ">=3.7" summary = "A very fast and expressive template engine." -groups = ["release"] +groups = ["release", "test"] dependencies = [ "MarkupSafe>=2.0", ] @@ -109,12 +269,26 @@ files = [ {file = "jinja2-3.1.4.tar.gz", hash = "sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369"}, ] +[[package]] +name = "markdown-it-py" +version = "3.0.0" +requires_python = ">=3.8" +summary = "Python port of markdown-it. Markdown parsing, done right!" +groups = ["test"] +dependencies = [ + "mdurl~=0.1", +] +files = [ + {file = "markdown-it-py-3.0.0.tar.gz", hash = "sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb"}, + {file = "markdown_it_py-3.0.0-py3-none-any.whl", hash = "sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1"}, +] + [[package]] name = "markupsafe" version = "2.1.5" requires_python = ">=3.7" summary = "Safely add untrusted strings to HTML/XML markup." -groups = ["release"] +groups = ["release", "test"] files = [ {file = "MarkupSafe-2.1.5-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:629ddd2ca402ae6dbedfceeba9c46d5f7b2a61d9749597d4307f943ef198fc1f"}, {file = "MarkupSafe-2.1.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:5b7b716f97b52c5a14bffdf688f971b2d5ef4029127f1ad7a513973cfd818df2"}, @@ -139,6 +313,27 @@ files = [ {file = "MarkupSafe-2.1.5.tar.gz", hash = "sha256:d283d37a890ba4c1ae73ffadf8046435c76e7bc2247bbb63c00bd1a709c6544b"}, ] +[[package]] +name = "mdurl" +version = "0.1.2" +requires_python = ">=3.7" +summary = "Markdown URL utilities" +groups = ["test"] +files = [ + {file = "mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8"}, + {file = "mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba"}, +] + +[[package]] +name = "mpmath" +version = "1.3.0" +summary = "Python library for arbitrary-precision floating-point arithmetic" +groups = ["test"] +files = [ + {file = "mpmath-1.3.0-py3-none-any.whl", hash = "sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c"}, + {file = "mpmath-1.3.0.tar.gz", hash = "sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f"}, +] + [[package]] name = "mypy" version = "1.11.1" @@ -175,6 +370,207 @@ files = [ {file = "mypy_extensions-1.0.0.tar.gz", hash = "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782"}, ] +[[package]] +name = "networkx" +version = "3.3" +requires_python = ">=3.10" +summary = "Python package for creating and manipulating graphs and networks" +groups = ["test"] +files = [ + {file = "networkx-3.3-py3-none-any.whl", hash = "sha256:28575580c6ebdaf4505b22c6256a2b9de86b316dc63ba9e93abde3d78dfdbcf2"}, + {file = "networkx-3.3.tar.gz", hash = "sha256:0c127d8b2f4865f59ae9cb8aafcd60b5c70f3241ebd66f7defad7c4ab90126c9"}, +] + +[[package]] +name = "numpy" +version = "2.0.1" +requires_python = ">=3.9" +summary = "Fundamental package for array computing in Python" +groups = ["test"] +files = [ + {file = "numpy-2.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:75b4e316c5902d8163ef9d423b1c3f2f6252226d1aa5cd8a0a03a7d01ffc6268"}, + {file = "numpy-2.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:6e4eeb6eb2fced786e32e6d8df9e755ce5be920d17f7ce00bc38fcde8ccdbf9e"}, + {file = "numpy-2.0.1-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:a1e01dcaab205fbece13c1410253a9eea1b1c9b61d237b6fa59bcc46e8e89343"}, + {file = "numpy-2.0.1-cp311-cp311-macosx_14_0_x86_64.whl", hash = "sha256:a8fc2de81ad835d999113ddf87d1ea2b0f4704cbd947c948d2f5513deafe5a7b"}, + {file = "numpy-2.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5a3d94942c331dd4e0e1147f7a8699a4aa47dffc11bf8a1523c12af8b2e91bbe"}, + {file = "numpy-2.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:15eb4eca47d36ec3f78cde0a3a2ee24cf05ca7396ef808dda2c0ddad7c2bde67"}, + {file = "numpy-2.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:b83e16a5511d1b1f8a88cbabb1a6f6a499f82c062a4251892d9ad5d609863fb7"}, + {file = "numpy-2.0.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1f87fec1f9bc1efd23f4227becff04bd0e979e23ca50cc92ec88b38489db3b55"}, + {file = "numpy-2.0.1-cp311-cp311-win32.whl", hash = "sha256:36d3a9405fd7c511804dc56fc32974fa5533bdeb3cd1604d6b8ff1d292b819c4"}, + {file = "numpy-2.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:08458fbf403bff5e2b45f08eda195d4b0c9b35682311da5a5a0a0925b11b9bd8"}, + {file = "numpy-2.0.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:6bf4e6f4a2a2e26655717a1983ef6324f2664d7011f6ef7482e8c0b3d51e82ac"}, + {file = "numpy-2.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7d6fddc5fe258d3328cd8e3d7d3e02234c5d70e01ebe377a6ab92adb14039cb4"}, + {file = "numpy-2.0.1-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:5daab361be6ddeb299a918a7c0864fa8618af66019138263247af405018b04e1"}, + {file = "numpy-2.0.1-cp312-cp312-macosx_14_0_x86_64.whl", hash = "sha256:ea2326a4dca88e4a274ba3a4405eb6c6467d3ffbd8c7d38632502eaae3820587"}, + {file = "numpy-2.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:529af13c5f4b7a932fb0e1911d3a75da204eff023ee5e0e79c1751564221a5c8"}, + {file = "numpy-2.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6790654cb13eab303d8402354fabd47472b24635700f631f041bd0b65e37298a"}, + {file = "numpy-2.0.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:cbab9fc9c391700e3e1287666dfd82d8666d10e69a6c4a09ab97574c0b7ee0a7"}, + {file = "numpy-2.0.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:99d0d92a5e3613c33a5f01db206a33f8fdf3d71f2912b0de1739894668b7a93b"}, + {file = "numpy-2.0.1-cp312-cp312-win32.whl", hash = "sha256:173a00b9995f73b79eb0191129f2455f1e34c203f559dd118636858cc452a1bf"}, + {file = "numpy-2.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:bb2124fdc6e62baae159ebcfa368708867eb56806804d005860b6007388df171"}, + {file = "numpy-2.0.1-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:61728fba1e464f789b11deb78a57805c70b2ed02343560456190d0501ba37b0f"}, + {file = "numpy-2.0.1-pp39-pypy39_pp73-macosx_14_0_x86_64.whl", hash = "sha256:12f5d865d60fb9734e60a60f1d5afa6d962d8d4467c120a1c0cda6eb2964437d"}, + {file = "numpy-2.0.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:eacf3291e263d5a67d8c1a581a8ebbcfd6447204ef58828caf69a5e3e8c75990"}, + {file = "numpy-2.0.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:2c3a346ae20cfd80b6cfd3e60dc179963ef2ea58da5ec074fd3d9e7a1e7ba97f"}, + {file = "numpy-2.0.1.tar.gz", hash = "sha256:485b87235796410c3519a699cfe1faab097e509e90ebb05dcd098db2ae87e7b3"}, +] + +[[package]] +name = "nvidia-cublas-cu12" +version = "12.1.3.1" +requires_python = ">=3" +summary = "CUBLAS native runtime libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_cublas_cu12-12.1.3.1-py3-none-manylinux1_x86_64.whl", hash = "sha256:ee53ccca76a6fc08fb9701aa95b6ceb242cdaab118c3bb152af4e579af792728"}, + {file = "nvidia_cublas_cu12-12.1.3.1-py3-none-win_amd64.whl", hash = "sha256:2b964d60e8cf11b5e1073d179d85fa340c120e99b3067558f3cf98dd69d02906"}, +] + +[[package]] +name = "nvidia-cuda-cupti-cu12" +version = "12.1.105" +requires_python = ">=3" +summary = "CUDA profiling tools runtime libs." +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_cuda_cupti_cu12-12.1.105-py3-none-manylinux1_x86_64.whl", hash = "sha256:e54fde3983165c624cb79254ae9818a456eb6e87a7fd4d56a2352c24ee542d7e"}, + {file = "nvidia_cuda_cupti_cu12-12.1.105-py3-none-win_amd64.whl", hash = "sha256:bea8236d13a0ac7190bd2919c3e8e6ce1e402104276e6f9694479e48bb0eb2a4"}, +] + +[[package]] +name = "nvidia-cuda-nvrtc-cu12" +version = "12.1.105" +requires_python = ">=3" +summary = "NVRTC native runtime libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-manylinux1_x86_64.whl", hash = "sha256:339b385f50c309763ca65456ec75e17bbefcbbf2893f462cb8b90584cd27a1c2"}, + {file = "nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-win_amd64.whl", hash = "sha256:0a98a522d9ff138b96c010a65e145dc1b4850e9ecb75a0172371793752fd46ed"}, +] + +[[package]] +name = "nvidia-cuda-runtime-cu12" +version = "12.1.105" +requires_python = ">=3" +summary = "CUDA Runtime native Libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_cuda_runtime_cu12-12.1.105-py3-none-manylinux1_x86_64.whl", hash = "sha256:6e258468ddf5796e25f1dc591a31029fa317d97a0a94ed93468fc86301d61e40"}, + {file = "nvidia_cuda_runtime_cu12-12.1.105-py3-none-win_amd64.whl", hash = "sha256:dfb46ef84d73fababab44cf03e3b83f80700d27ca300e537f85f636fac474344"}, +] + +[[package]] +name = "nvidia-cudnn-cu12" +version = "9.1.0.70" +requires_python = ">=3" +summary = "cuDNN runtime libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +dependencies = [ + "nvidia-cublas-cu12", +] +files = [ + {file = "nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl", hash = "sha256:165764f44ef8c61fcdfdfdbe769d687e06374059fbb388b6c89ecb0e28793a6f"}, + {file = "nvidia_cudnn_cu12-9.1.0.70-py3-none-win_amd64.whl", hash = "sha256:6278562929433d68365a07a4a1546c237ba2849852c0d4b2262a486e805b977a"}, +] + +[[package]] +name = "nvidia-cufft-cu12" +version = "11.0.2.54" +requires_python = ">=3" +summary = "CUFFT native runtime libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_cufft_cu12-11.0.2.54-py3-none-manylinux1_x86_64.whl", hash = "sha256:794e3948a1aa71fd817c3775866943936774d1c14e7628c74f6f7417224cdf56"}, + {file = "nvidia_cufft_cu12-11.0.2.54-py3-none-win_amd64.whl", hash = "sha256:d9ac353f78ff89951da4af698f80870b1534ed69993f10a4cf1d96f21357e253"}, +] + +[[package]] +name = "nvidia-curand-cu12" +version = "10.3.2.106" +requires_python = ">=3" +summary = "CURAND native runtime libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_curand_cu12-10.3.2.106-py3-none-manylinux1_x86_64.whl", hash = "sha256:9d264c5036dde4e64f1de8c50ae753237c12e0b1348738169cd0f8a536c0e1e0"}, + {file = "nvidia_curand_cu12-10.3.2.106-py3-none-win_amd64.whl", hash = "sha256:75b6b0c574c0037839121317e17fd01f8a69fd2ef8e25853d826fec30bdba74a"}, +] + +[[package]] +name = "nvidia-cusolver-cu12" +version = "11.4.5.107" +requires_python = ">=3" +summary = "CUDA solver native runtime libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +dependencies = [ + "nvidia-cublas-cu12", + "nvidia-cusparse-cu12", + "nvidia-nvjitlink-cu12", +] +files = [ + {file = "nvidia_cusolver_cu12-11.4.5.107-py3-none-manylinux1_x86_64.whl", hash = "sha256:8a7ec542f0412294b15072fa7dab71d31334014a69f953004ea7a118206fe0dd"}, + {file = "nvidia_cusolver_cu12-11.4.5.107-py3-none-win_amd64.whl", hash = "sha256:74e0c3a24c78612192a74fcd90dd117f1cf21dea4822e66d89e8ea80e3cd2da5"}, +] + +[[package]] +name = "nvidia-cusparse-cu12" +version = "12.1.0.106" +requires_python = ">=3" +summary = "CUSPARSE native runtime libraries" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +dependencies = [ + "nvidia-nvjitlink-cu12", +] +files = [ + {file = "nvidia_cusparse_cu12-12.1.0.106-py3-none-manylinux1_x86_64.whl", hash = "sha256:f3b50f42cf363f86ab21f720998517a659a48131e8d538dc02f8768237bd884c"}, + {file = "nvidia_cusparse_cu12-12.1.0.106-py3-none-win_amd64.whl", hash = "sha256:b798237e81b9719373e8fae8d4f091b70a0cf09d9d85c95a557e11df2d8e9a5a"}, +] + +[[package]] +name = "nvidia-nccl-cu12" +version = "2.20.5" +requires_python = ">=3" +summary = "NVIDIA Collective Communication Library (NCCL) Runtime" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_aarch64.whl", hash = "sha256:1fc150d5c3250b170b29410ba682384b14581db722b2531b0d8d33c595f33d01"}, + {file = "nvidia_nccl_cu12-2.20.5-py3-none-manylinux2014_x86_64.whl", hash = "sha256:057f6bf9685f75215d0c53bf3ac4a10b3e6578351de307abad9e18a99182af56"}, +] + +[[package]] +name = "nvidia-nvjitlink-cu12" +version = "12.6.20" +requires_python = ">=3" +summary = "Nvidia JIT LTO Library" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_nvjitlink_cu12-12.6.20-py3-none-manylinux2014_aarch64.whl", hash = "sha256:84fb38465a5bc7c70cbc320cfd0963eb302ee25a5e939e9f512bbba55b6072fb"}, + {file = "nvidia_nvjitlink_cu12-12.6.20-py3-none-manylinux2014_x86_64.whl", hash = "sha256:562ab97ea2c23164823b2a89cb328d01d45cb99634b8c65fe7cd60d14562bd79"}, + {file = "nvidia_nvjitlink_cu12-12.6.20-py3-none-win_amd64.whl", hash = "sha256:ed3c43a17f37b0c922a919203d2d36cbef24d41cc3e6b625182f8b58203644f6"}, +] + +[[package]] +name = "nvidia-nvtx-cu12" +version = "12.1.105" +requires_python = ">=3" +summary = "NVIDIA Tools Extension" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\"" +files = [ + {file = "nvidia_nvtx_cu12-12.1.105-py3-none-manylinux1_x86_64.whl", hash = "sha256:dc21cf308ca5691e7c04d962e213f8a4aa9bbfa23d95412f452254c2caeb09e5"}, + {file = "nvidia_nvtx_cu12-12.1.105-py3-none-win_amd64.whl", hash = "sha256:65f4d98982b31b60026e0e6de73fbdfc09d08a96f4656dd3665ca616a11e1e82"}, +] + [[package]] name = "packaging" version = "24.1" @@ -186,6 +582,22 @@ files = [ {file = "packaging-24.1.tar.gz", hash = "sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002"}, ] +[[package]] +name = "paramiko" +version = "3.4.1" +requires_python = ">=3.6" +summary = "SSH2 protocol library" +groups = ["test"] +dependencies = [ + "bcrypt>=3.2", + "cryptography>=3.3", + "pynacl>=1.5", +] +files = [ + {file = "paramiko-3.4.1-py3-none-any.whl", hash = "sha256:8e49fd2f82f84acf7ffd57c64311aa2b30e575370dc23bdb375b10262f7eac32"}, + {file = "paramiko-3.4.1.tar.gz", hash = "sha256:8b15302870af7f6652f2e038975c1d2973f06046cb5d7d65355668b3ecbece0c"}, +] + [[package]] name = "pluggy" version = "1.5.0" @@ -197,6 +609,61 @@ files = [ {file = "pluggy-1.5.0.tar.gz", hash = "sha256:2cffa88e94fdc978c4c574f15f9e59b7f4201d439195c3715ca9e2486f1d0cf1"}, ] +[[package]] +name = "pycparser" +version = "2.22" +requires_python = ">=3.8" +summary = "C parser in Python" +groups = ["lint", "test"] +files = [ + {file = "pycparser-2.22-py3-none-any.whl", hash = "sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc"}, + {file = "pycparser-2.22.tar.gz", hash = "sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6"}, +] + +[[package]] +name = "pygments" +version = "2.18.0" +requires_python = ">=3.8" +summary = "Pygments is a syntax highlighting package written in Python." +groups = ["test"] +files = [ + {file = "pygments-2.18.0-py3-none-any.whl", hash = "sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a"}, + {file = "pygments-2.18.0.tar.gz", hash = "sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199"}, +] + +[[package]] +name = "pynacl" +version = "1.5.0" +requires_python = ">=3.6" +summary = "Python binding to the Networking and Cryptography (NaCl) library" +groups = ["test"] +dependencies = [ + "cffi>=1.4.1", +] +files = [ + {file = "PyNaCl-1.5.0-cp36-abi3-macosx_10_10_universal2.whl", hash = "sha256:401002a4aaa07c9414132aaed7f6836ff98f59277a234704ff66878c2ee4a0d1"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.manylinux_2_24_aarch64.whl", hash = "sha256:52cb72a79269189d4e0dc537556f4740f7f0a9ec41c1322598799b0bdad4ef92"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a36d4a9dda1f19ce6e03c9a784a2921a4b726b02e1c736600ca9c22029474394"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl", hash = "sha256:0c84947a22519e013607c9be43706dd42513f9e6ae5d39d3613ca1e142fba44d"}, + {file = "PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:06b8f6fa7f5de8d5d2f7573fe8c863c051225a27b61e6860fd047b1775807858"}, + {file = "PyNaCl-1.5.0-cp36-abi3-musllinux_1_1_aarch64.whl", hash = "sha256:a422368fc821589c228f4c49438a368831cb5bbc0eab5ebe1d7fac9dded6567b"}, + {file = "PyNaCl-1.5.0-cp36-abi3-musllinux_1_1_x86_64.whl", hash = "sha256:61f642bf2378713e2c2e1de73444a3778e5f0a38be6fee0fe532fe30060282ff"}, + {file = "PyNaCl-1.5.0-cp36-abi3-win32.whl", hash = "sha256:e46dae94e34b085175f8abb3b0aaa7da40767865ac82c928eeb9e57e1ea8a543"}, + {file = "PyNaCl-1.5.0-cp36-abi3-win_amd64.whl", hash = "sha256:20f42270d27e1b6a29f54032090b972d97f0a1b0948cc52392041ef7831fee93"}, + {file = "PyNaCl-1.5.0.tar.gz", hash = "sha256:8ac7448f09ab85811607bdd21ec2464495ac8b7c66d146bf545b0f08fb9220ba"}, +] + +[[package]] +name = "pyproject-hooks" +version = "1.1.0" +requires_python = ">=3.7" +summary = "Wrappers to call pyproject.toml-based build backend hooks." +groups = ["test"] +files = [ + {file = "pyproject_hooks-1.1.0-py3-none-any.whl", hash = "sha256:7ceeefe9aec63a1064c18d939bdc3adf2d8aa1988a510afec15151578b232aa2"}, + {file = "pyproject_hooks-1.1.0.tar.gz", hash = "sha256:4b37730834edbd6bd37f26ece6b44802fb1c1ee2ece0e54ddff8bfc06db86965"}, +] + [[package]] name = "pytest" version = "8.3.2" @@ -271,31 +738,46 @@ files = [ {file = "python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427"}, ] +[[package]] +name = "rich" +version = "13.7.1" +requires_python = ">=3.7.0" +summary = "Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal" +groups = ["test"] +dependencies = [ + "markdown-it-py>=2.2.0", + "pygments<3.0.0,>=2.13.0", +] +files = [ + {file = "rich-13.7.1-py3-none-any.whl", hash = "sha256:4edbae314f59eb482f54e9e30bf00d33350aaa94f4bfcd4e9e3110e64d0d7222"}, + {file = "rich-13.7.1.tar.gz", hash = "sha256:9be308cb1fe2f1f57d67ce99e95af38a1e2bc71ad9813b0e247cf7ffbcc3a432"}, +] + [[package]] name = "ruff" -version = "0.5.6" +version = "0.5.7" requires_python = ">=3.7" summary = "An extremely fast Python linter and code formatter, written in Rust." groups = ["lint"] files = [ - {file = "ruff-0.5.6-py3-none-linux_armv6l.whl", hash = "sha256:a0ef5930799a05522985b9cec8290b185952f3fcd86c1772c3bdbd732667fdcd"}, - {file = "ruff-0.5.6-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:b652dc14f6ef5d1552821e006f747802cc32d98d5509349e168f6bf0ee9f8f42"}, - {file = "ruff-0.5.6-py3-none-macosx_11_0_arm64.whl", hash = "sha256:80521b88d26a45e871f31e4b88938fd87db7011bb961d8afd2664982dfc3641a"}, - {file = "ruff-0.5.6-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d9bc8f328a9f1309ae80e4d392836e7dbc77303b38ed4a7112699e63d3b066ab"}, - {file = "ruff-0.5.6-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:4d394940f61f7720ad371ddedf14722ee1d6250fd8d020f5ea5a86e7be217daf"}, - {file = "ruff-0.5.6-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:111a99cdb02f69ddb2571e2756e017a1496c2c3a2aeefe7b988ddab38b416d36"}, - {file = "ruff-0.5.6-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:e395daba77a79f6dc0d07311f94cc0560375ca20c06f354c7c99af3bf4560c5d"}, - {file = "ruff-0.5.6-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c476acb43c3c51e3c614a2e878ee1589655fa02dab19fe2db0423a06d6a5b1b6"}, - {file = "ruff-0.5.6-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:e2ff8003f5252fd68425fd53d27c1f08b201d7ed714bb31a55c9ac1d4c13e2eb"}, - {file = "ruff-0.5.6-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:c94e084ba3eaa80c2172918c2ca2eb2230c3f15925f4ed8b6297260c6ef179ad"}, - {file = "ruff-0.5.6-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:1f77c1c3aa0669fb230b06fb24ffa3e879391a3ba3f15e3d633a752da5a3e670"}, - {file = "ruff-0.5.6-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:f908148c93c02873210a52cad75a6eda856b2cbb72250370ce3afef6fb99b1ed"}, - {file = "ruff-0.5.6-py3-none-musllinux_1_2_i686.whl", hash = "sha256:563a7ae61ad284187d3071d9041c08019975693ff655438d8d4be26e492760bd"}, - {file = "ruff-0.5.6-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:94fe60869bfbf0521e04fd62b74cbca21cbc5beb67cbb75ab33fe8c174f54414"}, - {file = "ruff-0.5.6-py3-none-win32.whl", hash = "sha256:e6a584c1de6f8591c2570e171cc7ce482bb983d49c70ddf014393cd39e9dfaed"}, - {file = "ruff-0.5.6-py3-none-win_amd64.whl", hash = "sha256:d7fe7dccb1a89dc66785d7aa0ac283b2269712d8ed19c63af908fdccca5ccc1a"}, - {file = "ruff-0.5.6-py3-none-win_arm64.whl", hash = "sha256:57c6c0dd997b31b536bff49b9eee5ed3194d60605a4427f735eeb1f9c1b8d264"}, - {file = "ruff-0.5.6.tar.gz", hash = "sha256:07c9e3c2a8e1fe377dd460371c3462671a728c981c3205a5217291422209f642"}, + {file = "ruff-0.5.7-py3-none-linux_armv6l.whl", hash = "sha256:548992d342fc404ee2e15a242cdbea4f8e39a52f2e7752d0e4cbe88d2d2f416a"}, + {file = "ruff-0.5.7-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:00cc8872331055ee017c4f1071a8a31ca0809ccc0657da1d154a1d2abac5c0be"}, + {file = "ruff-0.5.7-py3-none-macosx_11_0_arm64.whl", hash = "sha256:eaf3d86a1fdac1aec8a3417a63587d93f906c678bb9ed0b796da7b59c1114a1e"}, + {file = "ruff-0.5.7-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a01c34400097b06cf8a6e61b35d6d456d5bd1ae6961542de18ec81eaf33b4cb8"}, + {file = "ruff-0.5.7-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:fcc8054f1a717e2213500edaddcf1dbb0abad40d98e1bd9d0ad364f75c763eea"}, + {file = "ruff-0.5.7-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:7f70284e73f36558ef51602254451e50dd6cc479f8b6f8413a95fcb5db4a55fc"}, + {file = "ruff-0.5.7-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:a78ad870ae3c460394fc95437d43deb5c04b5c29297815a2a1de028903f19692"}, + {file = "ruff-0.5.7-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:9ccd078c66a8e419475174bfe60a69adb36ce04f8d4e91b006f1329d5cd44bcf"}, + {file = "ruff-0.5.7-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:7e31c9bad4ebf8fdb77b59cae75814440731060a09a0e0077d559a556453acbb"}, + {file = "ruff-0.5.7-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8d796327eed8e168164346b769dd9a27a70e0298d667b4ecee6877ce8095ec8e"}, + {file = "ruff-0.5.7-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:4a09ea2c3f7778cc635e7f6edf57d566a8ee8f485f3c4454db7771efb692c499"}, + {file = "ruff-0.5.7-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:a36d8dcf55b3a3bc353270d544fb170d75d2dff41eba5df57b4e0b67a95bb64e"}, + {file = "ruff-0.5.7-py3-none-musllinux_1_2_i686.whl", hash = "sha256:9369c218f789eefbd1b8d82a8cf25017b523ac47d96b2f531eba73770971c9e5"}, + {file = "ruff-0.5.7-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:b88ca3db7eb377eb24fb7c82840546fb7acef75af4a74bd36e9ceb37a890257e"}, + {file = "ruff-0.5.7-py3-none-win32.whl", hash = "sha256:33d61fc0e902198a3e55719f4be6b375b28f860b09c281e4bdbf783c0566576a"}, + {file = "ruff-0.5.7-py3-none-win_amd64.whl", hash = "sha256:083bbcbe6fadb93cd86709037acc510f86eed5a314203079df174c40bbbca6b3"}, + {file = "ruff-0.5.7-py3-none-win_arm64.whl", hash = "sha256:2dca26154ff9571995107221d0aeaad0e75a77b5a682d6236cf89a58c70b76f4"}, + {file = "ruff-0.5.7.tar.gz", hash = "sha256:8dfc0a458797f5d9fb622dd0efc52d796f23f0a1493a9527f4e49a550ae9a7e5"}, ] [[package]] @@ -309,6 +791,57 @@ files = [ {file = "six-1.16.0.tar.gz", hash = "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926"}, ] +[[package]] +name = "sympy" +version = "1.13.2" +requires_python = ">=3.8" +summary = "Computer algebra system (CAS) in Python" +groups = ["test"] +dependencies = [ + "mpmath<1.4,>=1.1.0", +] +files = [ + {file = "sympy-1.13.2-py3-none-any.whl", hash = "sha256:c51d75517712f1aed280d4ce58506a4a88d635d6b5dd48b39102a7ae1f3fcfe9"}, + {file = "sympy-1.13.2.tar.gz", hash = "sha256:401449d84d07be9d0c7a46a64bd54fe097667d5e7181bfe67ec777be9e01cb13"}, +] + +[[package]] +name = "torch" +version = "2.4.0" +requires_python = ">=3.8.0" +summary = "Tensors and Dynamic neural networks in Python with strong GPU acceleration" +groups = ["test"] +dependencies = [ + "filelock", + "fsspec", + "jinja2", + "networkx", + "nvidia-cublas-cu12==12.1.3.1; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-cuda-cupti-cu12==12.1.105; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-cuda-nvrtc-cu12==12.1.105; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-cuda-runtime-cu12==12.1.105; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-cudnn-cu12==9.1.0.70; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-cufft-cu12==11.0.2.54; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-curand-cu12==10.3.2.106; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-cusolver-cu12==11.4.5.107; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-cusparse-cu12==12.1.0.106; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-nccl-cu12==2.20.5; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "nvidia-nvtx-cu12==12.1.105; platform_system == \"Linux\" and platform_machine == \"x86_64\"", + "sympy", + "triton==3.0.0; platform_system == \"Linux\" and platform_machine == \"x86_64\" and python_version < \"3.13\"", + "typing-extensions>=4.8.0", +] +files = [ + {file = "torch-2.4.0-cp311-cp311-manylinux1_x86_64.whl", hash = "sha256:e743adadd8c8152bb8373543964551a7cb7cc20ba898dc8f9c0cdbe47c283de0"}, + {file = "torch-2.4.0-cp311-cp311-manylinux2014_aarch64.whl", hash = "sha256:7334325c0292cbd5c2eac085f449bf57d3690932eac37027e193ba775703c9e6"}, + {file = "torch-2.4.0-cp311-cp311-win_amd64.whl", hash = "sha256:97730014da4c57ffacb3c09298c6ce05400606e890bd7a05008d13dd086e46b1"}, + {file = "torch-2.4.0-cp311-none-macosx_11_0_arm64.whl", hash = "sha256:f169b4ea6dc93b3a33319611fcc47dc1406e4dd539844dcbd2dec4c1b96e166d"}, + {file = "torch-2.4.0-cp312-cp312-manylinux1_x86_64.whl", hash = "sha256:997084a0f9784d2a89095a6dc67c7925e21bf25dea0b3d069b41195016ccfcbb"}, + {file = "torch-2.4.0-cp312-cp312-manylinux2014_aarch64.whl", hash = "sha256:bc3988e8b36d1e8b998d143255d9408d8c75da4ab6dd0dcfd23b623dfb0f0f57"}, + {file = "torch-2.4.0-cp312-cp312-win_amd64.whl", hash = "sha256:3374128bbf7e62cdaed6c237bfd39809fbcfaa576bee91e904706840c3f2195c"}, + {file = "torch-2.4.0-cp312-none-macosx_11_0_arm64.whl", hash = "sha256:91aaf00bfe1ffa44dc5b52809d9a95129fca10212eca3ac26420eb11727c6288"}, +] + [[package]] name = "towncrier" version = "24.7.1" @@ -324,6 +857,22 @@ files = [ {file = "towncrier-24.7.1.tar.gz", hash = "sha256:57a057faedabcadf1a62f6f9bad726ae566c1f31a411338ddb8316993f583b3d"}, ] +[[package]] +name = "triton" +version = "3.0.0" +summary = "A language and compiler for custom Deep Learning operations" +groups = ["test"] +marker = "platform_system == \"Linux\" and platform_machine == \"x86_64\" and python_version < \"3.13\"" +dependencies = [ + "filelock", +] +files = [ + {file = "triton-3.0.0-1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:5ce8520437c602fb633f1324cc3871c47bee3b67acf9756c1a66309b60e3216c"}, + {file = "triton-3.0.0-1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:34e509deb77f1c067d8640725ef00c5cbfcb2052a1a3cb6a6d343841f92624eb"}, + {file = "triton-3.0.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:cd34f19a8582af96e6291d4afce25dac08cb2a5d218c599163761e8e0827208e"}, + {file = "triton-3.0.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0d5e10de8c011adeb7c878c6ce0dd6073b14367749e34467f1cff2bde1b78253"}, +] + [[package]] name = "types-freezegun" version = "1.1.10" @@ -334,6 +883,20 @@ files = [ {file = "types_freezegun-1.1.10-py3-none-any.whl", hash = "sha256:fadebe72213e0674036153366205038e1f95c8ca96deb4ef9b71ddc15413543e"}, ] +[[package]] +name = "types-paramiko" +version = "3.4.0.20240423" +requires_python = ">=3.8" +summary = "Typing stubs for paramiko" +groups = ["lint"] +dependencies = [ + "cryptography>=37.0.0", +] +files = [ + {file = "types-paramiko-3.4.0.20240423.tar.gz", hash = "sha256:aaa98dda232c47886563d66743d3a8b66c432790c596bc3bdd3f17f91be2a8c1"}, + {file = "types_paramiko-3.4.0.20240423-py3-none-any.whl", hash = "sha256:c56e0d43399a1b909901b1e0375e0ff6ee62e16cd6e00695024abc2e9fe02035"}, +] + [[package]] name = "types-python-dateutil" version = "2.9.0.20240316" @@ -345,6 +908,17 @@ files = [ {file = "types_python_dateutil-2.9.0.20240316-py3-none-any.whl", hash = "sha256:6b8cb66d960771ce5ff974e9dd45e38facb81718cc1e208b10b1baccbfdbee3b"}, ] +[[package]] +name = "types-pyyaml" +version = "6.0.12.20240808" +requires_python = ">=3.8" +summary = "Typing stubs for PyYAML" +groups = ["lint"] +files = [ + {file = "types-PyYAML-6.0.12.20240808.tar.gz", hash = "sha256:b8f76ddbd7f65440a8bda5526a9607e4c7a322dc2f8e1a8c405644f9a6f4b9af"}, + {file = "types_PyYAML-6.0.12.20240808-py3-none-any.whl", hash = "sha256:deda34c5c655265fc517b546c902aa6eed2ef8d3e921e4765fe606fe2afe8d35"}, +] + [[package]] name = "types-requests" version = "2.32.0.20240712" @@ -364,7 +938,7 @@ name = "typing-extensions" version = "4.12.2" requires_python = ">=3.8" summary = "Backported and Experimental Type Hints for Python 3.8+" -groups = ["lint"] +groups = ["lint", "test"] files = [ {file = "typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d"}, {file = "typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8"}, diff --git a/pyproject.toml b/pyproject.toml index 1a8eed8..8855c90 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -21,6 +21,15 @@ dynamic = [ dependencies = [ ] +[project.optional-dependencies] +torch = [ + "torch>=1.10", +] +vllm = [ + "torch>=1.10", + "vllm>=0.4", +] + [project.urls] "Source" = "https://github.com/backend-developers-ltd/deterministic-ml" "Issue Tracker" = "https://github.com/backend-developers-ltd/deterministic-ml/issues" @@ -42,6 +51,11 @@ test = [ "pytest-apiver", "pytest-asyncio", "pytest-xdist", + "paramiko", + "numpy", + "torch", + "build>=1.2.1", + "rich>=13.7.1", ] lint = [ "codespell[toml]", @@ -50,6 +64,8 @@ lint = [ "types-freezegun", "types-python-dateutil", "types-requests", + "types-PyYAML", + "types-paramiko", ] release = [ "towncrier", @@ -131,5 +147,7 @@ module = [ "nox", "pytest", "tests.*", + "tools.*", + "vllm", ] ignore_missing_imports = true diff --git a/src/__init__.py b/src/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/src/deterministic_ml/_internal/deterministic.py b/src/deterministic_ml/_internal/deterministic.py new file mode 100644 index 0000000..65ae3f1 --- /dev/null +++ b/src/deterministic_ml/_internal/deterministic.py @@ -0,0 +1,51 @@ +import logging +import os +import random +import warnings + +LOG = logging.getLogger(__name__) + + +def set_deterministic(seed: int, *, any_card: bool = False): + LOG.info(f"Setting deterministic mode with seed {seed!r}, any_card={any_card!r}") + random.seed(seed) + try: + import numpy # noqa: F401 + except ImportError: + pass + else: + numpy.random.seed(seed) + + try: + set_deterministic_pytorch(seed, any_card=any_card) + except ModuleNotFoundError: + pass + + +def set_deterministic_pytorch(seed: int, *, any_card: bool = False): + """ + Set random seed for PyTorch and ensure deterministic operations. + + :param seed: + :param any_card: attempt to make it deterministic on any GPU. Most likely won't work, but will decrease performance. + :return: None + """ + cublas_workspace_config = os.environ.get("CUBLAS_WORKSPACE_CONFIG") + if cublas_workspace_config is None: + os.environ["CUBLAS_WORKSPACE_CONFIG"] = ":4096:8" + + import torch # noqa: F401 + + torch.manual_seed(seed) + torch.use_deterministic_algorithms(True, warn_only=True) + torch.backends.cudnn.deterministic = True + torch.backends.cudnn.benchmark = False + + if any_card: + warnings.warn( + 'Attempt to set "any_card" deterministic mode. ' + "This will further decrease performance and does not guarantee deterministic result across all GPUs.", + UserWarning, + ) + torch.backends.cudnn.allow_tf32 = False + torch.backends.cuda.matmul.allow_tf32 = False diff --git a/src/deterministic_ml/_internal/sysinfo.py b/src/deterministic_ml/_internal/sysinfo.py new file mode 100644 index 0000000..f78a976 --- /dev/null +++ b/src/deterministic_ml/_internal/sysinfo.py @@ -0,0 +1,215 @@ +import csv +import platform +import re +import shutil +import subprocess +import sys + + +def run_cmd(cmd): + proc = subprocess.run(cmd, shell=True, capture_output=True, check=False, text=True) + if proc.returncode != 0: + raise RuntimeError(f"run_cmd error {cmd=!r} {proc.returncode=} {proc.stdout=!r} {proc.stderr=!r}") + return proc.stdout + + +def get_machine_specs(): + """ + Get machine specs using various system commands. + + Heavily inspired by https://github.com/backend-developers-ltd/ComputeHorde/blob/e8c0f720c02cf3308627b2dae43b899e253b0b54/executor/app/src/compute_horde_executor/executor/management/commands/run_executor.py#L212 + MIT Licensed https://github.com/backend-developers-ltd/ComputeHorde/blob/e8c0f720c02cf3308627b2dae43b899e253b0b54/LICENSE + """ + data = {} + + data["docker_support"] = { + "runc": False, + "nvidia": False, + } + if shutil.which("docker"): + for runtime in ["runc", "nvidia"]: + try: + data["docker_support"][runtime] = ( + subprocess.check_output( + [ + "docker", + "run", + "--rm", + "--runtime", + runtime, + "busybox:stable-musl", + "echo", + "works", + ], + text=True, + stderr=subprocess.DEVNULL, + ).strip() + == "works" + ) + except subprocess.CalledProcessError: + pass + + data["gpu"] = {"count": 0, "details": []} + nvidia_cmd = "nvidia-smi --query-gpu=name,driver_version,name,memory.total,compute_cap,power.limit,clocks.gr,clocks.mem --format=csv" # noqa: E501 + if data["docker_support"]["nvidia"]: + nvidia_cmd = f"docker run --rm --runtime=nvidia --gpus all ubuntu:24.04 {nvidia_cmd}" + try: + nvidia_cmd_output = run_cmd(nvidia_cmd) + csv_data = csv.reader(nvidia_cmd_output.splitlines()) + header = [x.strip() for x in next(csv_data)] + for row in csv_data: + row = [x.strip() for x in row] + gpu_data = dict(zip(header, row)) + data["gpu"]["details"].append( + { + "name": gpu_data["name"], + "driver": gpu_data["driver_version"], + "capacity": gpu_data["memory.total [MiB]"].split(" ")[0], + "cuda": gpu_data["compute_cap"], + "power_limit": gpu_data["power.limit [W]"].split(" ")[0], + "graphics_speed": gpu_data["clocks.current.graphics [MHz]"].split(" ")[0], + "memory_speed": gpu_data["clocks.current.memory [MHz]"].split(" ")[0], + } + ) + data["gpu"]["count"] = len(data["gpu"]["details"]) + except Exception as exc: + # print(f'Error processing scraped gpu specs: {exc}', flush=True) + data["gpu_scrape_error"] = repr(exc) + + data["cpu"] = {"count": 0, "model": "", "clocks": []} + try: + lscpu_output = run_cmd("lscpu") + data["cpu"]["model"] = re.search(r"Model name:\s*(.*)$", lscpu_output, re.M).group(1) + data["cpu"]["count"] = int(re.search(r"CPU\(s\):\s*(.*)", lscpu_output).group(1)) + + cpu_data = run_cmd('lscpu --parse=MHZ | grep -Po "^[0-9,.]*$"').splitlines() + data["cpu"]["clocks"] = [float(x) for x in cpu_data] + except Exception as exc: + data["cpu_scrape_error"] = repr(exc) + + data["ram"] = {} + try: + with open("/proc/meminfo") as f: + meminfo = f.read() + + for name, key in [ + ("MemAvailable", "available"), + ("MemFree", "free"), + ("MemTotal", "total"), + ]: + data["ram"][key] = int(re.search(rf"^{name}:\s*(\d+)\s+kB$", meminfo, re.M).group(1)) + data["ram"]["used"] = data["ram"]["total"] - data["ram"]["free"] + except Exception as exc: + data["ram_scrape_error"] = repr(exc) + + data["hard_disk"] = {} + try: + disk_usage = shutil.disk_usage(".") + data["hard_disk"] = { + "total": disk_usage.total // 1024, # in kiB + "used": disk_usage.used // 1024, + "free": disk_usage.free // 1024, + } + except Exception as exc: + data["hard_disk_scrape_error"] = repr(exc) + + data["os"] = "" + try: + data["os"] = run_cmd('lsb_release -d | grep -Po "Description:\\s*\\K.*"').strip() + except Exception as exc: + data["os_scrape_error"] = repr(exc) + + return data + + +def get_python_env_specs(): + if shutil.which("uv") is None: + pip_cmd = [sys.executable, "-m", "pip"] + else: + pip_cmd = ["uv", "pip"] + + package_list = ( + subprocess.check_output( + [*pip_cmd, "freeze"], + stderr=subprocess.DEVNULL, + text=True, + ) + .strip() + .split("\n") + ) + + data = { + "version": sys.version, + "packages": package_list, + } + + return data + + +def get_system_info(): + """Gather system information.""" + system_info = { + "os": platform.system(), + "os_version": platform.version(), + "release": platform.release(), + "machine": platform.machine(), + "processor": platform.processor(), + } + + try: + dpkg_packages = ( + subprocess.check_output( + [ + "dpkg-query", + "-W", + "-f", + "${Package}==${Version}\n", + ], + text=True, + ) + .strip() + .splitlines() + ) + except subprocess.CalledProcessError: + pass + else: + system_info["dpkg_packages"] = dpkg_packages + + return system_info + + +def cuda_info(): + try: + import torch + except ImportError: + return {} + else: + return { + "cuda": torch.version.cuda, + "cudnn": torch.backends.cudnn.version(), + } + + +def get_specs(): + return { + "machine": get_machine_specs(), + "python": get_python_env_specs(), + "system": get_system_info(), + "cuda": cuda_info(), + } + + +def _main(): + specs = get_specs() + try: + import yaml + except ImportError: + import json + + json.dump(specs, sys.stdout, indent=2) + else: + yaml.safe_dump(specs, sys.stdout, sort_keys=True) + + +if __name__ == "__main__": + _main() diff --git a/src/deterministic_ml/_v2/__init__.py b/src/deterministic_ml/_v2/__init__.py new file mode 100644 index 0000000..2ea1bfc --- /dev/null +++ b/src/deterministic_ml/_v2/__init__.py @@ -0,0 +1,4 @@ +from deterministic_ml._internal.deterministic import ( + set_deterministic, + set_deterministic_pytorch, +) # noqa: F401 diff --git a/src/deterministic_ml/v1/__init__.py b/src/deterministic_ml/v1/__init__.py index e366e76..4e5ad02 100644 --- a/src/deterministic_ml/v1/__init__.py +++ b/src/deterministic_ml/v1/__init__.py @@ -1,3 +1,5 @@ """ Public interface of the deterministic_ml package. """ + +from deterministic_ml._v2 import * # noqa: F401, F403 diff --git a/tests/integration/.gitignore b/tests/integration/.gitignore new file mode 100644 index 0000000..8b22acb --- /dev/null +++ b/tests/integration/.gitignore @@ -0,0 +1 @@ +/results/ \ No newline at end of file diff --git a/tests/integration/.rsyncignore b/tests/integration/.rsyncignore new file mode 100644 index 0000000..d9603cb --- /dev/null +++ b/tests/integration/.rsyncignore @@ -0,0 +1,16 @@ +*.pyc +*.sqlite3 +*~ +*.egg-info/ +/.idea/ +.env +.venv +venv +media/ +.backups/ +.envrc +.pdm-python +.terraform.lock.hcl +.terraform/ +.nox/ +__pycache__ diff --git a/tests/integration/README.md b/tests/integration/README.md new file mode 100644 index 0000000..c9cd453 --- /dev/null +++ b/tests/integration/README.md @@ -0,0 +1,7 @@ +See [Manual Integration Testing](../../docs/MANUAL_INTEGRATION_TESTING.md) for info how to use this. + + + +## How it works + +Each experiment produces \ No newline at end of file diff --git a/tests/integration/__init__.py b/tests/integration/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/analyze.py b/tests/integration/analyze.py new file mode 100755 index 0000000..4627355 --- /dev/null +++ b/tests/integration/analyze.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +import argparse +import collections +import hashlib +import logging + +import tools.exp +import yaml +from rich import print + +LOG = logging.getLogger(__name__) + + +def hash_output(output: dict) -> str: + hasher = hashlib.blake2b() + json_str = yaml.dump(output, sort_keys=True) + hasher.update(json_str.encode("utf8")) + return hasher.hexdigest() + + +if __name__ == "__main__": + parser = argparse.ArgumentParser() + parser.add_argument("experiment", help="Experiment name") + args = parser.parse_args() + + experiment_name = args.experiment.split("/")[-1] + + runs_outputs = sorted( + tools.exp.get_experiment_runs(experiment_name), + key=lambda run: run.run_id, + ) + print(f"Found {len(runs_outputs)} runs with output.yaml for experiment {experiment_name!r}") + + runs_by_hash = collections.defaultdict(list) + for run_output in runs_outputs: + output_hash = hash_output(run_output.output) + runs_by_hash[output_hash].append(run_output) + + print(f"Found {len(runs_by_hash)} unique outputs for {len(runs_outputs)} runs") + for group_no, (output_hash, runs) in enumerate(runs_by_hash.items()): + print(f"[bold]Group {group_no}[/bold] hash={output_hash[:6]} runs={len(runs)}") + for run in runs: + print(f" - Run: {run.run_id} ({run.output_dir_path})") + print(f" - {run.sysinfo['cuda']}") + print(f" - {run.sysinfo['machine']['gpu']}") diff --git a/tests/integration/argparse b/tests/integration/argparse new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/experiments/__init__.py b/tests/integration/experiments/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/README.md b/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/README.md new file mode 100644 index 0000000..beca4ad --- /dev/null +++ b/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/README.md @@ -0,0 +1,3 @@ +Requirements: +* 60GB HDD +* 80GB GPU vRAM diff --git a/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/__init__.py b/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/__main__.py b/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/__main__.py new file mode 100644 index 0000000..77496cb --- /dev/null +++ b/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/__main__.py @@ -0,0 +1,107 @@ +import argparse +import contextlib +import io +import pathlib +import sys +import time +from pprint import pprint + +import torch +import vllm +import yaml +from deterministic_ml.v1 import set_deterministic +from vllm import SamplingParams + +SEED = 42 + +set_deterministic(SEED) + + +@contextlib.contextmanager +def timed(name): + print(f"Starting {name}") + start = time.time() + yield + took = time.time() - start + print(f"{name} took {took:.2f} seconds") + + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument("output_path", type=pathlib.Path, help="Path to save the output") + parser.add_argument("--model", default="casperhansen/llama-3-70b-instruct-awq", help="Model name") + args = parser.parse_args() + + gpu_count = torch.cuda.device_count() + print(f"{gpu_count=}") + + model_name = args.model + + with timed("model loading"): + model = vllm.LLM( + model=model_name, + # quantization="AWQ", + tensor_parallel_size=gpu_count, + # quantization="AWQ", # Ensure quantization is set if needed + # tensor_parallel_size=1, # Set according to the number of GPUs available + enforce_eager=True, # Ensure eager mode is enabled + ) + model.llm_engine.tokenizer.eos_token_id = 128009 + + def make_prompt(prompt): + role_templates = { + "system": "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n{{{{ {} }}}}<|eot_id|>", + "user": "<|start_header_id|>user<|end_header_id|>\n{{{{ {} }}}}<|eot_id|>", + "assistant": "<|start_header_id|>assistant<|end_header_id|>\n{{{{ {} }}}}<|eot_id|>", + "end": "<|start_header_id|>assistant<|end_header_id|>", + } + msgs = [ + {"role": "system", "content": "You are a helpful AI assistant"}, + {"role": "user", "content": prompt}, + ] + full_prompt = io.StringIO() + for msg in msgs: + full_prompt.write(role_templates[msg["role"]].format(msg["content"])) + full_prompt.write(role_templates["end"]) + return full_prompt.getvalue() + + sampling_params = SamplingParams( + max_tokens=4096, + temperature=1000, + top_p=0.1, + seed=SEED, + ) + + def generate_responses(prompts: list[str]): + requests = [make_prompt(prompt) for prompt in prompts] + response = model.generate(requests, sampling_params, use_tqdm=True) + return response + + import hashlib + + output_hashes = {} + prompts = [ + "Count to 1000, skip unpopular numbers", + "Describe justice system in UK vs USA in 2000-5000 words", + "Describe schooling system in UK vs USA in 2000-5000 words", + "Explain me some random problem for me in 2000-5000 words", + "Tell me entire history of USA", + "Write a ballad. Pick a random theme.", + "Write an epic story about a dragon and a knight", + "Write an essay about being a Senior developer.", + ] + + with timed(f"{len(prompts)} responses generation"): + for prompt, r in zip(prompts, generate_responses(prompts)): + hasher = hashlib.blake2b() + hasher.update(r.outputs[0].text.encode("utf8")) + output_hashes[prompt] = hasher.hexdigest() + sys.stderr.flush() + + pprint(output_hashes) + with open(args.output_path / "output.yaml", "w") as f: + yaml.safe_dump(output_hashes, f) + + +if __name__ == "__main__": + main() diff --git a/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/requirements.txt b/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/requirements.txt new file mode 100644 index 0000000..2158a90 --- /dev/null +++ b/tests/integration/experiments/vllm_llama_3_70b_instruct_awq/requirements.txt @@ -0,0 +1,4 @@ +setuptools +torch +pyyaml +vllm diff --git a/tests/integration/run.py b/tests/integration/run.py new file mode 100755 index 0000000..c91ec70 --- /dev/null +++ b/tests/integration/run.py @@ -0,0 +1,187 @@ +#!/usr/bin/env python3 +import argparse +import contextlib +import logging +import pathlib +import re +import subprocess +import tempfile +from datetime import datetime + +import tools.exp +import tools.ssh +import yaml +from rich.logging import RichHandler + +LOG = logging.getLogger(__name__) + +_THIS_DIR = pathlib.Path(__file__).parent +TOOLS_DIR = _THIS_DIR / "tools" +RSYNC_IGNORE_FILEPATH = _THIS_DIR / ".rsyncignore" + + +def slugify(value: str) -> str: + value = re.sub(r"[^\w\s-]", "", value.lower()) + return re.sub(r"[-\s]+", "-", value).strip("-_") + + +@contextlib.contextmanager +def build_deterministic_ml(): + with tempfile.TemporaryDirectory() as tmpdir: + subprocess.run( + ["python", "-m", "build", "--wheel", ".", "-o", tmpdir], + check=True, + cwd=_THIS_DIR.parent.parent, + ) + wheel = next(pathlib.Path(tmpdir).glob("deterministic_ml-*.whl")) + yield wheel + + +def prepare_env(remote_run_command, run_dir, experiment_dir_name): + remote_run_command( + f""" + set -exo pipefail + + curl -LsSf https://astral.sh/uv/install.sh | sh + export PATH=$HOME/.cargo/bin:$PATH + + cd {run_dir} + uv venv -p python3.11 --python-preference managed + source .venv/bin/activate + uv pip install \ + ./deterministic_ml*.whl \ + pyyaml \ + -r {experiment_dir_name}/requirements.txt + """ + ) + + +def main(): + parser = argparse.ArgumentParser(description="deterministic-ml experiment runner") + parser.add_argument("experiment", help="Experiment name") + parser.add_argument("-p", "--port", type=int, default=22, help="SSH port") + parser.add_argument("remote", help="user@host") + parser.add_argument("-c", "--comment", default="", help="Optional comment") + parser.add_argument( + "--remote-storage-path", + default="~/experiments/", + help="Remote storage path prefix for experiments' venv and other files", + ) + args = parser.parse_args() + + slug = args.comment.lower().replace(" ", "_") + timestamp = datetime.now().strftime("%Y-%m-%d_%H-%M-%S") + + local_experiment_dir = tools.exp.get_experiment_dir(args.experiment) + exp_run_id = f"{args.experiment}/{timestamp}_{slug}" + remote_experiment_dir = pathlib.Path(args.remote_storage_path) / exp_run_id + + remote_output_dir = remote_experiment_dir / "output" + local_output_dir = tools.exp.RESULTS_DIR_PATH / exp_run_id + local_output_dir.parent.mkdir(parents=True, exist_ok=True) + local_output_dir.mkdir() + + with (local_output_dir / "experiment.yaml").open("w") as f: + yaml.dump( + { + "experiment": args.experiment, + "experiment_hash": tools.exp.get_experiment_hash(args.experiment), + "comment": args.comment, + "timestamp": timestamp, + "slug": slug, + "run_id": exp_run_id, + }, + f, + ) + + # configure local logging to print to file and stderr + log_file = local_output_dir / "run.local.log" + logging.basicConfig( + level=logging.INFO, + format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", + handlers=[ + logging.FileHandler(log_file), + RichHandler(), + ], + ) + LOG.info("Starting experiment %s with comment: %s", args.experiment, args.comment) + LOG.info("Local log file: %s", log_file) + + port = args.port + if "@" in args.remote: + username, hostname = args.remote.split("@") + else: + username = None + hostname = args.remote + + ssh_client = tools.ssh.ssh_connect(username=username, hostname=hostname, port=port) + + def remote_exec_command(command, **kwargs): + return tools.ssh.remote_exec_command(ssh_client, command, **kwargs) + + def rsync_local_to_remote(local_path, remote_path, **kwargs): + return tools.ssh.rsync_local_to_remote( + local_path, + remote_path, + args.remote, + port, + exclude_from=RSYNC_IGNORE_FILEPATH, + **kwargs, + ) + + LOG.info("Syncing files to remote") + remote_exec_command(f"mkdir -p {remote_output_dir}") + rsync_local_to_remote(TOOLS_DIR, remote_experiment_dir) + rsync_local_to_remote(local_experiment_dir, remote_experiment_dir) + with build_deterministic_ml() as wheel: + rsync_local_to_remote(wheel, remote_experiment_dir) + + LOG.info("Setting up remote environment") + prepare_env(remote_exec_command, remote_experiment_dir, local_experiment_dir.name) + + LOG.info("Gathering system info") + in_env_cmd = f""" + set -exo pipefail + + cd {remote_experiment_dir} + export PATH=$HOME/.cargo/bin:$PATH + source .venv/bin/activate; + """ + + remote_exec_command( + f"{in_env_cmd} python -m deterministic_ml._internal.sysinfo > {remote_output_dir / 'sysinfo.yaml'}" + ) + + LOG.info("Running experiment code on remote") + + try: + remote_exec_command( + f"{in_env_cmd} python -m {local_experiment_dir.name} {remote_output_dir}" + f" | tee {remote_output_dir / 'stdout.txt'}" + ) + finally: + LOG.info("Syncing output back to local") + + tools.ssh.rsync_remote_to_local( + remote_output_dir, + local_output_dir, + args.remote, + port, + exclude_from=RSYNC_IGNORE_FILEPATH, + ) + + LOG.info("Done") + + +if __name__ == "__main__": + try: + main() + except subprocess.CalledProcessError as e: + LOG.error( + "Command failed: %r stdout: %r stderr: %r status: %r", + e.cmd, + e.stdout, + e.stderr, + e.returncode, + ) + raise diff --git a/tests/integration/tools/__init__.py b/tests/integration/tools/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/integration/tools/exp.py b/tests/integration/tools/exp.py new file mode 100644 index 0000000..5d5149a --- /dev/null +++ b/tests/integration/tools/exp.py @@ -0,0 +1,59 @@ +import dataclasses +import hashlib +import logging +import pathlib +from collections.abc import Iterable + +import yaml + +LOG = logging.getLogger(__name__) + +_TEST_TOP_PATH = pathlib.Path(__file__).parent.parent + +EXP_DIR_PATH = _TEST_TOP_PATH / "experiments" +RESULTS_DIR_PATH = _TEST_TOP_PATH / "results" + + +def get_experiment_dir(experiment_name: str) -> pathlib.Path: + exp_dir = EXP_DIR_PATH / experiment_name + if exp_dir.exists(): + return exp_dir + else: + raise FileNotFoundError(f"Experiment directory '{experiment_name}' not found in {EXP_DIR_PATH.absolute()!r}") + + +def get_experiment_hash(experiment_name: str) -> str: + exp_dir = get_experiment_dir(experiment_name) + hasher = hashlib.blake2b() + for file_path in sorted(exp_dir.glob("*.*")): + extension = file_path.suffix.lower() + if extension in {".pyc", ".pyo", ".md"}: + continue + with file_path.open("rb") as f: + hasher.update(f.read()) + return f"exp_hash_v1:{hasher.hexdigest()[:6]}" + + +@dataclasses.dataclass +class ExperimentRun: + run_id: str + output: dict + sysinfo: dict + output_dir_path: pathlib.Path + + +def get_experiment_runs(experiment_name: str) -> Iterable[ExperimentRun]: + runs_outputs = (RESULTS_DIR_PATH / experiment_name).glob("*/output.yaml") + + for run_output in runs_outputs: + run_id = run_output.parent.name + with run_output.open() as f: + output_data = yaml.safe_load(f) + with (run_output.parent / "sysinfo.yaml").open() as f: + sysinfo_data = yaml.safe_load(f) + yield ExperimentRun( + run_id=run_id, + output=output_data, + sysinfo=sysinfo_data, + output_dir_path=run_output.parent, + ) diff --git a/tests/integration/tools/ssh.py b/tests/integration/tools/ssh.py new file mode 100644 index 0000000..64b67b6 --- /dev/null +++ b/tests/integration/tools/ssh.py @@ -0,0 +1,87 @@ +import logging +import pathlib +import subprocess + +import paramiko + +LOG = logging.getLogger(__name__) + + +def rsync_local_to_remote( + local_path: pathlib.Path, + remote_path: pathlib.Path, + remote: str, + port: int = 22, + *, + exclude_from: pathlib.Path, + dir_content_only: bool = False, +): + local_path_str = str(local_path) if not dir_content_only else f"{local_path}/" + rsync_cmd = [ + "rsync", + "-av", + "-e", + f"ssh -p {port} -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no", + local_path_str, + f"{remote}:{remote_path}/", + ] + if exclude_from: + rsync_cmd.append(f"--exclude-from={exclude_from.absolute()}") + return subprocess.run(rsync_cmd, check=True, capture_output=True) + + +def rsync_remote_to_local( + remote_path: pathlib.Path, + local_path: pathlib.Path, + remote: str, + port: int = 22, + *, + exclude_from: pathlib.Path, +): + local_path.mkdir(parents=True, exist_ok=True) + + rsync_cmd = [ + "rsync", + "-av", + "-e", + f"ssh -p {port} -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no", + f"{remote}:{remote_path}/", + f"{local_path}/", + ] + if exclude_from: + rsync_cmd.append(f"--exclude-from={exclude_from.absolute()}") + return subprocess.run(rsync_cmd, check=True, capture_output=True) + + +class SilentMissingHostKeyPolicy(paramiko.MissingHostKeyPolicy): + def missing_host_key(self, client, hostname: str, key) -> None: + pass + + +def ssh_connect(hostname: str, port: int = 22, username: str | None = None) -> paramiko.SSHClient: + client = paramiko.SSHClient() + client.set_missing_host_key_policy(SilentMissingHostKeyPolicy()) + + client.connect( + hostname=hostname, + username=username, + port=port, + look_for_keys=True, + allow_agent=True, + ) + return client + + +def remote_exec_command(ssh_client: paramiko.SSHClient, command: str, **kwargs): + stdin, stdout, stderr = ssh_client.exec_command(command, **kwargs) + status_code = stdout.channel.recv_exit_status() + log_func = LOG.info if status_code == 0 else LOG.error + log_func( + "Command: %r stdout: %r stderr: %r status_code: %r", + command, + stdout.read().decode(errors="backslashreplace"), + stderr.read().decode(errors="backslashreplace"), + status_code, + ) + if status_code != 0: + raise subprocess.CalledProcessError(status_code, command, stdout.read(), stderr.read()) diff --git a/tests/unit/api/test_setup.py b/tests/unit/api/test_setup.py index 4343b66..09cb8f4 100644 --- a/tests/unit/api/test_setup.py +++ b/tests/unit/api/test_setup.py @@ -1,2 +1,5 @@ def test_apiver_exports(apiver_module): - assert sorted(name for name in dir(apiver_module) if not name.startswith("_")) == [] + assert sorted(name for name in dir(apiver_module) if not name.startswith("_")) == [ + "set_deterministic", + "set_deterministic_pytorch", + ] diff --git a/tests/unit/internal/test_deterministic.py b/tests/unit/internal/test_deterministic.py new file mode 100644 index 0000000..63caf8c --- /dev/null +++ b/tests/unit/internal/test_deterministic.py @@ -0,0 +1,32 @@ +import random + +import numpy +import pytest +import torch +from deterministic_ml._internal.deterministic import set_deterministic + + +@pytest.mark.parametrize( + "seed, expected_sequence", + [ + (0, [(864, 394), (684, 559), (44, 239)]), + (42, [(654, 114), (102, 435), (542, 67)]), + ], +) +def test_set_deterministic__random_sequence(seed, expected_sequence): + set_deterministic(seed) + + int_range = 0, 1000 + + sequence = [ + ( + random.randint(*int_range), + random.randint(*int_range), + ), + ( + numpy.random.randint(*int_range), + numpy.random.randint(*int_range), + ), + tuple(torch.randint(*int_range, (2,), dtype=torch.int).tolist()), + ] + assert sequence == expected_sequence diff --git a/tests/unit/internal/test_sysinfo.py b/tests/unit/internal/test_sysinfo.py new file mode 100644 index 0000000..26da95d --- /dev/null +++ b/tests/unit/internal/test_sysinfo.py @@ -0,0 +1,27 @@ +from deterministic_ml._internal.sysinfo import get_machine_specs, get_specs + + +def test_get_machine_specs(): + machine_specs = get_machine_specs() + assert machine_specs.keys() == { + "docker_support", + "gpu", + "gpu_scrape_error", + "cpu", + "ram", + "hard_disk", + "os", + } + assert machine_specs["cpu"]["count"] > 0 + assert machine_specs["ram"]["total"] > 0 + assert machine_specs["os"] + + +def test_get_specs(): + specs = get_specs() + assert specs.keys() == { + "cuda", + "machine", + "python", + "system", + }