From dbacff3672e5ec8b6c2de9b76ff3690c4955ede7 Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 14 Aug 2024 09:12:55 -0400 Subject: [PATCH 1/5] fix typos and remove private URL's --- README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 0fd3f00..1d2677e 100644 --- a/README.md +++ b/README.md @@ -2,13 +2,18 @@ ## Overview -The `nm-vllm` packages published in this repository are Neuralmagic Enterprise Edition of [vLLM](https://github.com/vllm-project/vllm). Packages are versioned Python wheels and docker images. These are released as "production level" official releases and "beta level" nightly releases. +The `nm-vllm` packages published in this repository are Neural Magic Enterprise Editions of [vLLM](https://github.com/vllm-project/vllm). Packages are versioned Python wheels and docker images. These are released as "production level" official releases and "beta level" nightly releases. -Official releases are made at the discretion of Neuralmagic, but typically track with `vllm` releases. These wheels are available via the official PyPI as well as [Neuralmagic's PyPI](https://pypi.neuralmagic.com). +Official releases are made at the discretion of Neural Magic, but typically track with `vllm` releases. These wheels are available via the official PyPI as well as [Neuralmagic's PyPI](https://pypi.neuralmagic.com). Nightly builds are released every night given green runs in automation. The wheels are available at [Neuralmagic's PyPI](https://pypi.neuralmagic.com). +## Benchmarks + +Please see how we are doing with our benchmark results [here]( https://neuralmagic.github.io/nm-vllm-certs/dev/bench/). + + ## Installation @@ -28,21 +33,16 @@ pip install nm-vllm[sparse] --extra-index-url https://pypi.neuralmagic.com/simpl ### Docker -The `nm-vllm-ent` [container registry](https://github.com/neuralmagic/nm-vllm-certs/pkgs/container/nm-vllm-ent) includes premade docker images. +The `nm-vllm-certs` [container registry](https://github.com/neuralmagic/nm-vllm-certs/pkgs/container/nm-vllm-certs) includes premade docker images. Launch the OpenAI-compatible server with: ```bash MODEL_ID=Qwen/Qwen2-0.5B-Instruct -docker run --gpus all --shm-size 2g ghcr.io/neuralmagic/nm-vllm-ent:latest --model $MODEL_ID +docker run --gpus all --shm-size 2g ghcr.io/neuralmagic/nm-vllm-certs:latest --model $MODEL_ID ``` -## Benchmarks - -Please see our benchmark results [here]( https://neuralmagic.github.io/nm-vllm-certs/dev/bench/). - - ## Models Neuralmagic maintains a variety of optimized models on our Hugging Face organization profiles: From adc0a1a15ec0ef3d2ef3cc1e6cd6b68ba1421b7d Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 14 Aug 2024 09:57:51 -0400 Subject: [PATCH 2/5] revert silly changes --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 1d2677e..0658063 100644 --- a/README.md +++ b/README.md @@ -33,13 +33,13 @@ pip install nm-vllm[sparse] --extra-index-url https://pypi.neuralmagic.com/simpl ### Docker -The `nm-vllm-certs` [container registry](https://github.com/neuralmagic/nm-vllm-certs/pkgs/container/nm-vllm-certs) includes premade docker images. +The `nm-vllm-certs` [container registry](https://github.com/neuralmagic/nm-vllm-certs/pkgs/container/nm-vllm-ent) includes premade docker images. Launch the OpenAI-compatible server with: ```bash MODEL_ID=Qwen/Qwen2-0.5B-Instruct -docker run --gpus all --shm-size 2g ghcr.io/neuralmagic/nm-vllm-certs:latest --model $MODEL_ID +docker run --gpus all --shm-size 2g ghcr.io/neuralmagic/nm-vllm-ent:latest --model $MODEL_ID ``` From f3c8bdf2a8f36a04e309c67242f031fe42470b3d Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 14 Aug 2024 09:59:00 -0400 Subject: [PATCH 3/5] missed one --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 0658063..b746348 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ pip install nm-vllm[sparse] --extra-index-url https://pypi.neuralmagic.com/simpl ### Docker -The `nm-vllm-certs` [container registry](https://github.com/neuralmagic/nm-vllm-certs/pkgs/container/nm-vllm-ent) includes premade docker images. +The `nm-vllm-ent` [container registry](https://github.com/neuralmagic/nm-vllm-certs/pkgs/container/nm-vllm-ent) includes premade docker images. Launch the OpenAI-compatible server with: From 6be9586d5a2231458f578258cf3a8e4a22e4fece Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 14 Aug 2024 10:32:37 -0400 Subject: [PATCH 4/5] Neural Magic not Neuralmagic --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index b746348..190ec0a 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,9 @@ ## Overview The `nm-vllm` packages published in this repository are Neural Magic Enterprise Editions of [vLLM](https://github.com/vllm-project/vllm). Packages are versioned Python wheels and docker images. These are released as "production level" official releases and "beta level" nightly releases. -Official releases are made at the discretion of Neural Magic, but typically track with `vllm` releases. These wheels are available via the official PyPI as well as [Neuralmagic's PyPI](https://pypi.neuralmagic.com). +Official releases are made at the discretion of Neural Magic, but typically track with `vllm` releases. These wheels are available via the official PyPI as well as [Neural Magic's PyPI](https://pypi.neuralmagic.com). -Nightly builds are released every night given green runs in automation. The wheels are available at [Neuralmagic's PyPI](https://pypi.neuralmagic.com). +Nightly builds are released every night given green runs in automation. The wheels are available at [Neural Magic's PyPI](https://pypi.neuralmagic.com). ## Benchmarks @@ -45,6 +45,6 @@ docker run --gpus all --shm-size 2g ghcr.io/neuralmagic/nm-vllm-ent:latest --mod ## Models -Neuralmagic maintains a variety of optimized models on our Hugging Face organization profiles: +Neural Magic maintains a variety of optimized models on our Hugging Face organization profiles: - [neuralmagic](https://huggingface.co/neuralmagic) - [nm-testing](https://huggingface.co/nm-testing) From 0895aa5c76df2126697e78bb5308c2a4b28d497b Mon Sep 17 00:00:00 2001 From: andy-neuma Date: Wed, 14 Aug 2024 18:41:54 -0400 Subject: [PATCH 5/5] PR feedback --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 190ec0a..52a5cca 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ ## Overview -The `nm-vllm` packages published in this repository are Neural Magic Enterprise Editions of [vLLM](https://github.com/vllm-project/vllm). Packages are versioned Python wheels and docker images. These are released as "production level" official releases and "beta level" nightly releases. +The `nm-vllm` packages published in this repository are Neural Magic Enterprise Editions of [vLLM](https://github.com/vllm-project/vllm). Packages are versioned Python wheels and Docker images. These are released as "production level" official releases and "beta level" nightly releases. Official releases are made at the discretion of Neural Magic, but typically track with `vllm` releases. These wheels are available via the official PyPI as well as [Neural Magic's PyPI](https://pypi.neuralmagic.com).