From 2be74543cba56becdd2d45d496524f2da236c2db Mon Sep 17 00:00:00 2001 From: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com> Date: Wed, 10 Apr 2024 10:05:55 -0400 Subject: [PATCH 1/5] Update README.md Updated Slack link --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 276323dd7b..380100b3dd 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ limitations under the License. Documentation - + Slack @@ -213,7 +213,7 @@ Confirm that telemetry is shut off through info logs streamed with engine invoca ### Get In Touch - [Contribution Guide](https://github.com/neuralmagic/deepsparse/blob/main/CONTRIBUTING.md) -- [Community Slack](https://join.slack.com/t/discuss-neuralmagic/shared_invite/zt-q1a1cnvo-YBoICSIw3L1dmQpjBeDurQ) +- [Community Slack](https://neuralmagic.com/community/) - [GitHub Issue Queue](https://github.com/neuralmagic/deepsparse/issues) - [Subscribe To Our Newsletter](https://neuralmagic.com/subscribe/) - [Blog](https://www.neuralmagic.com/blog/) From bcb6472b7b04ada46fb096b76e7ea9ca60a4467b Mon Sep 17 00:00:00 2001 From: Jeannie Finks <74554921+jeanniefinks@users.noreply.github.com> Date: Wed, 10 Apr 2024 13:05:23 -0400 Subject: [PATCH 2/5] Update CONTRIBUTING.md Fixing expired Slack link --- CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1acf0b1c09..a1b3e4ccc6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -77,7 +77,7 @@ For documentation edits, include: ## Question or Problem -- Sign up or log in to our [**Neural Magic Community Slack**](https://join.slack.com/t/discuss-neuralmagic/shared_invite/zt-q1a1cnvo-YBoICSIw3L1dmQpjBeDurQ). We are growing the community member by member and happy to see you there. Post all other questions including support or how to contribute. Don’t forget to search through existing discussions to avoid duplication! Thanks! +- Sign up or log in to our [**Neural Magic Community Slack**](https://neuralmagic.com/community/). We are growing the community member by member and happy to see you there. Post all other questions including support or how to contribute. Don’t forget to search through existing discussions to avoid duplication! Thanks! Post all other questions including support or how to contribute. Don’t forget to search through existing discussions to avoid duplication! Thanks! From c87e2f9d71461f006d09e1cf40aca97a9fea4f5e Mon Sep 17 00:00:00 2001 From: Rob Greenberg <100797996+rgreenberg1@users.noreply.github.com> Date: Thu, 11 Apr 2024 07:37:39 -0400 Subject: [PATCH 3/5] Updated Logo in README (#1642) --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 380100b3dd..59699a945a 100644 --- a/README.md +++ b/README.md @@ -16,9 +16,9 @@ limitations under the License.
-

- tool icon -   DeepSparse +

+ tool icon +   DeepSparse

Sparsity-aware deep learning inference runtime for CPUs

From cfe1baaa24e6154b9c8162f6bbce71da91c02079 Mon Sep 17 00:00:00 2001 From: Michael Goin Date: Fri, 12 Apr 2024 12:18:57 -0600 Subject: [PATCH 4/5] Update README.md --- src/deepsparse/transformers/README.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/deepsparse/transformers/README.md b/src/deepsparse/transformers/README.md index f1e38aee85..9630086df9 100644 --- a/src/deepsparse/transformers/README.md +++ b/src/deepsparse/transformers/README.md @@ -151,9 +151,12 @@ https://sparsezoo.neuralmagic.com/?useCase=text_generation) ```python from deepsparse import Pipeline -opt_pipeline = Pipeline.create(task="opt", model_path="zoo:opt-1.3b-opt_pretrain-quantW8A8") +llama_pipeline = Pipeline.create( + task="text-generation", + model_path="zoo:llama2-7b-ultrachat200k_llama2_pretrain-pruned50_quantized" +) -inference = opt_pipeline("Who is the president of the United States?") +inference = llama_pipeline("Who is the president of the United States?") >> 'The president of the United States is the head of the executive branch of government...' ``` @@ -163,7 +166,7 @@ Spinning up: ```bash deepsparse.server \ --task text-generation \ - --model_path zoo:opt-1.3b-opt_pretrain-pruned50_quantW8A8 + --model_path zoo:llama2-7b-ultrachat200k_llama2_pretrain-pruned50_quantized ``` Making a request: From 1ff44cddd797625ae9da44b2af0fadb7a7ace269 Mon Sep 17 00:00:00 2001 From: Michael Goin Date: Tue, 16 Apr 2024 14:33:58 -0400 Subject: [PATCH 5/5] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 59699a945a..e19ef6c39d 100644 --- a/README.md +++ b/README.md @@ -74,7 +74,7 @@ print(pipeline(prompt, max_new_tokens=75).generations[0].text) # Sparsity is the property of a matrix or other data structure in which a large number of elements are zero and a smaller number of elements are non-zero. In the context of machine learning, sparsity can be used to improve the efficiency of training and prediction. ``` -> [Check out the `TextGeneration` documentation for usage details.](https://github.com/neuralmagic/deepsparse/blob/main/docs/llms/text-generation-pipeline.md) +Check out the [`TextGeneration` documentation for usage details](https://github.com/neuralmagic/deepsparse/blob/main/docs/llms/text-generation-pipeline.md) and get the [latest sparsified LLMs on our HF Collection](https://huggingface.co/collections/neuralmagic/deepsparse-sparse-llms-659d61e81774dd48343642bf). ### Sparsity :handshake: Performance