🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.
-
Updated
May 14, 2025 - Python
#
text-generation-inference
Here are 11 public repositories matching this topic...
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
kubernetes inference huggingface llm modelscope llamacpp vllm text-generation-inference ollama sglang inference-platform
-
Updated
May 19, 2025 - Go
Amazon SageMaker Llama 2 Inference via Response Streaming
sagemaker sagemaker-endpoint response-streaming large-language-models text-generation-inference llama2 large-model-inference
-
Updated
Jun 28, 2024 - Jupyter Notebook
This repo introduces MagicData-CLAM, a Chinese SFT dataset, and provides to the community two relevant models that we finetuned. Contact business@magicdatatech.com for more information.
-
Updated
Aug 3, 2023 - Python
GUI version of text-generation-inference
-
Updated
Sep 1, 2023 - Python
This project demonstrates the process of fine-tuning the Qwen2.5-3B-Instruct model using GRPO (Generalized Reward Policy Optimization) on the GSM8K dataset.
python transformer adapters huggingface trl safetensors text-generation-inference unsloth qwen2-5 grpo
-
Updated
Apr 7, 2025 - Jupyter Notebook
RisuAI backend with python only. TextGen works, need more memory related updates
-
Updated
Jun 12, 2024 - Python
Deploy the Defog sqlcoder2 llm on Modal (https://modal.com) using Hugging Face Text Generation Inference (TGI)
sql code-generation text-to-sql huggingface llm text-generation-inference sqlcoder modal-labs defog-ai
-
Updated
Dec 12, 2023 - Python
Serve the AI Singapore SEA-LION model ⚛ with TGI
-
Updated
Apr 15, 2025 - Shell
This project demonstrates the process of fine-tuning the Qwen2.5-3B-Instruct model using GRPO (Generalized Reward Policy Optimization) on the GSM8K dataset.
python build openwrt transformer adapters nanopi huggingface trl nanopi-r1s nanopi-r1 safetensors text-generation-inference unsloth grpo
-
Updated
May 19, 2025 - Jupyter Notebook
Text Generation Interference example in Windows (docker, WSL is needed)
-
Updated
Jun 26, 2023 - Python
Improve this page
Add a description, image, and links to the text-generation-inference topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the text-generation-inference topic, visit your repo's landing page and select "manage topics."