Skip to content

Commit

Permalink
Notebook fixes for prompt completion (#16420)
Browse files Browse the repository at this point in the history
  • Loading branch information
raspawar authored Oct 8, 2024
1 parent 9c25736 commit 33402ae
Showing 1 changed file with 26 additions and 2 deletions.
28 changes: 26 additions & 2 deletions docs/docs/examples/llm/nvidia_text_completion.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Working with NVIDIA API Catalog"
"## Working with NVIDIA API Catalog\n",
"#### Usage of `use_chat_completions` argument: \n",
"Set None (default) to per-invocation decide on using /chat/completions vs /completions endpoints with query keyword arguments\n",
"\n",
"- set False to universally use /completions endpoint\n",
"- set True to universally use /chat/completions endpoint"
]
},
{
Expand All @@ -113,13 +118,22 @@
"llm = NVIDIA(model=\"bigcode/starcoder2-15b\", use_chat_completions=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Available Models\n",
"\n",
"`is_chat_model` can be used to get available text completion models"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"llm.available_models"
"print([model for model in llm.available_models if model.is_chat_model])"
]
},
{
Expand Down Expand Up @@ -206,6 +220,16 @@
"x = llm.stream_complete(prompt=\"# Reverse string in python:\", max_tokens=512)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for t in x:\n",
" print(t.delta, end=\"\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down

0 comments on commit 33402ae

Please sign in to comment.