Skip to content

Commit

Permalink
anthropic: ensure that we never send empty text content to the API (#718
Browse files Browse the repository at this point in the history
)

* anthropic: ensure that we never send empty text content to the API

* Update src/inspect_ai/model/_providers/anthropic.py

Co-authored-by: sdtblckgov <168540866+sdtblckgov@users.noreply.github.com>

---------

Co-authored-by: aisi-inspect <166920645+aisi-inspect@users.noreply.github.com>
Co-authored-by: sdtblckgov <168540866+sdtblckgov@users.noreply.github.com>
  • Loading branch information
3 people authored Oct 17, 2024
1 parent c2c9190 commit 2666c8b
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 3 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

- Cleanup Docker Containers immediately for samples with errors.
- Anthropic: remove stock tool use chain of thought prompt (many Anthropic models now do this internally, in other cases its better for this to be explicit rather than implicit).
- Anthropic: ensure that we never send empty text content to the API.
- Google: compatibility with google-generativeai v0.8.3
- Llama: remove extraneous <|start_header_id|>assistant<|end_header_id|> if it appears in an assistant message.
- Requirements: require semver>=3.0.0
Expand Down
11 changes: 8 additions & 3 deletions src/inspect_ai/model/_providers/anthropic.py
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,11 @@ def message_tool_choice(tool_choice: ToolChoice) -> message_create_params.ToolCh
return {"type": "auto"}


# text we insert when there is no content passed
# (as this will result in an Anthropic API error)
NO_CONTENT = "(no content)"


async def message_param(message: ChatMessage) -> MessageParam:
# no system role for anthropic (this is more like an assertion,
# as these should have already been filtered out)
Expand All @@ -412,7 +417,7 @@ async def message_param(message: ChatMessage) -> MessageParam:
if not content:
content = "error"
elif isinstance(message.content, str):
content = [TextBlockParam(type="text", text=message.content)]
content = [TextBlockParam(type="text", text=message.content or NO_CONTENT)]
else:
content = [
await message_param_content(content) for content in message.content
Expand Down Expand Up @@ -465,7 +470,7 @@ async def message_param(message: ChatMessage) -> MessageParam:

# normal text content
elif isinstance(message.content, str):
return MessageParam(role=message.role, content=message.content)
return MessageParam(role=message.role, content=message.content or NO_CONTENT)

# mixed text/images
else:
Expand Down Expand Up @@ -561,7 +566,7 @@ async def message_param_content(
content: Content,
) -> TextBlockParam | ImageBlockParam:
if isinstance(content, ContentText):
return TextBlockParam(type="text", text=content.text)
return TextBlockParam(type="text", text=content.text or NO_CONTENT)
else:
# resolve to url
image = content.image
Expand Down

0 comments on commit 2666c8b

Please sign in to comment.