Remove extraneous assistant message headers (#711)

* Remove extraneous assistant message headers

* Update CHANGELOG.md

---------

Co-authored-by: jjallaire-aisi <joseph.allaire@dsit.gov.uk>

CHANGELOG.md

@@ -5,6 +5,7 @@
 - Cleanup Docker Containers immediately for samples with errors.
 - Anthropic: remove stock tool use chain of thought prompt (many Anthropic models now do this internally, in other cases its better for this to be explicit rather than implicit).
 - Google: compatibility with google-generativeai v0.8.3
+- Llama: remove extraneous <|start_header_id|>assistant<|end_header_id|> if it appears in an assistant message.
 - Requirements: require semver>=3.0.0
 - Open log files in binary mode when reading headers (fixes ijson deprecation warning).
 - Bugfix: strip protocol prefix when resolving eval event content

src/inspect_ai/model/_providers/util/llama31.py

@@ -104,12 +104,16 @@ def parse_assistant_response(
 
             # return the message
             return ChatMessageAssistant(
-                content=content, tool_calls=tool_calls, source="generate"
+                content=filter_assistant_header(content),
+                tool_calls=tool_calls,
+                source="generate",
             )
 
         # otherwise this is just an ordinary assistant message
         else:
-            return ChatMessageAssistant(content=response, source="generate")
+            return ChatMessageAssistant(
+                content=filter_assistant_header(response), source="generate"
+            )
 
     @override
     def assistant_message(self, message: ChatMessageAssistant) -> ChatAPIMessage:
@@ -183,3 +187,7 @@ def parse_tool_call_content(content: str, tools: list[ToolInfo]) -> ToolCall:
             type="function",
             parse_error=parse_error,
         )
+
+
+def filter_assistant_header(message: str) -> str:
+    return re.sub(r"<\|start_header_id\|>assistant<\|end_header_id\|>", "", message)