feat: Support VertexAI for Claude #1138
Conversation
|
very cool thanks @yukukotani - Vertex is to google what bedrock is to aws! |
| async fn post(&self, payload: Value) -> Result<Value, ProviderError> { | ||
| let url = format!( | ||
| "https://{}/v1/projects/{}/locations/{}/publishers/anthropic/models/{}:streamRawPredict", | ||
| self.host, self.project_id, self.region, self.model.model_name |
There was a problem hiding this comment.
can we add some validation to the host? right now if the VERTEXAI_API_HOST has the prefix "https" then we will have the invalid url
|
|
||
| async fn post(&self, payload: Value) -> Result<Value, ProviderError> { | ||
| let url = format!( | ||
| "https://{}/v1/projects/{}/locations/{}/publishers/anthropic/models/{}:streamRawPredict", |
There was a problem hiding this comment.
is it possible to make it as a variable so we can change it to others, like mistral and lllam, both of these models are also supported in vertex: https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/mistral
| @@ -82,6 +86,7 @@ export const supported_providers = [ | |||
| 'Ollama', | |||
| 'OpenRouter', | |||
| 'Azure OpenAI', | |||
| 'Vertex AI', | |||
There was a problem hiding this comment.
the ui codebase requires more changes as right now we have a new provider and we need to ask user to input the vertex ai project and region. https://github.com/block/goose/tree/main/ui/desktop/src/components/settings/providers
yukukotani
force-pushed
the
vertex-ai
branch
2 times, most recently
from
d1d94d6 to
d2dc136
Compare
|
@yingjiehe-xyz Thank you for reviewing! I've added some fixes. |
| messages: &[Message], | ||
| tools: &[Tool], | ||
| ) -> Result<Value> { | ||
| let mut request = anthropic::create_request(model_config, system, messages, tools)?; |
There was a problem hiding this comment.
let's add one more check whether it is anthropic model? By checking the model_config name, we can let the user know right now, only anthropic model is support via vertex ai
| match status { | ||
| reqwest::StatusCode::OK => Ok(response_json), | ||
| reqwest::StatusCode::UNAUTHORIZED | reqwest::StatusCode::FORBIDDEN => { | ||
| Err(ProviderError::Authentication(format!( |
There was a problem hiding this comment.
can we add some debug info like https://github.com/block/goose/blob/main/crates/goose/src/providers/anthropic.rs#L108?
yukukotani
force-pushed
the
vertex-ai
branch
from
fb748d2 to
93b2678
Compare
|
ready for getting merged I believe |
Add support for Vertex AI.
I've added only Claude sonnet 3.5 in this PR but we can expand it to Gemini or other models later.