Skip to content

[EDU-6432] Edge AI reference #1624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ With Edge Functions for **Edge Application**, you can:
- Deploy functions easily.
- Make use of pair programming with the Chat GPT integration.
- Preview the outcome of the function live on the preview deployment.
- Run AI agents directly on Azion's distributed infrastructure with ultra-low latency.
- Build complete AI ecosystems.

---

Expand All @@ -48,6 +50,21 @@ By using the [Azion Runtime](/en/documentation/runtime/overview/) to develop you

---

## AI Framework Support

Edge Functions provides support for advanced AI workflows, enabling you to:

- Run AI agents directly on Azion's distributed infrastructure with ultra-low latency, building enterprise-grade solutions that operate at the edge with reliability and scale.
- Create sophisticated AI workflows using **LangGraph** and **LangChain** integrations, including advanced retrieval capabilities that combine vector and full-text search for complex use cases.
- Leverage native [Edge SQL](/en/documentation/products/store/edge-sql/) integration for RAG implementations and vector storage, along with MCP Servers that enable agent-to-agent collaboration using Google's Agent2Agent (A2A) protocol.
- Access state-of-the-art models like Mistral, Florence, Qwen, and others, that follow OpenAI's API standard, all while reducing complexity with a fully integrated AI infrastructure running at the edge of the network.

Azion's **Edge AI** allows you to run AI models on Azion Runtime, and can be used together with Edge Functions to create complex AI-powered applications.

<LinkButton link="/en/documentation/products/ai/edge-ai/" label="Learn more about Edge AI" severity="secondary" target="_blank" />

---

## JavaScript frameworks

### Next.js
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
---
title: BAAI/bge-reranker-v2-m3
description: >-
BAAI/bge-reranker-v2-m3 is a lightweight reranker model with strong multilingual capabilities.
meta_tags: 'edge ai, ai models, artificial intelligence, edge computing'
namespace: docs_edge_ai_models_baai_bge_reranker_v2_m3
permalink: /documentation/products/ai/edge-ai/models/baai-bge-reranker-v2-m3/
---

**BAAI/bge-reranker-v2-m3** is a lightweight reranker model with strong multilingual capabilities. It's easy to deploy and offers fast inference.

## Model details

| Category | Details |
|----------|---------|
| **Model Name** | BAAI/bge-reranker-v2-m3 |
| **Version** | Original |
| **Model Category** | Reranker |
| **Size** | 568M parameters |
| **HuggingFace Model** | [BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3) |
| **License** | [Apache 2.0](https://choosealicense.com/licenses/apache-2.0/) |

## Capabilities

| Feature | Details |
|---------|--------|
| Azion Long-term Support (LTS) | ✅ / ❌|
| Context Length | 8192 |
| Supports LoRA | ❌ |
| Input data | Text |

## Usage

### Rerank example

This is an example of a basic rerank request using this model:

```bash
curl -X POST \
http://endpoint-url/rerank \
-H 'Content-Type: application/json' \
-d '{
"model": "BAAI/bge-reranker-v2-m3",
"query": "What is deep learning?",
"documents": [
"Deep learning is a subset of machine learning that uses neural networks with many layers",
"The weather is nice today",
"Deep learning enables computers to learn from large amounts of data",
"I like pizza"
]
}'
```

### Score example

This is an example of a basic score request using this model:

```bash
curl -X POST \
http://endpoint-url/score \
-H 'Content-Type: application/json' \
-d '{
"model": "BAAI/bge-reranker-v2-m3",
"text_1": "What is deep learning?",
"text_2": [
"Deep learning is a subset of machine learning that uses neural networks with many layers",
"The weather is nice today",
"Deep learning enables computers to learn from large amounts of data",
"I like pizza"
]
}'
```

### Running with Edge Functions:

This is an example of how to run this model using Edge Functions:

```ts
const modelResponse = await Azion.AI.run("baai-bge-reranker-v2-m3", {
"query": "What is deep learning?",
"documents": [
"Deep learning is a subset of machine learning that uses neural networks with many layers",
"The weather is nice today",
"Deep learning enables computers to learn from large amounts of data",
"I like pizza"
]
})
```

## JSON schema

```json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"query",
"documents"
],
"properties": {
"query": {
"type": "string"
},
"documents": {
"type": "array",
"items": {
"type": "string"
}
},
"top_n": {
"type": "integer"
},
"max_tokens_per_doc": {
"type": "integer"
}
}
}
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
---
title: E5 Mistral 7B Instruct
description: >-
E5 Mistral 7B Instruct is optimized for English text embedding, multilingual processing, customization, and long input sequences.
meta_tags: 'edge ai, ai models, artificial intelligence, edge computing, mistral'
namespace: docs_edge_ai_models_e5_mistral_7b_instruct
permalink: /documentation/products/ai/edge-ai/models/e5-mistral-7b-instruct/
---

The **E5 Mistral 7B Instruct** model is optimized for English text embedding tasks, with capabilities for multilingual processing, flexible customization, and handling long input sequences, making it suitable for complex natural language processing applications.

## Model details

| Category | Details |
|----------|---------|
| **Model Name** | E5 Mistral 7B Instruct |
| **Version** | Original |
| **Model Category** | Embedding |
| **Size** | 7B parameters |
| **HuggingFace Model** | [e5-mistral-7b-instruct](https://huggingface.co/intfloat/e5-mistral-7b-instruct) |
| **OpenAI Compatible Endpoint**| [Embeddings](https://platform.openai.com/docs/api-reference/embeddings/create) |
| **License** | MIT |

## Capabilities

| Feature | Details |
|---------|--------|
| Azion Long-term Support (LTS) | ❌ |
| Context Length | 32728 |
| Supports LoRA | ✅ |
| Input Data | Text |

## Usage

### Embedding

This is an example of how to use this model to generate embeddings for text input:

```bash
curl http://endpoint-url/v1/embeddings \
-H "Content-Type: application/json" \
-d '{
"input": "The food was delicious and the waiter...",
"model": "intfloat/e5-mistral-7b-instruct",
"encoding_format": "float"
}'
```

### Running with Edge Functions:

This is and example of how to use this model with Edge Functions:

```ts
const modelResponse = await Azion.AI.run("intfloat-e5-mistral-7b-instruct", {
"input": "The food was delicious and the waiter...",
"encoding_format": "float"
})
```

## JSON schema

```json
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"required": [
"input"
],
"properties": {
"encoding_format": {
"type": "string",
"enum": [
"float",
"base64"
]
},
"dimensions": {
"type": "integer"
},
"input": {
"oneOf": [
{
"type": "string"
},
{
"type": "array",
"items": {
"oneOf": [
{
"type": "string"
},
{
"type": "integer"
},
{
"type": "array",
"items": {
"type": "integer"
}
}
]
}
}
]
}
}
}
```
Loading