# CreateAI API Documentation

> Build powerful AI applications with CreateAI

This file contains the full text of the CreateAI API documentation. For a short link index instead, fetch https://docs.aiml.asu.edu/llms-index.txt. The live model catalog is rendered client-side from the CreateAI API and is therefore not included here; see https://docs.aiml.asu.edu/models.md for the Models page.

## Introduction

# CreateAI API Documentation

Welcome to the CreateAI API documentation. This platform provides access to a wide range of AI models across multiple modalities, including text, audio, and vision. Whether you're looking to generate text, analyze images, or process audio, CreateAI has you covered.

## Getting Started

CreateAI offers a comprehensive suite of endpoints for various AI/ML tasks:

- **Query**: Text generation with a variety of models
- **Search**: Semantic search and RAG capabilities
- **Audio**: Speech to text transcription
- **Vision**: Image analysis and understanding
- **Speech**: Text-to-speech synthesis
- **Reranker**: Document reranking for better search results
- **Realtime**: Real-time speech to speech streaming connections
- **Embeddings**: Generate vector embeddings for text

## Quick Example

```python
import requests

url = "https://api-main.aiml.asu.edu/query" # Replace with the appropriate url of your environment (prod, beta, poc)
headers = {
    "Authorization": "Bearer your_project_service_token",
    "Content-Type": "application/json"
}

payload = {
    "query": "What is the capital of France?",
}

response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

## Authentication

All API requests authenticate with a Bearer token. Which token you use depends on what you're building.

| Token | Use it for | Identifies |
|-------|-----------|------------|
| **Service token** | Server-side and service-to-service calls | The project |
| **Project owner token** | Administrative work via the [Manage Project](/endpoints/manage-project) endpoint | The project owner |
| **Project web token** | Calls made on behalf of a signed-in ASU user | The user |

A service token never expires and carries your project's full quota, so it belongs on a server you control. A project web token is issued per user by [ASU single sign-on](/sso-redirect) and lasts 24 hours.

:::warning Don't ship a service token in a user-facing app
Anything that runs in a browser or on a user's device — including vibe-coded apps — exposes its credentials to whoever is using it. Use [ASU Sign-In](/sso-redirect) to get a per-user project web token instead, and keep service tokens server-side.
:::

See [Token Details](/tokens) for the full comparison.

## Put your app in front of ASU users

You don't need to build a login screen. Register a redirect URL on your project and CreateAI generates a login link that verifies the user's ASURITE and drops them on your page with a token identifying them.

1. An admin registers your redirect URL and enables the feature.
2. CreateAI generates a login link for that URL.
3. You share the link. Users sign in with their ASURITE.
4. They land on your page with a `projectWebToken` attached.

Read [ASU Sign-In & Redirect URLs](/sso-redirect) for redirect URL rules, multi-audience setups, and troubleshooting.

## For AI agents and LLMs

This documentation is published in a machine-readable form, so an agent can read it without scraping HTML.

| File | Contents |
|------|----------|
| [`/llms.txt`](https://docs.aiml.asu.edu/llms.txt) | The full text of every documentation page in one file |
| [`/llms-index.txt`](https://docs.aiml.asu.edu/llms-index.txt) | A short index linking each page, with a one-line summary |

Every page is also available as plain markdown by appending `.md` to its URL — for example [`/endpoints/query.md`](https://docs.aiml.asu.edu/endpoints/query.md).

:::note Naming
`/llms.txt` carries the whole corpus here, and `/llms-index.txt` is the short index. That's the reverse of the usual [llmstxt.org](https://llmstxt.org/) convention, so fetch `/llms-index.txt` first if you only want to find the right page.
:::

Each page in this site shows whether it's part of the corpus with a badge in its top-right corner.

## Base URL

| Environment | Base URL |
|-------------|----------|
| **Production** | `https://api-main.aiml.asu.edu` |
| **Beta** | `https://api-main-beta.aiml.asu.edu` |
| **POC** | `https://api-main-poc.aiml.asu.edu` |
## Next Steps

1. Get your [API access](/access)
2. Review the [available models](/models)
3. Pick the right token in [Token Details](/tokens)
4. Check [rate limits](/limits)
5. Read about [error handling](/errors)
6. Set up [ASU Sign-In](/sso-redirect) if real users will use your app
7. Learn about [going live](/going-live)
8. Explore the [OpenAI-Compatible API](/openai-compatible) for easy integration

## Need Help?

If you need assistance or want to request API access, visit the [Access Request](/access) page. You can also contact us at #createai-community-hub via slack or email us at aiacceleration@asu.edu

---

## Query

# Query Endpoint

Send natural language queries to LLMs with optional knowledge base search, conversation history, semantic caching, and advanced RAG capabilities. The Query endpoint supports multiple model providers and can be accessed via REST or WebSocket connections.

## Overview

The Query endpoint is the primary interface for sending prompts to large language models. It supports multiple providers and models, optional knowledge base search (RAG), conversation history, tool calling, structured output, and prompt enhancement. Queries can be sent over REST or WebSocket connections.

:::info Token Types
- **Developer Token:** Use `collection` in search_params for knowledge base search
- **Service Token:** Use `project_id` parameter at root level
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|-----------|
| **REST** | `POST` | `query` | `https://api-main.aiml.asu.edu/query` |
| **WebSocket** | — | `query` | `wss://apiws-main.aiml.asu.edu?access_token=YOUR_CREATEAI_TOKEN` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/query` |
| **WebSocket** | — | `query` |

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `endpoint` | string | Must be `"query"` |
| `action` | string | Must be `"query"` for websocket connections, ignored for REST |
| `request_source` | string | Must be `"override_params"` if using project service token and you would like to send your own parameters |
| `query` | string | The natural language query or prompt |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_provider` | string | Project default | The model provider (e.g., `"openai"`, `"aws"`). Can be overridden per request. |
| `model_name` | string | Project default | The model name (e.g., `"gpt4o"`, `"claude3_5_sonnet"`, `"nova-micro"`). Can be overridden per request. |
| `project_id` | string | — | Required when using developer token |
| `session_id` | string | — | Session identifier. Used with `enable_history` to maintain context across requests and `chat_upload` if uploading files for in-context use. |
| `query_id` | string | — | Unique query identifier only required when accessing in-context files. |
| `model_params` | object | Project default | Model configuration parameters ([see below](#model_params-object)). Can be overridden per request. |
| `enable_search` | boolean | Project default | Enable knowledge base search / RAG. Can be overridden per request. |
| `search_params` | object | Project default | Search configuration ([see below](#search_params-object-when-enable_search-is-true)). Required when `enable_search` is true. Can be overridden per request. |
| `enable_history` | boolean | Project default | Enable conversation history tracking. Pair with `session_id` to maintain context across multiple requests. Can be overridden per request. |
| `history` | array | — | Previous conversation history messages. This is only needed if you want to pass your custom history. |
| `response_format` | object | — | Response format configuration ([see below](#response_format-object)). Supported: `{"type": "json"}`|
| `enhance_prompt` | object | Project default | Prompt enhancement options ([see below](#enhance_prompt-object)). Can be overridden per request. |
| `eval_params` | object | — | Evaluation parameters ([see below](#eval_params-object)) |
| `chat_upload` | object | — | Upload files for in-context use ([see below](#chat_upload-object)) |

### `model_params` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `temperature` | float | Project default | Controls randomness (0.0–2.0). Lower = more deterministic. Can be overridden per request. |
| `system_prompt` | string | Project default | System-level instructions for the model. Can be overridden per request. |
| `top_p` | float | Project default | Nucleus sampling parameter (0.0–1.0). Can be overridden per request. |
| `top_k` | integer | — | Top-k sampling parameter |
| `thinking_level` | string | — | Thinking/reasoning depth: `"LOW"`, `"MEDIUM"`, `"HIGH"` |
| `tools` | array | — | Function/tool definitions for tool calling (see [Tool Calling](#tool-calling-function-calling) section) |
| `response_format` | object | — | Structured output schema (JSON Schema format, see [Structured Output](#structured-output-json-schema)) This is different from the `response_format` object used in the main query parameters. |

### `response_format` Object

| Parameter | Type | Description |
|-----------|------|-------------|
| `type` | string | **(Required)** Response format type: `"text"`, `"json"`, or `"openai"` |

### `search_params` Object (when enable_search is true)

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `collection` | string | — | Collection ID. Required when using developer token. |
| `retrieval_type` | string | Project default | Type of retrieval: `"chunk"`, `"document"`, or `"neighbor"`. Can be overridden per request. |
| `top_k` | integer | Project default | Number of search results to retrieve. Can be overridden per request. |
| `output_fields` | array | Project default | Fields to include in search results. Can be overridden per request. |
| `tags` | array | — | Filter by document tags |
| `source_names` | array | — | Filter by specific source file names |
| `expr` | string | — | Filter expression for metadata |
| `rerank` | boolean | `false` | Enable reranking on search results |
| `reranker_model` | string | — | Reranker model (e.g., `"cohere_rerank-3_5"`, `"amazon_rerank"`) |
| `reranker_provider` | string | — | Reranker provider (e.g., `"aws"`) |
| `top_n` | integer | — | Number of results after reranking |
| `advanced_rag` | string | `"false"` | Enable advanced RAG features (`"true"` / `"false"`) |
| `prompt_mode` | string | Project default | Prompt mode: `"restricted"`, `"unrestricted"`, or `"custom"`. Can be overridden per request. |
| `search_prompt` | string | Project default | Custom search prompt template (use `{data}` and `{query}` placeholders). Can be overridden per request. |

### `enhance_prompt` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `timezone` | string | Project default | Timezone for time-aware prompts (e.g., `"MST"`). Can be overridden per request. |
| `time` | boolean | Project default | Include current time in prompt. Can be overridden per request. |
| `date` | boolean | Project default | Include current date in prompt. Can be overridden per request. |
| `verbosity` | string | Project default | Response verbosity: `"brief"`, `"normal"`, `"detailed"`. Can be overridden per request. |

### `eval_params` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `context_utilization` | boolean | `false` | Enable context utilization evaluation |
| `prompt_guard` | boolean | `false` | Enable prompt guard evaluation |

### `chat_upload` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `images` | array | — | List of uploaded image filenames |
| `docs` | array | — | List of uploaded document filenames |
| `audios` | array | — | List of uploaded audio filenames |

## Basic Query Example (Developer Token)

```python
import requests

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}
dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is artificial intelligence?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "You are a helpful assistant.",
        "top_p": 0.01
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "What is artificial intelligence?",
  model_params: {
    temperature: 0.1,
    system_prompt: "You are a helpful assistant.",
    top_p: 0.01
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is artificial intelligence?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "You are a helpful assistant.",
      "top_p": 0.01},
    "response_format": {"type": "json"}
  }'
```

### Response

```json
{
  "response": {
    "response": "AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines designed to think and learn like humans. It encompasses various technologies, including machine learning, natural language processing, and robotics.",
    "metadata": {
      "sources": {},
      "query_id": "ada15f24b0aa4d9fac680c9f15b9f7e4",
      "usage_metric": {
        "input_token_count": 65,
        "output_token_count": 39,
        "input_token_cost": 0.000325,
        "output_token_cost": 0.000585,
        "input_token_details": {
          "user_query": 4,
          "incontext_text": 0,
          "system_prompt": 60,
          "knowledge_base": 0,
          "conversation_history": 1
        },
        "output_token_details": {
          "output_response": 39
        },
        "input_token_cost_details": {
          "user_query": 2e-05,
          "incontext_text": 0.0,
          "system_prompt": 0.0003,
          "knowledge_base": 0.0,
          "conversation_history": 5e-06
        },
        "output_token_cost_details": {
          "output_response": 0.000585
        },
        "total_token_count": 104,
        "total_token_cost": 0.00091
      }
    }
  },
  "file": "main.py",
  "query_id": "ada15f24b0aa4d9fac680c9f15b9f7e4"
}
```

## Query with Knowledge Base Search (RAG)

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "Why do generative models keep making up stuff?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "Please answer the question only based on the information from the knowledge base. Do not answer if there is no relevant information in the knowledge base.",
        "top_p": 0.01,
        "top_k": 2
    },
    "enable_search": True,
    "search_params": {
        "db_type": "opensearch",
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "top_k": 5,
        "retrieval_type": "chunk",
        "output_fields": [
            "source_name",
            "page_number",
            "content",
            "tags"
        ]
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "Why do generative models keep making up stuff?",
  model_params: {
    temperature: 0.1,
    system_prompt: "Please answer the question only based on the information from the knowledge base.",
    top_p: 0.01,
    top_k: 2
  },
  enable_search: true,
  search_params: {
    db_type: "opensearch",
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    top_k: 5,
    retrieval_type: "chunk",
    output_fields: ["source_name", "page_number", "content", "tags"]
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "Why do generative models keep making up stuff?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "Please answer the question only based on the information from the knowledge base.",
      "top_p": 0.01,
      "top_k": 2
    },
    "enable_search": true,
    "search_params": {
      "db_type": "opensearch",
      "collection": "a1a5059652d449c9a9c21c2388d42da5",
      "top_k": 5,
      "retrieval_type": "chunk",
      "output_fields": ["source_name", "page_number", "content", "tags"]},
    "response_format": {"type": "json"}
  }'
```

### Response

```json
{
  "response": {
    "response": "Generative models hallucinate because they are trained to predict statistically likely next tokens rather than retrieve verified facts. The model generates plausible-sounding text based on patterns learned during training, but it does not have a mechanism to verify the accuracy of its outputs against a ground truth source.",
    "metadata": {
      "sources": {
        "source_name": "Diffusion World Model.pdf",
        "page_number": 3,
        "content": "...",
        "tags": ["research"]
      },
      "query_id": "b3c9e8a1f2d74a5eb1234567890abcde",
      "usage_metric": {
        "input_token_count": 512,
        "output_token_count": 85,
        "total_token_count": 597,
        "total_token_cost": 0.00245
      }
    }
  },
  "file": "main.py",
  "query_id": "b3c9e8a1f2d74a5eb1234567890abcde"
}
```

## Query with RAG and Reranking

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What are the latest advances in reinforcement learning?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "Answer based on the knowledge base.",
        "top_p": 0.01
    },
    "enable_search": True,
    "search_params": {
        "db_type": "opensearch",
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "top_k": 5,
        "retrieval_type": "chunk",
        "output_fields": ["source_name", "page_number", "content"],
        "rerank": True,
        "reranker_model": "cohere_rerank-3_5",
        "reranker_provider": "aws",
        "top_n": 3
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "What are the latest advances in reinforcement learning?",
  model_params: {
    temperature: 0.1,
    system_prompt: "Answer based on the knowledge base.",
    top_p: 0.01
  },
  enable_search: true,
  search_params: {
    db_type: "opensearch",
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    top_k: 5,
    retrieval_type: "chunk",
    output_fields: ["source_name", "page_number", "content"],
    rerank: true,
    reranker_model: "cohere_rerank-3_5",
    reranker_provider: "aws",
    top_n: 3
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What are the latest advances in reinforcement learning?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "Answer based on the knowledge base.",
      "top_p": 0.01
    },
    "enable_search": true,
    "search_params": {
      "db_type": "opensearch",
      "collection": "a1a5059652d449c9a9c21c2388d42da5",
      "top_k": 5,
      "retrieval_type": "chunk",
      "output_fields": ["source_name", "page_number", "content"],
      "rerank": true,
      "reranker_model": "cohere_rerank-3_5",
      "reranker_provider": "aws",
      "top_n": 3
    },
    "response_format": {"type": "json"}
  }'
```

## Query with Project ID (Service Token)

```python
import requests
from config import headers, dev_url

endpoint = "/query"

# Use project_id with query action for project service token
payload = {
    "action": "query",
    "query": "Explain what this document is about.",
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  query: "Explain what this document is about.",
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_SERVICE_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "query": "Explain what this document is about.",
    "response_format": {"type": "json"}
  }'
```

## Query with Chat Upload

Upload documents, images, or audio files for in-context querying.

:::tip Upload Files First
Before using chat upload in queries, you must first upload your files using the [Chat Upload endpoint](/endpoints/manage-project#chat-upload). This returns a `query_id` and `session_id` that you'll use in the query below.
:::

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "session_id": "sessionid1234", # query id you sent during chat upload
    "query_id": "query_id_returned", # This is the query_id returned from the chat upload response
    "project_id": "cfbdd896380c4d2ba2aa319da9f9eba6",
    "query": "Explain what this CreateAI Chat.pdf is?",
    "response_format": {"type": "json"},
    "search_params": {
        "output_fields": ["source_name", "page_number", "tags", "url"]
    },
    "chat_upload": {
        "images": [],
        "docs": ["CreateAI Chat.pdf"],
        "audios": []
    }
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
    action: "query",
    session_id: "sessionid1234", // query id you sent during chat upload
    query_id: "query_id_returned", // This is the query_id returned from the chat upload response
    project_id: "cfbdd896380c4d2ba2aa319da9f9eba6",
    query: "Explain what this CreateAI Chat.pdf is?",
  response_format: { type: "json" },
  search_params: {
    output_fields: ["source_name", "page_number", "tags", "url"]
  },
  chat_upload: {
    images: [],
    docs: ["CreateAI Chat.pdf"],
    audios: []
  },
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_SERVICE_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "session_id": "sessionid1234", # query id you sent during chat upload
    "query_id": "query_id_returned", # This is the query_id returned from the chat upload response
    "project_id": "cfbdd896380c4d2ba2aa319da9f9eba6",
    "query": "Explain what this CreateAI Chat.pdf is?",
    "response_format": {"type": "json"},
    "search_params": {
      "output_fields": ["source_name", "page_number", "tags", "url"]
    },
    "chat_upload": {
      "images": [],
      "docs": ["CreateAI Chat.pdf"],
      "audios": []
    }
  }'
```

## Query with History and Prompt Enhancement

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "session_id": "session_id_here",
    "query": "Can you elaborate on that?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "You are a helpful assistant.",
        "top_p": 0.01
    },
    "enable_history": True,
    "history": [],
    "enhance_prompt": {
        "timezone": "MST",
        "time": True,
        "date": True,
        "verbosity": "brief"
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  session_id: "session_id_here",
  query: "Can you elaborate on that?",
  model_params: {
    temperature: 0.1,
    system_prompt: "You are a helpful assistant.",
    top_p: 0.01
  },
  enable_history: true,
  history: [],
  enhance_prompt: {
    timezone: "MST",
    time: true,
    date: true,
    verbosity: "brief"
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "session_id": "session_id_here",
    "query": "Can you elaborate on that?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "You are a helpful assistant.",
      "top_p": 0.01
    },
    "enable_history": true,
    "history": [],
    "enhance_prompt": {
      "timezone": "MST",
      "time": true,
      "date": true,
      "verbosity": "brief"
    },
    "response_format": {"type": "json"}
  }'
```

## Query with Custom Search Prompt

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "aws",
    "model_name": "nova-micro",
    "enable_search": True,
    "query": "What programs does ASU offer?",
    "search_params": {
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "output_fields": ["content"],
        "prompt_mode": "custom",
        "search_prompt": (
            "Your role is to support students, researchers, and professors at ASU "
            "by addressing their questions comprehensively. "
            "Use the Knowledge Base at your discretion to formulate responses. "
            "Knowledge Base: {data} Question: {query}"
        )
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "aws",
  model_name: "nova-micro",
  enable_search: true,
  query: "What programs does ASU offer?",
  search_params: {
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    output_fields: ["content"],
    prompt_mode: "custom",
    search_prompt:
      "Your role is to support students, researchers, and professors at ASU " +
      "by addressing their questions comprehensively. " +
      "Use the Knowledge Base at your discretion to formulate responses. " +
      "Knowledge Base: {data} Question: {query}"
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "aws",
    "model_name": "nova-micro",
    "enable_search": true,
    "query": "What programs does ASU offer?",
    "search_params": {
      "collection": "a1a5059652d449c9a9c21c2388d42da5",
      "output_fields": ["content"],
      "prompt_mode": "custom",
      "search_prompt": "Your role is to support students. Knowledge Base: {data} Question: {query}"
    },
    "response_format": {"type": "json"}
  }'
```

## Tool Calling (Function Calling)

Enable the model to call external functions/tools by defining them in `model_params.tools`. Follow this format as per OpenAI Function Calling guidelines. The model will respond with the function call and arguments in a structured format for your application to execute and return results.

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is the weather in New York?",
    "model_params": {
        "temperature": 0.1,
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "description": "Get current temperature for a given location.",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "location": {
                                "type": "string",
                                "description": "City and country e.g. Bogotá, Colombia"
                            }
                        },
                        "required": ["location"],
                        "additionalProperties": False
                    },
                    "strict": True
                }
            }
        ]
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "What is the weather in New York?",
  model_params: {
    temperature: 0.1,
    tools: [
      {
        type: "function",
        function: {
          name: "get_weather",
          description: "Get current temperature for a given location.",
          parameters: {
            type: "object",
            properties: {
              location: {
                type: "string",
                description: "City and country e.g. Bogotá, Colombia"
              }
            },
            required: ["location"],
            additionalProperties: false
          },
          strict: true
        }
      }
    ]
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is the weather in New York?",
    "model_params": {
      "temperature": 0.1,
      "tools": [
        {
          "type": "function",
          "function": {
            "name": "get_weather",
            "description": "Get current temperature for a given location.",
            "parameters": {
              "type": "object",
              "properties": {
                "location": {
                  "type": "string",
                  "description": "City and country e.g. New York, USA"
                }
              },
              "required": ["location"],
              "additionalProperties": false
            },
            "strict": true
          }
        }
      ]
    },
    "response_format": {"type": "json"}
  }'
```

## Structured Output (JSON Schema)

Request structured JSON output by specifying a JSON schema in `model_params.response_format`.

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "Solve 2x + 5 = 15 step by step",
    "model_params": {
        "temperature": 0.1,
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "math_response",
                "strict": True,
                "schema": {
                    "type": "object",
                    "properties": {
                        "steps": {
                            "type": "array",
                            "items": {
                                "type": "object",
                                "properties": {
                                    "explanation": {"type": "string"},
                                    "output": {"type": "string"}
                                },
                                "required": ["explanation", "output"],
                                "additionalProperties": False
                            }
                        },
                        "final_answer": {"type": "string"}
                    },
                    "required": ["steps", "final_answer"],
                    "additionalProperties": False
                }
            }
        }
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "Solve 2x + 5 = 15 step by step",
  model_params: {
    temperature: 0.1,
    response_format: {
      type: "json_schema",
      json_schema: {
        name: "math_response",
        strict: true,
        schema: {
          type: "object",
          properties: {
            steps: {
              type: "array",
              items: {
                type: "object",
                properties: {
                  explanation: { type: "string" },
                  output: { type: "string" }
                },
                required: ["explanation", "output"],
                additionalProperties: false
              }
            },
            final_answer: { type: "string" }
          },
          required: ["steps", "final_answer"],
          additionalProperties: false
        }
      }
    }
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

## Evaluation Parameters

Enable evaluation metrics on query responses by including `eval_params`.

```python
payload = {
    # ... other query parameters ...
    "eval_params": {
        "context_utilization": True,   # Evaluate how well the response uses provided context
    }
}
```

## Supported Transports

| Body Type | REST | WebSocket |
|-----------|------|-----------|
| `query` | Yes | Yes |
| `search` | Yes | No |
| `vision` | Yes | Yes |
| `audio` | Yes | Yes |
| `speech` | Yes | Yes |
| `image` | Yes | No |
| `rerank` | Yes | Yes |
| `realtime` | No | Yes |

## Response Format

The response contains a `response` object with the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `response` | string | The model's generated response text |
| `metadata` | object | Metadata about the query execution |
| `metadata.sources` | object | Source documents used (when search is enabled) |
| `metadata.query_id` | string | Unique identifier for this query |
| `metadata.usage_metric` | object | Token usage and cost breakdown |
| `file` | string | Handler file name |
| `query_id` | string | Unique query identifier |

### usage_metric Object

| Field | Type | Description |
|-------|------|-------------|
| `input_token_count` | integer | Total input tokens consumed |
| `output_token_count` | integer | Total output tokens generated |
| `input_token_cost` | float | Cost of input tokens |
| `output_token_cost` | float | Cost of output tokens |
| `total_token_count` | integer | Total tokens (input + output) |
| `total_token_cost` | float | Total cost |
| `input_token_details` | object | Breakdown: `user_query`, `incontext_text`, `system_prompt`, `knowledge_base`, `conversation_history` |
| `output_token_details` | object | Breakdown: `output_response` |

## Use Cases

- **Conversational AI** - Build chatbots and virtual assistants with history tracking
- **RAG Applications** - Query LLMs with knowledge base context for grounded responses
- **Document Q&A** - Upload and query documents directly via chat upload
- **Tool-Augmented Queries** - Enable LLMs to call external functions and APIs
- **Structured Data Extraction** - Extract structured JSON from unstructured text using JSON schemas
- **Custom Prompt Engineering** - Use custom search prompts with `prompt_mode: "custom"` for domain-specific RAG

:::warning Important
- Developer tokens must use `collection` in search_params when `enable_search` is true
- Service tokens must use `project_id` at the root level
- Use `source_names` (plural) in search_params, not `source_name`
- The `rerank`, `reranker_model`, and `reranker_provider` fields must all be set together for reranking
- When using `prompt_mode: "custom"`, include `{data}` and `{query}` placeholders in your `search_prompt`
:::

---

Stream real-time audio conversations with LLMs. Supports voice input/output, knowledge base search, and reranking. Available via **WebSocket only**.

### Endpoint

```
WebSocket route: query  (WebSocket only)
```

### Request Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `endpoint` | string | Yes | Must be `"realtime"` |
| `query` | string | Yes | The text query or instruction |
| `audio_file` | string | Yes | Base64-encoded audio file content |
| `model_provider` | string | Yes | The model provider (e.g., `"openai"`) |
| `model_name` | string | Yes | The model name |
| `session_id` | string | No | Session identifier |
| `project_id` | string | No | Project identifier |
| `model_params` | object | No | Model configuration parameters (see below) |
| `enable_search` | boolean | No | Enable knowledge base search (default: false) |
| `search_params` | object | No | Search configuration (see Query search_params) |
| `semantic_caching` | boolean | No | Enable semantic caching (default: false) |
| `enhance_prompt` | object | No | Prompt enhancement options |
| `enable_history` | boolean | No | Enable history tracking (default: false) |
| `response_format` | object | No | Response format configuration |

#### model_params Object (Realtime)

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `system_prompt` | string | No | System-level instructions |
| `voice` | string | No | Voice for audio output (e.g., `"alloy"`) |
| `input_audio_format` | string | No | Input audio format (e.g., `"pcm16"`, `"wav"`, `"mp3"`) |
| `output_audio_format` | string | No | Output audio format (e.g., `"pcm16"`, `"wav"`, `"mp3"`) |
| `temperature` | float | No | Controls randomness (0.0 - 2.0) |
| `modalities` | array | No | Output modalities: `"text"`, `"audio"`, or both |

### Realtime Example

```python
import base64
import json

# Read and encode audio file
with open("recording.wav", "rb") as audio_file:
    audio_data = audio_file.read()
encoded_audio = base64.b64encode(audio_data).decode("utf-8")

# WebSocket message payload
payload = {
    "endpoint": "realtime",
    "session_id": "sid_realtime_001",
    "query": "Answer the question in the audio",
    "audio_file": encoded_audio,
    "model_provider": "openai",
    "model_name": "gpt4o",
    "model_params": {
        "system_prompt": "You are a helpful assistant.",
        "voice": "alloy",
        "input_audio_format": "pcm16",
        "output_audio_format": "pcm16",
        "temperature": 0.7,
        "modalities": ["text", "audio"]
    },
    "enable_search": True,
    "search_params": {
        "db_type": "opensearch",
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "top_k": 5,
        "retrieval_type": "chunk",
        "output_fields": ["source_name", "page_number", "content"],
        "rerank": True,
        "reranker_model": "amazon_rerank",
        "reranker_provider": "aws"
    },
    "semantic_caching": False,
    "enhance_prompt": {
        "timezone": "MST",
        "time": True,
        "date": True,
        "verbosity": "brief"
    },
    "enable_history": True,
    "response_format": {"type": "json"}
}

# Send via WebSocket
# ws.send(json.dumps(payload))
```

```javascript
const fs = require("fs");

// Read and encode audio file
const audioData = fs.readFileSync("recording.wav");
const encodedAudio = audioData.toString("base64");

const payload = {
  endpoint: "realtime",
  session_id: "sid_realtime_001",
  query: "Answer the question in the audio",
  audio_file: encodedAudio,
  model_provider: "openai",
  model_name: "gpt4o",
  model_params: {
    system_prompt: "You are a helpful assistant.",
    voice: "alloy",
    input_audio_format: "pcm16",
    output_audio_format: "pcm16",
    temperature: 0.7,
    modalities: ["text", "audio"]
  },
  enable_search: true,
  search_params: {
    db_type: "opensearch",
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    top_k: 5,
    retrieval_type: "chunk",
    output_fields: ["source_name", "page_number", "content"],
    rerank: true,
    reranker_model: "amazon_rerank",
    reranker_provider: "aws"
  },
  semantic_caching: false,
  enhance_prompt: {
    timezone: "MST",
    time: true,
    date: true,
    verbosity: "brief"
  },
  enable_history: true,
  response_format: { type: "json" }
};

// Send via WebSocket
const ws = new WebSocket("wss://api.aiml-platform.com/v1");
ws.onopen = () => ws.send(JSON.stringify(payload));
ws.onmessage = (event) => console.log(JSON.parse(event.data));
```

:::info
The Realtime endpoint is available via **WebSocket only** — REST is not supported. It combines audio input/output with optional RAG for voice-based conversational AI.
:::

---

Compare responses from different models side-by-side. Uses the `query` action with a custom search prompt and is available via **WebSocket only**.

### Endpoint

```
WebSocket route: query_compare  (WebSocket only)
```

### Request Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `action` | string | Yes | Must be `"query"` |
| `query` | string | Yes | The query to compare across models |
| `model_provider` | string | Yes | The model provider (e.g., `"aws"`, `"openai"`) |
| `model_name` | string | Yes | The model name (e.g., `"nova-micro"`, `"claude3_5_sonnet"`, `"llama3-405b"`) |
| `session_id` | string | No | Session identifier |
| `enable_search` | boolean | No | Enable knowledge base search (default: false) |
| `search_params` | object | No | Search configuration with custom prompt support |
| `response_format` | object | No | Response format configuration |

#### search_params Object (Query Compare)

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `collection` | string | Conditional | Collection ID for knowledge base search |
| `output_fields` | array | No | Fields to include in search results. Also include any custom fields you added in the metadata during data upload. |
| `prompt_mode` | string | No | Prompt mode: `"default"` or `"custom"` |
| `search_prompt` | string | No | Custom prompt template with `{data}` and `{query}` placeholders |

---

## Related

- [Search](/endpoints/search)
- [Embeddings](/endpoints/embeddings)
- [Reranker](/endpoints/reranker)
- [Vision](/endpoints/vision)
- [Audio](/endpoints/audio)
- [Speech](/endpoints/speech)
- [Image](/endpoints/image)
- [Manage Project](/endpoints/manage-project)
- [Realtime](/endpoints/realtime)
- [Models](/models)

---

## Search

# Search Endpoint

Perform semantic search across your document collections with advanced filtering and retrieval options.

:::tip Supported Connections
**REST API**
:::

## Overview

The Search endpoint enables powerful semantic search capabilities across your document collections. Search through indexed documents using natural language queries with support for metadata filtering, different retrieval strategies, and custom output fields.

:::info Token Types
- **Service Token:** Use `project_id` parameter at root level
:::

## Endpoint

```
POST base_url/search
```

## Request Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `query` | string | Yes | The search query text |
| `search_params` | object | Yes | Search configuration parameters (see below) |
| `project_id` | string | Conditional | Required when using service token (instead of collection) |
| `embedding_name` | string | No | Embedding model to use (default: "ada") |
| `embedding_provider` | string | No | Embedding provider (default: "openai") |
| `semantic_caching` | boolean | No | Enable semantic caching (default: false) |

### search_params Object

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `retrieval_type` | string | No | Type of retrieval: "chunk", "document", or "neighbor" (default: "chunk") |
| `top_k` | integer | No | Number of results to return (1 to n) |
| `output_fields` | array | No | Fields to include in results. Default: ["source_name", "page_number", "content", "source_type"] |
| `expr` | string | No | Filter expression for metadata (see examples below) |
| `source_name` | array | No | Filter by specific source file names |
| `tags` | array | No | Filter by document tags |
| `db_type` | string | No | Database type (default: "opensearch") |
| `search_type` | string | No | Search type (default: "vector") |
| `advanced_rag` | boolean | No | Enable advanced RAG features (default: false) |
| `advanced_rag_config` | object | No | Configuration for advanced RAG |

## Retrieval Types

- **chunk** - Returns individual text chunks that match the query
- **document** - Returns entire documents that match the query
- **neighbor** - Returns neighboring chunks around the best matches

## Search with project_id (Service Token)

```python
import requests
from config import headers, dev_url

endpoint = "/search"

# Use project_id instead of collection when using service token
payload = {
    "query": "machine learning architectures",
    "project_id": "proj_abc123xyz456",
    "search_params": {
        "retrieval_type": "chunk",
        "top_k": 3,
        "output_fields": [
            "source_name",
            "page_number",
            "content"
        ],
    },
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main-poc.aiml.asu.edu/";

// Use project_id with service token
const payload = {
  query: "machine learning architectures",
  project_id: "proj_abc123xyz456",
  search_params: {
    retrieval_type: "chunk",
    top_k: 3,
    output_fields: ["source_name", "page_number", "content"]
  }
};

const response = await fetch(`${BASE_URL}/search`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_SERVICE_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main-poc.aiml.asu.edu/search \
  -H "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "machine learning architectures",
    "project_id": "proj_abc123xyz456",
    "search_params": {
      "retrieval_type": "chunk",
      "top_k": 3,
      "output_fields": ["source_name", "page_number", "content"]
    }
  }'
```

### Response

```json
{
  "response": [
    {
      "source_name": "architecture-guide.pdf",
      "page_number": 42,
      "score": 2.1234567,
      "content": "Sed ut perspiciatis unde omnis iste natus error sit voluptatem accusantium doloremque laudantium. Nemo enim ipsam voluptatem quia voluptas sit aspernatur aut odit aut fugit."
    },
    {
      "source_name": "deep-learning-overview.pdf",
      "page_number": 15,
      "score": 1.9876543,
      "content": "At vero eos et accusamus et iusto odio dignissimos ducimus qui blanditiis praesentium voluptatum deleniti atque corrupti quos dolores et quas molestias excepturi sint occaecati."
    },
    {
      "source_name": "neural-networks.pdf",
      "page_number": 8,
      "score": 1.8765432,
      "content": "Temporibus autem quibusdam et aut officiis debitis aut rerum necessitatibus saepe eveniet ut et voluptates repudiandae sint et molestiae non recusandae."
    }
  ]
}
```

## Expression Filtering (expr)

Use boolean expressions to filter results based on metadata fields. When filtering on custom metadata, use `metadata.field_name` syntax.

### Expression Examples

| Expression | Description |
|------------|-------------|
| `course_level == 'level_5'` | Exact match on a field |
| `metadata.degree_type == 'graduate'` | Filter on custom metadata field |
| `tags != 'Module 0' && tags != 'Syllabus'` | Exclude content with both tags |
| `tags == 'Module 0' && tags == 'Syllabus'` | Include only content with both tags |
| `tags != 'Module 0' \|\| tags != 'Syllabus'` | Exclude content with either tag |
| `tags == 'Module 0' \|\| tags == 'Syllabus'` | Include content with either tag |
| `page_number > 10 && page_number < 50` | Numeric range filtering |

## Advanced Search with Metadata Filtering

```python
import requests
from config import headers, dev_url

endpoint = "/search"

# Search with custom metadata fields and filtering
payload = {
    "query": "psychology and computer science",
    "search_params": {
        "collection": "a6bb05d46c97476d8b3dbc86cc2e2e9e",
        "retrieval_type": "neighbor",
        "top_k": 3,
        "output_fields": [
            "source_name",
            "content",
            "course_name",
            "course_id",
            "course_code",
            "course_level"
        ],
        "expr": "course_level == 'level_5'"
    },
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main-poc.aiml.asu.edu/";

const payload = {
  query: "psychology and computer science",
  search_params: {
    collection: "a6bb05d46c97476d8b3dbc86cc2e2e9e",
    retrieval_type: "neighbor",
    top_k: 3,
    output_fields: [
      "source_name",
      "content",
      "course_name",
      "course_id",
      "course_code",
      "course_level"
    ],
    expr: "course_level == 'level_5'"
  }
};

const response = await fetch(`${BASE_URL}/search`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main-poc.aiml.asu.edu/search \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "psychology and computer science",
    "search_params": {
      "collection": "a6bb05d46c97476d8b3dbc86cc2e2e9e",
      "retrieval_type": "neighbor",
      "top_k": 3,
      "output_fields": [
        "source_name",
        "content",
        "course_name",
        "course_id",
        "course_code",
        "course_level"
      ],
      "expr": "course_level == \"level_5\""
    }
  }'
```

### Response

```json
{
  "response": [
    {
      "source_name": "course-catalog-2024.json",
      "course_name": "Advanced Cognitive Psychology",
      "course_id": "crs_789xyz",
      "course_code": "PSY501",
      "course_level": "level_5",
      "score": 2.3456789,
      "content": "Quis autem vel eum iure reprehenderit qui in ea voluptate velit esse quam nihil molestiae consequatur. Itaque earum rerum hic tenetur a sapiente delectus."
    },
    {
      "source_name": "course-catalog-2024.json",
      "course_name": "Software Architecture Patterns",
      "course_id": "crs_456abc",
      "course_code": "CSE502",
      "course_level": "level_5",
      "score": 2.1987654,
      "content": "Nam libero tempore, cum soluta nobis est eligendi optio cumque nihil impedit quo minus id quod maxime placeat facere possimus, omnis voluptas assumenda est."
    },
    {
      "source_name": "course-catalog-2024.json",
      "course_name": "Computational Psychology",
      "course_id": "crs_123def",
      "course_code": "PSY510",
      "course_level": "level_5",
      "score": 2.0876543,
      "content": "Omnis dolor repellendus. Temporibus autem quibusdam et aut officiis debitis aut rerum necessitatibus saepe eveniet ut voluptates repudiandae."
    }
  ]
}
```

## Response Format

The response contains a `response` array with matching results. Each result includes:

- `content` - The matched text content
- `score` - Relevance score (higher is better)
- `source_name` - Source document filename
- `page_number` - Page number (if applicable)
- Any additional fields specified in `output_fields`

## Use Cases

- **Knowledge Base Search** - Search through documentation and support articles
- **Document Q&A** - Find relevant passages to answer user questions
- **Content Discovery** - Help users discover related content based on interests
- **Academic Research** - Search through papers, courses, and educational content
- **Compliance & Legal** - Search through contracts and legal documents with metadata filtering

:::warning Important
- Service tokens must use `project_id` at the root level
- When using metadata filters with `expr`, prefix custom metadata fields with "metadata."
- Default output fields are: source_name, page_number, content, source_type
:::

## Related

- [Embeddings](/endpoints/embeddings)
- [Reranker](/endpoints/reranker)
- [Query Endpoint](/endpoints/query)

---

## Audio (Transcription)

# Audio Endpoint

Transcribe audio files to text using speech-to-text models.

## Overview

The Audio endpoint enables transcription of audio files to text using advanced speech-to-text models. It supports both REST and WebSocket connections for real-time or batch transcription.

:::info Token Types
**Service Token** is supported for this endpoint.
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|-----------|
| **REST** | `POST` | `audio` | `https://api-main.aiml.asu.edu` |
| **WebSocket** | — | `audio` | `wss://apiws-main.aiml.asu.edu?access_token=YOUR_CREATEAI_TOKEN` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/query` |
| **WebSocket** | — | `query` |

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `endpoint` | string | Must be `"audio"` |
| `action` | string | Must be `"query"` for websocket connections, ignored for REST |
| `request_source` | string | Must be `"override_params"` if using project service token |
| `query` | string | Instruction for the transcription (e.g., `"transcribe this"`) |
| `audio_file` | string | Base64-encoded audio file content |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_provider` | string | Project default | The model provider (e.g., `"openai"`). Can be overridden per request. |
| `model_name` | string | Project default | The model name (e.g., `"whisper-1"`). Can be overridden per request. |
| `session_id` | string | — | Session identifier. Used with `enable_history` to maintain context across requests. |
| `model_params` | object | Project default | Model configuration parameters ([see below](#model_params-object-audio)). Can be overridden per request. |
| `enable_history` | boolean | `false` | Enable history tracking. Pair with `session_id` to maintain context across multiple requests. |
| `response_format` | object | — | Response format configuration. Supported: `{"type": "json"}` |

### `model_params` Object (Audio)

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `system_prompt` | string | Project default | System-level instructions. Can be overridden per request. |
| `timestamp_granularities` | array | — | Timestamp detail levels: `"word"`, `"segment"`, or both |

## Audio Example

```python
import requests
import base64
headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

# Read and encode audio file
with open("speech.mp3", "rb") as audio_file:
    audio_data = audio_file.read()
encoded_audio = base64.b64encode(audio_data).decode("utf-8")

payload = {
    "action": "query", # required for websocket connections, ignored for REST
    "endpoint": "audio",
    "query": "transcribe this",
    "audio_file": encoded_audio,
    "model_provider": "openai",
    "model_name": "whisper-1",
    "model_params": {
        "system_prompt": "Transcribe the audio accurately.",
        "timestamp_granularities": ["word", "segment"]
    },
    "enable_history": False,
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api.aiml-platform.com/v1";
const fs = require("fs");

// Read and encode audio file
const audioData = fs.readFileSync("speech.mp3");
const encodedAudio = audioData.toString("base64");

const payload = {
  endpoint: "audio",
  query: "transcribe this",
  audio_file: encodedAudio,
  model_provider: "openai",
  model_name: "whisper-1",
  model_params: {
    system_prompt: "Transcribe the audio accurately.",
    timestamp_granularities: ["word", "segment"]
  },
  enable_history: false,
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/audio`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
# First encode your audio file to base64
AUDIO_BASE64=$(base64 -i speech.mp3)

curl -X POST https://api.aiml-platform.com/v1/audio \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"endpoint\": \"audio\",
    \"query\": \"transcribe this\",
    \"audio_file\": \"$AUDIO_BASE64\",
    \"model_provider\": \"openai\",
    \"model_name\": \"whisper-1\",
    \"model_params\": {
      \"system_prompt\": \"Transcribe the audio accurately.\",
      \"timestamp_granularities\": [\"word\", \"segment\"]
    },
    \"enable_history\": false,
    \"response_format\": {\"type\": \"json\"}
  }"
```

## Audio with Chat Upload

You can also reference previously uploaded audio files instead of sending base64 content. See [Manage Project - Chat Upload](/endpoints/manage-project#chat-upload) for details on uploading files.

```python
payload = {
    "endpoint": "audio",
    "query": "transcribe this",
    "session_id": "unique-session-id-123", # required for chat upload to maintain context
    "query_id": "unique-query-id-456", # query id returned from chat upload api
    "audio_file": encoded_audio,
    "model_provider": "openai",
    "model_name": "whisper-1",
    "model_params": {
        "system_prompt": "Transcribe the audio accurately.",
        "timestamp_granularities": ["word", "segment"]
    },
    "chat_upload": {
        "audios": ["meeting-recording.mp3"]
    },
    "response_format": {"type": "json"}
}
```

---

## Vision

# Vision Endpoint

Analyze images using vision-capable LLMs, with optional knowledge base search for context-augmented responses.

## Overview

The Vision endpoint enables image analysis using vision-capable language models. Analyze images, extract information, and answer questions about visual content with optional knowledge base integration for context-augmented responses.

:::info Token Types
**Service Token** is supported for this endpoint.
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|----------|
| **REST** | `POST` | `vision` | `https://api-main.aiml.asu.edu` |
| **WebSocket** | `WSS` | `query` | `wss://apiws-main.aiml.asu.edu?access_token=YOUR_CREATEAI_TOKEN` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/query` |
| **WebSocket** | `WSS` | `query` |

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `endpoint` | string | Must be `"vision"` |
| `request_source` | string | Must be `"override_params"` if using project service token |
| `query` | string | Question or instruction about the image |
| `image_file` | string | Base64-encoded image file content |

### Override (Project Defaults)

These parameters use your project's default configuration if omitted. Include them to override per request.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_provider` | string | Project default | The model provider (e.g., `"openai"`) |
| `model_name` | string | Project default | The model name (e.g., `"gpt4o"`) |
| `model_params` | object | Project default | Model configuration parameters ([see below](#model_params-object)) |
| `enable_history` | boolean | Project default | Enable history tracking |
| `enable_search` | boolean | Project default | Enable knowledge base search |
| `search_params` | object | Project default | Search configuration (see [Query search_params](query#search_params-object-when-enable_search-is-true)) |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `project_id` | string | — | Project identifier |
| `session_id` | string | — | Session identifier |
| `query_id` | string | — | Unique query identifier |
| `response_format` | object | — | Response format configuration. Supported: `{"type": "json"}` |
| `chat_upload` | object | — | Upload images/docs for context ([see manage-project](manage-project#chat-upload)) |

### `model_params` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `system_prompt` | string | — | System-level instructions for the model |

## Vision Example

```python
import requests
import base64

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

# Read and encode image file
with open("diagram.png", "rb") as image_file:
    image_data = image_file.read()
encoded_image = f"data:image/png;base64,{base64.b64encode(image_data).decode('utf-8')}"

payload = {
    "endpoint": "vision",                                            # required
    "request_source": "override_params",                             # required if using project service token
    "query": "What is shown in this image?",                         # required
    "image_file": encoded_image,                                      # required
    "model_provider": "openai",                                      # override (uses project default if omitted)
    "model_name": "gpt4o",                                           # override (uses project default if omitted)
    "model_params": {                                                 # override (uses project default if omitted)
        "system_prompt": "You are a helpful assistant that describes images."
    },
    "enable_history": False,                                          # override (uses project default if omitted)
    "response_format": {"type": "json"}                               # optional
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)
const fs = require("fs");

// Read and encode image file
const imageData = fs.readFileSync("diagram.png");
const encodedImage = imageData.toString("base64");

const payload = {
  endpoint: "vision",                                              // required
  request_source: "override_params",                                // required if using project service token
  query: "What is shown in this image?",                            // required
  image_file: encodedImage,                                         // required
  model_provider: "openai",                                         // override (uses project default if omitted)
  model_name: "gpt4o",                                              // override (uses project default if omitted)
  model_params: {                                                    // override (uses project default if omitted)
    system_prompt: "You are a helpful assistant that describes images."
  },
  enable_history: false,                                             // override (uses project default if omitted)
  response_format: { type: "json" }                                  // optional
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
# First encode your image to base64
IMAGE_BASE64=$(base64 -i diagram.png)

curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{
    \"endpoint\": \"vision\",
    \"request_source\": \"override_params\",
    \"query\": \"What is shown in this image?\",
    \"image_file\": \"$IMAGE_BASE64\",
    \"model_provider\": \"openai\",
    \"model_name\": \"gpt4o\",
    \"model_params\": {
      \"system_prompt\": \"You are a helpful assistant that describes images.\"
    },
    \"enable_history\": false,
    \"response_format\": {\"type\": \"json\"}
  }"
```

## Vision with Knowledge Base Search

Combine image analysis with knowledge base context for richer responses:

```python
import requests
import base64

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

with open("chart.png", "rb") as image_file:
    image_data = image_file.read()
encoded_image = f"data:image/png;base64,{base64.b64encode(image_data).decode('utf-8')}"

payload = {
    "endpoint": "vision",                                            # required
    "request_source": "override_params",                             # required if using project service token
    "query": "Analyze this chart using the research data.",          # required
    "image_file": encoded_image,                                      # required
    "model_provider": "openai",                                      # override (uses project default if omitted)
    "model_name": "gpt4o",                                           # override (uses project default if omitted)
    "model_params": {                                                 # override (uses project default if omitted)
        "system_prompt": "Analyze images using knowledge base context."
    },
    "enable_search": True,                                            # override (uses project default if omitted)
    "search_params": {                                                # override (uses project default if omitted)
        "db_type": "opensearch",
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "top_k": 5,
        "retrieval_type": "chunk",
        "output_fields": ["source_name", "page_number", "content"]
    },
    "enable_history": False,                                          # override (uses project default if omitted)
    "response_format": {"type": "json"}                               # optional
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)
const fs = require("fs");

const imageData = fs.readFileSync("chart.png");
const encodedImage = imageData.toString("base64");

const payload = {
  endpoint: "vision",                                              // required
  request_source: "override_params",                                // required if using project service token
  query: "Analyze this chart using the research data.",            // required
  image_file: encodedImage,                                         // required
  model_provider: "openai",                                         // override (uses project default if omitted)
  model_name: "gpt4o",                                              // override (uses project default if omitted)
  model_params: {                                                    // override (uses project default if omitted)
    system_prompt: "Analyze images using knowledge base context."
  },
  enable_search: true,                                               // override (uses project default if omitted)
  search_params: {                                                   // override (uses project default if omitted)
    db_type: "opensearch",
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    top_k: 5,
    retrieval_type: "chunk",
    output_fields: ["source_name", "page_number", "content"]
  },
  enable_history: false,                                             // override (uses project default if omitted)
  response_format: { type: "json" }                                  // optional
};

const response = await fetch(`${BASE_URL}/vision`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

## Vision with Chat Upload

Reference previously uploaded images instead of inline base64:

```python
payload = {
    "endpoint": "vision",                                            # required
    "request_source": "override_params",                             # required if using project service token
    "query": "Describe the uploaded image.",                         # required
    "image_file": encoded_image,                                      # required
    "model_provider": "openai",                                      # override (uses project default if omitted)
    "model_name": "gpt4o",                                           # override (uses project default if omitted)
    "model_params": {"system_prompt": "Describe images clearly."},    # override (uses project default if omitted)
    "chat_upload": {                                                  # optional
        "images": ["architecture-diagram.jpg"],
        "docs": []
    },
    "response_format": {"type": "json"}                               # optional
}
```

---

## Speech (Text-to-Speech)

# Speech Endpoint

Convert text to speech using text-to-speech models.

## Overview

The Speech endpoint converts text to natural-sounding speech using advanced text-to-speech models. Generate audio output with multiple voice options and natural prosody for various use cases.

:::info Token Types
**Service Token** is supported for this endpoint.
:::

:::danger Agentic projects cannot generate speech
If your project has **Enable External Tools** turned on (for example, web search), your project runs as an *agentic* experience. Speech generation models are **not supported** in agentic mode today, so these requests will fail.

You do not need to change your project settings. Send `"request_source": "override_params"` together with `"agentic": false` to run that single request in non-agentic mode:

```json
{
  "endpoint": "speech",
  "request_source": "override_params",
  "agentic": false,
  "query": "Hello, welcome to the AI platform. How can I help you today?",
  "model_provider": "openai",
  "model_name": "tts1",
  "voice": "alloy"
}
```

`agentic: false` switches your project settings to non-agentic for that request only. Use it only for multimodal generation calls (image, speech) — leave it out of normal `query` calls so your project's tools stay available.

Using the [OpenAI-compatible API](/openai-compatible#createai-parameters-via-extra_body)? Pass the same two fields through `extra_body`.
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|----------|
| **REST** | `POST` | `speech` | `https://api-main.aiml.asu.edu` |
| **WebSocket** | `WSS` | `query` | `wss://apiws-main.aiml.asu.edu?access_token=YOUR_CREATEAI_TOKEN` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/query` |
| **WebSocket** | `WSS` | `query` |

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `endpoint` | string | Must be `"speech"` |
| `request_source` | string | Must be `"override_params"` if using project service token |
| `query` | string | The text to convert to speech |

### Override (Project Defaults)

These parameters use your project's default configuration if omitted. Include them to override per request.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_provider` | string | Project default | The model provider (e.g., `"openai"`) |
| `model_name` | string | Project default | The model name (e.g., `"tts1"`) |
| `voice` | string | Project default | Voice identifier (e.g., `"alloy"`, `"echo"`, `"fable"`, `"onyx"`, `"nova"`, `"shimmer"`) |
| `model_params` | object | Project default | Model configuration parameters ([see below](#model_params-object)) |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `agentic` | boolean | Project default | Set to `false` to run the request in non-agentic mode. **Required if your project has External Tools enabled** — see the note above. |
| `project_id` | string | — | Project identifier |
| `session_id` | string | — | Session identifier |

### `model_params` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `system_prompt` | string | — | System-level instructions |

## Speech Example

```python
import requests

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

payload = {
    "endpoint": "speech",                                            # required
    "request_source": "override_params",                             # required if using project service token
    "query": "Hello, welcome to the AI platform. How can I help you today?",  # required
    "model_provider": "openai",                                      # override (uses project default if omitted)
    "model_name": "tts1",                                            # override (uses project default if omitted)
    "voice": "alloy",                                                # override (uses project default if omitted)
    "model_params": {                                                 # override (uses project default if omitted)
        "system_prompt": "Speak clearly and naturally."
    },
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  endpoint: "speech",                                              // required
  request_source: "override_params",                                // required if using project service token
  query: "Hello, welcome to the AI platform. How can I help you today?",  // required
  model_provider: "openai",                                         // override (uses project default if omitted)
  model_name: "tts1",                                               // override (uses project default if omitted)
  voice: "alloy",                                                   // override (uses project default if omitted)
  model_params: {                                                    // override (uses project default if omitted)
    system_prompt: "Speak clearly and naturally."
  },
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "speech",
    "request_source": "override_params",
    "query": "Hello, welcome to the AI platform. How can I help you today?",
    "model_provider": "openai",
    "model_name": "tts1",
    "voice": "alloy",
    "model_params": {
      "system_prompt": "Speak clearly and naturally."
    }
  }'
```

## Available Voices

| Voice | Description |
|-------|-------------|
| `alloy` | Neutral, balanced tone |
| `echo` | Warm, conversational |
| `fable` | Expressive, storytelling |
| `onyx` | Deep, authoritative |
| `nova` | Friendly, upbeat |
| `shimmer` | Soft, gentle |

---

## Image (Generation)

# Image Endpoint

Generate images from text prompts using image generation models.

## Overview

The Image endpoint generates images from text prompts using state-of-the-art image generation models. Create high-quality, photorealistic images or stylized artwork based on natural language descriptions.

:::info Token Types
**Service Token** is supported for this endpoint.
:::

:::danger Agentic projects cannot generate images
If your project has **Enable External Tools** turned on (for example, web search), your project runs as an *agentic* experience. Image generation models are **not supported** in agentic mode today, so these requests will fail.

You do not need to change your project settings. Send `"request_source": "override_params"` together with `"agentic": false` to run that single request in non-agentic mode:

```json
{
  "endpoint": "image",
  "request_source": "override_params",
  "agentic": false,
  "query": "Generate an image of a futuristic university campus at sunset",
  "model_provider": "gcp-deepmind",
  "model_name": "geminiflash2_5_image",
  "model_params": {
    "system_prompt": "Generate high-quality, photorealistic images."
  },
  "enable_history": false,
  "response_format": {"type": "json"}
}
```

`agentic: false` switches your project settings to non-agentic for that request only. Use it only for multimodal generation calls (image, speech) — leave it out of normal `query` calls so your project's tools stay available.

Using the [OpenAI-compatible API](/openai-compatible#createai-parameters-via-extra_body)? Pass the same two fields through `extra_body`.
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|-----------|
| **REST** | `POST` | `image` | `https://api-main.aiml.asu.edu` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/query` |

:::info
The Image endpoint is available via **REST only** — WebSocket is not supported for image generation.
:::

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `endpoint` | string | Must be `"image"` |
| `request_source` | string | Must be `"override_params"` if using project service token |
| `query` | string | The text prompt describing the image to generate |

### Override (Project Defaults)

These parameters use your project's default configuration if omitted. Include them to override per request.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_provider` | string | Project default | The model provider (e.g., `"gcp-deepmind"`) |
| `model_name` | string | Project default | The model name (e.g., `"geminiflash2_5_image"`) |
| `model_params` | object | Project default | Model configuration parameters ([see below](#model_params-object-image)) |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `agentic` | boolean | Project default | Set to `false` to run the request in non-agentic mode. **Required if your project has External Tools enabled** — see the note above. |
| `session_id` | string | — | Session identifier. Used with `enable_history` to maintain context across requests. |
| `project_id` | string | — | Project identifier. Required when using service token. |
| `enable_history` | boolean | `false` | Enable history tracking. Pair with `session_id` to maintain context across multiple requests. |
| `response_format` | object | — | Response format configuration. Supported: `{"type": "json"}` |

### `model_params` Object (Image)

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `system_prompt` | string | Project default | System-level instructions for image generation. Can be overridden per request. |

## Image Example

```python
import requests

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

payload = {
    "endpoint": "image",                                           # required
    "request_source": "override_params",                            # required if using project service token
    "query": "Generate an image of a futuristic university campus at sunset",  # required
    "model_provider": "gcp-deepmind",                                     # override (uses project default if omitted)
    "model_name": "geminiflash2_5_image",                                         # override (uses project default if omitted)
    "model_params": {                                                # override (uses project default if omitted)
        "system_prompt": "Generate high-quality, photorealistic images."
    },
    "enable_history": False,                                         # optional
    "response_format": {"type": "json"}                              # optional
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  endpoint: "image",                                             // required
  request_source: "override_params",                              // required if using project service token
  query: "Generate an image of a futuristic university campus at sunset",  // required
  model_provider: "gcp-deepmind",                                       // override (uses project default if omitted)
  model_name: "geminiflash2_5_image",                                           // override (uses project default if omitted)
  model_params: {                                                  // override (uses project default if omitted)
    system_prompt: "Generate high-quality, photorealistic images."
  },
  enable_history: false,                                           // optional
  response_format: { type: "json" }                                // optional
};

const response = await fetch(`${BASE_URL}/image`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
# endpoint, request_source, query = required
# model_provider, model_name, model_params = override (uses project default if omitted)
# enable_history, response_format = optional

curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "image",
    "request_source": "override_params",
    "query": "Generate an image of a futuristic university campus at sunset",
    "model_provider": "gcp-deepmind",
    "model_name": "geminiflash2_5_image",
    "model_params": {
      "system_prompt": "Generate high-quality, photorealistic images."
    },
    "enable_history": false,
    "response_format": {"type": "json"}
  }'
```

## Related

- [Query](/endpoints/query)
- [Vision](/endpoints/vision)
- [Models](/models)

---

## Reranker

# Reranker Endpoint

Improve search relevance by reranking documents using advanced AI models.

## Overview

The Reranker endpoint improves search quality by reordering a list of documents based on their relevance to a query. It uses advanced AI models to analyze the semantic relationship between your query and each document, returning them ordered by relevance score.

Use reranking as a second-stage ranker after initial retrieval to significantly boost result quality. This is particularly valuable for improving search accuracy in RAG (Retrieval-Augmented Generation) pipelines and semantic search applications.

:::info Token Types
**Service Token** is supported for this endpoint.
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|----------|
| **REST** | `POST` | `query` | `https://api-main.aiml.asu.edu/query` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/query` |

:::info Available Models
- **amazon_reranker** — AWS-powered reranking model
- **cohere_reranker-3_5** — Cohere's latest reranking model (via AWS)

*Both models are provided by AWS*
:::

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `endpoint` | string | Must be `"rerank"` |
| `request_source` | string | Must be `"override_params"` if using project service token |
| `query` | string | The search query to match against documents |
| `model_params` | object | Model configuration containing documents ([see below](#model_params-object)) |

### Override (Project Defaults)

These parameters use your project's default configuration if omitted. Include them to override per request.

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_provider` | string | Project default | Provider name (e.g., `"aws"`) |
| `model_name` | string | Project default | Model name: `"amazon_reranker"` or `"cohere_reranker-3_5"` |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `session_id` | string | — | Session identifier |
| `response_format` | object | — | Response format configuration. Supported: `{"type": "json"}` |

### `model_params` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `documents` | array | — | **(Required)** Array of text documents to rerank (max 100) |
| `top_n` | integer | `10` | Number of top results to return |

## Basic Example - amazon_reranker

```python
import requests

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

url = dev_url + endpoint
payload = {
    "endpoint": "rerank",                                            # required
    "request_source": "override_params",                             # required if using project service token
    "query": "what is the research focus of Dr. Katsanos",           # required
    "model_provider": "aws",                                         # override (uses project default if omitted)
    "model_name": "amazon_reranker",                                  # override (uses project default if omitted)
    "model_params": {                                                 # required
        "top_n": 3,
        "documents": [
            "Dr. Katsanos is an expert in exercise physiology, focused on metabolic responses to human obesity...",
            "Blue Cross Blue Shield Arizona offers health insurance and related services...",
            "CHiR serves individuals and organizations in need of comprehensive health care information..."
        ]
    }
}

response = requests.post(url, json=payload, headers=headers)
print(response.status_code)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  endpoint: "rerank",                                              // required
  request_source: "override_params",                                // required if using project service token
  query: "what is the research focus of Dr. Katsanos",             // required
  model_provider: "aws",                                            // override (uses project default if omitted)
  model_name: "amazon_reranker",                                    // override (uses project default if omitted)
  model_params: {                                                    // required
    top_n: 3,
    documents: [
      "Dr. Katsanos is an expert in exercise physiology...",
      "Blue Cross Blue Shield Arizona offers health insurance...",
      "CHiR serves individuals and organizations..."
    ]
  }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "rerank",
    "request_source": "override_params",
    "query": "what is the research focus of Dr. Katsanos",
    "model_provider": "aws",
    "model_name": "amazon_reranker",
    "model_params": {
      "top_n": 3,
      "documents": [
        "Dr. Katsanos is an expert in exercise physiology...",
        "Blue Cross Blue Shield Arizona offers health insurance...",
        "CHiR serves individuals and organizations..."
      ]
    }
  }'
```

### Response

```json
{
  "response": [
    {
      "index": 0,
      "relevance_score": 0.8326452942108686
    },
    {
      "index": 2,
      "relevance_score": 0.00015720345248929426
    },
    {
      "index": 1,
      "relevance_score": 6.643433442159853e-06
    }
  ],
  "metadata": {
    "sources": {},
    "query_id": "31a33bb99e53473ab040da18a1492156",
    "usage_metric": {
      "input_token_count": 37,
      "output_token_count": 67,
      "total_token_count": 104,
      "total_token_cost": 9.3e-05
    }
  }
}
```

## Example - cohere_reranker-3_5

```python
import requests

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

url = dev_url + endpoint
payload = {
    "endpoint": "rerank",                                            # required
    "request_source": "override_params",                             # required if using project service token
    "query": "machine learning best practices",                      # required
    "model_provider": "aws",                                         # override (uses project default if omitted)
    "model_name": "cohere_reranker-3_5",                             # override (uses project default if omitted)
    "model_params": {                                                 # required
        "top_n": 5,
        "documents": [
            "Document about supervised learning techniques...",
            "Guide to neural network architectures...",
            "Overview of data preprocessing methods...",
            "Deep dive into gradient descent optimization...",
            "Tutorial on cross-validation strategies..."
        ]
    }
}

response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  endpoint: "rerank",                                              // required
  request_source: "override_params",                                // required if using project service token
  query: "machine learning best practices",                        // required
  model_provider: "aws",                                            // override (uses project default if omitted)
  model_name: "cohere_reranker-3_5",                                // override (uses project default if omitted)
  model_params: {                                                    // required
    top_n: 5,
    documents: [
      "Document about supervised learning techniques...",
      "Guide to neural network architectures...",
      "Overview of data preprocessing methods...",
      "Deep dive into gradient descent optimization...",
      "Tutorial on cross-validation strategies..."
    ]
  }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "rerank",
    "request_source": "override_params",
    "query": "machine learning best practices",
    "model_provider": "aws",
    "model_name": "cohere_reranker-3_5",
    "model_params": {
      "top_n": 5,
      "documents": [
        "Document about supervised learning...",
        "Guide to neural network architectures...",
        "Overview of data preprocessing methods...",
        "Deep dive into gradient descent...",
        "Tutorial on cross-validation strategies..."
      ]
    }
  }'
```

### Response

```json
{
  "response": [
    {
      "index": 3,
      "relevance_score": 0.9125
    },
    {
      "index": 0,
      "relevance_score": 0.8847
    },
    {
      "index": 1,
      "relevance_score": 0.8653
    },
    {
      "index": 4,
      "relevance_score": 0.7892
    },
    {
      "index": 2,
      "relevance_score": 0.7541
    }
  ],
  "metadata": {
    "query_id": "abc123def456",
    "usage_metric": {
      "total_token_count": 156,
      "total_token_cost": 0.000142
    }
  }
}
```

## How It Works

1. Perform initial search using BM25, embeddings, or other methods
2. Pass the top 50-100 documents to the reranker endpoint
3. Receive reordered results with relevance scores (0 to 1)
4. Display the top N results to users

## Response Format

The response contains a `response` array with reranked results and a `metadata` object.

### Response Fields

| Field | Type | Description |
|-------|------|-------------|
| `response` | array | Array of reranked results, ordered by relevance (highest first) |
| `response[].index` | integer | Original index of the document in the input array |
| `response[].relevance_score` | float | Relevance score between 0 and 1 (higher is more relevant) |
| `metadata` | object | Additional information including query_id and usage metrics |

## Error Handling

If you use an incorrect model name format, you'll receive a 500 error:

```python
# Incorrect model name format
{
  "model_name": "aws_reranker"  # Wrong!
}

# Response:
{
  "message": "An error has occurred: aws_reranker is not supported as query model."
}

# Correct model names:
{
  "model_name": "amazon_reranker"      # Correct
  # or
  "model_name": "cohere_reranker-3_5"  # Correct
}
```

## Benefits & Use Cases

- **Higher Accuracy:** More precise relevance scoring than first-stage retrieval
- **Better Rankings:** Considers query-document interactions more deeply using transformers
- **Easy Integration:** Drop-in improvement for existing search and RAG pipelines
- **RAG Enhancement:** Improve context quality for LLM responses
- **Semantic Search:** Better understand user intent beyond keyword matching

:::tip Best Practice
For optimal results, pass 50-100 documents from your initial search to the reranker and return the top 10-20 using the `top_n` parameter.
:::

:::warning Important
- Always use `model_provider: "aws"`
- Model names must be exactly `amazon_reranker` or `cohere_reranker-3_5`
- Maximum 100 documents per request
- Documents array goes inside `model_params`, not at root level
- Relevance scores range from 0 to 1 (higher is better)
:::

## Related

- [Search Endpoint](/endpoints/search)
- [Query Endpoint](/endpoints/query)

---

## Realtime

# Realtime Endpoint

Stream real-time audio conversations with LLMs. Supports voice input/output, knowledge base search, and reranking. Available via **WebSocket only**.

:::warning Important
The Realtime endpoint requires sending audio in **chunks** via WebSocket. Each chunk must include the `action: "query"` field and use the `websocket_chunk` parameter. This is not a traditional request-response pattern.
:::

## Endpoint

```
WebSocket route: query  (WebSocket only)
```

## Request Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `action` | string | Yes | Must be `"query"` (required on every chunk) |
| `endpoint` | string | Yes | Must be `"realtime"` |
| `query` | string | Yes | The text query or instruction |
| `model_provider` | string | Yes | The model provider (e.g., `"openai"`) |
| `model_name` | string | Yes | The model name (e.g., `"gpt4o_mini_realtime"`) |
| `session_id` | string | Yes | Session identifier for multi-turn conversations |
| `websocket_chunk` | object | Yes | Chunk metadata and audio data (see below) |
| `project_id` | string | No | Project identifier |
| `model_params` | object | No | Model configuration parameters (see below) |
| `enable_search` | boolean | No | Enable knowledge base search (default: false) |
| `search_params` | object | No | Search configuration (see Query search_params) |
| `semantic_caching` | boolean | No | Enable semantic caching (default: false) |
| `enhance_prompt` | object | No | Prompt enhancement options |
| `enable_history` | boolean | No | Enable history tracking (default: false) |
| `response_format` | object | No | Response format configuration |

### websocket_chunk Object

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `total_chunks` | integer | Yes | Total number of chunks in the audio stream |
| `chunk_number` | integer | Yes | Current chunk number (1-indexed) |
| `chunk` | string | Yes | Base64-encoded audio chunk |
| `payload_hash` | string | Yes | SHA-1 hash of the complete audio data |

### model_params Object (Realtime)

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `system_prompt` | string | No | System-level instructions |
| `voice` | string | No | Voice for audio output (e.g., `"alloy"`) |
| `input_audio_format` | string | No | Input audio format: `"pcm16"`, `"g711_ulaw"`, or `"g711_alaw"` |
| `output_audio_format` | string | No | Output audio format: `"pcm16"`, `"g711_ulaw"`, or `"g711_alaw"` |
| `temperature` | float | No | Controls randomness (0.0 - 1.0) |
| `modalities` | array | No | Output modalities: `"text"`, `"audio"`, or both |

## How It Works

The Realtime endpoint uses a **chunking approach** to stream audio data:

1. **Connect to WebSocket** with your access token in the URL query parameter
2. **Encode your audio** to base64 (WAV format recommended for processing)
3. **Split the base64 data** into chunks (e.g., 20,000 characters each)
4. **Calculate SHA-1 hash** of the complete audio data for verification
5. **Send each chunk** via WebSocket with metadata (chunk number, total chunks, payload hash)
6. **Include `action: "query"`** on every chunk message
7. **Receive streaming responses** containing audio and/or text in real-time

Each chunk message must preserve the complete payload structure, including `action`, `endpoint`, and `websocket_chunk`.

:::tip WebSocket URL Format
```
wss://apiws-main.aiml.asu.edu/?access_token=YOUR_TOKEN
wss://apiws-main-beta.aiml.asu.edu/?access_token=YOUR_TOKEN  (Beta environment)
```
:::

## Realtime Example

```python
import websocket
import base64
from hashlib import sha1
import json
import time
import uuid
from pydub import AudioSegment
from pydub.playback import play

def play_audio_from_base64(base64_audio: str):
    """Decode and play base64-encoded PCM audio"""
    pcm_audio = base64.b64decode(base64_audio)
    print(f"Base64 audio length: {len(base64_audio)}")
    print(f"Decoded PCM length: {len(pcm_audio)}")

    # Ensure even number of bytes for 16-bit audio
    if len(pcm_audio) % 2 != 0:
        pcm_audio = pcm_audio[:-1]

    audio = AudioSegment(
        data=pcm_audio,
        sample_width=2,  # 16-bit
        frame_rate=24000,  # 24kHz
        channels=1
    )
    play(audio)

def convert_audio_to_base64(audio_path):
    """Convert audio file to base64 and generate hash"""
    with open(audio_path, "rb") as audio_file:
        audio_data = audio_file.read()
        return (
            base64.b64encode(audio_data).decode("utf-8"),
            sha1(audio_data).hexdigest(),
        )

def slice_base64(base64_data, chunk_size=20000):
    """Split base64 string into chunks"""
    return [
        base64_data[i:i + chunk_size]
        for i in range(0, len(base64_data), chunk_size)
    ]

def send_audio_chunks(
    audio_path,
    websocket_url,
    session_id=None,
    delay=0.1,
    last_chunk_delay=0.1
):
    """Send audio file in chunks via WebSocket and receive streaming response"""

    # Convert audio to base64 and get hash
    base64_audio, audio_hash = convert_audio_to_base64(audio_path)
    print("Audio converted to base64")
    print("Length:", len(base64_audio))
    print("Hash:", audio_hash)
    print("Preview:", base64_audio[:100])

    # Split into chunks
    audio_chunks = slice_base64(base64_audio)
    total_chunks = len(audio_chunks)

    # Connect to WebSocket
    ws = websocket.create_connection(websocket_url)
    print(f"Connected to WebSocket: {websocket_url}")

    # Send each chunk
    for i, chunk in enumerate(audio_chunks):
        payload = {
            "action": "query",
            "endpoint": "realtime",
            "query": "Respond in english only",
            "model_provider": "openai",
            "model_name": "gpt4o_mini_realtime",
            "model_params": {
                "voice": "alloy",
                "input_audio_format": "pcm16",
                "output_audio_format": "pcm16",
                "temperature": 0.7,
                "modalities": ["text", "audio"],
            },
            "session_id": session_id,
            "response_format": {"type": "json"},
            "websocket_chunk": {
                "total_chunks": total_chunks,
                "chunk_number": i + 1,
                "chunk": chunk,
                "payload_hash": audio_hash,
            },
        }

        ws.send(json.dumps(payload))
        print(f"Sent chunk {i + 1} of {total_chunks}")

        # Add delay between chunks
        if i == total_chunks - 2:
            time.sleep(last_chunk_delay)
        else:
            time.sleep(delay)

    # Receive streaming response
    try:
        while True:
            start_time = time.time()
            response = ws.recv()
            print("Received response:", response)

            # Handle audio response
            if "audio_response" in response:
                audio_response = json.loads(response).get("audio_response")
                print("Audio response length:", len(audio_response))
                play_audio_from_base64(audio_response)

            print("Response time:", time.time() - start_time)

            # Check for end of stream
            if "<EOS>" in json.loads(response).get("response", ""):
                ws.close()
                break

    except websocket.WebSocketConnectionClosedException as e:
        print("WebSocket closed:", e)
    finally:
        ws.close()

# Usage
audio_path = "audio.wav"  # Replace with your audio file path
websocket_url = "wss://apiws-main.aiml.asu.edu/?access_token=YOUR_TOKEN"
session_id = uuid.uuid4().hex  # Generate unique session ID

send_audio_chunks(audio_path, websocket_url, session_id)
```

```javascript
const fs = require("fs");
const crypto = require("crypto");
const WebSocket = require("ws");
const { v4: uuidv4 } = require("uuid");

/**
 * Convert audio file to base64 and generate hash
 */
function convertAudioToBase64(audioPath) {
  const audioData = fs.readFileSync(audioPath);
  const base64Audio = audioData.toString("base64");
  const audioHash = crypto.createHash("sha1").update(audioData).digest("hex");
  return { base64Audio, audioHash };
}

/**
 * Split base64 string into chunks
 */
function sliceBase64(base64Data, chunkSize = 20000) {
  const chunks = [];
  for (let i = 0; i < base64Data.length; i += chunkSize) {
    chunks.push(base64Data.slice(i, i + chunkSize));
  }
  return chunks;
}

/**
 * Send audio chunks via WebSocket and receive streaming response
 */
function sendAudioChunks(audioPath, websocketUrl, sessionId, delay = 100) {
  const { base64Audio, audioHash } = convertAudioToBase64(audioPath);
  console.log("Audio converted to base64");
  console.log("Length:", base64Audio.length);
  console.log("Hash:", audioHash);
  console.log("Preview:", base64Audio.substring(0, 100));

  const audioChunks = sliceBase64(base64Audio);
  const totalChunks = audioChunks.length;

  const ws = new WebSocket(websocketUrl);

  ws.on("open", () => {
    console.log(`Connected to WebSocket: ${websocketUrl}`);

    // Send each chunk with delay
    let chunkIndex = 0;
    const sendNextChunk = () => {
      if (chunkIndex < totalChunks) {
        const payload = {
          action: "query",
          endpoint: "realtime",
          query: "Respond in english only",
          model_provider: "openai",
          model_name: "gpt4o_mini_realtime",
          model_params: {
            voice: "alloy",
            input_audio_format: "pcm16",
            output_audio_format: "pcm16",
            temperature: 0.7,
            modalities: ["text", "audio"]
          },
          session_id: sessionId,
          response_format: { type: "json" },
          websocket_chunk: {
            total_chunks: totalChunks,
            chunk_number: chunkIndex + 1,
            chunk: audioChunks[chunkIndex],
            payload_hash: audioHash
          }
        };

        ws.send(JSON.stringify(payload));
        console.log(`Sent chunk ${chunkIndex + 1} of ${totalChunks}`);

        chunkIndex++;
        setTimeout(sendNextChunk, delay);
      }
    };

    sendNextChunk();
  });

  ws.on("message", (data) => {
    const startTime = Date.now();
    const response = JSON.parse(data);
    console.log("Received response:", response);

    // Handle audio response
    if (response.audio_response) {
      const audioBase64 = response.audio_response;
      console.log("Audio response length:", audioBase64.length);

      // Decode base64 to buffer and save or play
      const pcmBuffer = Buffer.from(audioBase64, "base64");
      // Here you would play the audio using a library like 'play-sound' or 'speaker'
    }

    console.log("Response time:", Date.now() - startTime, "ms");

    // Check for end of stream
    if (response.response && response.response.includes("<EOS>")) {
      ws.close();
    }
  });

  ws.on("close", () => {
    console.log("WebSocket connection closed");
  });

  ws.on("error", (error) => {
    console.error("WebSocket error:", error);
  });
}

// Usage
const audioPath = "audio.wav";  // Replace with your audio file path
const websocketUrl = "wss://apiws-main.aiml.asu.edu/?access_token=YOUR_TOKEN";
const sessionId = uuidv4().replace(/-/g, "");  // Generate unique session ID

sendAudioChunks(audioPath, websocketUrl, sessionId);
```

:::info Key Points
- **WebSocket only** — REST is not supported
- **Chunking required** — Audio must be sent in chunks via the `websocket_chunk` parameter
- **Session persistence** — Use the same `session_id` for multi-turn conversations
- **Action field mandatory** — Every chunk must include `"action": "query"`
- **Endpoint is "realtime"** — This is important for routing the request correctly
- **Supported audio formats** — `pcm16`, `g711_ulaw`, `g711_alaw` (wav/mp3 not supported for input/output formats)
- **Temperature range** — Valid values are 0.0 to 1.0 (not 2.0)
:::

---

## Embeddings

# Embeddings Endpoint

Generate vector embeddings for text, enabling semantic search and similarity comparisons.

## Overview

The Embeddings endpoint generates vector embeddings for text, enabling semantic search, similarity comparisons, clustering, and other machine learning tasks. Convert text into dense vector representations that capture semantic meaning.

:::info Token Types
- **Service Token:** Supported by default
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|----------|
| **REST** | `POST` | `embeddings` | `https://api-main.aiml.asu.edu/embeddings` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/embeddings` |

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `request_source` | string | Must be `"override_params"` if using project service token |
| `query` | string or array of strings | The text to generate embeddings for |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `embeddings_provider` | string | Project default | The model provider (e.g., `"openai"`). Can be overridden per request. |
| `embeddings_model` | string | Project default | The model name (e.g., `"te3s"`). Can be overridden per request. |

## Example

```python
import requests

url = "https://api-main.aiml.asu.edu/embeddings" # Replace with the appropriate url of your environment (prod, beta, poc)
headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

json_payload = {
    "embeddings_model": "te3s",
    "embeddings_provider": "openai",
    "query": "The quick brown fox jumps over the lazy dog"
}

response = requests.post(url, headers=headers, json=json_payload)
result = response.json()
print(result)
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  embeddings_model: "te3s",
  embeddings_provider: "openai",
  query: "The quick brown fox jumps over the lazy dog"
};

const response = await fetch(`${BASE_URL}/embeddings`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/embeddings \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "embeddings_model": "te3s",
    "embeddings_provider": "openai",
    "query": "The quick brown fox jumps over the lazy dog"
  }'
```

## Batch Processing

```python
import requests

url = "https://api-main.aiml.asu.edu/embeddings" # Replace with the appropriate url of your environment (prod, beta, poc)
headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

texts = [
    "First document",
    "Second document",
    "Third document"
]

for text in texts:
    json_payload = {
        "embeddings_model": "te3s",
        "embeddings_provider": "openai",
        "query": text
    }
    response = requests.post(url, headers=headers, json=json_payload)
    print(response.json())
```

## Available Models

| Provider | Model Name | Description | Output Vector Size | Input Context Window | Extra Parameters |
|----------|------------|-------------|-----------------|------------------|
| **openai** | `te3s` | Small embedding model for general use cases | {512, 1536} | 8192 tokens | Batching support
| **openai** | `te3l` | Larger embedding model for high quality embeddings | {256, 1024, 3072} | 8191 tokens | Batching support
| **openai** | `ada` | OpenAI's popular embedding model | 1536 | 8191 tokens | Batching support
| **aws** | `titan` | AWS Titan text embedding model v1 | 1536 | 8191 tokens |
| **aws** | `titan2` | AWS Titan text v2 | 1024 | 8191 tokens | dimensions and normalize
| **aws** | `titan-multimodal-g1` | AWS Titan multimodal embedding model | {256, 384, 1024} | 128
| **aws** | `ce-english` | Cohere English embedding model | 1024 | 512 tokens |
| **aws** | `ce-multilingual` | Cohere multilingual embedding model | 1024 | 512 tokens |
| **gcp** | `gecko` | GCP Gecko embedding model | 3072 | 768 tokens |
| **gcp** | `gecko-2` | GCP Gecko embedding model v2 | 3072 | 8191 tokens |
| **gcp** | `gecko-3` | GCP Gecko embedding model v3 | 3072 | 8191 tokens |
| **gcp** | `gecko-multilingual` | GCP Gecko multilingual embedding model |
| **gcp-deepmind** | `gemini-embedding-001` | DeepMind Gemini embedding model | {768, 1536, 3072} | 8191 tokens | Batching support, dimensions, task_type |

## Use Cases

- **Semantic Search**: Find relevant documents
- **Clustering**: Group similar content
- **Recommendations**: Content recommendations
- **Anomaly Detection**: Find outliers
- **Classification**: Text classification
- **RAG**: Retrieval Augmented Generation

## Similarity Calculation

```python
import numpy as np
import requests

url = "https://api-main.aiml.asu.edu/embeddings" # Replace with the appropriate url of your environment (prod, beta, poc)
headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

def get_embedding(text):
    json_payload = {
        "embeddings_model": "te3s",
        "embeddings_provider": "openai",
        "query": text
    }
    response = requests.post(url, headers=headers, json=json_payload)
    return response.json()

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Compare two texts
emb1 = get_embedding("Python programming")
emb2 = get_embedding("Coding in Python")

# similarity = cosine_similarity(emb1, emb2)
# print(f"Similarity: {similarity}")  # High value means similar
```

## Best Practices

1. Use batch processing for efficiency
2. Cache embeddings to avoid recomputation
3. Choose appropriate model for your use case
4. Normalize embeddings for cosine similarity

## Related

- [Search](/endpoints/search)
- [Models](/models)
- [Reranker](/endpoints/reranker)

---

## Manage Project


Unified API for managing projects, knowledge base, behaviors, and user access along with uploading incontext files.

:::tip Supported Connections
**REST API**
:::

## Overview

The Manage Project endpoint uses a unified structure with different `resource` and `method` combinations to control all aspects of your AI projects.

## Base URL

| Environment | Method | URL |
|-------------|--------|-----|
| **Production** | `POST` | `https://api-main.aiml.asu.edu/project` |
| **Beta** | `POST` | `https://api-main-beta.aiml.asu.edu/project` |
| **POC** | `POST` | `https://api-main-poc.aiml.asu.edu/project` |

:::warning Project Owner Token Required
All operations require a [project owner token](/tokens#3-project-owner-token) for authentication. Project owner token is linked to a specific project and grants full access to manage that project. Ensure you include the token in the `Authorization` header of your requests.

```bash
Authorization: Bearer owner_...
Content-Type: application/json
```
:::

## Request Structure

All requests follow this pattern unless otherwise specified:

```json
{
  "resource": "resource_type",  // project, data, user, access
  "method": "action",           // describe, update, list, add, remove, etc.
  "details": {
    // Operation-specific parameters
  }
}
```
---

## Project Operations

### Describe Project

Get complete project details and configuration.

```json
{
  "resource": "project",
  "method": "describe",
  "details": {
    "project_id": "your_project_id"
  }
}
```

### Update Project

Update any aspect of your project. **Only include fields you want to change.** For a list of all updatable fields, see [Update Project Fields](#update-project-fields) below.

```json
{
  "resource": "project",
  "method": "update",
  "details": {
    "project_id": "your_project_id",
    "project_name": "Updated Name",
    "model_name": "gpt4",
    "model_provider": "openai"
  }
}
```

```json
{
  "resource": "project",
  "method": "update",
  "details": {
    "project_id": "your_project_id",
    "model_params": {
      "custom_system": true,
      "system_prompt": "You are a helpful assistant...",
      "temperature": 0.7,
      "max_tokens": 2000
    }
  }
}
```

```json
{
  "resource": "project",
  "method": "update",
  "details": {
    "project_id": "your_project_id",
    "interface": {
      "title": "My AI Assistant",
      "description": "Welcome message",
      "enable_upload": true,
      "enable_voice": false,
      "enable_session_panel": true,
      "input_placeholder": "Ask me anything...",
      "disclaimer": "This is an AI assistant...",
      "starter_groups": [
        {
          "title": "Getting Started",
          "starters": [
            "How can you help me?",
            "What can I ask?"
          ]
        }
      ]
    }
  }
}
```

```json
{
  "resource": "project",
  "method": "update",
  "details": {
    "project_id": "your_project_id",
    "enable_search": true,
    "search_params": {
      "top_k": 5,
      "prompt_mode": "restricted",      // or "unrestricted"
      "retrieval_type": "document",     // or "neighbor"
      "search_prompt": "Use the following context...",
      "output_fields": ["source_name", "page_number"]
    }
  }
}
```

```json
{
  "resource": "project",
  "method": "update",
  "details": {
    "project_id": "your_project_id",
    "public_interface": true,
    "unrestricted_access": {
      "asu": true  // Share with all ASU users
    }
  }
}
```

**Response:** Returns complete project configuration including all settings.

### Publish Project

Make your project live and accessible to shared users.

```json
{
  "resource": "project",
  "method": "publish",
  "details": {
    "project_id": "your_project_id"
  }
}
```

**Response:**
```json
{
  "message": "Project published successfully"
}
```

---

## Data Operations (Knowledge Base)

### Upload Files

Upload documents to your project's knowledge base.

```json
{
  "resource": "data",
  "method": "upload",
  "details": {
    "project_id": "your_project_id",
    "db_type": "opensearch",
    "files": [
      {
        "file_name": "document.pdf",
        "search_tags": ["tag1", "tag2"],
        "selected": true,
        "visible": true,
"chunk_size": 768,
                    "chunk_overlap": 256,
        "metadata": {
          "author": "John Doe",
          "title": "Important Document"
        }
      }
    ]
  }
}
```
**Note:** `chunk_size` and `chunk_overlap` parameters control how documents are split for retrieval. Adjust these based on your document structure and retrieval needs. Recommended settings are typically around `chunk_size: 768` and `chunk_overlap: 256`, but you can experiment within the following limits:
- chunk size = 10 to 4096
- chunk overlap = 8 to 4094

**Response:** Returns presigned S3 URLs for uploading files.
```json
{
  "files": {
    "document.pdf": {
      "url": "https://s3.amazonaws.com/...",
      "fields": {
        "key": "...",
        "AWSAccessKeyId": "...",
        "policy": "...",
        "signature": "..."
      }
    }
  }
}
```

### Upload from URLs

Add content by URL (web scraping with Firecrawl).

```json
{
  "resource": "data",
  "method": "upload",
  "details": {
    "project_id": "your_project_id",
    "db_type": "opensearch",
    "urls": [
      {
        "url_name": "https://example.com/page",
        "search_tags": ["web", "docs"],
        "selected": true,
        "visible": true,
        "depth_level": 2,        // How many levels to crawl
        "url_document": true,
"chunk_size": 768,
                    "chunk_overlap": 256,
        "metadata": {
          "source": "Example Website",
          "category": "Reference"
        }
      }
    ]
  }
}
```

### List Files

Get all files and their upload/indexing status.

```json
{
  "resource": "data",
  "method": "list",
  "details": {
    "project_id": "your_project_id",
    "db_type": "opensearch"
  }
}
```

**Response:**
```json
{
  "files": [
    {
      "file_name": "document.pdf",
      "file_id": "",
      "file_size": 10928,
      "upload_type": "local",
      "search_status": "searchable",  // Make sure this field is "searchable" before using in RAG
      "selected": true,
      "visible": true,
      "search_tags": [],
      "notes": "",
      "links": [],
      "metadata": {},
      "time_loaded": 1771953011,
      "time_modified": null,
      "time_start": 1771953008
    }
  ],
  "urls": [
    {
      "url_name": "https://example.com",
      "sync_status": "complete",  // syncing, complete, failed
      "search_status": "searchable",
      "depth_level": 2,
      "visible": true,
      "url_document": true,
      "depth_enabled": true,
      "sync_failed_message": "",
      "upload_type": "firecrawl",
      "file_size": 0,
      "notes": "",
      "links": [],
      "search_tags": [],
      "metadata": {},
      "search_status": "updating",
      "selected": true,
      "time_modified": null,
      "time_loaded": null,
      "time_start": 1771954155,
      "time_synced": 1771954155
    }
  ]
}
```

### Delete Files

Remove files from knowledge base.

```json
{
  "resource": "data",
  "method": "delete",
  "details": {
    "project_id": "your_project_id",
    "db_type": "opensearch",
    "files": [
      {
        "file_name": "document.pdf"
      }
    ]
  }
}
```

:::warning After deleting a file, make sure the file is not present in the list data api call. Only then attempt to reupload the file if needed. Deletion can take up to a minute to reflect in the system.
:::

### Chat Upload

Upload files during a chat session (for temporary use in conversation).

:::info Session-Scoped Access
Chat upload files are **only accessible by the user who uploaded them in that specific session**. This differs from knowledge base documents, which are accessible by anyone with access to the project. Use chat upload for temporary, user-specific file interactions.
:::

```json
{
  "resource": "data",
  "method": "chat_upload",
  "details": {
    "project_id": "your_project_id",
    "session_id": "your_session_id", # required for chat upload to maintain context
    "db_type": "opensearch",
    "files": [
      {
        "file_name": "image.png"
      }
    ]
  }
}
```

**Response:** Returns presigned S3 URL and a `query_id` for tracking.

Once uploaded, you can reference these files in your queries. See [Query with Chat Upload](/endpoints/query#query-with-chat-upload) for usage examples.

### List Chat Assets

Check status of chat-uploaded files.

```json
{
  "resource": "data",
  "method": "list_assets",
  "details": {
    "project_id": "your_project_id",
    "session_id": "your_session_id",
    "db_type": "opensearch"
  }
}
```

---

## User Operations

### Describe User

Get user details before sharing a project (validates user exists).

```json
{
  "resource": "user",
  "method": "describe",
  "details": {
    "users": ["email@asu.edu", "another@asu.edu"]
  }
}
```

**Response:**
```json
{
  "users": [
    {
      "id": "user_id",
      "name": "John Doe",
      "email": "email@asu.edu",
      "phone": null,
      "photo_url": "https://...",
      "auth_apps": {
        "asugpt": true,
        "platform": true,
        "compare": true,
        "syllabot": true
      }
    }
  ]
}
```

---

## Access Operations

### Add User Access

Share your project with users by granting them specific roles.

```json
{
  "resource": "access",
  "method": "add",
  "details": {
    "project_id": "your_project_id",
    "users": [
      {
        "user_id": "user_asurite_id",
        "role": "viewer"  // viewer, editor, or owner
      }
    ]
  }
}
```

**Roles:**
- `viewer` - Can chat with the project
- `editor` - Can modify project settings
- `owner` - Full administrative access

**Response:** Returns list of all users with access to the project.

### Remove User Access

Revoke a user's access to your project.

```json
{
  "resource": "access",
  "method": "remove",
  "details": {
    "project_id": "your_project_id",
    "users": [
      {
        "user_id": "user_asurite_id",
        "role": "viewer"
      }
    ]
  }
}
```

---

## Common Parameters

### Update Project Fields

These are all the fields you can include in `details` when calling `resource: "project"`, `method: "update"`. **Only include fields you want to change.**

#### Top-Level Fields

| Parameter | Type | Description |
|-----------|------|-------------|
| `project_id` | string | **(Required)** Your project identifier |
| `project_name` | string | Display name for the project |
| `description` | string | Project description |
| `model_name` | string | AI model to use (e.g., `"gpt4_1-mini"`) |
| `model_provider` | string | Model provider (e.g., `"openai"`) |
| `public_interface` | boolean | Make the chat interface publicly accessible |
| `enable_history` | boolean | Enable conversation history tracking |
| `semantic_caching` | boolean | Enable semantic caching for repeated queries |
| `enable_search` | boolean | Enable knowledge base search (RAG) |
| `use_cases` | array | List of use case tags |

#### `model_params` Object

| Parameter | Type | Description |
|-----------|------|-------------|
| `custom_system` | boolean | Use a custom system prompt |
| `system_prompt` | string | Instructions for the AI model |
| `temperature` | number | Response randomness (0–2) |
| `max_tokens` | number | Maximum response length |
| `top_p` | number | Nucleus sampling (0–1) |

#### `interface` Object

| Parameter | Type | Description |
|-----------|------|-------------|
| `title` | string | Chat interface title |
| `description` | string | Welcome message |
| `enable_upload` | boolean | Allow file uploads in chat |
| `enable_voice` | boolean | Enable voice input |
| `enable_session_panel` | boolean | Show conversation history panel |
| `input_placeholder` | string | Placeholder text for the input field |
| `disclaimer` | string | Terms or disclaimer message |
| `starter_groups` | array | Conversation starter groups (see below) |

**`starter_groups` item:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `title` | string | Group title |
| `starters` | array | List of starter prompt strings |

#### `search_params` Object

| Parameter | Type | Description |
|-----------|------|-------------|
| `top_k` | number | Number of documents to retrieve |
| `prompt_mode` | string | `"restricted"` or `"unrestricted"` |
| `retrieval_type` | string | `"chunk"` or `"document"` |
| `search_prompt` | string | Template for RAG context. Use `{data}` and `{query}` placeholders. |
| `output_fields` | array | Metadata fields to return (e.g., `["source_name", "page_number", "content"]`) |
| `expr` | string | Filter expression for search results |

#### `enhance_prompt` Object

| Parameter | Type | Description |
|-----------|------|-------------|
| `time_zone` | string | Time zone for prompt context (e.g., `"MST"`) |
| `include_time` | boolean | Include current time in prompt context |
| `include_date` | boolean | Include current date in prompt context |
| `verbosity` | string | Response verbosity level |

---

## Quick Reference

| Operation | Resource | Method |
|-----------|----------|--------|
| Get project details | `project` | `describe` |
| Update configuration | `project` | `update` |
| Publish project | `project` | `publish` |
| Upload files | `data` | `upload` |
| List knowledge base | `data` | `list` |
| Delete files | `data` | `delete` |
| Chat file upload | `data` | `chat_upload` |
| List chat assets | `data` | `list_assets` |
| Get user info | `user` | `describe` |
| Grant access | `access` | `add` |
| Revoke access | `access` | `remove` |

:::tip Best Practices
- Only include fields you want to change in update requests
- Check file `search_status` after upload to ensure indexing completes
- Use `describe user` before adding access to validate email addresses
- Publish project after making changes to make them live
:::

## Related

- [Token Details](/tokens) - Learn about project owner tokens
- [Query Endpoint](/endpoints/query) - Make chat requests
- [Search Endpoint](/endpoints/search) - Query knowledge base directly

---

## Available Models



---

## Token Details


Understand the different types of API tokens and how to use them securely.

## Using API Tokens

```python
import requests
import os

url = "https://api-main-poc.aiml.asu.edu/query"

# Production with service token
prod_headers = {
    "Authorization": f"Bearer {os.getenv('AIML_PROJECT_SERVICE_TOKEN')}",  # eyjh...
    "Content-Type": "application/json"
}

# Service token uses project settings — only query is needed
prod_payload = {
    "query": "Hello"
}

response = requests.post(url, headers=prod_headers, json=prod_payload)
print(response.json().get("response", ""))
```

**Response:**
```json
{
  "response": "Hello! How can I assist you today?"
}
```

## Token Types

CreateAI uses three types of tokens for authentication, each designed for specific use cases and security requirements.

## 1. Service Token

Service tokens are production-grade keys for service-to-service authentication in deployed applications.

### Characteristics

- Associated with specific services or applications
- Rate limits are directly tied to the project limits
- Scoped to specific endpoints and permissions
- No automatic expiration
- Token uses the project’s configured model and provider settings
- Token can only be used to interact with an LLM via the API, and cannot be used to access project management endpoints

### When to Use

- Production applications and services

## 2. Project Owner Token

Project owner tokens have full administrative access to manage projects, sharing settings, and knowledge bases. These should be used very carefully.

### Characteristics

- Full administrative privileges to the project
- Can modify project settings

### When to Use

- When you need to update project configurations, manage knowledge bases, control user access, or publish settings via API
- If you need the chat upload endpoint, it requires a project owner token for uploading files to the chat
- [Manage Project endpoint](/endpoints/manage-project) operations (requires owner token)

:::note Project Management Access
Only project owner tokens can access the [Manage Project endpoint](/endpoints/manage-project) for updating project configurations, managing knowledge bases, controlling user access, and publishing workflows. Developer and service tokens do not have these administrative privileges.
:::

## Token Security Best Practices

### Storage

- Never commit tokens to version control
- Use environment variables or secret managers
- Encrypt tokens at rest
- Use separate tokens for each environment

### Usage

- Always use HTTPS for API requests or Websocket if streaming is needed
- Revoke tokens immediately if compromised

### Access Control

- Follow principle of least privilege
- Restrict project owner token usage

---

## Access Request

# Request API Access

Request your API key directly within CreateAI — no external forms needed.

## How to Request Your API Key

1. Go to your **Project** in CreateAI
2. Navigate to **Profile → API Keys**
3. Click **Request API Key**
4. Submit the form

Your API key will typically be available within **24–48 hours** (often sooner).

:::tip Need it sooner?
If your request is urgent, message our team directly and we'll expedite it.
:::

:::warning Token Security
Never lose or expose your API token. Treat it like a password:
- Store tokens securely (environment variables, secret managers)
- Never commit tokens to version control
- Don't share tokens in chat, email, or screenshots
- Rotate tokens immediately if compromised
:::

## What You'll Receive

After approval, your API key will be available in your project dashboard:

### Service Token

By default, **service tokens** provide access to:

- REST API for all endpoints
- WebSocket streaming for supported endpoints
- All available AI models
- Standard rate limits (750,000 tokens/minute)

### Project Owner Token

If you need to manage project settings, you'll also receive a **project owner token** that allows you to:

- Update project configuration
- Manage model access
- Configure interface settings
- Control project sharing
- Update knowledge base

See [Token Details](/tokens) for more information on token types.

## Learn More

For a walkthrough of the access process, see the **[API Access Guide](https://docs.google.com/presentation/d/1_OXB4p01HUbZZHeI-aTFEGfiRjYCFdx7/edit?slide=id.p1#slide=id.p1)**.

## Questions?

If you have questions about API access, contact:

- **Email**: aiacceleration@asu.edu or sjain238@asu.edu
- **Office Hours**: Monday-Friday, 9 AM - 5 PM MST

## After Receiving Access

Once you receive your token:

1. Review the [Token Details](/tokens) documentation
2. Explore the [API Endpoints](/endpoints/query)
3. Explore the [OpenAI-Compatible API](/openai-compatible) for easy integration
4. Check [Rate Limits](/limits)
5. Read about [Going Live](/going-live)

---

## Error Handling


## Error Response Format

All errors return a JSON object with an `error` key containing the error message:

```json
{
  "error": "Error message describing what went wrong"
}
```

## Rate Limit Error (Most Important)

If you receive a rate limit error, **wait 60 seconds** before making another request.

```json
{
  "error": "Rate limit exceeded"
}
```

### Rate Limits

- Rate limits are set per project in **tokens per minute**
- Each project has its own rate limit configuration
- To increase your rate limit, contact support: aiacceleration@asu.edu

### Handling Rate Limits

```python
import time
import requests

url = "https://api-main.aiml.asu.edu/query"
headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

def make_request_with_rate_limit_handling():
    try:
        response = requests.post(url, headers=headers, json={"query": "Hello"})
        data = response.json()

        if "error" in data and "rate limit" in data["error"].lower():
            print("Rate limit exceeded. Waiting 60 seconds...")
            time.sleep(60)
            # Retry the request
            response = requests.post(url, headers=headers, json={"query": "Hello"})
            return response.json()

        return data
    except Exception as e:
        print(f"Request failed: {e}")
        return None
```

## Common Errors

| Error | Description | Solution |
|-------|-------------|----------|
| `Invalid API key` | Missing or incorrect token | Check your token is valid and properly formatted |
| `Rate limit exceeded` | Too many requests | Wait 60 seconds before retrying |
| `Invalid request parameters` | Malformed request | Verify request matches API specification |
| `Model not found` | Invalid model name | Check [available models](/models) |
| Model not supported in agentic mode | Your project has External Tools (e.g. web search) enabled, which makes it agentic. Image and speech generation are not supported in agentic mode. | Send `"request_source": "override_params"` and `"agentic": false` on the request — see [Image](/endpoints/image) or [Speech](/endpoints/speech) |

## Agentic Mode and Multimodal Generation

Turning on **Enable External Tools** for a project (for example, web search) makes it an agentic experience. Image and speech generation models are not supported in agentic mode today.

You do not have to change your project settings to work around this. Add `"agentic": false` alongside `"request_source": "override_params"` to make a single request run in non-agentic mode:

```json
{
  "endpoint": "image",
  "request_source": "override_params",
  "agentic": false,
  "query": "Generate an image of a futuristic university campus at sunset",
  "model_provider": "gcp-deepmind",
  "model_name": "geminiflash2_5_image",
  "model_params": {
    "system_prompt": "Generate high-quality, photorealistic images."
  },
  "enable_history": false,
  "response_format": {"type": "json"}
}
```

On the [OpenAI-compatible API](/openai-compatible#createai-parameters-via-extra_body), send the same two fields through the SDK's `extra_body` parameter:

```python
response = client.chat.completions.create(
    model="gcp-deepmind/geminiflash2_5_image",
    messages=[{"role": "user", "content": "Generate an image of a futuristic university campus at sunset"}],
    extra_body={
        "request_source": "override_params",
        "agentic": False,
    },
)
```

Only use `agentic: false` for multimodal generation requests. Normal `query` calls should leave it out so your project's tools remain available.

## Support

For persistent errors or to request a rate limit increase:
- Email: aiacceleration@asu.edu

---

## Going Live


Checklist and best practices for deploying your application to production.

## Pre-Launch Checklist

### 1. Authentication

- [ ] Store tokens in environment variables
- [ ] Never commit tokens to version control
- [ ] Confirm no service token is reachable from the browser or a user's device
- [ ] Use a per-user [project web token](/sso-redirect) for anything user-facing
- [ ] Restrict project owner tokens to administrative tasks

:::danger Never put a service token in a custom application
A service token never expires and carries your project's entire quota. If it ships inside a custom app — a web frontend, a mobile app, a desktop tool, anything vibe-coded — every user of that app can extract it from network traffic or bundled source. Whoever has it can spend your whole quota, and you cannot tell which user made which call.

Service tokens belong on a server you control, where the token stays in your environment and never reaches the client.

For user-facing apps, use [ASU Sign-In](/sso-redirect) instead. Each signed-in user arrives with their own `projectWebToken` that identifies them and expires in 24 hours, so calls are attributable per user and a leaked token has a limited blast radius.
:::

### 2. Error Handling

- [ ] Implement retry logic with exponential backoff
- [ ] Handle all error status codes properly
- [ ] Set up error logging and monitoring
- [ ] Configure alerting for critical errors

### 3. Rate Limiting

- [ ] Understand your quota limits
- [ ] Implement request throttling
- [ ] Handle 429 responses gracefully
- [ ] Monitor usage metrics

### 4. Security

- [ ] Use HTTPS for all requests
- [ ] Validate and sanitize user inputs
- [ ] Implement proper CORS policies
- [ ] Set up API key rotation

### 5. Monitoring

- [ ] Track API response times
- [ ] Monitor error rates
- [ ] Set up usage dashboards
- [ ] Configure performance alerts

### 6. Testing

- [ ] Test all endpoints in staging
- [ ] Load test with production-like traffic
- [ ] Verify error handling
- [ ] Test failover scenarios

### 7. ASU Sign-In

Skip this section if your application has no end users of its own.

- [ ] Register your production redirect URL with the platform team
- [ ] Confirm the redirect URL is a complete HTTPS address, not a bare domain
- [ ] Register the specific path you need rather than the whole site
- [ ] Verify the generated login link lands users on the right page
- [ ] Read the `projectWebToken` once on page load, then clear it from the address bar
- [ ] Handle token expiry by sending the user back through the login link
- [ ] Distribute the correct login link to each audience if you registered several

See [ASU Sign-In & Redirect URLs](/sso-redirect) for the full flow and its troubleshooting table.

:::note Redirect URLs are registered by an admin
Project owners and editors can't add or change redirect URLs themselves. Request yours before launch day rather than during it.
:::

## Environment Configuration

### Development

```bash
# .env.development
VITE_ENV=dev
VITE_API_BASE_URL=https://api-dev-poc.aiml.asu.edu
VITE_DEV_TOKEN=dev_your_token_here
```

### Production

```bash
# .env.production (never commit!)
VITE_ENV=prod
VITE_API_BASE_URL=https://api-main.aiml.asu.edu
AIML_API_KEY=your_production_token_here
```

## Deployment Best Practices

### 1. Gradual Rollout

- Start with a small percentage of traffic
- Monitor error rates and performance
- Gradually increase traffic
- Have a rollback plan ready

### 2. Health Checks

Implement health checks to monitor API connectivity. Run this server-side, where the service token stays in your environment:

```python
import requests
import os

def health_check():
    try:
        response = requests.post(
            "https://api-main.aiml.asu.edu/query",
            headers={
                "Authorization": f"Bearer {os.environ['AIML_SERVICE_TOKEN']}",
                "Content-Type": "application/json"
            },
            json={"query": "test"},
            timeout=10
        )
        response.raise_for_status()
        return True
    except:
        return False
```

## Support

For production support:

- **Email**: aiacceleration@asu.edu
- **Escalation**: aiacceleration@asu.edu
- **Slack Channel**: #createai-community-hub (ASU ET Slack)

---

## ASU Sign-In & Redirect URLs

# Custom Redirect URLs with ASU Single Sign-On

Control where users land after signing in, and receive a token that identifies them.

## Overview

Redirect URLs let your project send signed-in users to a specific page instead of the default app page. When someone signs in through your project's login link, CreateAI verifies their ASU credentials and sends them to your registered redirect URL with a token attached.

You don't need to build a sign-in screen. CreateAI handles authentication end to end.

This is the recommended way to put a vibe-coded app in front of real ASU users: build your pages, register the page you want people to land on, then share the login link CreateAI gives you.

:::note What this does and doesn't do
Redirect URLs control **where** a signed-in user lands. They don't change **who** is allowed to sign in — your project's existing access rules still apply.
:::

## How it works

1. An admin registers one or more redirect URLs on your project and turns the feature on.
2. CreateAI generates a login link for each registered redirect URL.
3. You share the login link with your users.
4. A user opens the link and signs in with their ASURITE.
5. On success, the user lands on your redirect URL with a token attached.

:::info Who can register a redirect URL
Currently only an admin can do this. Project owners and editors can't register or change redirect URLs on their own — reach out to the platform team to get yours added.
:::

## Get your login link

CreateAI generates a ready-to-use login link for every registered redirect URL. For example:

```text
https://weblogin.asu.edu/cas/login?service=https://auth-main-poc.aiml.asu.edu/app/?aid=ExampleAppId123%26eid=ExampleEncodedProjectId789%26redirect=https://app-poc.aiml.asu.edu/example-project-page
```

Copy this link and share it directly — there's nothing to build or modify. When a user signs in successfully, they land on the redirect URL at the end of the link (here, `https://app-poc.aiml.asu.edu/example-project-page`) with a token attached to the address.

:::note
A redirect URL only works once an admin has added it to your project's registered redirect URL list. Once registered, its login link is generated automatically, and links for your other registered redirect URLs keep working unaffected.
:::

## Understand the token

After sign-in, your redirect URL receives a token as a query parameter:

```text
https://myapp.example.com/dashboard/?projectWebToken=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
```

| Property | Detail |
| --- | --- |
| Identifies | The signed-in user |
| Valid for | 24 hours |
| Renewal | None — send the user back through the login link for a new token |
| Also used for | Authenticating calls to your project's API on the user's behalf |

Read the token once when the page loads, store it the way you'd store a normal session, then remove it from the visible address bar.

## Use multiple redirect URLs

Your project isn't limited to one redirect URL. Register as many as you need — one per audience, course, app, or environment. Each gets its own login link.

| Redirect URL | Audience |
| --- | --- |
| `https://myapp.example.edu/student` | Students |
| `https://myapp.example.edu/instructor` | Instructors |
| `https://myapp.example.edu/admin` | Admins |

One redirect URL is set as the default, used when a login link doesn't specify one. Every other registered URL still works — each just has its own distinct login link. Distribute the matching link to each audience.

### Guidelines

- Register the exact page you need rather than a bare domain. A registered path such as `https://myapp.example.edu/student` permits that page and anything nested under it, like `/student/dashboard`. Registering a bare domain such as `https://myapp.example.edu` permits landing anywhere on that site.
- Sign-in can succeed even when a redirect URL isn't registered, but the user won't be sent anywhere afterward.
- Registering a new redirect URL has no effect on redirect URLs you've already registered.

:::warning
Each redirect URL must be a complete, working HTTPS address — for example, `https://myapp.example.edu/dashboard`.
:::

## Troubleshooting

| Status | Cause | Resolution |
| --- | --- | --- |
| 400 Redirect mismatch | The redirect URL isn't registered on the project | Confirm it's in your project's redirect list, and that you used a platform-generated login link |
| 401 Unauthorized | The user isn't authorized for this project | Grant the user access to the project |
| 403 Forbidden | The application is disabled | Contact platform support |
| 404 Not Found | No redirect URL is configured, or the project ID doesn't resolve | Confirm the feature is enabled and at least one redirect URL is registered |
| 503 Service Unavailable | Temporary maintenance | Retry after a short wait |

## Related

- [Token Details](/tokens) — how project tokens differ from service tokens
- [Going Live](/going-live) — production checklist before you share your link widely
- [Error Handling](/errors) — full list of API error responses

---

## OpenAI-Compatible API


CreateAI provides an OpenAI-compatible API, allowing you to use the [OpenAI SDK](https://platform.openai.com/docs/libraries) (Python, Node.js, etc.) or any OpenAI-compatible client to interact with all supported models.

:::caution Work in Progress
This API is a work in progress and support will increase in the future. Currently we only support chat completions and NOT responses API. If you are facing any issues, please reach out on Slack. Any feedback is appreciated!
:::

:::tip Why use the OpenAI-compatible API?
- Drop-in replacement — use existing OpenAI SDK code with minimal changes
- Supports chat completions, embeddings, streaming, and tool calls
- Works with any OpenAI-compatible library or framework (LangChain, LlamaIndex, etc.)
:::

## Base URL

Point your OpenAI client to the CreateAI base URL for your environment:

```
https://api-main-poc.aiml.asu.edu/v1
```

```
https://api-main-beta.aiml.asu.edu/v1
```

```
https://api-main.aiml.asu.edu/v1
```

## Authentication

Use your current CreateAI token as the API key. **Service tokens** is supported. See [Token Details](/tokens) for how to obtain and manage tokens.

Your current token will work with the OpenAI-compatible API, so no additional steps are needed if you already have access. Please reach out via slack on #createai-community-hub if you have any issues or questions about access.

:::info Token Types
- **Service Token**: Pass `"defaults"` as the `model` parameter to use your project's configured model and settings.
:::

## Model Format

Models are specified as `provider/model_name`. Pass `"defaults"` to use your project's configured model (service tokens only).

| Model | Format |
|-------|--------|
| GPT-4o | `openai/gpt4o` |
| GPT-4.1 | `openai/gpt4_1` |
| GPT-5 Mini | `openai/gpt5_mini` |
| GPT-5.3 Instant | `openai/gpt5_3-instant` |
| Claude 4 Opus | `aws/claude4_opus` |
| Claude 4 Sonnet | `aws/claude4_sonnet` |
| Claude 3 Haiku | `aws/claude3_haiku` |
| Gemini Pro 3 | `gcp-deepmind/geminipro3` |
| Gemini Pro 3.1 | `gcp-deepmind/geminipro3_1` |
| Gemma 4 31B IT | `asu-air/gemma4_31b_it` |

Use the [List Models](#list-models) endpoint to see all available models.

:::note
The Responses API is not currently supported. Use the Chat Completions API for all requests.
:::

## Quick Start

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",  # service token
)

# With a service token, use "defaults" to use your project's configured model
response = client.chat.completions.create(
    model="defaults", # or openai/gpt4o
    messages=[{"role": "user", "content": "Hello!"}],
)

print(response.choices[0].message.content)
```

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api-main-poc.aiml.asu.edu/v1",
  apiKey: "YOUR_CREATEAI_TOKEN", // service token
});

const response = await client.chat.completions.create({
  model: "openai/gpt4o",
  messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0].message.content);
```

```bash
curl -X POST https://api-main-poc.aiml.asu.edu/v1/chat/completions \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt4o",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'
```

## Endpoints

### Chat Completions

`POST /v1/chat/completions`

Generates a model response for the given conversation. Supports streaming, system prompts, tool calls, and all standard OpenAI chat completion parameters.

#### Request Body

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `model` | string | Yes | Model in `provider/model_name` format, or `"defaults"` to use project settings |
| `messages` | array | Yes | Array of message objects with `role` and `content` |
| `stream` | boolean | No | Enable streaming (`true`/`false`). Default: `false` |
| `temperature` | number | No | Sampling temperature (0–2) |
| `max_tokens` | number | No | Maximum tokens to generate |
| `top_p` | number | No | Nucleus sampling parameter |
| `frequency_penalty` | number | No | Frequency penalty (-2.0 to 2.0) |
| `presence_penalty` | number | No | Presence penalty (-2.0 to 2.0) |
| `tools` | array | No | List of tools the model may call |
| `tool_choice` | string | No | Controls tool usage (`auto`, `none`, etc.) |

#### Non-Streaming Example

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
)

response = client.chat.completions.create(
    model="openai/gpt4o",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of Arizona?"},
    ],
    temperature=0.7,
    max_tokens=256,
)

print(response.choices[0].message.content)
print(f"Usage: {response.usage}")
```

```bash
curl -X POST https://api-main-poc.aiml.asu.edu/v1/chat/completions \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is the capital of Arizona?"}
    ],
    "temperature": 0.7,
    "max_tokens": 256
  }'
```

#### Streaming Example

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
)

stream = client.chat.completions.create(
    model="openai/gpt4o",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
    if chunk.usage:
        print(f"\n\nUsage: {chunk.usage}")
```

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api-main-poc.aiml.asu.edu/v1",
  apiKey: "YOUR_CREATEAI_TOKEN",
});

const stream = await client.chat.completions.create({
  model: "openai/gpt4o",
  messages: [{ role: "user", content: "Tell me a story" }],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);
}
```

```bash
curl -N -X POST https://api-main-poc.aiml.asu.edu/v1/chat/completions \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt4o",
    "messages": [{"role": "user", "content": "Tell me a story"}],
    "stream": true
  }'
```

#### Tool Calls (Function Calling)

```python
from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a given city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "City name, e.g. 'Phoenix'",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                    },
                },
                "required": ["city"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="openai/gpt4o",
    messages=[{"role": "user", "content": "What's the weather in Phoenix?"}],
    tools=tools,
    tool_choice="auto",
)

message = response.choices[0].message
if message.tool_calls:
    for tool_call in message.tool_calls:
        args = json.loads(tool_call.function.arguments)
        print(f"Function: {tool_call.function.name}, Args: {args}")
else:
    print(message.content)
```

```bash
curl -X POST https://api-main-poc.aiml.asu.edu/v1/chat/completions \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt4o",
    "messages": [{"role": "user", "content": "What'\''s the weather in Phoenix?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get the current weather for a given city.",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {"type": "string"},
            "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
          },
          "required": ["city"]
        }
      }
    }],
    "tool_choice": "auto"
  }'
```

### List Models

`GET /v1/models`

Returns a list of all available models.

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
)

models = client.models.list()
for model in models.data:
    print(f"{model.id} (owned by: {model.owned_by})")
```

```bash
curl https://api-main-poc.aiml.asu.edu/v1/models \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN"
```

**Response:**

```json
{
  "object": "list",
  "data": [
    {
      "id": "openai/gpt4o",
      "object": "model",
      "created": 1716000000,
      "owned_by": "openai"
    },
    {
      "id": "aws/claude4_sonnet",
      "object": "model",
      "created": 1716000000,
      "owned_by": "aws"
    }
  ]
}
```

### Embeddings

`POST /v1/embeddings`

Generates embedding vectors for the given input text.

#### Request Body

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `model` | string | Yes | Model in `provider/model_name` format (e.g., `openai/te3s`, `openai/te3l`) |
| `input` | string or array | Yes | Text to embed |
| `encoding_format` | string | No | Format of the embeddings (`float` or `base64`) |

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
)

response = client.embeddings.create(
    model="openai/te3s",
    input="The quick brown fox jumps over the lazy dog",
)

print(f"Embedding dimension: {len(response.data[0].embedding)}")
print(f"Usage: {response.usage}")
```

```bash
curl -X POST https://api-main-poc.aiml.asu.edu/v1/embeddings \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/te3s",
    "input": "The quick brown fox jumps over the lazy dog"
  }'
```

**Response:**

```json
{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.0023064255, -0.009327292, ...]
    }
  ],
  "model": "openai/te3s",
  "usage": {
    "prompt_tokens": 9,
    "total_tokens": 9
  }
}
```

## Optional Headers

Any CreateAI specific payload can be passed as headers for additional functionality:

| Header | Description |
|--------|-------------|
| `session_id` | Session identifier for conversation tracking |
| `project_id` | Project identifier |
| `enable_history` | Enable conversation history (`true`/`false`) |
| `enable_search` | Enable knowledge base search (`true`/`false`) |

NOTE: These headers are optional and can be used to enhance functionality but are not required for basic API usage. The headers override project-level defaults when provided.

## CreateAI Parameters via `extra_body`

The OpenAI SDK does not know about CreateAI specific parameters, but it does let you merge extra fields into the request body. In Python, pass them through `extra_body`; the values are sent as top-level JSON keys alongside `model` and `messages`.

```python
response = client.chat.completions.create(
    model="defaults",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={
        "request_source": "override_params",
        "agentic": False,
    },
)
```

In Node.js there is no `extra_body` — pass the fields directly in the request object (TypeScript users may need a cast, since the properties are not in the SDK's types):

```javascript
const response = await client.chat.completions.create({
  model: "defaults",
  messages: [{ role: "user", content: "Hello!" }],
  request_source: "override_params",
  agentic: false,
});
```

With cURL, add them as normal top-level JSON keys:

```bash
curl -X POST https://api-main-poc.aiml.asu.edu/v1/chat/completions \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "defaults",
    "messages": [{"role": "user", "content": "Hello!"}],
    "request_source": "override_params",
    "agentic": false
  }'
```

:::danger Agentic projects and multimodal generation
If your project has **Enable External Tools** turned on (for example, web search), your project runs as an *agentic* experience. Image and speech generation models are **not supported** in agentic mode today, so those requests will fail.

You do not need to change your project settings. Send `request_source: "override_params"` and `agentic: false` through `extra_body` to run that single request in non-agentic mode:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
)

response = client.chat.completions.create(
    model="gcp-deepmind/geminiflash2_5_image",
    messages=[{"role": "user", "content": "Generate an image of a futuristic university campus at sunset"}],
    extra_body={
        "request_source": "override_params",
        "agentic": False,
    },
)
```

`agentic: false` switches your project settings to non-agentic for that request only — it does not change the project. Use it only for multimodal generation calls; leave it out of normal chat completions so your project's tools stay available.
:::

## Environment-Specific Base URLs

| Environment | Base URL |
|-------------|----------|
| Production | `https://api-main.aiml.asu.edu/v1` |
| Beta | `https://api-main-beta.aiml.asu.edu/v1` |
| POC | `https://api-main-poc.aiml.asu.edu/v1` |

## Framework Integration

The OpenAI-compatible API works with popular frameworks out of the box:

### LangChain

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
    model="openai/gpt4o",
)

response = llm.invoke("What is the capital of Arizona?")
print(response.content)
```

### LlamaIndex

```python
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    api_base="https://api-main-poc.aiml.asu.edu/v1",
    api_key="YOUR_CREATEAI_TOKEN",
    model="openai/gpt4o",
)

response = llm.complete("What is the capital of Arizona?")
print(response.text)
```

## Support

Need help?

- Check [Error Handling](/errors)
- Review [Token Details](/tokens)
- Slack: #createai-community-hub

---

## Rate Limits

# Rate Limits & Quotas

Understanding rate limits helps you build applications that use the API efficiently and reliably.

## Rate Limit Tiers

Every project by default is rate limited to 750,000 tokens per minute. Rate limits are enforced based on your token type:

| Token Type | Tokens/Minute |
|------------|---------------|
| Service    | 750,000       |

### 429 Response

When you exceed the rate limit, you'll receive an error response like this:

```json
{
  "error": "Rate limit exceeded..."
}
```

### Best Practices

#### 1. Exponential Backoff

```python
import time
import random
import requests

url = "https://api-main-poc.aiml.asu.edu/query"
headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

def make_request_with_backoff(max_retries=5):
    for attempt in range(max_retries):
        try:
            response = requests.post(
                url, headers=headers,
                json={"query": "Hello"}, timeout=30
            )
            response.raise_for_status()
            return response.json()
        except requests.exceptions.HTTPError as e:
            if e.response.status_code == 429 and attempt < max_retries - 1:
                wait_time = (2 ** attempt) + random.uniform(0, 1)
                time.sleep(wait_time)
            else:
                raise
```

#### 2. Request Throttling

```python
from ratelimit import limits, sleep_and_retry
import requests

@sleep_and_retry
@limits(calls=60, period=60)  # 60 calls per minute
def call_api(query):
    return requests.post(
        "https://api-main-poc.aiml.asu.edu/query",
        headers=headers,
        json={"query": query}
    )
```

#### 3. Batch Processing

```python
import requests

url = "https://api-main-poc.aiml.asu.edu/embeddings"
headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}

# Process texts individually
for text in texts:
    response = requests.post(url, headers=headers, json={
        "endpoint": "embeddings",
        "model_name": "text-embedding-3-small",
        "model_provider": "openai",
        "query": text
    })
```

## Token Limits

Different models have different context window limits:

| Model | Max Tokens | Context Window |
|-------|------------|----------------|
| GPT-4 | 8,192 | 8,192 |
| GPT-4-32k | 32,768 | 32,768 |
| GPT-3.5-Turbo | 4,096 | 4,096 |
| GPT-3.5-Turbo-16k | 16,384 | 16,384 |

Find out more about token limits in the [Model Documentation](/models).

## Increasing Limits

Need higher limits? Contact us:

1. Explain your use case
2. Provide usage projections
3. Describe your application
4. Request specific limit increases

**Email**: aiacceleration@asu.edu with subject "Rate Limit Increase Request"

## Best Practices Summary

1. Implement exponential backoff
2. Use request throttling
3. Batch requests when possible
4. Cache responses appropriately
5. Track usage metrics
6. Plan for peak traffic
7. Test rate limit handling

## Cost Optimization

### Tips to Reduce Usage

- **Cache responses** for repeated queries
- **Use appropriate models** (don't use GPT-4 when GPT-3.5 suffices)
- **Optimize prompts** to reduce token usage
- **Implement deduplication** to avoid redundant requests
- **Set max_tokens** appropriately
- **Use streaming** for better user experience without extra costs

## Support

Questions about rate limits?

- Review the [Error Handling](/errors) guide
- Check [Token Details](/tokens)
- **Email**: aiacceleration@asu.edu or sjain238@asu.edu

---

## Connect

# Connection Methods

CreateAI supports multiple connection methods to suit different use cases.

## REST API

The primary method for interacting with the platform is through our REST API.

### Base URL

```
https://api-main-poc.aiml.asu.edu/
```

```
https://api-main-beta.aiml.asu.edu/
```

```
https://api-main.aiml.asu.edu/
```

### Making Requests

:::info Token Types
- **Service Token**: Only `query` is required — your project settings determine the model.
:::

```python
import requests

url = "https://api-main-poc.aiml.asu.edu/query"
headers = {
    "Authorization": "Bearer YOUR_SERVICE_TOKEN",
    "Content-Type": "application/json"
}

json_payload = {
    "query": "what is your name?"
}

response = requests.post(url, headers=headers, json=json_payload)
response_json = response.json()
result = response_json.get("response", "")
print(result)
```

```bash
# With service token (project settings determine model)
curl https://api-main-poc.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "what is your name?"
  }'

```

## WebSocket (Realtime API)

For real-time streaming and bidirectional communication, use the WebSocket endpoint.

### Connection URL

```
wss://apiws-main-poc.aiml.asu.edu
```

```
wss://apiws-main-beta.aiml.asu.edu
```

```
wss://apiws-main.aiml.asu.edu
```

### Authentication

Authenticate by passing your API token as a query parameter in the WebSocket URL and as a Bearer token in the headers.

### Example

```python
import websocket
import json

token = "YOUR_CREATEAI_TOKEN"
wss_url = f"wss://apiws-main-poc.aiml.asu.edu/?access_token={token}"

headers = {
    "Authorization": f"Bearer {token}",
}

payload = {
    "action": "query",
    "model_name": "gpt4o",
    "model_provider": "openai",
    "query": "Hello"
}

def on_message(ws, message):
    print(f"Received message: {message}")
    try:
        data = json.loads(message)
        if 'connection_id' in data:
            print(f"Connection ID: {data['connection_id']}")
    except json.JSONDecodeError:
        print("Non-JSON message received")

def on_error(ws, error):
    print(f"Error: {error}")

def on_close(ws, close_status_code, close_msg):
    print(f"Connection closed with status: {close_status_code}, message: {close_msg}")

def on_open(ws):
    print("Connection opened")
    ws.send(json.dumps(payload))

ws = websocket.WebSocketApp(
    wss_url,
    on_message=on_message,
    on_error=on_error,
    on_close=on_close,
    header=headers
)

ws.on_open = on_open
ws.run_forever()
```

### Use Cases

- Real-time chat applications
- Streaming audio/video processing
- Interactive voice assistants
- Live transcription
- Collaborative tools

## Streaming

For endpoints that support streaming, you can receive responses incrementally using the WebSocket connection:

```python
import websocket
import json

token = "YOUR_CREATEAI_TOKEN"
wss_url = f"wss://apiws-main-poc.aiml.asu.edu/?access_token={token}"

headers = {
    "Authorization": f"Bearer {token}",
}

payload = {
    "action": "query",
    "model_name": "gpt4o",
    "model_provider": "openai",
    "query": "Tell me a story"
}

def on_message(ws, message):
    data = json.loads(message)
    if "response" in data:
        print(data["response"], end="")

def on_open(ws):
    ws.send(json.dumps(payload))

ws = websocket.WebSocketApp(
    wss_url,
    on_message=on_message,
    on_open=on_open,
    header=headers
)

ws.run_forever()
```

## Environment-Specific URLs

Different environments are available:

| Environment | Base URL |
|-------------|----------|
| Production | `https://api-main.aiml.asu.edu/` |
| Beta | `https://api-main-beta.aiml.asu.edu/` |
| POC | `https://api-main-poc.aiml.asu.edu/` |
| Development | `https://api-dev-poc.aiml.asu.edu/` |

## Connection Best Practices

### 1. Use Connection Pooling

```python
import requests

# Create a session for connection pooling
session = requests.Session()
session.headers.update({
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
})

# Reuse the session for multiple requests
response = session.post(
    "https://api-main-poc.aiml.asu.edu/query",
    json={"query": "Hello"}
)
```

### 2. Set Timeouts

```python
import requests

response = requests.post(
    "https://api-main-poc.aiml.asu.edu/query",
    headers=headers,
    json=json_payload,
    timeout=30  # 30 seconds
)
```

### 3. Handle Network Errors

```python
import requests

try:
    response = requests.post(url, headers=headers, json=json_payload, timeout=30)
    response.raise_for_status()
    result = response.json()
except requests.exceptions.ConnectionError as e:
    print(f"Connection error: {e}")
except requests.exceptions.Timeout as e:
    print(f"Timeout error: {e}")
except requests.exceptions.HTTPError as e:
    print(f"HTTP error: {e}")
```

## Troubleshooting

### Connection Refused

- Check your network connectivity
- Verify the base URL is correct
- Ensure firewall allows outbound HTTPS

### SSL Certificate Errors

- Update your SSL certificates
- Check system time is correct
- Verify your SDK version is up to date

### Timeout Errors

- Increase timeout values
- Check for network latency
- Try with a simpler request

## Support

Need help connecting?

- Check [Error Handling](/errors)
- Review [Authentication](/tokens)
- Contact: aiacceleration@asu.edu