# Query

> Send natural language queries to LLMs with optional knowledge base search, history tracking, and multimodal support.

# Query Endpoint

Send natural language queries to LLMs with optional knowledge base search, conversation history, semantic caching, and advanced RAG capabilities. The Query endpoint supports multiple model providers and can be accessed via REST or WebSocket connections.

## Overview

The Query endpoint is the primary interface for sending prompts to large language models. It supports multiple providers and models, optional knowledge base search (RAG), conversation history, tool calling, structured output, and prompt enhancement. Queries can be sent over REST or WebSocket connections.

:::info Token Types
- **Developer Token:** Use `collection` in search_params for knowledge base search
- **Service Token:** Use `project_id` parameter at root level
:::

## Base URL

| Protocol | Method | Route | Final URL |
|----------|--------|-------|-----------|
| **REST** | `POST` | `query` | `https://api-main.aiml.asu.edu/query` |
| **WebSocket** | — | `query` | `wss://apiws-main.aiml.asu.edu?access_token=YOUR_CREATEAI_TOKEN` |

## Endpoint

| Protocol | Method | Route |
|----------|--------|-------|
| **REST** | `POST` | `base_url/query` |
| **WebSocket** | — | `query` |

## Request Parameters

### Required

| Parameter | Type | Description |
|-----------|------|-------------|
| `endpoint` | string | Must be `"query"` |
| `action` | string | Must be `"query"` for websocket connections, ignored for REST |
| `request_source` | string | Must be `"override_params"` if using project service token and you would like to send your own parameters |
| `query` | string | The natural language query or prompt |

### Optional

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `model_provider` | string | Project default | The model provider (e.g., `"openai"`, `"aws"`). Can be overridden per request. |
| `model_name` | string | Project default | The model name (e.g., `"gpt4o"`, `"claude3_5_sonnet"`, `"nova-micro"`). Can be overridden per request. |
| `project_id` | string | — | Required when using developer token |
| `session_id` | string | — | Session identifier. Used with `enable_history` to maintain context across requests and `chat_upload` if uploading files for in-context use. |
| `query_id` | string | — | Unique query identifier only required when accessing in-context files. |
| `model_params` | object | Project default | Model configuration parameters ([see below](#model_params-object)). Can be overridden per request. |
| `enable_search` | boolean | Project default | Enable knowledge base search / RAG. Can be overridden per request. |
| `search_params` | object | Project default | Search configuration ([see below](#search_params-object-when-enable_search-is-true)). Required when `enable_search` is true. Can be overridden per request. |
| `enable_history` | boolean | Project default | Enable conversation history tracking. Pair with `session_id` to maintain context across multiple requests. Can be overridden per request. |
| `history` | array | — | Previous conversation history messages. This is only needed if you want to pass your custom history. |
| `response_format` | object | — | Response format configuration ([see below](#response_format-object)). Supported: `{"type": "json"}`|
| `enhance_prompt` | object | Project default | Prompt enhancement options ([see below](#enhance_prompt-object)). Can be overridden per request. |
| `eval_params` | object | — | Evaluation parameters ([see below](#eval_params-object)) |
| `chat_upload` | object | — | Upload files for in-context use ([see below](#chat_upload-object)) |

### `model_params` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `temperature` | float | Project default | Controls randomness (0.0–2.0). Lower = more deterministic. Can be overridden per request. |
| `system_prompt` | string | Project default | System-level instructions for the model. Can be overridden per request. |
| `top_p` | float | Project default | Nucleus sampling parameter (0.0–1.0). Can be overridden per request. |
| `top_k` | integer | — | Top-k sampling parameter |
| `thinking_level` | string | — | Thinking/reasoning depth: `"LOW"`, `"MEDIUM"`, `"HIGH"` |
| `tools` | array | — | Function/tool definitions for tool calling (see [Tool Calling](#tool-calling-function-calling) section) |
| `response_format` | object | — | Structured output schema (JSON Schema format, see [Structured Output](#structured-output-json-schema)) This is different from the `response_format` object used in the main query parameters. |

### `response_format` Object

| Parameter | Type | Description |
|-----------|------|-------------|
| `type` | string | **(Required)** Response format type: `"text"`, `"json"`, or `"openai"` |

### `search_params` Object (when enable_search is true)

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `collection` | string | — | Collection ID. Required when using developer token. |
| `retrieval_type` | string | Project default | Type of retrieval: `"chunk"`, `"document"`, or `"neighbor"`. Can be overridden per request. |
| `top_k` | integer | Project default | Number of search results to retrieve. Can be overridden per request. |
| `output_fields` | array | Project default | Fields to include in search results. Can be overridden per request. |
| `tags` | array | — | Filter by document tags |
| `source_names` | array | — | Filter by specific source file names |
| `expr` | string | — | Filter expression for metadata |
| `rerank` | boolean | `false` | Enable reranking on search results |
| `reranker_model` | string | — | Reranker model (e.g., `"cohere_rerank-3_5"`, `"amazon_rerank"`) |
| `reranker_provider` | string | — | Reranker provider (e.g., `"aws"`) |
| `top_n` | integer | — | Number of results after reranking |
| `advanced_rag` | string | `"false"` | Enable advanced RAG features (`"true"` / `"false"`) |
| `prompt_mode` | string | Project default | Prompt mode: `"restricted"`, `"unrestricted"`, or `"custom"`. Can be overridden per request. |
| `search_prompt` | string | Project default | Custom search prompt template (use `{data}` and `{query}` placeholders). Can be overridden per request. |

### `enhance_prompt` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `timezone` | string | Project default | Timezone for time-aware prompts (e.g., `"MST"`). Can be overridden per request. |
| `time` | boolean | Project default | Include current time in prompt. Can be overridden per request. |
| `date` | boolean | Project default | Include current date in prompt. Can be overridden per request. |
| `verbosity` | string | Project default | Response verbosity: `"brief"`, `"normal"`, `"detailed"`. Can be overridden per request. |

### `eval_params` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `context_utilization` | boolean | `false` | Enable context utilization evaluation |
| `prompt_guard` | boolean | `false` | Enable prompt guard evaluation |

### `chat_upload` Object

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `images` | array | — | List of uploaded image filenames |
| `docs` | array | — | List of uploaded document filenames |
| `audios` | array | — | List of uploaded audio filenames |

## Basic Query Example (Developer Token)

```python
import requests

headers = {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
}
dev_url = "https://api-main.aiml.asu.edu" # Replace with the appropriate url of your environment (prod, beta, poc)
endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is artificial intelligence?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "You are a helpful assistant.",
        "top_p": 0.01
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "What is artificial intelligence?",
  model_params: {
    temperature: 0.1,
    system_prompt: "You are a helpful assistant.",
    top_p: 0.01
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is artificial intelligence?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "You are a helpful assistant.",
      "top_p": 0.01},
    "response_format": {"type": "json"}
  }'
```

### Response

```json
{
  "response": {
    "response": "AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines designed to think and learn like humans. It encompasses various technologies, including machine learning, natural language processing, and robotics.",
    "metadata": {
      "sources": {},
      "query_id": "ada15f24b0aa4d9fac680c9f15b9f7e4",
      "usage_metric": {
        "input_token_count": 65,
        "output_token_count": 39,
        "input_token_cost": 0.000325,
        "output_token_cost": 0.000585,
        "input_token_details": {
          "user_query": 4,
          "incontext_text": 0,
          "system_prompt": 60,
          "knowledge_base": 0,
          "conversation_history": 1
        },
        "output_token_details": {
          "output_response": 39
        },
        "input_token_cost_details": {
          "user_query": 2e-05,
          "incontext_text": 0.0,
          "system_prompt": 0.0003,
          "knowledge_base": 0.0,
          "conversation_history": 5e-06
        },
        "output_token_cost_details": {
          "output_response": 0.000585
        },
        "total_token_count": 104,
        "total_token_cost": 0.00091
      }
    }
  },
  "file": "main.py",
  "query_id": "ada15f24b0aa4d9fac680c9f15b9f7e4"
}
```

## Query with Knowledge Base Search (RAG)

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "Why do generative models keep making up stuff?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "Please answer the question only based on the information from the knowledge base. Do not answer if there is no relevant information in the knowledge base.",
        "top_p": 0.01,
        "top_k": 2
    },
    "enable_search": True,
    "search_params": {
        "db_type": "opensearch",
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "top_k": 5,
        "retrieval_type": "chunk",
        "output_fields": [
            "source_name",
            "page_number",
            "content",
            "tags"
        ]
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "Why do generative models keep making up stuff?",
  model_params: {
    temperature: 0.1,
    system_prompt: "Please answer the question only based on the information from the knowledge base.",
    top_p: 0.01,
    top_k: 2
  },
  enable_search: true,
  search_params: {
    db_type: "opensearch",
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    top_k: 5,
    retrieval_type: "chunk",
    output_fields: ["source_name", "page_number", "content", "tags"]
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "Why do generative models keep making up stuff?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "Please answer the question only based on the information from the knowledge base.",
      "top_p": 0.01,
      "top_k": 2
    },
    "enable_search": true,
    "search_params": {
      "db_type": "opensearch",
      "collection": "a1a5059652d449c9a9c21c2388d42da5",
      "top_k": 5,
      "retrieval_type": "chunk",
      "output_fields": ["source_name", "page_number", "content", "tags"]},
    "response_format": {"type": "json"}
  }'
```

### Response

```json
{
  "response": {
    "response": "Generative models hallucinate because they are trained to predict statistically likely next tokens rather than retrieve verified facts. The model generates plausible-sounding text based on patterns learned during training, but it does not have a mechanism to verify the accuracy of its outputs against a ground truth source.",
    "metadata": {
      "sources": {
        "source_name": "Diffusion World Model.pdf",
        "page_number": 3,
        "content": "...",
        "tags": ["research"]
      },
      "query_id": "b3c9e8a1f2d74a5eb1234567890abcde",
      "usage_metric": {
        "input_token_count": 512,
        "output_token_count": 85,
        "total_token_count": 597,
        "total_token_cost": 0.00245
      }
    }
  },
  "file": "main.py",
  "query_id": "b3c9e8a1f2d74a5eb1234567890abcde"
}
```

## Query with RAG and Reranking

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What are the latest advances in reinforcement learning?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "Answer based on the knowledge base.",
        "top_p": 0.01
    },
    "enable_search": True,
    "search_params": {
        "db_type": "opensearch",
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "top_k": 5,
        "retrieval_type": "chunk",
        "output_fields": ["source_name", "page_number", "content"],
        "rerank": True,
        "reranker_model": "cohere_rerank-3_5",
        "reranker_provider": "aws",
        "top_n": 3
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "What are the latest advances in reinforcement learning?",
  model_params: {
    temperature: 0.1,
    system_prompt: "Answer based on the knowledge base.",
    top_p: 0.01
  },
  enable_search: true,
  search_params: {
    db_type: "opensearch",
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    top_k: 5,
    retrieval_type: "chunk",
    output_fields: ["source_name", "page_number", "content"],
    rerank: true,
    reranker_model: "cohere_rerank-3_5",
    reranker_provider: "aws",
    top_n: 3
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What are the latest advances in reinforcement learning?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "Answer based on the knowledge base.",
      "top_p": 0.01
    },
    "enable_search": true,
    "search_params": {
      "db_type": "opensearch",
      "collection": "a1a5059652d449c9a9c21c2388d42da5",
      "top_k": 5,
      "retrieval_type": "chunk",
      "output_fields": ["source_name", "page_number", "content"],
      "rerank": true,
      "reranker_model": "cohere_rerank-3_5",
      "reranker_provider": "aws",
      "top_n": 3
    },
    "response_format": {"type": "json"}
  }'
```

## Query with Project ID (Service Token)

```python
import requests
from config import headers, dev_url

endpoint = "/query"

# Use project_id with query action for project service token
payload = {
    "action": "query",
    "query": "Explain what this document is about.",
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  query: "Explain what this document is about.",
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_SERVICE_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "query": "Explain what this document is about.",
    "response_format": {"type": "json"}
  }'
```

## Query with Chat Upload

Upload documents, images, or audio files for in-context querying.

:::tip Upload Files First
Before using chat upload in queries, you must first upload your files using the [Chat Upload endpoint](/endpoints/manage-project#chat-upload). This returns a `query_id` and `session_id` that you'll use in the query below.
:::

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "session_id": "sessionid1234", # query id you sent during chat upload
    "query_id": "query_id_returned", # This is the query_id returned from the chat upload response
    "project_id": "cfbdd896380c4d2ba2aa319da9f9eba6",
    "query": "Explain what this CreateAI Chat.pdf is?",
    "response_format": {"type": "json"},
    "search_params": {
        "output_fields": ["source_name", "page_number", "tags", "url"]
    },
    "chat_upload": {
        "images": [],
        "docs": ["CreateAI Chat.pdf"],
        "audios": []
    }
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
    action: "query",
    session_id: "sessionid1234", // query id you sent during chat upload
    query_id: "query_id_returned", // This is the query_id returned from the chat upload response
    project_id: "cfbdd896380c4d2ba2aa319da9f9eba6",
    query: "Explain what this CreateAI Chat.pdf is?",
  response_format: { type: "json" },
  search_params: {
    output_fields: ["source_name", "page_number", "tags", "url"]
  },
  chat_upload: {
    images: [],
    docs: ["CreateAI Chat.pdf"],
    audios: []
  },
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_SERVICE_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_SERVICE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "session_id": "sessionid1234", # query id you sent during chat upload
    "query_id": "query_id_returned", # This is the query_id returned from the chat upload response
    "project_id": "cfbdd896380c4d2ba2aa319da9f9eba6",
    "query": "Explain what this CreateAI Chat.pdf is?",
    "response_format": {"type": "json"},
    "search_params": {
      "output_fields": ["source_name", "page_number", "tags", "url"]
    },
    "chat_upload": {
      "images": [],
      "docs": ["CreateAI Chat.pdf"],
      "audios": []
    }
  }'
```

## Query with History and Prompt Enhancement

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "session_id": "session_id_here",
    "query": "Can you elaborate on that?",
    "model_params": {
        "temperature": 0.1,
        "system_prompt": "You are a helpful assistant.",
        "top_p": 0.01
    },
    "enable_history": True,
    "history": [],
    "enhance_prompt": {
        "timezone": "MST",
        "time": True,
        "date": True,
        "verbosity": "brief"
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  session_id: "session_id_here",
  query: "Can you elaborate on that?",
  model_params: {
    temperature: 0.1,
    system_prompt: "You are a helpful assistant.",
    top_p: 0.01
  },
  enable_history: true,
  history: [],
  enhance_prompt: {
    timezone: "MST",
    time: true,
    date: true,
    verbosity: "brief"
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "session_id": "session_id_here",
    "query": "Can you elaborate on that?",
    "model_params": {
      "temperature": 0.1,
      "system_prompt": "You are a helpful assistant.",
      "top_p": 0.01
    },
    "enable_history": true,
    "history": [],
    "enhance_prompt": {
      "timezone": "MST",
      "time": true,
      "date": true,
      "verbosity": "brief"
    },
    "response_format": {"type": "json"}
  }'
```

## Query with Custom Search Prompt

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "aws",
    "model_name": "nova-micro",
    "enable_search": True,
    "query": "What programs does ASU offer?",
    "search_params": {
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "output_fields": ["content"],
        "prompt_mode": "custom",
        "search_prompt": (
            "Your role is to support students, researchers, and professors at ASU "
            "by addressing their questions comprehensively. "
            "Use the Knowledge Base at your discretion to formulate responses. "
            "Knowledge Base: {data} Question: {query}"
        )
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "aws",
  model_name: "nova-micro",
  enable_search: true,
  query: "What programs does ASU offer?",
  search_params: {
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    output_fields: ["content"],
    prompt_mode: "custom",
    search_prompt:
      "Your role is to support students, researchers, and professors at ASU " +
      "by addressing their questions comprehensively. " +
      "Use the Knowledge Base at your discretion to formulate responses. " +
      "Knowledge Base: {data} Question: {query}"
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "aws",
    "model_name": "nova-micro",
    "enable_search": true,
    "query": "What programs does ASU offer?",
    "search_params": {
      "collection": "a1a5059652d449c9a9c21c2388d42da5",
      "output_fields": ["content"],
      "prompt_mode": "custom",
      "search_prompt": "Your role is to support students. Knowledge Base: {data} Question: {query}"
    },
    "response_format": {"type": "json"}
  }'
```

## Tool Calling (Function Calling)

Enable the model to call external functions/tools by defining them in `model_params.tools`. Follow this format as per OpenAI Function Calling guidelines. The model will respond with the function call and arguments in a structured format for your application to execute and return results.

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is the weather in New York?",
    "model_params": {
        "temperature": 0.1,
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "description": "Get current temperature for a given location.",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "location": {
                                "type": "string",
                                "description": "City and country e.g. Bogotá, Colombia"
                            }
                        },
                        "required": ["location"],
                        "additionalProperties": False
                    },
                    "strict": True
                }
            }
        ]
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "What is the weather in New York?",
  model_params: {
    temperature: 0.1,
    tools: [
      {
        type: "function",
        function: {
          name: "get_weather",
          description: "Get current temperature for a given location.",
          parameters: {
            type: "object",
            properties: {
              location: {
                type: "string",
                description: "City and country e.g. Bogotá, Colombia"
              }
            },
            required: ["location"],
            additionalProperties: false
          },
          strict: true
        }
      }
    ]
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

```bash
curl -X POST https://api-main.aiml.asu.edu/query \
  -H "Authorization: Bearer YOUR_CREATEAI_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "What is the weather in New York?",
    "model_params": {
      "temperature": 0.1,
      "tools": [
        {
          "type": "function",
          "function": {
            "name": "get_weather",
            "description": "Get current temperature for a given location.",
            "parameters": {
              "type": "object",
              "properties": {
                "location": {
                  "type": "string",
                  "description": "City and country e.g. New York, USA"
                }
              },
              "required": ["location"],
              "additionalProperties": false
            },
            "strict": true
          }
        }
      ]
    },
    "response_format": {"type": "json"}
  }'
```

## Structured Output (JSON Schema)

Request structured JSON output by specifying a JSON schema in `model_params.response_format`.

```python
import requests
from config import headers, dev_url

endpoint = "/query"

payload = {
    "action": "query",
    "model_provider": "openai",
    "model_name": "gpt4o",
    "query": "Solve 2x + 5 = 15 step by step",
    "model_params": {
        "temperature": 0.1,
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "math_response",
                "strict": True,
                "schema": {
                    "type": "object",
                    "properties": {
                        "steps": {
                            "type": "array",
                            "items": {
                                "type": "object",
                                "properties": {
                                    "explanation": {"type": "string"},
                                    "output": {"type": "string"}
                                },
                                "required": ["explanation", "output"],
                                "additionalProperties": False
                            }
                        },
                        "final_answer": {"type": "string"}
                    },
                    "required": ["steps", "final_answer"],
                    "additionalProperties": False
                }
            }
        }
    },
    "response_format": {"type": "json"}
}

url = dev_url + endpoint
response = requests.post(url, json=payload, headers=headers)
print(response.json())
```

```javascript
const BASE_URL = "https://api-main.aiml.asu.edu"; // Replace with the appropriate url of your environment (prod, beta, poc)

const payload = {
  action: "query",
  model_provider: "openai",
  model_name: "gpt4o",
  query: "Solve 2x + 5 = 15 step by step",
  model_params: {
    temperature: 0.1,
    response_format: {
      type: "json_schema",
      json_schema: {
        name: "math_response",
        strict: true,
        schema: {
          type: "object",
          properties: {
            steps: {
              type: "array",
              items: {
                type: "object",
                properties: {
                  explanation: { type: "string" },
                  output: { type: "string" }
                },
                required: ["explanation", "output"],
                additionalProperties: false
              }
            },
            final_answer: { type: "string" }
          },
          required: ["steps", "final_answer"],
          additionalProperties: false
        }
      }
    }
  },
  response_format: { type: "json" }
};

const response = await fetch(`${BASE_URL}/query`, {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_CREATEAI_TOKEN",
    "Content-Type": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(data);
```

## Evaluation Parameters

Enable evaluation metrics on query responses by including `eval_params`.

```python
payload = {
    # ... other query parameters ...
    "eval_params": {
        "context_utilization": True,   # Evaluate how well the response uses provided context
    }
}
```

## Supported Transports

| Body Type | REST | WebSocket |
|-----------|------|-----------|
| `query` | Yes | Yes |
| `search` | Yes | No |
| `vision` | Yes | Yes |
| `audio` | Yes | Yes |
| `speech` | Yes | Yes |
| `image` | Yes | No |
| `rerank` | Yes | Yes |
| `realtime` | No | Yes |

## Response Format

The response contains a `response` object with the following fields:

| Field | Type | Description |
|-------|------|-------------|
| `response` | string | The model's generated response text |
| `metadata` | object | Metadata about the query execution |
| `metadata.sources` | object | Source documents used (when search is enabled) |
| `metadata.query_id` | string | Unique identifier for this query |
| `metadata.usage_metric` | object | Token usage and cost breakdown |
| `file` | string | Handler file name |
| `query_id` | string | Unique query identifier |

### usage_metric Object

| Field | Type | Description |
|-------|------|-------------|
| `input_token_count` | integer | Total input tokens consumed |
| `output_token_count` | integer | Total output tokens generated |
| `input_token_cost` | float | Cost of input tokens |
| `output_token_cost` | float | Cost of output tokens |
| `total_token_count` | integer | Total tokens (input + output) |
| `total_token_cost` | float | Total cost |
| `input_token_details` | object | Breakdown: `user_query`, `incontext_text`, `system_prompt`, `knowledge_base`, `conversation_history` |
| `output_token_details` | object | Breakdown: `output_response` |

## Use Cases

- **Conversational AI** - Build chatbots and virtual assistants with history tracking
- **RAG Applications** - Query LLMs with knowledge base context for grounded responses
- **Document Q&A** - Upload and query documents directly via chat upload
- **Tool-Augmented Queries** - Enable LLMs to call external functions and APIs
- **Structured Data Extraction** - Extract structured JSON from unstructured text using JSON schemas
- **Custom Prompt Engineering** - Use custom search prompts with `prompt_mode: "custom"` for domain-specific RAG

:::warning Important
- Developer tokens must use `collection` in search_params when `enable_search` is true
- Service tokens must use `project_id` at the root level
- Use `source_names` (plural) in search_params, not `source_name`
- The `rerank`, `reranker_model`, and `reranker_provider` fields must all be set together for reranking
- When using `prompt_mode: "custom"`, include `{data}` and `{query}` placeholders in your `search_prompt`
:::

---

Stream real-time audio conversations with LLMs. Supports voice input/output, knowledge base search, and reranking. Available via **WebSocket only**.

### Endpoint

```
WebSocket route: query  (WebSocket only)
```

### Request Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `endpoint` | string | Yes | Must be `"realtime"` |
| `query` | string | Yes | The text query or instruction |
| `audio_file` | string | Yes | Base64-encoded audio file content |
| `model_provider` | string | Yes | The model provider (e.g., `"openai"`) |
| `model_name` | string | Yes | The model name |
| `session_id` | string | No | Session identifier |
| `project_id` | string | No | Project identifier |
| `model_params` | object | No | Model configuration parameters (see below) |
| `enable_search` | boolean | No | Enable knowledge base search (default: false) |
| `search_params` | object | No | Search configuration (see Query search_params) |
| `semantic_caching` | boolean | No | Enable semantic caching (default: false) |
| `enhance_prompt` | object | No | Prompt enhancement options |
| `enable_history` | boolean | No | Enable history tracking (default: false) |
| `response_format` | object | No | Response format configuration |

#### model_params Object (Realtime)

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `system_prompt` | string | No | System-level instructions |
| `voice` | string | No | Voice for audio output (e.g., `"alloy"`) |
| `input_audio_format` | string | No | Input audio format (e.g., `"pcm16"`, `"wav"`, `"mp3"`) |
| `output_audio_format` | string | No | Output audio format (e.g., `"pcm16"`, `"wav"`, `"mp3"`) |
| `temperature` | float | No | Controls randomness (0.0 - 2.0) |
| `modalities` | array | No | Output modalities: `"text"`, `"audio"`, or both |

### Realtime Example

```python
import base64
import json

# Read and encode audio file
with open("recording.wav", "rb") as audio_file:
    audio_data = audio_file.read()
encoded_audio = base64.b64encode(audio_data).decode("utf-8")

# WebSocket message payload
payload = {
    "endpoint": "realtime",
    "session_id": "sid_realtime_001",
    "query": "Answer the question in the audio",
    "audio_file": encoded_audio,
    "model_provider": "openai",
    "model_name": "gpt4o",
    "model_params": {
        "system_prompt": "You are a helpful assistant.",
        "voice": "alloy",
        "input_audio_format": "pcm16",
        "output_audio_format": "pcm16",
        "temperature": 0.7,
        "modalities": ["text", "audio"]
    },
    "enable_search": True,
    "search_params": {
        "db_type": "opensearch",
        "collection": "a1a5059652d449c9a9c21c2388d42da5",
        "top_k": 5,
        "retrieval_type": "chunk",
        "output_fields": ["source_name", "page_number", "content"],
        "rerank": True,
        "reranker_model": "amazon_rerank",
        "reranker_provider": "aws"
    },
    "semantic_caching": False,
    "enhance_prompt": {
        "timezone": "MST",
        "time": True,
        "date": True,
        "verbosity": "brief"
    },
    "enable_history": True,
    "response_format": {"type": "json"}
}

# Send via WebSocket
# ws.send(json.dumps(payload))
```

```javascript
const fs = require("fs");

// Read and encode audio file
const audioData = fs.readFileSync("recording.wav");
const encodedAudio = audioData.toString("base64");

const payload = {
  endpoint: "realtime",
  session_id: "sid_realtime_001",
  query: "Answer the question in the audio",
  audio_file: encodedAudio,
  model_provider: "openai",
  model_name: "gpt4o",
  model_params: {
    system_prompt: "You are a helpful assistant.",
    voice: "alloy",
    input_audio_format: "pcm16",
    output_audio_format: "pcm16",
    temperature: 0.7,
    modalities: ["text", "audio"]
  },
  enable_search: true,
  search_params: {
    db_type: "opensearch",
    collection: "a1a5059652d449c9a9c21c2388d42da5",
    top_k: 5,
    retrieval_type: "chunk",
    output_fields: ["source_name", "page_number", "content"],
    rerank: true,
    reranker_model: "amazon_rerank",
    reranker_provider: "aws"
  },
  semantic_caching: false,
  enhance_prompt: {
    timezone: "MST",
    time: true,
    date: true,
    verbosity: "brief"
  },
  enable_history: true,
  response_format: { type: "json" }
};

// Send via WebSocket
const ws = new WebSocket("wss://api.aiml-platform.com/v1");
ws.onopen = () => ws.send(JSON.stringify(payload));
ws.onmessage = (event) => console.log(JSON.parse(event.data));
```

:::info
The Realtime endpoint is available via **WebSocket only** — REST is not supported. It combines audio input/output with optional RAG for voice-based conversational AI.
:::

---

Compare responses from different models side-by-side. Uses the `query` action with a custom search prompt and is available via **WebSocket only**.

### Endpoint

```
WebSocket route: query_compare  (WebSocket only)
```

### Request Parameters

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `action` | string | Yes | Must be `"query"` |
| `query` | string | Yes | The query to compare across models |
| `model_provider` | string | Yes | The model provider (e.g., `"aws"`, `"openai"`) |
| `model_name` | string | Yes | The model name (e.g., `"nova-micro"`, `"claude3_5_sonnet"`, `"llama3-405b"`) |
| `session_id` | string | No | Session identifier |
| `enable_search` | boolean | No | Enable knowledge base search (default: false) |
| `search_params` | object | No | Search configuration with custom prompt support |
| `response_format` | object | No | Response format configuration |

#### search_params Object (Query Compare)

| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `collection` | string | Conditional | Collection ID for knowledge base search |
| `output_fields` | array | No | Fields to include in search results. Also include any custom fields you added in the metadata during data upload. |
| `prompt_mode` | string | No | Prompt mode: `"default"` or `"custom"` |
| `search_prompt` | string | No | Custom prompt template with `{data}` and `{query}` placeholders |

---

## Related

- [Search](/endpoints/search)
- [Embeddings](/endpoints/embeddings)
- [Reranker](/endpoints/reranker)
- [Vision](/endpoints/vision)
- [Audio](/endpoints/audio)
- [Speech](/endpoints/speech)
- [Image](/endpoints/image)
- [Manage Project](/endpoints/manage-project)
- [Realtime](/endpoints/realtime)
- [Models](/models)