InstaFuel_Chatbot_public/docs/api_usage.md

# InstaFuel Chatbot API Usage

This reference is aimed at frontend engineers who call the FastAPI backend defined in `src/api/server.py`. It covers the required environment variables, how to run the server locally, and the full contract for every HTTP endpoint.

## 1. Runtime Requirements

| Variable | Required | Description |
| --- | --- | --- |
| `MODEL_PROVIDER` | ✅ | Provider segment used by `llm.models.Model` for chat completions (e.g., `openrouter`). |
| `MODEL_NAME` | ✅ | Chat completion model identifier (e.g., `meta-llama/llama-3.1-70b-instruct`). |
| `OPENROUTER_API_KEY` | ✅ | API key passed to OpenRouter-compatible chat completions. |
| `QDRANT_URL` / `QDRANT_HOST` | ✅ | Connection info for the Qdrant instance that hosts the `products` collection. |
| `QDRANT_API_KEY` | ➖ | Only needed if your Qdrant instance enforces authentication. |
| `EMBED_BASE_URL` | ➖ | Override for the embedding HTTP endpoint (defaults to `http://127.0.0.1:11434/v1` for Ollama). |
| `EMBED_API_KEY` | ➖ | API key forwarded to the embedding endpoint (defaults to `ollama`). |
| `CHAT_STORE_PATH` | ➖ | DuckDB connection string. Examples: `data/chat.duckdb` (local), `md:my_db` (MotherDuck), `s3://bucket/chat.duckdb` (S3). Defaults to `data/chat_store.duckdb`. |
| `MOTHERDUCK_TOKEN` | ➖ | Authentication token for MotherDuck cloud service (only when using `md:` protocol). |

Create a `.env` file in the repository root and populate these values. The server automatically loads it on startup.

## 2. Starting the Server

From the repository root:

```powershell
uvicorn src.api.server:app --host 0.0.0.0 --port 8000 --reload
```

Once running, `GET /health` should respond with `{ "status": "ok" }` when the QueryEngine and Qdrant dependencies are ready.

## 3. Endpoint Summary

| Method | Path | Description |
| --- | --- | --- |
| `GET` | `/health` | Lightweight readiness check used by load balancers and uptime probes. |
| `POST` | `/chat` | Sends a user message, runs intent detection + reasoning, and returns the assistant reply plus any referenced product metadata. |

All endpoints accept and return JSON. Unless specified otherwise, include `Content-Type: application/json` and `Accept: application/json` headers.

## 4. Endpoint Details

### 4.1 `GET /health`

Returns the readiness of the QueryEngine stack.

```json
{ "status": "ok" }
```

When the underlying models are still loading you will receive `{ "status": "starting" }`.

**Example request**

```powershell
curl --request GET "http://localhost:8000/health"
```

If you deploy behind a custom domain, replace the host accordingly.

### 4.2 `POST /chat`

Generates an assistant reply, enforces the new concise/wholesome response policies, and returns structured product metadata referenced in the answer.

**Headers**

| Header | Value |
| --- | --- |
| `Content-Type` | `application/json` |
| `Accept` | `application/json` |

**Request Body**

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `message` | `string` | ✅ | The user's latest utterance. Empty strings are rejected. |
| `conversation_id` | `string` | ➖* | Provide when resuming a chat. Leave blank on the very first turn to have the backend mint one. |
| `user_id` | `string` | ➖* | Required whenever you send an existing `conversation_id`. The backend generates both IDs the first time and echoes them in every response. |
| `user_context` | `string` | ➖ | Optional preferences, goals, or profile data that should condition the reasoning. When sent, set `use_user_context=true` so it is persisted. |
| `use_user_context` | `boolean` | ➖ | Defaults to `false`. Toggle to `true` when you want the supplied `user_context` JSON/text to be merged into the per-user DuckDB store. |

`conversation_id` and `user_id` work as a pair: the chat store refuses to continue a conversation if the IDs do not match (details in section 6). The backend keeps only the 10 most recent turns per conversation to limit prompt size.

**Response Body**

```jsonc
{
  "conversation_id": "api-9c941072-1c48-4d6b-9c88-872ec3e8e9e4",
  "user_id": "user-54a2d4ce-5b0a-43e9-8bf0-8fa20087d0e1",
  "response": "Here's a concise, friendly plan...",
  "product_suggestions": [
    {
      "name": "Advance Curcumin",
      "product_url": "https://example.com/curcumin",
      "image_url": "https://cdn.example.com/curcumin.png",
      "notes": null
    }
  ]
}
```

The `product_suggestions` array lists only the products explicitly referenced by the assistant (detected via the server-side summarisation step added in the latest QueryEngine update). Fields:

- `name`: Product name mentioned in the response.
- `product_url`: Landing page or PDP URL, if available in the catalog metadata.
- `image_url`: Image preview URL, when supplied.
- `notes`: Optional text snippets surfaced by the LLM (e.g., quick benefit callouts).

If the assistant did not recommend a product, the array will be empty.

**Example request**

```powershell
# First turn (IDs auto-generated in the response)
curl --request POST "http://localhost:8000/chat" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --data '{
    "message": "I need a muscle gain stack and my knees are sensitive.",
    "use_user_context": true,
    "user_context": "{\"age\":28,\"training_level\":\"intermediate\"}"
  }'

# Follow-up turn (must echo BOTH IDs returned earlier)
curl --request POST "http://localhost:8000/chat" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --data '{
    "message": "Great, can you add a recovery drink?",
    "conversation_id": "api-9c941072-1c48-4d6b-9c88-872ec3e8e9e4",
    "user_id": "user-54a2d4ce-5b0a-43e9-8bf0-8fa20087d0e1"
  }'
```

## 5. Response Modes: Sync vs Streaming

The backend exposes two styles of response generation through `QueryEngine`:

| Mode | Backend method | How it works | API surface | When to use |
| --- | --- | --- | --- | --- |
| Synchronous (default) | `get_response_with_suggestions` | Runs the full reasoning chain, waits for the final assistant reply, and summarises referenced products. | `POST /chat` (blocking HTTP request) | Chat widgets or automations that can wait ~2-6 seconds for a complete answer. |
| Streaming | `stream_response` | Async generator that yields low-latency tokens plus a final `product_suggestions` event. | Expose via WebSocket/SSE endpoint (not included yet). | Live typing indicators, conversational UIs that show the answer as it is generated. |

### 5.1 Calling the synchronous endpoint

```ts
// Example using fetch in a React app
const payload = {
  message: "Plan a stack for lean bulk",
  conversation_id: sessionId, // reuse the value returned by previous calls
  user_context: "29y/o lifter, lactose intolerant"
};

const response = await fetch("https://api.instafuel.ai/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Accept": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
// data.response -> assistant text
// data.product_suggestions -> array shown in merchandising panel
```

The Express/Next.js server pattern is the same—perform the POST, forward the JSON, and map `product_suggestions` to your merchandising component.

### 5.2 Hooking up a streaming endpoint

If you need token-by-token streaming, add a WebSocket or Server-Sent Events (SSE) route in the FastAPI server that wraps `QueryEngine.stream_response`:

```python
@app.websocket("/chat/stream")
async def stream_chat(ws: WebSocket):
    await ws.accept()
    async for event in query_engine.stream_response(query, history, ctx):
        await ws.send_json(event.model_dump())
```

`stream_response` yields these event types:

- `response_start`: signals when the assistant started thinking.
- `response`: contains `data.content` chunks to append to your transcript.
- `product_suggestions`: emitted after the model finishes; identical payload to the synchronous endpoint.
- `response_end`: indicates completion (close the stream once you see it).

On the frontend you can consume an SSE endpoint like this:

```ts
const events = new EventSource("https://api.instafuel.ai/chat/stream?message=..." );

events.addEventListener("response", evt => {
  const { content } = JSON.parse(evt.data);
  appendPartial(content);
});

events.addEventListener("product_suggestions", evt => {
  const { suggestions } = JSON.parse(evt.data);
  showProducts(suggestions);
});

events.addEventListener("response_end", () => events.close());
```

Until you expose that streaming route, the `/chat` endpoint remains the canonical synchronous interface.

## 6. Error Handling

| Status | Cause | Notes |
| --- | --- | --- |
| `400` | Missing/empty `message`, or `conversation_id` provided without `user_id`. | Validate payloads before sending. |
| `403` | `conversation_id` does not belong to the supplied `user_id`. | Happens when clients mix user sessions. Surface a “refresh chat” UI. |
| `404` | Unknown `conversation_id`. | The server has no record of that chat for the given user. Start a fresh conversation. |
| `503` | QueryEngine not ready yet. | Retry after the server finishes booting models/Qdrant. |
| `502` | LLM returned an empty string. | Logged server-side for investigation. |
| `500` | Unexpected runtime failure. | See server logs for a stack trace. |

FastAPI automatically returns a JSON payload shaped like `{ "detail": "..." }` for the errors above.

## 7. Working With Conversations & Chat Store

- Persist BOTH `conversation_id` and `user_id` from the first response. Every subsequent call must echo both or the chat store will reject the turn.
- All turns (plus optional user context) are persisted in DuckDB. The database can be hosted locally, on MotherDuck cloud, S3, or network-mounted storage (see deployment options in README.md).
- Context fields merge by name: when you resend the same key (e.g., `goal`) the value from the latest message timestamp wins.
- Set `use_user_context=true` only when you intend to store/update context; otherwise the text is ignored for privacy.
- The backend stores at most the last 10 turns per conversation to keep LLM prompts efficient—mirror this trimming strategy client-side if you keep your own transcript.
- Sensitive-topic disclaimers and wholesome tone rules are already enforced in the server prompts, so clients do not need additional censorship logic.

## 8. Smoke Testing

Use the lightweight `scripts/smoke_test.py` or the CLI (`python run_chat.py`) to verify that your environment variables and embeddings work before deploying the API server.