InstaFuel_Chatbot_public/docs/api_usage.md

InstaFuel Chatbot API Usage

This reference is aimed at frontend engineers who call the FastAPI backend defined in src/api/server.py. It covers the required environment variables, how to run the server locally, and the full contract for every HTTP endpoint.

1. Runtime Requirements

Variable	Required	Description
MODEL_PROVIDER	✅	Provider segment used by llm.models.Model for chat completions (e.g., openrouter).
MODEL_NAME	✅	Chat completion model identifier (e.g., meta-llama/llama-3.1-70b-instruct).
OPENROUTER_API_KEY	✅	API key passed to OpenRouter-compatible chat completions.
QDRANT_URL / QDRANT_HOST	✅	Connection info for the Qdrant instance that hosts the products collection.
QDRANT_API_KEY	➖	Only needed if your Qdrant instance enforces authentication.
EMBED_BASE_URL	➖	Override for the embedding HTTP endpoint (defaults to http://127.0.0.1:11434/v1 for Ollama).
EMBED_API_KEY	➖	API key forwarded to the embedding endpoint (defaults to ollama).
CHAT_STORE_PATH	➖	DuckDB connection string. Examples: data/chat.duckdb (local), md:my_db (MotherDuck), s3://bucket/chat.duckdb (S3). Defaults to data/chat_store.duckdb.
MOTHERDUCK_TOKEN	➖	Authentication token for MotherDuck cloud service (only when using md: protocol).

Create a .env file in the repository root and populate these values. The server automatically loads it on startup.

2. Starting the Server

From the repository root:

uvicorn src.api.server:app --host 0.0.0.0 --port 8000 --reload

Once running, GET /health should respond with { "status": "ok" } when the QueryEngine and Qdrant dependencies are ready.

3. Endpoint Summary

Method	Path	Description
GET	/health	Lightweight readiness check used by load balancers and uptime probes.
POST	/chat	Sends a user message, runs intent detection + reasoning, and returns the assistant reply plus any referenced product metadata.

All endpoints accept and return JSON. Unless specified otherwise, include Content-Type: application/json and Accept: application/json headers.

4. Endpoint Details

4.1 GET /health

Returns the readiness of the QueryEngine stack.

{ "status": "ok" }

When the underlying models are still loading you will receive { "status": "starting" }.

Example request

curl --request GET "http://localhost:8000/health"

If you deploy behind a custom domain, replace the host accordingly.

4.2 POST /chat

Generates an assistant reply, enforces the new concise/wholesome response policies, and returns structured product metadata referenced in the answer.

Headers

Header	Value
Content-Type	application/json
Accept	application/json

Request Body

Field	Type	Required	Description
message	string	✅	The user's latest utterance. Empty strings are rejected.
conversation_id	string	➖*	Provide when resuming a chat. Leave blank on the very first turn to have the backend mint one.
user_id	string	➖*	Required whenever you send an existing conversation_id. The backend generates both IDs the first time and echoes them in every response.
user_context	string	➖	Optional preferences, goals, or profile data that should condition the reasoning. When sent, set use_user_context=true so it is persisted.
use_user_context	boolean	➖	Defaults to false. Toggle to true when you want the supplied user_context JSON/text to be merged into the per-user DuckDB store.

conversation_id and user_id work as a pair: the chat store refuses to continue a conversation if the IDs do not match (details in section 6). The backend keeps only the 10 most recent turns per conversation to limit prompt size.

Response Body

{
  "conversation_id": "api-9c941072-1c48-4d6b-9c88-872ec3e8e9e4",
  "user_id": "user-54a2d4ce-5b0a-43e9-8bf0-8fa20087d0e1",
  "response": "Here's a concise, friendly plan...",
  "product_suggestions": [
    {
      "name": "Advance Curcumin",
      "product_url": "https://example.com/curcumin",
      "image_url": "https://cdn.example.com/curcumin.png",
      "notes": null
    }
  ]
}

The product_suggestions array lists only the products explicitly referenced by the assistant (detected via the server-side summarisation step added in the latest QueryEngine update). Fields:

• name: Product name mentioned in the response.
• product_url: Landing page or PDP URL, if available in the catalog metadata.
• image_url: Image preview URL, when supplied.
• notes: Optional text snippets surfaced by the LLM (e.g., quick benefit callouts).

If the assistant did not recommend a product, the array will be empty.

Example request

# First turn (IDs auto-generated in the response)
curl --request POST "http://localhost:8000/chat" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --data '{
    "message": "I need a muscle gain stack and my knees are sensitive.",
    "use_user_context": true,
    "user_context": "{\"age\":28,\"training_level\":\"intermediate\"}"
  }'

# Follow-up turn (must echo BOTH IDs returned earlier)
curl --request POST "http://localhost:8000/chat" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --data '{
    "message": "Great, can you add a recovery drink?",
    "conversation_id": "api-9c941072-1c48-4d6b-9c88-872ec3e8e9e4",
    "user_id": "user-54a2d4ce-5b0a-43e9-8bf0-8fa20087d0e1"
  }'

5. Response Modes: Sync vs Streaming

The backend exposes two styles of response generation through QueryEngine:

Mode	Backend method	How it works	API surface	When to use
Synchronous (default)	get_response_with_suggestions	Runs the full reasoning chain, waits for the final assistant reply, and summarises referenced products.	POST /chat (blocking HTTP request)	Chat widgets or automations that can wait ~2-6 seconds for a complete answer.
Streaming	stream_response	Async generator that yields low-latency tokens plus a final product_suggestions event.	Expose via WebSocket/SSE endpoint (not included yet).	Live typing indicators, conversational UIs that show the answer as it is generated.

5.1 Calling the synchronous endpoint

// Example using fetch in a React app
const payload = {
  message: "Plan a stack for lean bulk",
  conversation_id: sessionId, // reuse the value returned by previous calls
  user_context: "29y/o lifter, lactose intolerant"
};

const response = await fetch("https://api.instafuel.ai/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Accept": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
// data.response -> assistant text
// data.product_suggestions -> array shown in merchandising panel

The Express/Next.js server pattern is the same—perform the POST, forward the JSON, and map product_suggestions to your merchandising component.

5.2 Hooking up a streaming endpoint

If you need token-by-token streaming, add a WebSocket or Server-Sent Events (SSE) route in the FastAPI server that wraps QueryEngine.stream_response:

@app.websocket("/chat/stream")
async def stream_chat(ws: WebSocket):
    await ws.accept()
    async for event in query_engine.stream_response(query, history, ctx):
        await ws.send_json(event.model_dump())

stream_response yields these event types:

• response_start: signals when the assistant started thinking.
• response: contains data.content chunks to append to your transcript.
• product_suggestions: emitted after the model finishes; identical payload to the synchronous endpoint.
• response_end: indicates completion (close the stream once you see it).

On the frontend you can consume an SSE endpoint like this:

const events = new EventSource("https://api.instafuel.ai/chat/stream?message=..." );

events.addEventListener("response", evt => {
  const { content } = JSON.parse(evt.data);
  appendPartial(content);
});

events.addEventListener("product_suggestions", evt => {
  const { suggestions } = JSON.parse(evt.data);
  showProducts(suggestions);
});

events.addEventListener("response_end", () => events.close());

Until you expose that streaming route, the /chat endpoint remains the canonical synchronous interface.

6. Error Handling

Status	Cause	Notes
400	Missing/empty message, or conversation_id provided without user_id.	Validate payloads before sending.
403	conversation_id does not belong to the supplied user_id.	Happens when clients mix user sessions. Surface a “refresh chat” UI.
404	Unknown conversation_id.	The server has no record of that chat for the given user. Start a fresh conversation.
503	QueryEngine not ready yet.	Retry after the server finishes booting models/Qdrant.
502	LLM returned an empty string.	Logged server-side for investigation.
500	Unexpected runtime failure.	See server logs for a stack trace.

FastAPI automatically returns a JSON payload shaped like { "detail": "..." } for the errors above.

7. Working With Conversations & Chat Store

• Persist BOTH conversation_id and user_id from the first response. Every subsequent call must echo both or the chat store will reject the turn.
• All turns (plus optional user context) are persisted in DuckDB. The database can be hosted locally, on MotherDuck cloud, S3, or network-mounted storage (see deployment options in README.md).
• Context fields merge by name: when you resend the same key (e.g., goal) the value from the latest message timestamp wins.
• Set use_user_context=true only when you intend to store/update context; otherwise the text is ignored for privacy.
• The backend stores at most the last 10 turns per conversation to keep LLM prompts efficient—mirror this trimming strategy client-side if you keep your own transcript.
• Sensitive-topic disclaimers and wholesome tone rules are already enforced in the server prompts, so clients do not need additional censorship logic.

8. Smoke Testing

Use the lightweight scripts/smoke_test.py or the CLI (python run_chat.py) to verify that your environment variables and embeddings work before deploying the API server.