InstaFuel_Chatbot_public/api_usage.md

InstaFuel Chatbot API Usage

This reference is aimed at frontend engineers who call the FastAPI backend defined in src/api/server.py. It covers the required environment variables, how to run the server locally, and the full contract for every HTTP endpoint.

1. Runtime Requirements

Variable	Required	Description
MODEL_PROVIDER	✅	Provider segment used by llm.models.Model for chat completions (e.g., openrouter).
MODEL_NAME	✅	Chat completion model identifier (e.g., meta-llama/llama-3.1-70b-instruct).
OPENROUTER_API_KEY	✅	API key passed to OpenRouter-compatible chat completions.
QDRANT_URL / QDRANT_HOST	✅	Connection info for the Qdrant instance that hosts the products collection.
QDRANT_API_KEY	➖	Only needed if your Qdrant instance enforces authentication.
EMBED_BASE_URL	➖	Override for the embedding HTTP endpoint (defaults to http://127.0.0.1:11434/v1 for Ollama).
EMBED_API_KEY	➖	API key forwarded to the embedding endpoint (defaults to ollama).

Create a .env file in the repository root and populate these values. The server automatically loads it on startup.

2. Starting the Server

From the repository root:

uvicorn src.api.server:app --host 0.0.0.0 --port 8000 --reload

Once running, GET /health should respond with { "status": "ok" } when the QueryEngine and Qdrant dependencies are ready.

3. Endpoint Summary

Method	Path	Description
GET	/health	Lightweight readiness check used by load balancers and uptime probes.
POST	/chat	Sends a user message, runs intent detection + reasoning, and returns the assistant reply plus any referenced product metadata.

All endpoints accept and return JSON. Unless specified otherwise, include Content-Type: application/json and Accept: application/json headers.

4. Endpoint Details

4.1 GET /health

Returns the readiness of the QueryEngine stack.

{ "status": "ok" }

When the underlying models are still loading you will receive { "status": "starting" }.

Example request

curl --request GET "http://localhost:8000/health"

If you deploy behind a custom domain, replace the host accordingly.

4.2 POST /chat

Generates an assistant reply, enforces the new concise/wholesome response policies, and returns structured product metadata referenced in the answer.

Headers

Header	Value
Content-Type	application/json
Accept	application/json

Request Body

Field	Type	Required	Description
message	string	✅	The user's latest utterance. Empty strings are rejected.
conversation_id	string	➖	Use the same value on subsequent calls to preserve turn history. One is created automatically if omitted.
user_context	string	➖	Optional preferences, goals, or profile data that should condition the reasoning.

Response Body

{
  "conversation_id": "api-9c941072-1c48-4d6b-9c88-872ec3e8e9e4",
  "response": "Here's a concise, friendly plan...",
  "product_suggestions": [
    {
      "name": "Advance Curcumin",
      "product_url": "https://example.com/curcumin",
      "image_url": "https://cdn.example.com/curcumin.png",
      "notes": null
    }
  ]
}

The product_suggestions array lists only the products explicitly referenced by the assistant (detected via the server-side summarisation step added in the latest QueryEngine update). Fields:

• name: Product name mentioned in the response.
• product_url: Landing page or PDP URL, if available in the catalog metadata.
• image_url: Image preview URL, when supplied.
• notes: Optional text snippets surfaced by the LLM (e.g., quick benefit callouts).

If the assistant did not recommend a product, the array will be empty.

Example request

curl --request POST "http://localhost:8000/chat" \
  --header "Content-Type: application/json" \
  --header "Accept: application/json" \
  --data '{
    "message": "I need a muscle gain stack and my knees are sensitive.",
    "conversation_id": "demo-user-123",
    "user_context": "28 y/o male, intermediate lifter"
  }'

5. Response Modes: Sync vs Streaming

The backend exposes two styles of response generation through QueryEngine:

Mode	Backend method	How it works	API surface	When to use
Synchronous (default)	get_response_with_suggestions	Runs the full reasoning chain, waits for the final assistant reply, and summarises referenced products.	POST /chat (blocking HTTP request)	Chat widgets or automations that can wait ~2-6 seconds for a complete answer.
Streaming	stream_response	Async generator that yields low-latency tokens plus a final product_suggestions event.	Expose via WebSocket/SSE endpoint (not included yet).	Live typing indicators, conversational UIs that show the answer as it is generated.

5.1 Calling the synchronous endpoint

// Example using fetch in a React app
const payload = {
  message: "Plan a stack for lean bulk",
  conversation_id: sessionId, // reuse the value returned by previous calls
  user_context: "29y/o lifter, lactose intolerant"
};

const response = await fetch("https://api.instafuel.ai/chat", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Accept": "application/json"
  },
  body: JSON.stringify(payload)
});

const data = await response.json();
// data.response -> assistant text
// data.product_suggestions -> array shown in merchandising panel

The Express/Next.js server pattern is the same—perform the POST, forward the JSON, and map product_suggestions to your merchandising component.

5.2 Hooking up a streaming endpoint

If you need token-by-token streaming, add a WebSocket or Server-Sent Events (SSE) route in the FastAPI server that wraps QueryEngine.stream_response:

@app.websocket("/chat/stream")
async def stream_chat(ws: WebSocket):
    await ws.accept()
    async for event in query_engine.stream_response(query, history, ctx):
        await ws.send_json(event.model_dump())

stream_response yields these event types:

• response_start: signals when the assistant started thinking.
• response: contains data.content chunks to append to your transcript.
• product_suggestions: emitted after the model finishes; identical payload to the synchronous endpoint.
• response_end: indicates completion (close the stream once you see it).

On the frontend you can consume an SSE endpoint like this:

const events = new EventSource("https://api.instafuel.ai/chat/stream?message=..." );

events.addEventListener("response", evt => {
  const { content } = JSON.parse(evt.data);
  appendPartial(content);
});

events.addEventListener("product_suggestions", evt => {
  const { suggestions } = JSON.parse(evt.data);
  showProducts(suggestions);
});

events.addEventListener("response_end", () => events.close());

Until you expose that streaming route, the /chat endpoint remains the canonical synchronous interface.

6. Error Handling

Status	Cause	Notes
400	Missing/empty message.	Validate input before sending.
503	QueryEngine not ready yet.	Retry after the server finishes booting models/Qdrant.
502	LLM returned an empty string.	Logged server-side for investigation.
500	Unexpected runtime failure.	See server logs for a stack trace.

FastAPI automatically returns a JSON payload shaped like { "detail": "..." } for the errors above.

7. Working With Conversations

• Always persist the conversation_id returned by the API and reuse it on the next request to keep follow-up context.
• The backend stores only the 10 most recent turns per conversation to keep prompts efficient—trim your own chat transcripts similarly if you mirror them client-side.
• Sensitive-topic disclaimers and wholesome tone rules are already enforced in the server prompts, so clients do not need additional censorship logic.

8. Smoke Testing

Use the lightweight scripts/smoke_test.py or the CLI (python run_chat.py) to verify that your environment variables and embeddings work before deploying the API server.