InstaFuel_Chatbot_public/docs/token_analysis.md

Token Consumption Analysis Report

1. Executive Summary

This report analyzes the token usage across 8 different interaction modes within the conversational AI system.

Key Findings:

• Overall Average: The system consumes an average of 4,772 tokens per turn.
• Performance Baseline: There is a high fixed "overhead" cost. The Intent Classification stage runs on every query and consistently consumes ~2,350 tokens, regardless of the complexity of the user request.
• The Outlier: The recommendation_request mode is significantly heavier than all other interactions, consuming 10,038 tokens (approx. 2.1x the overall average) due to additional processing stages (Query Enhancement and Product Suggestions).
• Prompt vs. Completion: The usage is heavily weighted towards Prompt tokens (Context), with Completion tokens remaining relatively low for most tasks, except for recommendations.

---

2. Detailed Breakdown by Intent

The following table summarizes token usage for each tested intent, sorted from lowest to highest consumption.

Intent Type	Input Sample	Total Tokens	vs. Average
greeting	Hello there!	3,281	-31.2%
goodbye	Thanks, bye!	3,561	-25.4%
general_question	What is the capital of France?	4,081	-14.5%
faq	What are your shipping policies?	4,082	-14.5%
product_enquiry	Do you have any protein powder?	4,106	-13.9%
complaint	I want to return my order...	4,512	-5.4%
order_status	Where is my order...	4,518	-5.3%
recommendation_request	Recommend me a pre-workout...	10,038	+110.3%

---

3. Stage Component Analysis

The total token count is driven by specific processing stages.

A. Fixed Overhead (Intent Classification)

• Average Usage: 2,349 tokens.
• Observation: This stage is remarkably consistent (Variance < 1%).
• Implication: Every single user message, even "Hello", incurs a minimum cost of ~2.3k tokens just to determine what the user wants.

B. Response Generation

• Average Usage: 2,111 tokens.
• Observation: This varies significantly based on the intent.
  • Simple interactions (Greeting/Goodbye) average ~1,000 tokens.
  • Complex interactions (Order status/Complaints) average ~2,160 tokens.
  • Recommendations spike to 5,194 tokens (likely due to including retrieved context in the prompt).

C. Specialized Stages (The "Recommendation" Cost)

The recommendation_request is the only intent triggering these additional stages, adding 2,491 tokens to the flow:

• Query Enhancement: 876 tokens.
• Product Suggestions: 1,615 tokens.

4. Recommendations & Optimization

Based on this data, the following optimizations are suggested:

1. Optimize Intent Classification: The ~2,350 token overhead for Intent Classification is the largest inefficiency for short queries.

   • Action: Review the Intent Prompt. It likely contains too many few-shot examples or unnecessary instructions. Reducing this by 50% would save ~1,000 tokens per turn across the board.

2. Review Recommendation Logic: The recommendation_request uses 5,194 tokens in the "Response Generation" stage, plus 1,615 in "Product Suggestions."

   • Action: Ensure that the "Product Suggestions" (likely a database retrieval) aren't being duplicated into the "Response Generation" context unnecessarily.

3. Prompt/Completion Ratio:

   • Prompt: Average ~4,700
   • Completion: Average ~100 (excluding recommendation)
   • Action: The system is heavily read-heavy. Ensure you are using a model optimized for context-window pricing (input tokens) rather than generation pricing.