How did I reduce token cost by 90%?

Zero-Token Architectures, Intelligent Caching, and Context Optimisation to reduce token cost

How the Multi-Agent Research Assistant Achieves Up to 90% Token Savings While Improving Performance and Scalability

Written by - Shivam Suraj Dube

Introduction

Large Language Models (LLMs) have transformed how modern AI systems perform research, analysis, reasoning, and content generation. As organizations increasingly adopt multi-agent architectures, a new challenge emerges: rapidly escalating token costs.

In a typical multi-agent workflow, agents continuously exchange prompts, search results, conversational history, and intermediate outputs. While this improves task specialization, it also creates a hidden problem known as token bloat. The result is higher API bills, slower response times, increased latency, and frequent rate-limiting issues.

The Multi-Agent Research Assistant was designed with a different philosophy:

Optimize architecture first, then optimize prompts.

By combining rule-based agents, intelligent caching, context compression, deduplication algorithms, structured prompting, and hybrid local/cloud model routing, the system achieves up to 90% token reduction and significant operational cost savings without sacrificing output quality.

The Hidden Cost Problem in Multi-Agent Systems

Most developers focus on model selection while overlooking the real source of expense:

Repeated Context Transfer

Every agent often receives:

• Previous conversation history

• Search results

• User instructions

• Intermediate reasoning outputs

As workflows grow, context size increases exponentially.

Example

A single-agent workflow may consume:

• 2,000 input tokens

• 500 output tokens

A four-agent pipeline can easily consume:

• 8,000–12,000 input tokens

• 2,000+ output tokens

for the exact same task.

Without optimization, costs scale faster than capability.

Cost Optimisation Strategy #1 to reduce token cost:

Zero-Token Architecture

Eliminate Unnecessary LLM Calls

The cheapest token is the one you never generate.

Many AI applications use LLMs for tasks that can be handled more efficiently using deterministic code.

Manager Agent

Instead of using an LLM for:

• Query planning

• Topic classification

• Search generation

the system uses:

• Keyword detection

• Sector mapping

• Predefined templates

Result:

 Zero tokens consumed Instant execution Consistent outputs.

This helps in the objective which is "reduce token cost".

Analyst Agent

Instead of prompting an LLM to extract:

• Revenue

• Market capitalization

• Employee count

• Headquarters

• CEO information

the system relies on regex-based extraction and pattern matching.

Result:

 0 API cost Faster processing Higher reliability

This architectural decision alone eliminates thousands of tokens from every workflow.

Cost Optimization Strategy #2: Intelligent Search Caching to reduce token cost -

Search grounding is one of the most expensive operations in modern AI systems.

Running the same search repeatedly wastes both money and tokens.

SearchCache Implementation

The Multi-Agent Research Assistant stores:

• Search summaries

• Source references

• Token statistics

inside a local cache.

Features

• SHA-256 query hashing

• File-based persistence

• Thread-safe architecture

• 24-hour cache validity

When identical searches occur, the system retrieves results directly from cache instead of calling the model again.

Business Impact

Benefits include:

• Near-zero cost for repeated queries

• Faster response times

• Reduced API usage

• Improved scalability

For frequently searched topics, cache hits can reduce operational costs by more than 80%.

This helps in the objective which is "reduce token cost".

Cost Optimization Strategy #3: Context Compression to reduce token cost

One of the largest contributors to token expenses is excessive context.

Most search results contain:

• Duplicate information

• Repeated facts

• Low-value content

• Verbose explanations

Passing all of this downstream dramatically increases costs.

Character Budgeting

The system applies strict limits:

Component	Maximum Characters
Search Summary	1,000
Analyst Input	3,000

Only the most relevant information proceeds through the pipeline.

Benefits

• Smaller prompts

• Lower API costs

• Faster generation

• Better signal-to-noise ratio

Cost Optimization Strategy #4: Deduplication Algorithms to reduce token cost

Multiple search queries frequently return identical facts.

Without filtering, the same information is paid for multiple times.

The project uses a word-overlap comparison algorithm that removes content exceeding a 60% similarity threshold.

Before Deduplication

• 10 search results

• 3,500 characters

• Significant redundancy

After Deduplication

• 10 search results

• 1,800 characters

• Unique insights only

Result:

 Reduced prompt size Lower token consumption Better output quality

This optimization saves hundreds of tokens per research cycle.

This helps in the objective which is "reduce token cost".

Cost Optimization Strategy #5: Output Budget Control to reduce token cost

Many AI applications generate far more text than necessary.

Long outputs directly increase API costs.

Strict Agent Budgets

The system enforces token limits:

Agent	Token Budget
Manager	200
Searcher	300
Analyst	300
Writer	1,200

This prevents runaway generations while ensuring concise, focused outputs.

Additional Savings

The project also uses:

• Temperature = 0.0

• Thinking Budget = 0

These settings reduce unnecessary reasoning tokens and eliminate verbose responses.

The result is predictable, low-cost generation across every workflow.

Cost Optimization Strategy #6: Structured Prompt Engineering to reduce token cost -

Prompt structure directly influences model efficiency.

The system uses XML-style tags to organize information:

• Topic

• Themes

• Facts

• Products

• Raw Data

Structured prompts help models process information more efficiently and reduce instruction ambiguity.

Benefits

• Better instruction adherence

• Reduced hallucinations

• Lower prompt complexity

• Improved token efficiency

Well-structured prompts often achieve the same output quality using fewer tokens.

This helps in the objective which is "reduce token cost".

Cost Optimization Strategy #7: Hybrid Local-Cloud AI Routing to reduce token cost -

For organizations operating at scale, API expenses can become substantial.

The Multi-Agent Research Assistant supports a hybrid architecture using Ollama.

Cloud Models (Gemini)

Used only for:

• Search grounding

• Internet-connected retrieval

Local Models (Gemma via Ollama)

Used for:

• Analysis

• Summarization

• Report generation

Financial Impact

Traditional Workflow:

• Search → Analysis → Writing

• All performed through paid APIs

Hybrid Workflow:

• Search performed in cloud

• Analysis performed locally

Result:

 Up to 70–90% lower operational costs Reduced API dependency Unlimited local processing

In many cases, synthesis costs are reduced to $0.00 per run.

This helps in the objective which is "reduce token cost".

Measuring Cost Savings with Token Auditing

Optimization should be measurable.

The project includes a real-time TokenTracker that monitors:

• Input tokens

• Output tokens

• Cached tokens

• Cost estimates

• Cache hit rates

• Agent-level usage

Example Cost Report

Agent	Input	Output	Total	Cost
Searcher	1542	240	1782	$0.001063
Writer	2125	450	2575	$0.001313
Total	3667	690	4357	$0.002376

This visibility enables continuous optimization and prevents hidden cost growth.

Results: Up to 90% Cost Reduction

By combining these optimization techniques, the Multi-Agent Research Assistant achieves:

Financial Benefits

• Up to 90% token reduction

• Significant API cost savings

• Near-zero repeated query costs

Performance Benefits

• Faster response times

• Lower latency

• Reduced rate-limiting issues

Engineering Benefits

• Deterministic workflows

• Better scalability

• Improved reliability

• Production-ready architecture

Conclusion

Building successful AI systems is no longer just about selecting the most powerful model. The real competitive advantage lies in designing architectures that maximize intelligence while minimizing cost.

Through zero-token agents, intelligent caching, context compression, deduplication, structured prompting, and hybrid model routing, the Multi-Agent Research Assistant demonstrates how organizations can dramatically reduce token consumption while improving performance and scalability.

As AI adoption accelerates, cost-efficient architectures will become a critical requirement rather than a competitive advantage.

The future of AI is not just smarter models—it is smarter system design.

Key Takeaway

A 90% reduction in token usage does not come from better prompts alone. It comes from eliminating unnecessary LLM calls, optimizing context flow, and engineering every stage of the pipeline for efficiency.

— Written by Shivam Suraj Dube