Mikhail IvanovAll case studies / AI Assistants & Agents
LLM parts search audit cuts query cost by 16.7% without losing accuracy
Industry: Auto parts marketplaceFocus: AI assistants and agents
−16.7%cost per query: $0.1353 → $0.1128
0OEM numbers lost on the control set after optimization
3 stagesaudit, accuracy improvements, cost reduction; all paid
Before
- The client asked for an analysis of accuracy issues in their AI-powered auto parts search and a plan for improvements
- Search built on Node.js: LLM in Yandex AI Studio plus web search, without a vector index or reference database
- Payment for cost reduction was tied to the measured percentage of savings
What we built
- Pipeline audit: query understanding, sources and their trust levels, candidate extraction and ranking
- Benchmark setup: reference query set, exact@1, hit@5, MRR metrics, repeated runs to account for noise
- Vehicle generation detection by VIN and part compatibility checks were implemented
- Cost: removed an unnecessary 23.7 thousand-character knowledge base from the system prompt, compressed JSON, deduplicated queries, cached the static prefix
- Every change passed an accuracy gate; a shortened prompt reduced accuracy and was rolled back
How it works
QueryVIN or text
→
NormalizationLLM parses the query
→
Web searchsources with trust levels
→
Extraction and rankingOEM number candidates
→
Benchmark setupaccuracy and cost of each run
Before and after
| What | Before | After |
|---|---|---|
| Query cost | $0.1353 | $0.1128 |
| Cached prompt tokens | 0 | 173 thousand |
Result
- The audit was accepted and the client closed the order
- The improvements and cost reduction were paid for, and acceptance documents were signed
- The benchmark caught a reranker regression before production deployment, and the change was rolled back
Technology
What matters in a project like this. The main cost was not web requests (31%), but model tokens (69%). Query deduplication saved exactly zero, while prompt cleanup and prefix caching produced the savings.
Have a similar project?
Describe it in two lines in Telegram. We'll explain how we'd approach it, how long it would take, and what it would cost.
Similar case studies
Auto repair shop
A bot sends the shop owner a daily summary, while AI finds discrepancies in the numbers
every daydaily summary for the owner and mechanic workload in the morning
SaaS for local businesses: review responses
SaaS collects reviews from 8 platforms and drafts AI responses for each business niche
8 platformsreviews collected from them
Education startup (EdTech)
MVP platform where AI creates teaching materials, with a quality judge and data isolation
550+ testsindependent verifier run, types passed without errors