LLM & RAG integration
Getting a model to answer questions about your own content is easy to prototype and hard to trust. The difference is retrieval quality and grounding — whether the answer came from your documents, and whether the user can check.
Why the first RAG prototype disappoints
The standard recipe — chunk every document, embed it, retrieve the top few matches, paste them into a prompt — works impressively in a demo and then produces confidently wrong answers on the questions people actually ask.
Usually the model is not at fault. The retrieval returned the wrong passages, or returned the right ones alongside three contradictory older versions, or the answer needed information spread across four documents that no single similarity search would surface together.
Fixing that is a search engineering problem more than an AI one: how content is chunked, how vector and keyword search are combined, how results are re-ranked, how permissions are enforced, and how the system behaves when the answer genuinely is not in the corpus.
What we build
Hybrid retrieval
Vector similarity combined with keyword search, because embeddings miss exact identifiers — part numbers, policy codes, names — and keyword search misses paraphrase. Both, then re-ranked.
Structure-aware chunking
Splitting that respects document structure rather than character counts, so a clause keeps its heading and a table keeps its header row. This single decision moves answer quality more than model choice usually does.
Grounded citations
Every claim traceable to the source passage, shown to the user. It makes the system checkable, and it makes hallucination visible instead of invisible.
Permission-aware search
Retrieval filtered by the asking user's access rights, enforced at query time. Without this a knowledge assistant becomes an efficient way to leak internal documents.
Freshness and versioning
Superseded documents demoted or excluded, so the system does not confidently quote a policy that was replaced two years ago.
Evaluation on your questions
A test set of real questions with verified answers, scored on every change. It is the only way to know whether a tweak to chunking or a model upgrade actually helped.
Typical stack
Chosen per project, not by habit. If your team already runs something that works, we use it.
- Vector storage
- pgvector
- Qdrant
- Pinecone
- Elasticsearch with dense vectors
- Retrieval
- hybrid dense + BM25
- cross-encoder re-ranking
- query rewriting
- metadata filtering
- Sources
- SharePoint
- Confluence
- Google Drive
- S3
- databases
- ticketing systems
- Evaluation
- retrieval precision and recall
- answer groundedness scoring
- regression suites
How a RAG project runs
We build the evaluation set before the system. Fifty real questions with verified answers, collected from the people who will use it, is what turns "it feels better" into a measurement.
- 01
Corpus and question audit
What content exists, what state it is in, who may see what, and the real questions users need answered. Content problems surface here — and they are usually the actual blocker.
- 02
Retrieval first
We tune retrieval until the right passages come back reliably, measured, before worrying about answer phrasing. Generation quality is capped by retrieval quality.
- 03
Grounded generation
Answer synthesis with citations, refusal behaviour when the corpus does not contain the answer, and tone matched to the audience.
- 04
Ship and monitor
Deployed with query logging, so the questions it fails on become the next iteration rather than a complaint.
Common questions
Do we need to reorganise our documents first?
Usually not reorganise, but expect to prune. Duplicated and superseded documents are the most common cause of poor answers, and identifying them is part of the corpus audit. Perfect document hygiene is not a prerequisite.
Can it respect our existing permissions?
Yes, and it should. Retrieval is filtered by the asking user's access rights at query time, mirroring your existing groups rather than inventing a parallel permission model.
How do you stop it making things up?
Three things: retrieval good enough that the answer is genuinely in context, prompting and validation that require citations, and explicit refusal behaviour when the corpus does not support an answer. Measured by groundedness scoring on the evaluation set, not by hope.
Which model should we use?
It matters less than people expect — retrieval quality dominates. We build model-agnostic so you can switch as the market moves, and we will pick based on your data residency needs, latency budget and cost per query rather than benchmark rankings.
What does it cost to run?
Ongoing cost is mostly per-query inference plus vector storage, and it is predictable once volume is known. We instrument cost per query from day one so you can see it, and caching common questions typically removes a large share of it.
Start with a scoping call.
Thirty minutes, no obligation. If we are not the right fit we will tell you on the call rather than after a proposal.
Related services
AI agent development
Most AI pilots stall because a demo that answers questions is not a system that does work. We build agents tha…
AI chatbots & assistants
The bar for a customer-facing assistant is not "can it hold a conversation" — it is whether it resolves the qu…
AI readiness audit
A short, fixed-fee engagement that answers three questions: where AI would actually pay off in your business, …