The five areas that decide go-live
| Area | In the demo | In production |
|---|---|---|
| Documents | A curated folder that answered well | The whole estate — duplicates, old versions, conflicting sources |
| Permissions | One user’s view of everything | Each user sees only what they may — kept in step with the source systems |
| Integrations | Answers drawn from documents | Answers that depend on live systems of record, each with its own owner and sign-in |
| Cost and speed | A handful of questions | Real volume, real latency expectations, a monthly bill someone owns |
| What “good” means | Impressive answers in a meeting | A written definition, agreed before more is built, that go-live is judged against |
None of these is solved by changing the model.
What Bedrock gives you — and what it leaves to you
AWS’s documentation is clear about the building blocks:
- Knowledge bases with metadata filtering. You can filter retrieved results on document fields and attributes — the documentation’s own examples are recency and category (AWS, Configure and customize queries and response generation). Which documents each user may see is a decision you make and encode.
- Guardrails with a contextual grounding check. It is designed “to detect and filter hallucinations in model responses when a reference source and a user query is provided”, and supports summarisation, paraphrasing and question answering — “Conversational QA / Chatbot use cases are not supported” (AWS, contextual grounding check).
Both checked 5 October 2026. That second point matters for many pilots: if what you built is a chat assistant, check which safeguards actually apply to it before you rely on them.
A way forward: one workflow
- Pick the one workflow that justified the pilot — answering policy questions for support staff, drafting replies from a product manual, finding clauses across contracts.
- Fix the documents that workflow reads, not the whole estate: one source of truth per topic, owners for each.
- Settle permissions for that workflow — who may ask, and what each person’s answers may draw on.
- Agree in writing what a good result is before anything more is built.
- Decide — widen, change approach or stop — on the result, not on the demo.
When the answer must be the same every time
Some steps behind a RAG assistant — an eligibility rule, a fee calculation, a policy decision — need the same answer for the same input and a trace back to the rule that produced it. Those are better as compiled logic than as generated text.
For that, Pricient quotes IOanyT’s Proof Sprint: in three to four weeks, one workflow is compiled — LLMs write and test the logic at build time, and deterministic code runs it, so the same input gives the same answer (what Compiled AI means). It ends with a consistency and accuracy report on your own data and a written go / no-go, you keep the code either way, and the price is fixed and agreed at scoping.
Who signs off
The team that builds the assistant should not be the one that certifies it. Independent review exists for exactly that reason.
How Pricient handles a stalled Bedrock pilot
Describe what you built and where it stalled. A person reads it; this lane’s own intake is not live yet. You receive the specification and a gap list — documents, permissions, integrations, cost, what “good” means — before anyone quotes a sprint.