CloudFlowRAG LabView sources
RAG RELIABILITY LAB

Can an AI answer only when the evidence is there?

Test grounded answers, inspect the exact passage, and watch the system refuse what CloudFlow's documents cannot support.

5 source documents3 evidence statesExact passage citations
Live evidence check
Question

How is customer data protected?

Grounded answerSupported

CloudFlow encrypts customer data at rest using AES-256 and in transit using TLS 1.2 or higher. Access is limited through role-based controls and least-privilege permissions.

Evidence

The answer is directly grounded in the cited passage.

Information Security Policy3.2 Encryption and key management
Customer data is encrypted at rest using AES-256 and in transit using TLS 1.2 or higher. Production access is governed by role-based access control and the principle of least privilege.

The answer is directly supported by one passage.

01
ASK THE LAB

Start with the evidence, not the answer.

Choose a test or ask your own question. Every test runs through the live retrieval and evidence-judging pipeline. The displayed quotation is copied from the retrieved source—not written by the model.

Suggested questions
Supported

The answer is directly grounded in the cited passage.

Needs Review

Relevant evidence exists, but it does not fully support the requested claim.

No Evidence

The available documents do not contain enough evidence to answer.

Question

How is customer data protected?

Grounded answerSupported

CloudFlow encrypts customer data at rest using AES-256 and in transit using TLS 1.2 or higher. Access is limited through role-based controls and least-privilege permissions.

Evidence

The answer is directly grounded in the cited passage.

Information Security Policy3.2 Encryption and key management
Customer data is encrypted at rest using AES-256 and in transit using TLS 1.2 or higher. Production access is governed by role-based access control and the principle of least privilege.

The answer is directly supported by one passage.

02
HOW IT WORKS

Retrieval and generation are separate jobs.

1

Retrieve passages

Search the approved CloudFlow library with transparent keyword and concept matching, then send only the top passages forward.

2

Judge the evidence

Check whether the passages actually answer the question, only partially relate to it, or provide no usable evidence.

3

Answer or refuse

Generate a structured answer with an evidence status and exact citation—or refuse rather than fill the gap with a guess.

03
CONTROLLED SOURCE LIBRARY

Five fictional documents. One auditable answer boundary.

These documents were written for the lab and intentionally include similar terminology, plan limits, and near-matching facts that can confuse weak retrieval systems.

01Product and Features GuideVersion 3.2 · July 2026

CloudFlow’s supported workflows, plan boundaries, integrations, and AI-assisted features.

Indexed sections
  • Platform overview
  • Core workflows
  • Integrations
  • AI-assisted features
  • Product limitations
02Information Security PolicyVersion 4.1 · June 2026

Encryption, access controls, vulnerability management, logging, and incident response.

Indexed sections
  • Security governance
  • Encryption
  • Access control
  • Monitoring
  • Incident response
03Privacy and Data Handling PolicyVersion 2.7 · July 2026

Customer roles, data use, retention, deletion, subprocessors, and AI data handling.

Indexed sections
  • Data roles
  • Collection and use
  • Retention and deletion
  • Subprocessors
  • AI data handling
04Customer Support and Service-Level PolicyVersion 3.0 · May 2026

Support hours, severity levels, response targets, availability, exclusions, and credits.

Indexed sections
  • Support coverage
  • Severity definitions
  • Response targets
  • Availability target
  • Maintenance and exclusions
05Business Continuity and Disaster Recovery PlanVersion 2.4 · April 2026

Resilience architecture, backup cadence, recovery objectives, testing, and crisis roles.

Indexed sections
  • Continuity scope
  • Resilience design
  • Backup strategy
  • Recovery objectives
  • Testing and governance
04
EVALUATION

Reliability will be measured, not claimed.

Run the same nine labeled questions through retrieval and the model. The report keeps failed examples visible instead of presenting only the best answers.

Ready to runRetrieval accuracy

Did the expected supporting source appear in the top results?

Ready to runRefusal accuracy

Did unsupported questions receive a refusal instead of a guess?

Ready to runCitation support

Does every factual answer trace to the displayed passage?

Ready to runValid outputs

Did every response conform to the required structured schema?

Nine labeled cases · four reliability metrics

This run uses live model responses, so a future model or prompt change can improve—or reduce—the score.

05
WHERE IT BREAKS

The most useful demo includes the failures.

RAG can fail before the model ever writes an answer. The lab exposes those failure modes so a reviewer can see the tradeoffs.

Near-match trap

Thirty days sounds definitive.

One privacy passage mentions a 30-day retrieval window, but other records follow 60-, 90-, and 365-day schedules.

Expected response: Needs Review
Metric confusion

Response time is not recovery time.

A one-hour Priority 1 support response can be incorrectly cited as the disaster-recovery objective.

Expected response: cite the BCDR plan
Plausible invention

A common integration still needs proof.

Salesforce sounds reasonable for a SaaS platform, but the CloudFlow documents never confirm that integration.

Expected response: No Evidence
PROJECT NOTE

Built to make AI reliability understandable in a sales conversation.

CloudFlow is fictional. The project was designed and built by Jacob Lavian using AI-assisted development, with the architecture, evaluation choices, limitations, and source material documented for honest discussion in technical and customer-facing interviews.