AletheionGuard

Verdict System

How AletheionGuard decides between ACCEPT, MAYBE, and REFUSED based on Q1, Q2, and Height

✅

ACCEPT

Low uncertainty. Model is confident and likely correct.

Conditions:
Q1 < 0.35 AND
Q2 < 0.35 AND
Total Uncertainty < 0.30
⚠️

MAYBE

Moderate uncertainty. Question may be ambiguous.

Conditions:
Q1 ≥ 0.35 OR
0.30 ≤ Total Uncertainty < 0.60
🚫

REFUSED

High uncertainty. Model lacks knowledge or hallucination risk.

Conditions:
Q2 ≥ 0.35 OR
Total Uncertainty ≥ 0.60

Decision Logic

# Official verdict decision logic
u = 1.0 - height # Total uncertainty = sqrt(Q1² + Q2²)
if q2 >= 0.35 or u >= 0.60:
verdict = "REFUSED" # High epistemic OR total uncertainty
elif q1 >= 0.35 or (0.30 <= u < 0.60):
verdict = "MAYBE" # High aleatoric OR moderate total uncertainty
else:
verdict = "ACCEPT" # Low uncertainty

Key Insight

The order matters: REFUSED takes priority over MAYBE. If epistemic uncertainty (Q2) is high, the verdict is always REFUSED regardless of Q1, because the model fundamentally lacks knowledge.

Verdict Breakdown

✅ ACCEPT - High Confidence

What it Means

  • • Question is clear and unambiguous
  • • Model has strong knowledge about the topic
  • • Low risk of hallucination
  • • Response is likely accurate

Recommended Action

  • • ✅ Safe to use response automatically
  • • ✅ Can be shown to end users
  • • ✅ Minimal human review needed
  • • ✅ High trust in model output
# Example: ACCEPT verdict
question = "What is 2 + 2?"
result = auditor.evaluate(question)
Q1: 0.05 # Clear question
Q2: 0.03 # Model knows math
Height: 0.94
Verdict: ACCEPT

⚠️ MAYBE - Moderate Uncertainty

What it Means

  • • Question may be ambiguous or subjective
  • • Multiple valid interpretations possible
  • • Model has knowledge but context is unclear
  • • Inherent data noise (high Q1)

Recommended Action

  • • 🔍 Ask user for clarification
  • • 🔍 Request more specific question
  • • 🔍 Show response with disclaimer
  • • 🔍 Include confidence score in UI
# Example: MAYBE verdict (high Q1)
question = "What's the capital of Netherlands?"
result = auditor.evaluate(question)
Q1: 0.42 # Ambiguous (Amsterdam vs The Hague)
Q2: 0.12 # Model knows both cities
Height: 0.56
Verdict: MAYBE

🚫 REFUSED - High Epistemic Uncertainty

What it Means

  • • Model lacks sufficient knowledge
  • • High risk of hallucination
  • • Out-of-distribution query
  • • Topic outside training data

Recommended Action

  • • 🛑 Do NOT use response automatically
  • • 🛑 Escalate to human expert
  • • 🛑 Retrieve additional context (RAG)
  • • 🛑 Flag for manual review
# Example: REFUSED verdict (high Q2)
question = "What will Bitcoin cost tomorrow?"
result = auditor.evaluate(question)
Q1: 0.22 # Question is clear
Q2: 0.68 # Future is unknowable
Height: 0.28
Verdict: REFUSED

Decision Tree Visualization

Start
|
Is Q2 ≥ 0.35?
/ \
YES NO
| |
REFUSED Is total_u ≥ 0.60?
/ \
YES NO
| |
REFUSED Is Q1 ≥ 0.35?
/ \
YES NO
| |
MAYBE Is 0.30 ≤ total_u < 0.60?
/ \
YES NO
| |
MAYBE ACCEPT

Threshold Reference

MetricThresholdIf ExceededReason
Q2 (Epistemic)≥ 0.35REFUSEDModel lacks knowledge
Total Uncertainty≥ 0.60REFUSEDCombined uncertainty too high
Q1 (Aleatoric)≥ 0.35MAYBEQuestion is ambiguous
Total Uncertainty0.30 - 0.59MAYBEModerate uncertainty
All MetricsBelow thresholdsACCEPTLow uncertainty, high confidence

Real-World Examples

Healthcare Q&A

Q: "What is normal heart rate?"

Q1: 0.12, Q2: 0.08, Height: 0.86

✅ ACCEPT - Safe to answer

Q: "Is my headache serious?"

Q1: 0.45, Q2: 0.18, Height: 0.52

⚠️ MAYBE - Need more context

Q: "Should I take drug X?"

Q1: 0.22, Q2: 0.71, Height: 0.26

🚫 REFUSED - See doctor

Customer Support Automation

Q: "How do I reset password?"

Q1: 0.08, Q2: 0.05, Height: 0.91

✅ ACCEPT - Auto-respond

Q: "Why was I charged?"

Q1: 0.38, Q2: 0.25, Height: 0.55

⚠️ MAYBE - Ask for details

Q: "Legal issue with contract"

Q1: 0.15, Q2: 0.82, Height: 0.17

🚫 REFUSED - Escalate to agent

RAG System Optimization

Verdict: ACCEPT (Q2 = 0.08)

Model has sufficient knowledge. Return answer directly without additional retrieval.

Verdict: MAYBE (Q1 = 0.41)

Question is ambiguous. Ask user to clarify before retrieving documents.

Verdict: REFUSED (Q2 = 0.67)

High epistemic uncertainty. Trigger additional retrieval (k=10 → k=20) to provide more context.

Implementation Example

from aletheion_guard import EpistemicAuditor
auditor = EpistemicAuditor()
def handle_query(query: str, llm_response: str):
# Audit the LLM response
audit = auditor.evaluate(llm_response)
if audit.verdict == "ACCEPT":
return {
"response": llm_response,
"confidence": audit.height,
"action": "show_to_user"
}
elif audit.verdict == "MAYBE":
return {
"response": llm_response,
"confidence": audit.height,
"action": "ask_clarification",
"message": "Could you provide more details?"
}
else: # REFUSED
return {
"response": "I don't have enough confidence to answer this.",
"confidence": audit.height,
"action": "escalate_to_expert",
"q2": audit.q2 # For debugging
}

Next Steps

🧠 Epistemic Uncertainty

Understand Q2 and why it triggers REFUSED

📊 Q1 vs Q2 Deep Dive

Mathematical foundations and thresholds

💻 Code Examples

Implement verdict-based routing

🎮 Try It Live

See verdicts in real-time

Questions about Verdicts?

Learn how to tune thresholds for your specific use case

Contact SupportAPI ReferenceCommunity Forum