Handbook
Validation, Verification, and Assurance Intelligence
Establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation…
Updated
Definition
Establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status.
Coverage role
Provides the independent assurance layer for every other intelligence. Verification asks whether declared requirements were met; validation asks whether the result is fit for its intended use and stakeholder need.
Scope
Included
- validation object and intended use
- acceptance and assurance criteria
- claim and requirement traceability
- inspection analysis demonstration and testing
- formal and property-based checks
- evidence triangulation
- adversarial and boundary testing
- reproducibility and independence
- discrepancy adjudication
- residual risk and release status
Excluded and non-goals
- self-certification by the generating model
- using a checklist as proof without evidence
- claiming legal regulatory or professional certification without authority
- hiding failed tests
- validating outside the declared version scope population or operating conditions
- confusing plausibility fluency confidence or popularity with correctness
Boundary rules
- Route to general.validation-verification when the requested outcome requires one or more declared capabilities and the output can be represented by its canonical artifacts.
- Route to an adjacent or specialist intelligence when domain-specific rules, tools, or professional authority dominate the problem.
- Use general.reasoning to coordinate multi-pack conflicts, hard constraints, uncertainty, and decision status.
- Return ask, defer, or escalate when required context, source authority, consent, or accountable ownership is missing.
Capability semantics
| Capability ID | Name | Semantic intent |
|---|---|---|
capability.general.validation-verification.define_validation_object_and_stakes |
Define Validation Object And Stakes | Identify exactly what version claim artifact model process or system is being checked, for whom, for what use, and at what consequence level. |
capability.general.validation-verification.derive_assurance_criteria |
Derive Assurance Criteria | Convert needs requirements invariants policies hazards and claims into observable pass fail or inconclusive criteria. |
capability.general.validation-verification.build_traceability_matrix |
Build Traceability Matrix | Link each material claim or requirement to evidence, method, validator, owner, result, and unresolved gap. |
capability.general.validation-verification.select_verification_methods |
Select Verification Methods | Choose inspection analysis demonstration testing formal proof simulation peer review or independent reproduction according to claim type and risk. |
capability.general.validation-verification.validate_intended_use_and_stakeholder_fit |
Validate Intended Use And Stakeholder Fit | Check that the result addresses the real need, population, context, usability, accessibility, and unacceptable outcomes. |
capability.general.validation-verification.run_boundary_adversarial_and_robustness_tests |
Run Boundary Adversarial And Robustness Tests | Test edge cases counterexamples perturbations misuse scenarios distribution shifts and failure conditions. |
capability.general.validation-verification.triangulate_and_reproduce_evidence |
Triangulate And Reproduce Evidence | Compare independent sources methods tools reviewers or runs and distinguish reproduction from replication. |
capability.general.validation-verification.adjudicate_discrepancies |
Adjudicate Discrepancies | Classify failures as defect evidence gap criterion problem environment issue or accepted deviation with owner and expiry. |
capability.general.validation-verification.issue_bounded_validation_status |
Issue Bounded Validation Status | Report pass fail conditional inconclusive or not-tested status, validation domain, residual risk, release decision, and reopen triggers. |
Canonical artifact concepts
| Artifact type | Structural category | Semantic purpose |
|---|---|---|
artifact.general.validation-verification.validation_charter |
brief | Validation Charter for Validation, Verification, and Assurance Intelligence: A bounded specification of purpose, stakeholders, requirements, constraints, assumptions, and success criteria. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.assurance_and_acceptance_criteria |
acceptance_contract | Assurance And Acceptance Criteria for Validation, Verification, and Assurance Intelligence: A traceable requirement and acceptance contract linking each obligation to an observable criterion, verifier, owner, and status. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.claim_requirement_traceability_matrix |
model | Claim Requirement Traceability Matrix for Validation, Verification, and Assurance Intelligence: A purpose-bounded representation of entities, relationships, assumptions, boundaries, and evidence of adequacy. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.verification_and_validation_plan |
plan | Verification And Validation Plan for Validation, Verification, and Assurance Intelligence: A governed action design that sequences work, ownership, dependencies, timing, contingencies, and review. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.test_and_inspection_suite |
structured_output | Test And Inspection Suite for Validation, Verification, and Assurance Intelligence: A typed domain output that preserves purpose, context, evidence, assumptions, status, and next action. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.evidence_triangulation_matrix |
model | Evidence Triangulation Matrix for Validation, Verification, and Assurance Intelligence: A purpose-bounded representation of entities, relationships, assumptions, boundaries, and evidence of adequacy. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.adversarial_and_robustness_report |
analysis | Adversarial And Robustness Report for Validation, Verification, and Assurance Intelligence: An evidence-linked assessment that states criteria, findings, uncertainty, limitations, status, and action. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.discrepancy_and_deviation_log |
ledger | Discrepancy And Deviation Log for Validation, Verification, and Assurance Intelligence: A controlled collection of uniquely identified entries with ownership, provenance, status, and review state. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.validation_report |
analysis | Validation Report for Validation, Verification, and Assurance Intelligence: An evidence-linked assessment that states criteria, findings, uncertainty, limitations, status, and action. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.residual_risk_and_release_decision |
decision | Residual Risk And Release Decision for Validation, Verification, and Assurance Intelligence: A bounded decision or routing record that exposes alternatives, governing constraints, evidence, rationale, ownership, status, and reopening conditions. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
artifact.general.validation-verification.intended_use_and_stakeholder_fit_assessment |
analysis | Intended Use And Stakeholder Fit Assessment for Validation, Verification, and Assurance Intelligence: An evidence-linked assessment that states criteria, findings, uncertainty, limitations, status, and action. It supports the pack purpose of establish warranted confidence in a claim, model, plan, decision, artifact, process, or system by defining criteria, tracing evidence, applying independent checks, exposing discrepancies, and issuing a bounded validation status. |
Method families and limits
| Method ID | Method | Use when | Avoid when |
|---|---|---|---|
method.general.validation-verification.requirements_verification_and_intended_use_validation |
requirements verification and intended-use validation | Use when a claim, requirement, artifact, process, model, or system needs bounded evidence of conformance and fitness for intended use. | Avoid self-certification by the generator alone, tests without traceability to claims, or a pass status that omits untested scope and residual risk. |
method.general.validation-verification.inspection_analysis_demonstration_and_test |
inspection analysis demonstration and test | Use when a claim, requirement, artifact, process, model, or system needs bounded evidence of conformance and fitness for intended use. | Avoid self-certification by the generator alone, tests without traceability to claims, or a pass status that omits untested scope and residual risk. |
method.general.validation-verification.formal_proof_and_constraint_solving |
formal proof and constraint solving | Use when variables, domains, rules, invariants, or proof obligations can be represented formally and a witness, counterexample, or unsatisfied core is decision-relevant. | Avoid encoding ambiguous natural-language rules as hard constraints without review or treating solver satisfiability as evidence that the real-world model is complete. |
method.general.validation-verification.property_based_and_metamorphic_testing |
property-based and metamorphic testing | Use when correctness can be probed through invariants, generated cases, transformations, reference outputs, or independent implementations. | Avoid relying on a narrow golden set, nondeterministic oracles without tolerance, or generated cases that violate the real input domain. |
method.general.validation-verification.golden_set_and_differential_testing |
golden-set and differential testing | Use when correctness can be probed through invariants, generated cases, transformations, reference outputs, or independent implementations. | Avoid relying on a narrow golden set, nondeterministic oracles without tolerance, or generated cases that violate the real input domain. |
method.general.validation-verification.peer_review_and_independent_verification_and_validation |
peer review and independent verification and validation | Use when a claim, requirement, artifact, process, model, or system needs bounded evidence of conformance and fitness for intended use. | Avoid self-certification by the generator alone, tests without traceability to claims, or a pass status that omits untested scope and residual risk. |
method.general.validation-verification.source_triangulation |
source triangulation | Use when the reliability of a conclusion depends on locating original evidence, comparing independent sources, preserving scope, and identifying conflicts. | Avoid equating repetition with independence, citing summaries for claims available in primary sources, or merging conflicting evidence into a single unsupported score. |
method.general.validation-verification.reproduction_replication_and_calibration |
reproduction replication and calibration | Use when confidence depends on repeating a result, independently reimplementing it, comparing it with reference data, or estimating systematic measurement error. | Avoid calling a same-code rerun an independent replication, hiding deviations from the original conditions, or treating agreement within one dataset as general validity. |
method.general.validation-verification.red_team_boundary_and_misuse_testing |
red-team boundary and misuse testing | Use when a claim, requirement, artifact, process, model, or system needs bounded evidence of conformance and fitness for intended use. | Avoid self-certification by the generator alone, tests without traceability to claims, or a pass status that omits untested scope and residual risk. |
Pack-specific invariants
The pack also inherits the core invariants in core/core_invariants.yaml.
| Invariant ID | Statement | Failure action |
|---|---|---|
invariant.general.validation-verification.01_verification_and_validation_are_distinguished_and_both_ref |
Verification and validation are distinguished and both reference an explicit object version and intended use. | repair_or_conditional_pass |
invariant.general.validation-verification.02_criteria_are_defined_before_final_evaluation_or_any_post_h |
Criteria are defined before final evaluation or any post-hoc criteria are labeled and justified. | repair_or_conditional_pass |
invariant.general.validation-verification.03_every_material_pass_claim_has_traceable_evidence_from_a_me |
Every material pass claim has traceable evidence from a method appropriate to the claim type. | repair_or_conditional_pass |
invariant.general.validation-verification.04_generator_self_confidence_is_never_sufficient_validation_e |
Generator self-confidence is never sufficient validation evidence. | repair_or_conditional_pass |
invariant.general.validation-verification.05_failed_skipped_flaky_inconclusive_and_out_of_scope_checks |
Failed skipped flaky inconclusive and out-of-scope checks remain visible. | repair_or_conditional_pass |
invariant.general.validation-verification.06_validator_independence_and_strength_are_proportional_to_co |
Validator independence and strength are proportional to consequence and conflict of interest. | repair_or_conditional_pass |
invariant.general.validation-verification.07_the_status_is_bounded_to_declared_conditions_population_da |
The status is bounded to declared conditions population data version and time window. | repair_or_conditional_pass |
invariant.general.validation-verification.08_residual_uncertainty_risk_accepted_deviations_and_reopenin |
Residual uncertainty risk accepted deviations and reopening triggers accompany release. | repair_or_conditional_pass |
invariant.general.validation-verification.09_certification_accreditation_and_professional_sign_off_are |
Certification accreditation and professional sign-off are claimed only when the responsible authority provides them. | repair_or_conditional_pass |
Dependencies and relations
Operational general intelligences: general.question-framing
Foundation concepts: general.logical-constraint, general.epistemic-evidence, general.metacognitive, general.diagnostic, general.strategic-adversarial, general.tool-computational, general.ethical-normative
Life intelligences: None
Source roles
| Source ID | Claim roles | Runtime resolution required |
|---|---|---|
nasa.systems_engineering |
concept_definition, engineering_method, verification_and_validation | False |
nasa.systems_modeling |
concept_definition, engineering_method, verification_and_validation | False |
nap.reproducibility_replicability |
concept_definition, research_and_evidence_practice | False |
nist.engineering_statistics |
concept_definition, measurement_or_security_practice, risk_control | False |
Risk and authority boundary
Risk class: moderate
- The result would be represented as regulatory professional clinical legal financial safety or security certification.
- Independent evidence is unavailable for a high-consequence claim yet release is requested.
- Validation reveals a credible immediate hazard rights violation security exposure or material deception.
Accountable people retain consent, value, regulated, irreversible, and residual-risk decisions. This semantic page does not claim a deployed implementation.
Maturity
semantic: S3_reviewable_definition
contract: C3_testable_contract
implementation: I0_not_included
evidence: E2_source_roles_mapped
governance: G2_controls_and_review_defined
overlay: O2_paired_seed_complete
Paired overlay
Runtime maturity
- Capabilities: 9
- Technical floor→ceiling:
I1_runtime→I2_integrated - Autonomy floor→ceiling:
A0_assisted→A0_assisted - I1_runtime count: 8
- Evidence class:
contract: 8,semantic: 1 - Full hierarchy: MATURITY-HIERARCHY.md
Evaluation suite status
- CI status:
pass - Suite:
evaluation_suites/validation-verification.yaml - Cases defined: 7
- Capabilities under test: 9