Procurement Guide

1. 1

   Open the vendor's trust page

   Every certified vendor has a public attestation URL. If a vendor can't produce one, that's your answer.

2. 2

   Confirm Verified vs Assessed

   Verified means MedAttest administered the run and captured evidence. Assessed means the vendor uploaded their own results. Treat them very differently.

3. 3

   Check the tier AND the gates

   The FACT-A / B / C badge is the headline, but the real assurance comes from the non-negotiable gates: zero severity-4 (critical) omissions and zero confirmed cross-patient containment leaks.

4. 4

   Confirm the model version matches what you'd deploy

   A FACT certification is bound to the exact model version tested. If the vendor has shipped an update since, the cert no longer applies — require re-certification.

5. 5

   Read sample size and confidence intervals

   Look at n (assertions and encounters) and the width of the Wilson CI. A small-sample “0% fabrication” is not the same as a large clean sample.

6. 6

   Download the artifacts for your file

   Pull the public PDF report and the CHAI Model Card–shaped JSON export. They are designed to drop straight into your committee packet and your governance system of record.

A trust page is the single artifact that anchors the buying decision. Every certified vendor gets one, at a stable URL, and every element on it is signed and versioned.

Pick the lowest tier that is acceptable for your deployment pattern, not your wish list. Tier is a floor, not a guarantee for any specific workflow — and a single missed severity-4 (patient-safety-critical) fact, or any confirmed cross-patient containment leak, blocks every tier including FACT-C.

For decision-support workflows (e.g. prior authorization), decision accuracy is the primary metric: ≥95% for FACT-A, ≥90% for FACT-B, ≥80% for FACT-C — with the fabrication and omission gates still applied to the written rationale.

FACT distinguishes two assurance levels. They are not interchangeable, and a procurement team should be explicit about which one is acceptable.

If your committee is making a multi-year commitment or auto-actioning outputs in the chart, require Verified.

Every rate on a FACT attestation carries a 95% Wilson confidence interval, and tiering gates on the conservative (upper) bound, not the point estimate. A small-sample “0% fabrication” cannot buy a Tier A — the upper bound is still wide. Only a large, clean sample narrows the bound enough.

• Always read n assertions and n encounters alongside any headline rate.
• A 0% point estimate on 30 assertions has an upper bound near 11% — that does not pass FACT-A.
• Two vendors with identical headline rates can have very different real-world risk if their sample sizes differ.

• !Self-published “accuracy” benchmarks with no independent ground truth.
• !A cert that does not name the exact model version tested.
• !Assessed evidence presented as if it were Verified.
• !Headline rates without sample sizes or confidence intervals.
• !Refusal to be tested, or insisting on grading their own outputs.
• !Numbers that change silently between vendor decks and the attestation page.

Copy these clauses directly into your RFP or master services agreement. They are designed to be enforceable against a vendor’s public attestation.

Vendor shall maintain a current MedAttest FACT attestation at tier FACT-B or higher for the specific product and workflow deployed at [Health System]. The attestation shall be publicly accessible at a stable URL and shall be referenced by URL in this Agreement.

The FACT attestation referenced under this Agreement shall be issued at the “Verified” assurance level, meaning the certification run was administered by MedAttest via vendor API or proctored UI session. “Assessed” (vendor-submitted) attestations do not satisfy this requirement.

The model version, build identifier, and configuration profile listed on the active FACT attestation shall match the model version, build identifier, and configuration profile deployed in the [Health System] environment. Vendor shall not deploy a different model version without delivering an updated FACT attestation issued against that version.

Any material change to the underlying model, prompting strategy, or post-processing that affects clinical output shall require a new FACT certification run, completed and published prior to the change being deployed to [Health System] environments.

Vendor warrants that the active FACT attestation reflects zero severity-4 (patient-safety-critical) omissions and zero confirmed cross-patient containment findings during the certification run. Loss of either gate, on any subsequent re-certification, constitutes a material breach.

Upon request, Vendor shall provide the public FACT PDF report and the CHAI Model Card–shaped JSON export for the active attestation, suitable for ingestion into [Health System]’s AI governance system of record.

• What is your current FACT tier, and is it Verified or Assessed?
• What is the URL of your public trust page?
• What exact model version is named on the attestation, and is it the same build we would deploy?
• When was the attestation issued, and what is your re-certification cadence?
• Were there any severity-4 omissions or containment findings on the certification run, in any version?
• Can you provide the CHAI Model Card–shaped JSON export for ingestion into our governance system?
• If we update your model in production, what is your contractual commitment to re-certify before deployment?

• SOC 2 analogy. Verified attestations are the Type II analogue (continuous, proctored evidence); Assessed attestations are the Type I analogue (point-in-time, vendor-submitted). Treat them with the same skepticism.
• CHAI Model Card. Every FACT attestation includes a machine-readable export shaped to CHAI’s Model Card guidance, suitable for AI governance committees and inventory systems.
• FDA & standards momentum. FACT is an independent attestation standard, aligned with the direction of clinical AI oversight. It is not a regulatory approval and does not substitute for one where one is required.

Fabrication

A claim in the AI output that is not supported by the source encounter — a hallucination.

Omission

A clinically relevant fact present in the source that the AI failed to capture. Scored by severity.

Unsupported inference

A conclusion the AI drew that goes beyond what the source evidence supports, without inventing a new fact.

Containment

Whether information from one patient's encounter ever leaks into another patient's output. A hard pass/fail gate.

Severity

1 Minor (excluded), 2 Relevant, 3 Significant (always adjudicated), 4 Critical (a single miss blocks certification).

Assurance level

Verified (MedAttest-administered) vs Assessed (vendor-submitted).

Attestation

The signed, versioned trust page + PDF + JSON export issued for a vendor/workflow at a point in time.