Evaluating COI Software: 15 Questions Every Buyer Should Ask

Buying COI compliance software is not like buying most SaaS products. The consequences of choosing the wrong platform are not inconvenience — they are uninsured losses, audit failures, and compliance gaps that create real financial exposure. The demo always looks good. The questions below help you see past the demo.

These 15 questions are organized into five categories. For each question, we explain why it matters and what good versus bad answers look like.

Extraction and Accuracy (Questions 1–4)

Data extraction is the foundation. If the software cannot accurately read a certificate, everything built on top of that data — compliance checks, gap detection, expiration tracking — is unreliable.

1. What is your field-level extraction accuracy on real-world certificates?

Why it matters: Marketing materials often cite accuracy numbers tested on clean, high-resolution, standard-format certificates. Real-world certificates include faxed copies, low-resolution scans, handwritten annotations, stamps, and non-standard layouts. The accuracy that matters is accuracy on the documents your vendors actually submit.

Good answer: "Our field-level accuracy is 96–99% across production certificates, measured on a rolling sample of 10,000+ certificates per quarter. We publish our methodology and can provide accuracy reports by field type."

Bad answer: "We're 99.9% accurate" (without methodology), or "Our AI is trained on millions of documents" (volume of training data is not the same as measured production accuracy).

What to do: Ask for accuracy metrics broken down by field type (limits, dates, endorsements, Description of Operations). Accuracy on numerical fields (limits) is typically higher than accuracy on free-text fields (Description of Operations). Ask for production accuracy — not lab accuracy.

2. How do you handle poor-quality documents?

Why it matters: In the real world, certificates arrive as fourth-generation faxes, photos taken with a phone camera, heavily compressed JPEGs, and PDFs that were scanned at 72 DPI. If the system cannot process these, your compliance team will spend their time manually entering data from exactly the documents that are hardest to read.

Good answer: "We process all document qualities and assign confidence scores per field. Fields below our confidence threshold are flagged for human review. We show the analyst exactly which fields need attention, with the original document visible for comparison."

Bad answer: "We reject low-quality documents and ask the vendor to resubmit" (this creates operational friction and delays) or "Our OCR handles everything" (basic OCR fails badly on poor-quality documents).

3. Can you extract and interpret the Description of Operations field?

Why it matters: The Description of Operations / Locations / Vehicles section at the bottom of the ACORD 25 form is the most important — and most variable — field on any certificate. It contains critical compliance information: Additional Insured status, Waiver of Subrogation, Primary and Non-Contributory language, project-specific references, and notice provisions.

This field is free text. Agents write it differently every time. "AI: ABC Corp per written contract" and "Additional Insured as required by contract: ABC Corporation" and "Certificate holder is included as Additional Insured, Waiver of Subrogation applies, coverage is primary and non-contributory" are all conveying similar information in completely different formats.

Good answer: "We use AI language models to parse the Description of Operations and extract structured compliance data — Additional Insured status, Waiver of Subrogation, Primary & Non-Contributory, Notice of Cancellation, and project references. We handle the common language variations and flag unusual phrasing for review."

Bad answer: "We capture the full text of the Description field" (capturing text is not the same as interpreting it — if the system cannot extract structured data from free text, your analysts are still reading it manually).

4. How do you handle non-ACORD forms and international certificates?

Why it matters: Not every certificate is an ACORD 25. You will encounter ACORD 27 (Evidence of Property Insurance), ACORD 28 (Evidence of Commercial Property Insurance), carrier-specific certificate forms, and — for international vendors — certificates issued in completely different formats (Lloyd's of London certificates, certificates from European or Asian carriers).

Good answer: "We support ACORD 25, 27, and 28 natively. For non-ACORD forms, our AI model reads the document structure and extracts equivalent fields. For international certificates, we support [specific formats] and flag unknown formats for manual review with field mapping tools."

Bad answer: "We only support ACORD 25" (too limited for most organizations) or "We support all formats" (unlikely — ask for specifics).

Compliance Engine (Questions 5–8)

Once data is extracted, it must be checked against your requirements. The compliance engine determines whether a certificate meets your standards.

5. Can I define custom compliance requirements by vendor type, project, and jurisdiction?

Why it matters: A single set of insurance requirements does not fit all vendors. A janitorial company and a structural demolition contractor need different coverage types and limits. A vendor working in New York has different requirements than one working in Ohio (monopolistic WC state). Your compliance engine must support this complexity.

Good answer: "You can create unlimited requirement templates organized by vendor tier, project, trade, or jurisdiction. Templates are assigned to vendor records and applied automatically during compliance checks. You can override at the individual vendor level when needed."

Bad answer: "We have one default requirement set that applies to all vendors" (too rigid) or "Our team configures your requirements during onboarding" (if you cannot modify requirements yourself, you are dependent on their support team for routine changes).

6. How does your system handle endorsement language variations?

Why it matters: Insurance agents use dozens of different phrasings to indicate the same endorsement. "Additional Insured per CG 20 10" and "AI status granted per written contract" and "Certificate holder is included as additional insured on a primary and non-contributory basis" all convey Additional Insured status, but with different specificity and legal strength.

Good answer: "Our compliance engine uses AI to interpret endorsement language and classify it by type and strength. We distinguish between blanket Additional Insured (per written contract), scheduled Additional Insured (specifically named), and endorsement-form-referenced AI (citing a specific ISO form number). Each classification can be mapped to your acceptance criteria."

Bad answer: "We check for keywords like 'additional insured' in the Description field" (keyword matching produces both false positives and false negatives — it misses abbreviations like "AI" and triggers on negations like "additional insured status is NOT provided").

7. Do you support state-specific requirements?

Why it matters: Workers' Compensation in Ohio must come from the state fund. California requires Stop Gap endorsements for monopolistic state coverage. State minimum auto limits vary from $15,000 (Pennsylvania) to $500,000 (Michigan). Cancellation notice periods are statutory in many states. If your vendors operate across multiple states, your compliance engine must know the rules for each one.

Good answer: "We maintain a state requirement database covering monopolistic WC states, statutory cancellation notice periods, contractor licensing requirements, and state-specific endorsement rules. These are applied automatically based on the work state assigned to the vendor record."

Bad answer: "Our platform applies your organizational requirements uniformly" (meaning state-specific variation is your problem to manage manually).

8. What happens when the system cannot determine compliance with confidence?

Why it matters: No extraction system is perfect. No compliance engine can interpret every edge case. What matters is how the system handles uncertainty. Does it guess? Does it default to "compliant" (dangerous) or "non-compliant" (annoying but safe)? Does it escalate to a human reviewer with context?

Good answer: "Fields or compliance determinations below our confidence threshold are routed to your compliance team for review. The reviewer sees the extracted data, the original document, the specific requirement that could not be verified, and the AI's best interpretation with a confidence score."

Bad answer: "Our AI handles everything automatically" (no system should make final compliance determinations without a human-in-the-loop for low-confidence results).

Operations (Questions 9–11)

These questions address the day-to-day experience of using the platform.

9. Do you offer a vendor portal?

Why it matters: Without a vendor portal, every certificate collection is a manual email exchange. Your team sends a request, the vendor (or their agent) sends a PDF, your team processes it, finds gaps, emails the gaps back, waits for a corrected certificate, and repeats. A vendor portal shifts this to self-service: vendors see their requirements, upload certificates, view their compliance status, and receive automated gap notifications.

Good answer: "Our vendor portal shows each vendor their specific requirements, allows direct certificate upload (by the vendor or their agent/broker), displays real-time compliance status with specific gap details, and sends automated renewal reminders. Vendors can access their history and download past certificates."

Bad answer: "Vendors can email certificates to a dedicated inbox" (this is not a portal — it is an email alias).

10. How do expiration alerts work?

Why it matters: The single most common COI compliance failure is a missed expiration. The system's alerting cadence, escalation rules, and notification channels determine whether expirations are caught or missed.

Good answer: "Configurable alert cadence (default: 60, 30, 14 days before expiration). Alerts go to your team and to the vendor (via portal notification and email). Escalation rules allow you to define what happens if the vendor does not respond: automatic follow-up at intervals, status change to non-compliant, notification to the project or property manager."

Bad answer: "We send an email when a certificate is about to expire" (a single notification without follow-up or escalation is barely better than a calendar reminder).

11. What reporting and export capabilities are available?

Why it matters: You need to produce compliance reports for audits, for management, for investors, and for your own operational visibility. The software should make this trivial.

Good answer: "Pre-built reports for compliance status (by vendor, project, property, portfolio), expiration forecasts, gap analysis, historical compliance trends, and audit-ready exports. Custom report builder for ad hoc queries. All reports exportable to PDF and CSV. API access for feeding data into BI tools."

Bad answer: "You can export a CSV of your vendor list" (this is a data dump, not a reporting capability).

Security and Compliance (Questions 12–13)

COI certificates contain business-sensitive information: policy numbers, coverage limits, legal entity names, broker details. The platform that stores this data must meet enterprise security standards.

12. What security certifications do you hold?

Why it matters: SOC 2 Type II is the baseline expectation for any SaaS platform handling business-sensitive data. It demonstrates that the vendor has implemented and maintained controls for security, availability, processing integrity, confidentiality, and privacy — and that those controls have been independently audited.

Good answer: "SOC 2 Type II certified, with annual re-certification. We can provide our most recent SOC 2 report under NDA. We also support [ISO 27001 / HIPAA / other relevant certifications] for clients with specific regulatory requirements."

Bad answer: "We take security very seriously" (without certifications, this is meaningless) or "We're working toward SOC 2" (pre-certification means the controls have not been independently verified).

13. Where is data stored, and what is your data retention and deletion policy?

Why it matters: You need to know where your vendor data lives (US data centers? specific cloud provider?), how long it is retained, and what happens when you leave the platform. Can you export all your data? Is it deleted upon request?

Good answer: "Data stored in [specific cloud provider] US data centers with encryption at rest and in transit. Configurable retention periods. Full data export available at any time in standard formats. Data deleted within 30 days of contract termination upon request, with certificate of deletion provided."

Bad answer: "Data is stored securely in the cloud" (vague) or no clear answer on data portability (potential vendor lock-in).

Pricing (Questions 14–15)

14. What is the pricing model?

Why it matters: COI software pricing models vary: per vendor, per certificate, per user, flat fee, or hybrid. The model that works best depends on your organization's volume and growth trajectory. Per-certificate pricing penalizes high-volume operations. Per-vendor pricing is predictable. Per-user pricing limits adoption.

Good answer: Clear pricing with a model that aligns with your usage pattern. Transparent tiers. No hidden fees for features that are standard elsewhere (API access, vendor portal, reporting).

Bad answer: "Contact us for pricing" with no published information (suggests the pricing is negotiable enough to be inconsistent, or high enough that they do not want to publish it). Also watch for: per-certificate pricing with no cap (costs escalate unpredictably), or feature-gating core capabilities (vendor portal, API) behind expensive tiers.

15. What is the total cost of implementation?

Why it matters: The subscription price is not the total cost. Implementation includes: onboarding and configuration, data migration from your existing system (importing vendor records, historical certificates), user training, integration setup, and customization of requirement templates.

Good answer: "Implementation is included in the first-year subscription" or "Implementation is a one-time fee of $X, which covers configuration, data migration, training, and integration setup. Typical implementation takes 2–4 weeks."

Bad answer: "Implementation is scoped separately" with no estimate (this can be a significant hidden cost — some platforms charge $10,000–$50,000+ for implementation, data migration, and custom configuration).

How to Use This Checklist

1. Send these questions to vendors in writing before the demo. Their written responses are more reliable than verbal answers during a sales presentation.
2. Compare responses side by side. Create a scoring matrix: each question gets a score of 0 (bad/no answer), 1 (acceptable), or 2 (excellent).
3. Weight the categories based on your priorities. If you have 500+ vendors, extraction accuracy (Questions 1–4) is paramount. If you operate in 10+ states, state-specific compliance (Question 7) is critical.
4. Request a proof of concept. Upload 10–20 of your actual certificates (not clean samples) and evaluate extraction accuracy, compliance checking, and the user experience.
5. Talk to references. Ask the vendor for references in your industry and at your scale. Ask those references about the gaps between the sales pitch and the reality.

---

The right COI compliance software pays for itself through labor savings and risk reduction. The wrong software creates a false sense of security — showing green compliance statuses while gaps persist in the underlying data. These 15 questions help you tell the difference.