Quality Assurance Standard Operating Procedure

1. Purpose & Scope

This Standard Operating Procedure (SOP) defines how Quality Assurance (QA) reviews are planned, executed, reviewed, and closed across customer support interactions. It applies to both self-serve client teams running their own QA programs and managed-service (Done-For-You) specialists operating QA on behalf of clients.

The objective is to deliver fair, consistent, and transparent agent feedback that drives measurable performance improvement — without the overhead of manual spreadsheet grading or biased one-off reviews.

2. Roles & Responsibilities

• Reviewer / Specialist — Scores tickets using the scorecard, completes WIN coaching notes, and submits the review for approval.
• Secondary Auditor (AI Advanced) — Reviews a percentage of AI-generated scores, agreeing, overriding, or flagging for discussion to eliminate single-person bias.
• Leadership / Team Admin — Reviews and approves all scores before they are finalized, resolves agent disputes, and owns program configuration.
• Agent — Receives scored feedback, can view full scoring rationale, and may dispute a score they believe is unfair.
• Admin — Owns client and team configuration, Zendesk connections, scorecard templates, and role/permission assignments.

3. The QA Cycle

The QA cycle repeats on a defined cadence (daily, weekly, or monthly) set during program setup. Each cycle moves through seven phases: Program Setup → Queue Generation → Scoring → Leadership Review → Agent View & Dispute → Dispute Resolution → Finalization & Reporting.

Phase 1 — Program Setup

1. 1Connect your help desk (Zendesk): navigate to Zendesk Integration and authenticate with your subdomain and API token.
2. 2Build or select a scorecard: define 5–15 quality questions, assign max points per question, set the pass threshold (typically 80%), and add channel-specific custom instructions if needed.
3. 3Create a QA Program: name the program, set the review frequency (daily/weekly/monthly), reviews per agent per period, the pass threshold, and link the scorecard.
4. 4Define sampling rules: filter by channel, tags, groups, agents, keywords, and ticket status (default: solved). Set the sample size per generation (e.g., 5 tickets per agent).
5. 5Optional — enable automation: auto-generate the queue and/or auto-score with AI on your chosen cadence.

Phase 2 — Queue Generation

Generate the review queue from the QA Program. The system samples tickets matching your sampling rules and presents a list ready for scoring. Queues can be generated manually or automatically on schedule.

Phase 3 — Scoring

Manual scoring: open each ticket, answer each scorecard question using the rubric (scoring types include 2-1-0-NA, yes-no, 1-0-NA, and pass-fail-na), and submit. An overall score and pass/fail status are calculated automatically against the threshold.

AI scoring: submit the queue to AI. The AI analyzes each ticket's transcript and proposes a score with reasoning and a confidence level (high/medium/low). AI excels at text-based interactions (email, chat, social); phone recordings are scored manually.

For every review, complete the WIN coaching notes: What Worked, Improvement Area, and Next Step.

Phase 4 — Leadership Review & Approval

No score is finalized until leadership reviews and approves it. Leadership examines the AI's reasoning (or the manual reviewer's answers), the agent's dispute notes (if any), and the WIN coaching summary, then accepts or overrides the score.

In AI Advanced, secondary auditors review AI scores first; leadership then reviews both the AI score and the auditor's feedback before finalizing.

Phase 5 — Agent View & Dispute

Once approved, agents see their scored tickets, the full scoring explanation, and the WIN coaching notes. If an agent disagrees with a score, they can submit a dispute note for leadership review.

Phase 6 — Dispute Resolution

Disputed scores route to the Specialist Dispute Queue. The reviewer or leadership reviews the agent's note, re-examines the ticket against the scorecard, and either upholds or adjusts the score. The resolution is recorded and communicated back to the agent.

Phase 7 — Finalization & Reporting

Approved and dispute-resolved scores are finalized. Finalized reviews feed the QA Metrics dashboard, which tracks pass rates, average scores, top failure reasons, and trends over time. Use these metrics to calibrate the scorecard, coach agents, and report to leadership.

4. Pass / Fail Threshold

The default pass threshold is 80%. A ticket scoring at or above the threshold is marked Pass; below is marked Fail. The threshold is configurable per scorecard and per program.

5. Review Frequency

• Daily — high-volume or high-stakes support operations needing rapid feedback loops.
• Weekly — the most common cadence; balances coaching timeliness with reviewer workload.
• Monthly — lower-volume teams or leadership-level program overviews.

6. Dispute Handling

Agents may dispute any score they believe is inaccurate or unfair. Disputes are submitted with a written note. Leadership (or a designated specialist) reviews each dispute in the Dispute Review queue, re-scores if warranted, and documents the outcome. Dispute patterns are monitored to identify scorecard ambiguity or reviewer calibration gaps.

7. WIN Coaching Framework

Every review includes WIN coaching notes — the standard feedback format delivered to agents:

• W — What Worked: the specific behavior or skill the agent demonstrated well.
• I — Improvement Area: the one highest-impact area to focus on next.
• N — Next Step: a concrete, actionable step the agent can take to improve.

8. Tier Differences

• Free Scorecard — manual one-time scoring; results emailed to the agent; no dispute workflow or historical tracking.
• AI Standard — automated sampling, AI-proposed scores, leadership approval, agent dispute workflow, and a metrics dashboard.
• AI Advanced — adds a secondary auditor layer between AI scoring and leadership approval for enterprise-grade fairness, plus team calibration tracking.

9. Record Keeping & Data Retention

All QA reviews, AI scores, dispute responses, and coaching notes are stored as records linked to the originating ticket and agent. Finalized reviews remain accessible for historical tracking and reporting. Zendesk-imported tickets retain their original metadata (channel, tags, assignee, status) for audit traceability.

10. Tools Reference

• Quick Ticket Audit — /tools/qa-review (Free manual scoring)
• Quality Program Builder — /tools/qa-program-setup (Scorecards, programs, queues)
• Execute QA — /tools/execute-qa (AI scoring and queue management)
• Automate QA — /tools/qa-automate (Scheduled generation and auto-scoring)
• QA Review Hub — /tools/qa-review-hub (Flagged scores, disputes, finalized QA)
• View QA Metrics — /tools/qa-reports (Pass rates, trends, failure reasons)
• Zendesk Integration — /tools/zendesk (Connect, import, sample)