How to QA every support call instead of a sample

Why sampling falls short

Traditional call quality assurance works on a sample. A QA analyst pulls a few calls per agent each week, listens to them against a checklist, and records a score. The method made sense when listening was the only way to know what happened on a call, and a person can only listen to so many hours.

The weakness is that the calls that matter most are the rare ones. A wrong answer about a refund, a promise the agent had no authority to make, a customer who says they are leaving: these are exactly the calls a small random sample is likely to miss. Agents also know their score depends on which calls happened to be picked, which makes the feedback feel like luck rather than a fair picture of their work.

Reading every call changes the job of the QA team. Instead of hunting for problems, reviewers check the calls that were already flagged and spend their time on judgment and coaching.

Scorecards that belong to each department

A billing support line, a technical support queue and an inside sales team should not be graded on the same form. In Callens, scorecards use each department's own criteria and weights. The billing team can weight correct policy explanation heavily; the sales team can weight discovery and next steps.

Because every call is read, the scorecard applies to every call the department takes, not to the handful that were sampled. Numbers roll up by company, department or person, with weekly benchmarks against the department or the company that are withheld when a group is too small to be fair.

Measured criteria and judged criteria

A good scorecard separates what can be counted from what needs interpretation, and shows its working for both.

• Measured criteria are talk ratio, open questions, interruptions and the longest monologue, drawn from the 22 conversation measures. They are computed from word-level timings, not from a language model, and the scorecard shows the number.
• Judged criteria are questions like "did the agent confirm the issue was resolved?" or "did the agent explain the cancellation terms?". The scorecard shows the quote each judgment rests on, so a reviewer can agree or disagree with evidence in front of them.
• Every model-produced line is checked against a real segment of the transcript before it is stored, and each claim points to the second of the recording.

Policy and compliance checks

Callens types risks as churn, compliance, competitor, pricing or expectations, and grades each one low, medium or high. A compliance risk on a support call might be an agent promising a refund outside policy, or skipping a required statement. A churn risk might be a customer saying they are comparing other providers.

Signed webhooks (HMAC-SHA256) fire on completion, on high risk, on a policy breach or on a low score. That lets a team route those calls into the tools where reviewers and supervisors already work, instead of waiting for the next sampling round. Every change is logged field by field, and access to recordings has its own audit trail.

From scores to follow-up and coaching

Quality assurance is only useful if something happens afterwards. Commitments, meaning what was promised, by whom and by when, are lifted out of each call into one team queue, so a promised callback does not depend on the agent's memory. Coaching plans set one active target per person and measure it against the calls that follow, which shows whether the feedback changed anything.

Roles are owner, admin, manager, reviewer and viewer, so QA reviewers can work through calls without needing admin rights. Any call exports as a PDF report, with or without the transcript, and any filtered list of calls exports as CSV for a weekly QA review.