AI call QA glossary
The terms that come up when a QA programme moves from manual sampling to automated scoring, defined plainly and without vendor language.
Automated call scoring
Software evaluates a recorded call against a defined standard and produces a result without a person listening to it first. The standard is usually the organisation’s own scorecard, not a generic model of good service.
Scorecard
The set of criteria a call is judged against, with the weight and pass threshold for each. A scorecard is the organisation’s statement of what good looks like, and automated scoring is only as sound as the scorecard it is pointed at.
Evaluation
A single pass of a scorecard over a single call, producing a score plus the per-criterion judgements behind it. One call reviewed against a ten-point scorecard yields ten judgements, not one.
Evidence-linked scoring
Each score is stored with the part of the transcript it was derived from, so a disputed result can be shown rather than argued.
Pass mark
The threshold a call must reach to be considered acceptable. Setting it is a business decision rather than a technical one, and it should be documented so a failing result can be explained.
Fatal error
A criterion where any single occurrence fails the call regardless of the total score, such as a missed disclosure, a prohibited phrase or an unresolved compliance statement.
Confidence score
A measure of how certain the system is about a judgement. It is not a measure of whether the call was good, because a high-confidence wrong answer is still wrong.
Population
Every call the result is meant to describe, usually all recorded calls in the period. It is not every call, because a recording opt-out means some calls are never recorded.
Sample size
The number of calls that must be reviewed for a score to be statistically valid at a stated confidence and margin of error. It is a calculated number, not a convention.
Margin of error
How far a sampled result may sit from the true figure. A five per cent margin on a score of 82 means the real figure is likely between 77 and 87.
Coverage
Reviewed calls as a proportion of the population. At 5,000 calls a month, a statistically valid sample of 357 is about seven per cent coverage.
Finite population correction
An adjustment applied when the sample is a large fraction of the population, so that sample size converges instead of growing without limit.
QA calibration
The process of aligning reviewers so the same call receives the same score from different people. Calibration measures reviewer agreement and is the check that keeps a scorecard honest.
Inter-rater reliability
The degree to which independent reviewers reach the same judgement on the same call. Low reliability means the score is measuring the reviewer rather than the call.
Exception handling
The route a flagged call takes to a human decision. The software surfaces the call and the person decides the outcome.
Human in the loop
A person retains authority over the final judgement. It is distinct from a person merely glancing at an automated result without the ability to change it.
Conduct risk
The risk that a customer interaction itself breaches an obligation, such as mis-selling, unsuitable advice, prohibited pressure or a missed disclosure.
Automated decision-making transparency
The requirement to disclose the kinds of personal information used by computer programs and the kinds of decisions those programs support or make, commencing 10 December 2026 under Australian privacy law.
If a term here is missing and you need it defined for a procurement document, ask and we will add it.