Quality Metrics
Quality metrics is the organisation's ability to instrument its own quality system: selecting the indicators that genuinely reflect its state and trajectory, producing them reliably from trustworthy data, and — the part that makes this a capability rather than a dashboard — acting on what they show. ICH Q10 expects process performance and product quality to be monitored and the results to flow into management review, and the periodic product review that 21 CFR 211 requires serves the same end product by product. The craft is composition: lagging measures of outcome balanced with leading measures of condition, so that the picture predicts as well as records.
What this page does not claim
A capability is something an organization must be able to do; it is not a maturity score and not an assessment domain. The scored domains measure how consistently capabilities are performed, they do not map one-to-one, and nothing on this page rates your organization.
What this capability is
Quality metrics is the organisation's ability to instrument its own quality system: selecting the indicators that genuinely reflect its state and trajectory, producing them reliably from trustworthy data, and — the part that makes this a capability rather than a dashboard — acting on what they show. ICH Q10 expects process performance and product quality to be monitored and the results to flow into management review, and the periodic product review that 21 CFR 211 requires serves the same end product by product. The craft is composition: lagging measures of outcome balanced with leading measures of condition, so that the picture predicts as well as records.
The capability's permanent hazard is that measures change behaviour. A closure-time target reliably produces on-time shallow investigations; a deviation-count target reliably produces unrecorded deviations. Mature organisations design against this — pairing speed metrics with quality metrics, watching for the distortions each target invites, and revising measures that have started managing the organisation instead of informing it. They also resist metric sprawl: a small set, well-chosen, honestly produced, and actually discussed beats a hundred-tile dashboard that no decision has ever touched.
WHY IT MATTERS
- Attention and resources follow what is measured. An unmeasured backlog, an untrended recurrence rate, an invisible training gap — each is a risk the organisation has chosen not to see, and the choosing was done by the metric set.
- Metrics are the shared language between quality and general management. Trends, capability figures, and cost-of-quality numbers translate quality-system state into the terms in which budgets and priorities are decided — without them, quality argues from anecdote.
- Badly designed metrics are worse than none: they actively teach the organisation to hide deviations, close investigations early, and game thresholds. Every target should be assumed to distort until it has been examined for how.
- Regulatory thinking leans increasingly on measurement as evidence of oversight maturity — the product quality review, management review inputs, and quality-culture assessment all presuppose an organisation that can measure itself honestly.
[ POSITION IN THE FRAMEWORK ]
7 DIMENSIONS · 23 LINKSQuality metrics instruments the framework itself: leading and lagging indicators chosen to predict rather than flatter, produced from source-system data, and used to direct attention and resources to where risk actually sits.
06 · QUALITY MATURITY — QUALITY METRICS, REACTIVE TO ADAPTIVE
Metrics exist where a report demands them, assembled by hand, defined inconsistently between sites, and reviewed as history. Targets, where present, drive the numbers rather than the condition the numbers were meant to represent.
A defined metric set with owners and definitions is produced on a cadence and reviewed in quality forums. It is dominated by lagging counts — deviations raised, CAPAs closed — and its relationship to decisions is weak: the numbers are discussed, then the meeting moves on.
The set balances leading and lagging indicators, definitions are standardised across sites, and data flows from source systems rather than spreadsheet assembly. Metrics carry response rules — thresholds that trigger analysis — and known gaming risks are countered by paired measures.
Metrics are analysed, not just reported: correlations across indicators expose systemic causes, site and product comparisons direct investment, and the metric system itself is reviewed — the set periodically pruned and redesigned as the organisation's questions change.
Measurement approaches foresight: patterns across the metric base flag emerging risk before thresholds are breached, the cost and value of quality are quantified credibly enough to shape strategy, and the organisation demonstrably reallocates effort on metric evidence — the instrument steering, not merely displaying.
SPEQ’s shared five-stage progression, labelled synthesis — not the FDA QMM rating scale. Where does your organization sit? Score your quality system →
07 · REGULATORY & EVIDENCE
GOVERNING STANDARDS · 3
Derived from the 3 standards SPEQ maps to this subject, across 3 regulatory bodies: FDA, ICH, ISO.
RECORDS & OBJECTIVE EVIDENCE
- A defined metric set with documented definitions and owners
- Product quality reviews performed per product, on schedule
- Trend reports flowing into management review as required inputs
- Thresholds with defined response rules and evidence of response
- Metric data traceable to source systems, not manual assembly
COMMON INSPECTION FINDINGS
- Annual product reviews late, incomplete, or missing products
- Metric definitions drifting between sites until numbers cannot compare
- Thresholds breached with no documented analysis or action
- Closure-time targets met by investigations closed shallow at the deadline
- Dashboards reviewed on cadence with no decision ever recorded
HOW YOU’D SEE WHERE YOU SIT
- Pick a metric on the dashboard and ask for the last decision it changed — a measure that has never altered attention, resources, or practice is decoration.
- Whether deviation counts fell during a period when the process demonstrably did not improve — the signature of a reporting culture responding to a target.
- How a metric is produced: flowing from the system of record, or assembled monthly in a spreadsheet whose owner is the only person who understands it.
- Whether any metric has been retired or redesigned in the last two years — a static set means no one is asking whether the instrument still measures what matters.
- The balance of the set: count the leading indicators, and note whether investigation-quality measures exist alongside investigation-speed ones.
Observable behaviours, not a self-rating — what a capability looks like from the outside, the same way SPEQ’s Quality Culture assessment reads behaviour rather than felt safety.
FREQUENTLY ASKED
What is the difference between leading and lagging quality indicators?
Lagging indicators measure outcomes that have already happened: batch rejections, complaints, recalls, deviations raised. They are reliable, auditable, and too late — by the time they move, the cause has been operating for months. Leading indicators measure the conditions that produce those outcomes: training currency, CAPA backlog age, trend-signal response times, the ratio of problems found internally versus externally. They are noisier and need more interpretation, but they move first. A mature metric set holds both deliberately: lagging measures to confirm where you are, leading measures to warn where you are going — and the discipline to act on the second before the first confirms it.
Can quality metrics be gamed — and what do you do about it?
Assume every target distorts behaviour until you have worked out how, because the classic failures are well documented: closure-time targets that buy shallow investigations, deviation-count goals that suppress reporting, right-first-time figures polished by reclassifying what counts. The countermeasures are design, not exhortation. Pair every speed measure with a quality measure so gaming one degrades the other visibly. Derive numbers from source systems, not manual assembly. Periodically audit the metric itself — sample the records behind a good number. And read a sudden improvement with the same suspicion as a sudden decline: real processes rarely improve as fast as reporting habits do.
How is this capability measured in the maturity assessment?
Through the Risk Management & CAPA domain, which observes whether the organisation's signals — trends, backlogs, recurrence — are measured honestly and drive proportionate action. The distinction between the two is the point of the model: quality metrics is a capability, the function of instrumenting the quality system; the domain is a measurement axis scoring how consistently signal becomes action. An organisation can run an impressive dashboard and still score low, because the questions test what the numbers change, not what they display. Use this ladder to design the instrument; use the assessment to test whether it steers.
MEASURED THROUGH THE MATURITY ASSESSMENT
This capability is about what you must be able to do. How consistently you do it is what the maturity assessment scores — through the domain below.
Contributes to the FDA QMM practice area Management Commitment to Quality (a SPEQ mapping).
Score your quality system →