Case 5 · Data quality as a KPI¶
Data used on this page
Service desk tickets - synthetic · Maintenance work orders - synthetic · AI4I 2020 Predictive Maintenance Dataset - real public data · Online Retail II - real public data · NPS survey (simulated) - synthetic. Details on the data sources page. With synthetic data the findings illustrate the method; they are not claims about a real organisation.
Context¶
Cases 1-4 computed 24 KPIs on five datasets. Each of those KPIs silently assumes the data is complete, unique, valid and on time. This case measures those assumptions with the same discipline as any other KPI: a catalog definition, a target, an owner and a rule file (kpis/dq_rules.yaml) that works as a small data contract per dataset. It is the bridge between KPI design and data governance: every rule has an owner who fixes it at the source.
Question¶
Can we trust the data behind cases 1-4 - and which defect changes a business KPI?
KPIs · landing data, pooled over all datasets¶
Pooled figures are weighted by rows (the transaction log dominates them), so the per-dataset grid below is the view that matters.
Finding¶
Show the data behind this chart
| label | dimension | passed | total | value | status |
|---|---|---|---|---|---|
| Service tickets | completeness | 64137 | 64266 | 0.9980 | green |
| Service tickets | uniqueness | 10658 | 10711 | 0.9951 | amber |
| Service tickets | validity | 10690 | 10711 | 0.9980 | green |
| Service tickets | timeliness | 10456 | 10711 | 0.9762 | green |
| Maintenance work orders | completeness | 4580 | 4580 | 1.0000 | green |
| Maintenance work orders | uniqueness | 913 | 916 | 0.9967 | amber |
| Maintenance work orders | validity | 903 | 916 | 0.9858 | green |
| Maintenance work orders | timeliness | 846 | 907 | 0.9327 | amber |
| Process sensor data (AI4I 2020) | completeness | 70000 | 70000 | 1.0000 | green |
| Process sensor data (AI4I 2020) | uniqueness | 10000 | 10000 | 1.0000 | green |
| Process sensor data (AI4I 2020) | validity | 10000 | 10000 | 1.0000 | green |
| Process sensor data (AI4I 2020) | timeliness | 0 | 0 | – | n/a |
| Sales transactions (Online Retail II) | completeness | 6161219 | 6404226 | 0.9621 | amber |
| Sales transactions (Online Retail II) | uniqueness | 1033036 | 1067371 | 0.9678 | red |
| Sales transactions (Online Retail II) | validity | 1055384 | 1067371 | 0.9888 | green |
| Sales transactions (Online Retail II) | timeliness | 0 | 0 | – | n/a |
| NPS survey responses (simulated) | completeness | 5284 | 5284 | 1.0000 | green |
| NPS survey responses (simulated) | uniqueness | 1308 | 1321 | 0.9902 | amber |
| NPS survey responses (simulated) | validity | 1316 | 1321 | 0.9962 | green |
| NPS survey responses (simulated) | timeliness | 1295 | 1321 | 0.9803 | green |
Dataset-level completeness of the transactions looks acceptable (96.2%) because six columns are averaged - the exact misreading listed in the catalog. One column carries the whole problem.
Not measurable: Process sensor data (AI4I 2020), Sales transactions (Online Retail II) carry no load timestamp, so their timeliness cannot be computed. The grid says so instead of showing a made-up 100% - a missing audit column is itself a finding.
Decision¶
- Make the customer id mandatory, or reward it (loyalty account, faster checkout). It is the one defect that changes business KPIs: retention, repeat rate and RFM are computed on identified customers only.
- De-duplicate at ingestion with the business keys of
dq_rules.yaml; duplicates inflate ticket and work-order counts in every service and maintenance KPI. - Mobile work-order logging for the M08, M07 crews - paper work orders entered days later make MTTR and PM compliance lag and hide the night shift's real performance.
- Add a load timestamp to every source so timeliness can be measured everywhere.
- Publish the grid with owners: each red cell has a named data steward and a due date.
How we keep measuring¶
| KPI | Owner | Cadence | Target | Trigger for action |
|---|---|---|---|---|
| Completeness | Data Steward | Every load, per required column | ≥ 99.0% | Required column below 95% |
| Uniqueness | Data Steward | Every load | ≥ 99.9% | Any duplicate business key |
| Validity | Data Steward | Every load, per rule | ≥ 98.0% | Rule failure rate above 2% |
| Timeliness | Data Engineer | Daily, per source | ≥ 95.0% | Below 90% within the window |
The checks run in the same pipeline that computes the business KPIs, so a red data-quality cell is visible before anyone reads a business number built on it.