Uniqueness¶
Is each real-world record stored exactly once?
Current value (data_quality case data): 96.8% - ✕ Act. Open the case.
| Field | Value |
|---|---|
| Definition | Distinct business keys divided by rows. |
| Formula | count(distinct key) / count(rows) |
| Unit | % |
| Granularity | Dataset x load; key defined in kpis/dq_rules.yaml. |
| Direction | Higher is better ↑ |
| Target and thresholds | ≥ 99.9% (amber ≥ 99.0%) |
| Parameters | – |
| Owner | Data Steward |
| Source | All datasets of this repository (rules in kpis/dq_rules.yaml) |
| Input | table any, columns per kpis/dq_rules.yaml |
| Recommended chart | heatmap - Same grid as the other dimensions so they can be compared. |
| Related KPIs | Completeness |
How to read it¶
Duplicates silently inflate counts: one duplicated ticket is one extra "resolved" ticket in every service KPI.
Common misreadings¶
- Exact-row duplicates are only part of it; the same entity under two keys needs fuzzy matching.
Worked example (from the test suite)¶
| ticket_id |
|---|
| 1 |
| 2 |
| 2 |
| 3 |
| 3 |
Parameters: key = ['ticket_id']
Expected result: 60.00%. Distinct keys 1, 2, 3 = 3 of 5 rows = 0.6.
This example is asserted in tests/test_formulas.py against the Python function.
Implementations¶
Python function (the single implementation used by the site):
@kpi("dq_uniqueness")
def dq_uniqueness(df: pd.DataFrame, key: list[str]) -> float:
return safe_ratio(*uniqueness_counts(df, key))