Lab · Master data management

Who is who?
MDM, in your browser.

The same customer often appears in several systems — spelt differently, with dates in different formats, and addresses that disagree. Master data management finds the matches, builds one trusted 'golden record' per customer, and groups customers into households. Move the sliders and watch every decision change.

See the same engine in Python →
  1. 01

    Messy source data

    Ten records from three systems: policy, claims and marketing.

  2. 02

    Matching

    Every pair is scored field by field. High scores merge, the grey zone goes to a data steward, low scores stay apart.

  3. 03

    Survivorship

    Matched records become one golden record. By default the policy system — the highest-quality source — wins every field it holds.

  4. 04

    Households

    Golden records at the same address are grouped into households.

10

Source records

7

Customers

4

Households

2

Pairs for steward review

How much each field counts

35
30
20
15

Decision thresholds

90%
70%

Try lowering the merge threshold to 80% — and watch two different people become one.

Survivorship rules

Name comes from
Address comes from

Trust order: Policy, then Claims, then Marketing. Date of birth and email always come from the most trusted source.

Step 01

Messy source data

Ten records from three systems. Spot the spelling variants, date formats and abbreviations.

RecordNameDate of birthAddressEmailUpdated
P1PolicyJonathan Smith1980-03-1214 Oak Lane, Leeds, LS1 4ABjon.smith@example.com2016-01-10
C1ClaimsJon Smith12/03/198014 Oak Ln, Leeds LS1 4AB—2017-06-02
M1MarketingJ. Smyth12-03-198014 Oak Lane LS14ABjon.smith@example.com2017-09-15
P2PolicySarah Smith1982-09-0514 Oak Lane, Leeds, LS1 4ABsarah.smith@example.com2016-01-10
M2MarketingSarah Smith05/09/198222 Elm Road, Leeds, LS2 7QTsarah.smith@example.com2017-11-20
P3PolicyPriya Patel1975-07-213 Mill Street, York, YO1 6AApriya.patel@example.com2015-04-18
C2ClaimsPriya Patell21/07/19753 Mill St, York YO1 6AA—2016-08-30
P4PolicyRavi Patel1974-02-143 Mill Street, York, YO1 6AAravi.patel@example.com2015-04-18
M3MarketingDaniel Evans30/11/19908 Station Road, Bath, BA1 1AAdan.evans@example.com2017-02-11
C3ClaimsDanielle Evans1991-11-308 Station Rd, Bath BA1 1AA—2017-05-07

Step 02

Matching

Every pair of records is compared field by field. Fields missing on either side don't count. Shown: pairs scoring 40% or more.

  • P3 Priya PatelC2 Priya Patell

    99%Match
    Name
    98%
    Date of birth
    100%
    Address
    100%
    Email
    —

  • P1 Jonathan SmithC1 Jon Smith

    98%Match
    Name
    94%
    Date of birth
    100%
    Address
    100%
    Email
    —

  • P1 Jonathan SmithM1 J. Smyth

    96%Match
    Name
    88%
    Date of birth
    100%
    Address
    100%
    Email
    100%

  • C1 Jon SmithM1 J. Smyth

    95%Match
    Name
    88%
    Date of birth
    100%
    Address
    100%
    Email
    —

  • P2 Sarah SmithM2 Sarah Smith

    83%Steward review
    Name
    100%
    Date of birth
    100%
    Address
    14%
    Email
    100%

  • M3 Daniel EvansC3 Danielle Evans

    82%Steward review
    Name
    98%
    Date of birth
    50%
    Address
    100%
    Email
    —

  • C2 Priya PatellP4 Ravi Patel

    58%No match
    Name
    83%
    Date of birth
    0%
    Address
    100%
    Email
    —

  • P3 Priya PatelP4 Ravi Patel

    50%No match
    Name
    85%
    Date of birth
    0%
    Address
    100%
    Email
    0%

  • P1 Jonathan SmithP2 Sarah Smith

    49%No match
    Name
    82%
    Date of birth
    0%
    Address
    100%
    Email
    0%

  • C1 Jon SmithP2 Sarah Smith

    48%No match
    Name
    60%
    Date of birth
    0%
    Address
    100%
    Email
    —

Step 03

Golden records

Automatically matched records merge into one trusted customer. Each field shows the source that won under your survivorship rules; addresses are standardised.

  • Jonathan Smith

    from P1 + C1 + M1

    Name
    Jonathan SmithPolicy
    Born
    12 Mar 1980Policy
    Address
    14 Oak Lane Leeds, LS1 4ABPolicy
    Email
    jon.smith@example.comPolicy

  • Sarah Smith

    from P2

    Name
    Sarah SmithPolicy
    Born
    5 Sep 1982Policy
    Address
    14 Oak Lane Leeds, LS1 4ABPolicy
    Email
    sarah.smith@example.comPolicy

  • Sarah Smith

    from M2

    Name
    Sarah SmithMarketing
    Born
    5 Sep 1982Marketing
    Address
    22 Elm Road Leeds, LS2 7QTMarketing
    Email
    sarah.smith@example.comMarketing

  • Priya Patel

    from P3 + C2

    Name
    Priya PatelPolicy
    Born
    21 Jul 1975Policy
    Address
    3 Mill Street York, YO1 6AAPolicy
    Email
    priya.patel@example.comPolicy

  • Ravi Patel

    from P4

    Name
    Ravi PatelPolicy
    Born
    14 Feb 1974Policy
    Address
    3 Mill Street York, YO1 6AAPolicy
    Email
    ravi.patel@example.comPolicy

  • Daniel Evans

    from M3

    Name
    Daniel EvansMarketing
    Born
    30 Nov 1990Marketing
    Address
    8 Station Road Bath, BA1 1AAMarketing
    Email
    dan.evans@example.comMarketing

  • Danielle Evans

    from C3

    Name
    Danielle EvansClaims
    Born
    30 Nov 1991Claims
    Address
    8 Station Road Bath, BA1 1AAClaims
    Email
    —

Step 04

Households

Customers sharing an address — same house number and postcode — are grouped into a household.

  • Household of 2

    14 Oak Lane Leeds, LS1 4AB

    • Jonathan Smith
    • Sarah Smith

  • Single household

    22 Elm Road Leeds, LS2 7QT

    • Sarah Smith

  • Household of 2

    3 Mill Street York, YO1 6AA

    • Priya Patel
    • Ravi Patel

  • Household of 2

    8 Station Road Bath, BA1 1AA

    • Daniel Evans
    • Danielle Evans

From the real world

At Allianz UK I led customer data and MDM on IBM MDM Server. Addresses were standardised against Royal Mail's PAF, the policy system was the trusted source for the golden record, and the merge and review thresholds were tuned against known duplicates and checked by sampling with data stewards — who also worked the review queue. Together with data cleansing, that improved matching rates by 21%.

A simplified illustration with fictional data. It shows the concepts — probabilistic matching, survivorship and householding — not any employer's algorithm.

The Allianz story →

Contact

Let's talk.

I'm based in London, UK. Whether it's a leadership role, an advisory engagement or a transformation that needs shaping — my inbox is open.

srini.vankee@gmail.com
LinkedIn