Full Citation
Title: Linking Individuals Across Historical Sources: a Fully Automated Approach
Citation Type: Working Paper
Publication Year: 2018
ISBN:
ISSN:
DOI:
NSFID:
PMCID:
PMID:
Abstract: Linking individuals across historical datasets relies on information such as name and age that is both non-unique and prone to enumeration and transcription errors. These errors make it impossible to find the correct match with certainty. In the first part of the paper, we suggest a fully automated probabilistic method for linking historical datasets that enables researchers to create samples at the frontier of minimizing type I (false positives) and type II (false negatives) errors. The first step guides researchers in the choice of which variables to use for linking. The second step uses the Expectation-Maximization (EM) algorithm, a standard tool in statistics, to compute the probability that each two records correspond to the same individual. The third step suggests how to use these estimated probabilities to choose which records to use in the analysis. In the second part of the paper, we apply the method to link historical population censuses in the US and Norway, and use these samples to estimate measures of intergenerational occupational mobility. The estimates using our method are remarkably similar to the ones using IPUMS', which relies on hand linking to create a training sample. We created a Stata command that implements this method.
Url: https://www.nber.org/papers/w24324.pdf
User Submitted?: No
Authors: Abramitzky, Ran; Mill, Roy; Perez, Santiago
Series Title:
Publication Number: 24324
Institution: National Bureau of Economic Research
Pages:
Publisher Location: Cambridge, MA
Data Collections: IPUMS USA, IPUMS International
Topics: Other
Countries: Norway