BIBLIOGRAPHY

Publications, working papers, and other research using data resources from IPUMS.

Full Citation

Title: Linking Individuals Across Historical Sources: a Fully Automated Approach

Citation Type: Working Paper

Publication Year: 2018

Abstract: Linking individuals across historical datasets relies on information such as name and age that is both non-unique and prone to enumeration and transcription errors. These errors make it impossible to find the correct match with certainty. In the first part of the paper, we suggest a fully automated probabilistic method for linking historical datasets that enables researchers to create samples at the frontier of minimizing type I (false positives) and type II (false negatives) errors. The first step guides researchers in the choice of which variables to use for linking. The second step uses the Expectation-Maximization (EM) algorithm, a standard tool in statistics, to compute the probability that each two records correspond to the same individual. The third step suggests how to use these estimated probabilities to choose which records to use in the analysis. In the second part of the paper, we apply the method to link historical population censuses in the US and Norway, and use these samples to estimate measures of intergenerational occupational mobility. The estimates using our method are remarkably similar to the ones using IPUMS', which relies on hand linking to create a training sample. We created a Stata command that implements this method.

Url: https://www.nber.org/papers/w24324.pdf

User Submitted?: No

Authors: Abramitzky, Ran; Mill, Roy; Perez, Santiago

Series Title:

Publication Number: 24324

Institution: National Bureau of Economic Research

Pages:

Publisher Location: Cambridge, MA

Data Collections: IPUMS USA, IPUMS International

Topics: Other

Countries: Norway

IPUMS NHGIS NAPP IHIS ATUS Terrapop