Consistently evaluating data linkage classification results

ObjectivesData linkage is commonly viewed as the problem of classifying record pairs into matches and non-matches. In situations where ground truth data are available, performance measures such as precision, recall, F-measure, sensitivity, and specificity, are commonly used to evaluate the quality of matches obtained with a trained data linkage classifier. MethodsComparing multiple classifiers using such measures can, however, lead to inconsistent evaluation because for a given measure the same numerical result can be obtained from different classification outcomes. This can cause a suboptimal classifier being selected and potentially result in linked data sets of poor quality. To overcome this problem, we propose the Consistent Record Linkage (CRL) measure, an application focused evaluation method that ensures data linkage classifiers are assessed in a fair and transparent way. The CRL-measure allows the definition of maximum acceptable error rates, and it provides information about the robustness of a classifier based on identified classification thresholds. ResultsUsing both synthetic and real-world data sets, we illustrate how the CRL-measure can provide more detailed informa