Full Citation

Title: Statistically Sound Exploratory Rule Discovery

Citation Type: Miscellaneous

Publication Year: 2004

ISBN:

ISSN:

DOI:

NSFID:

PMCID:

PMID:

Abstract: Association rule discovery and other exploratory rule discovery techniques explore large search spaces of potential rules to find those that appear interesting by some user-selected criterion of interestingness. Due to the large number of rules considered, they suffer from an extreme risk of type-1 error, that is, of finding rules that appear due to chance alone to satisfy the interestingness criteria on the sample data. This paper proposes a technique to overcome this problem by using holdout data for statistical evaluation. Experiments demonstrate that standard exploratory rule discovery can result in large numbers of rules that are rejected when subjected to statistical evaluation on holdout data. They also reveal that modification of the rule discovery process to anticipate subsequent statistical evaluation can increase the number of rules that satisfy an inter-estingness criterion that are accepted by statistical evaluation on holdout data.

Url: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.201.243&rep=rep1&type=pdf

User Submitted?: No

Authors: Webb, Geoffrey I

Publisher: Monash University

Data Collections: IPUMS USA

Topics: Methodology and Data Collection

Countries: