2009 TellMeSomethingIDontKnowRandomi

From GM-RKB
Jump to: navigation, search

Subject Headings:

Notes

Cited By

Quotes

Author Keywords

Abstract

There is a wide variety of data mining methods available, and it is generally useful in exploratory data analysis to use many different methods for the same dataset. This, however, leads to the problem of whether the results found by one method are a reflection of the phenomenon shown by the results of another method, or whether the results depict in some sense unrelated properties of the data. For example, using clustering can give indication of a clear cluster structure, and computing correlations between variables can show that there are many significant correlations in the data. However, it can be the case that the correlations are actually determined by the cluster structure.

In this paper, we consider the problem of randomizing data so that previously discovered patterns or models are taken into account. The randomization methods can be used in iterative data mining. At each step in the data mining process, the randomization produces random samples from the set of data matrices satisfying the already discovered patterns or models. That is, given a data set and some statistics (e.g., cluster centers or co-occurrence counts) of the data, the randomization methods sample data sets having similar values of the given statistics as the original data set. We use Metropolis sampling based on local swaps to achieve this. We describe experiments on real data that demonstrate the usefulness of our approach. Our results indicate that in many cases, the results of, e.g., clustering actually imply the results of, say, frequent pattern discovery.

References

,

 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
2009 TellMeSomethingIDontKnowRandomiSami Hanhijärvi
Markus Ojala
Niko Vuokko
Kai Puolamäki
Nikolaj Tatti
Heikki Mannila
Tell Me Something I Don't Know: Randomization Strategies for Iterative Data MiningKDD-2009 Proceedings10.1145/1557019.15570652009