qimie2011

Last news

QIMIE'11 informal proceedings available

Invited talk by Nitesh Chawla:
Evaluation Conundrum in Machine Learning/Data Mining.

Accepted papers list is available.

Due to the extension of submission date the author notifications will be sent on Feb.6, 2011. We apology for the inconvenience from the extended notification date.

Submission webpage is open (deadline: January 10, extended January 15, 2011).

Invited talk by Longbing Cao:
Knowledge Actionability: Evaluation and Practices.

Best papers will be published in Frontiers of Computer Science in China Journal, Springer.

 

QIMIE'11 Description

 

There are a lot of data mining algorithms and methodologies for various fields and various problematic. Each data mining researcher/practitioner is faced with assessing the performance of his own solution(s) in order to make comparisons with state of the art approaches. He should also describe the intrinsic quality of the discovered patterns. Which methodology, which benchmarks, which measures of performance, which tools, which measures of interest, etc., should be used, and why? Every one should answer the previous questions, and assessing the quality and the performance is a critical issue.

The second Quality issues, measures of interestingness and evaluation of data mining models workshop (QIMIE'11) will focus on these questions and should be of great interest for a large panel of data miners. As a whole, QIMIE'11 intend to be a forum for a community-wide discussion of these issues and to contribute to a deep cross-fertilization within a large panel of researchers/practitioners attending PAKDD'11. Thus we strongly encourage interested peoples to propose topics and main themes that should be discussed within QIMIE'11. For that purpose, we opened a blog http://quality-evaluation-data-mining-models.blogspot.com/.

Following QIMIE'09 (organized in conjunction with the 13th Pacific-Asia Conference on Knowledge Discovery and Data Mining, April 2009, Bangkok, Thailand), the main themes of QIMIE'11 will focus on the theory, the techniques and the practices that can ensure the discovered knowledge is of quality. It will thus cover the problem of measuring quality of patterns, the evaluation of data mining models and the links between the discovery stage and the quality assessment stage.

QIMIE'11 is organized in association with the next PAKDD'11 conference (15th Pacific-Asia Conference on Knowledge Discovery and Data Mining, Shenzhen, China, May 24-27, 2011), a major international conference in the areas of data mining and knowledge discovery.

Major topics will include but are not limited to the following:

  • objective measures of interest (for individual rules, for rules basis)
  • subjective measures of interest and quality based on human knowledge, quality of ontologies, actionable rules (for individual rules, for rules basis)
  • algorithmic properties of measures of interest
  • comparison of algorithm: issues with benchmarks, experiments and parameters tuning questioning also reproducibility of data mining results, the need of new data sets which match new problems, methodologies, statistical tests, etc.
  • robustness evaluation and statistical evaluation
  • graphical tools like ROC, cost curves
  • special issues: imbalanced data, very large data set, very high dimensional data, changing environments, lack of training data, sample selection bias, graph data, etc.
  • special issues in specialized domains: bio-informatics, security, information retrieval, sequential and time series data, social networks, etc.
  • etc.

From the previous list of key topics, although not exhaustive, and from recent publications and related workshops questioning the usefulness of research in machine learning and data mining, one can identify five major themes (to be extended, all propositions are welcome):

  • properties of objective measures of interest (for individual rules, for rules basis) which leads to the problem on how to choose, depending of the user's goal and other factors, an appropriate interestingness measure in order to filter the huge amount of individual rules or to evaluate a set of rules; this theme is of great interest to machine learning and data mining community and has focussed a lot of works. QIMIE'11 intend to continue and enlarge this theme.
  • algorithmic properties of interestingness measures which leads to the problem of how to mine efficiently interesting patterns i.e. can we use interestingness measures as soon as possible (as well as the well-known support) to reduce both the time to mine databases and the number of founded patterns? This question has attracted a lot of works but is still a very challenging problem. QIMIE'11 intend to develop this original theme that, from the best of our knowledge, is not deeply treated in any event.
  • properties of subjective measures of interest, integration of domain knowledge, quality based on human knowledge, quality of ontologies, actionable rules
  • challenges with new data and new problems; these challenges are generally the subjects of fruitful specialized workshops (e.g. very large and very high dimensional data, imbalanced data, etc. and related specialized domains like bio-informatics, life sciences in a broad manner, etc.). QIMIE'11 intend to extent the relations between these challenges that are generally treated in a separate way.
  • evaluation and comparison of algorithms which lead to debate on how an algorithm should be evaluated, on which properties (e.g. accuracy, conciseness, specificity, sensitivity, etc.), on which trade-off between the different type of errors for multiple simultaneous hypothesis testing, on how to construct new evaluation measures?, etc.; clearly the debate within the machine learning and data mining community into how we evaluate new algorithms is not close. QIMIE'11 intend to extent the relations between these challenges and the three previous themes.

Stéphane Lallich, ERIC, Université Lyon 2, stephane.lallich@univ-lyon2.fr
Philippe Lenca, Lab-STICC, Telecom Bretagne, philippe.lenca@telecom-bretagne.eu