Rule-based Recommender Systems for the Web of Data
A challenge of RuleML 2015, August 2015 Berlin
This challenge has two focus areas:
- rule learning algorithms applied on recommender problems
- using the linked open data cloud for feature set extension
The challenge uses a semantically enriched version of the MovieLens dataset.
In addition to the standard metrics of recommender performance, the
challenge aims to assess the understandability of the rule set
generated by the participating rule-based systems.
Motivation and Objectives
Many modern recommender systems rely on machine learning algorithms to learn user preferences. While these generally provide the best results, they also usually act as a black-box solution which does not provide a human-understandable explanation. However, the ability to explain a specific recommendation is a mandatory requirement in some application domains.
The participating systems are requested to find and recommend a
limited set of 5 items that best match a user profile.
- The participants will be provided with a semantically enriched
version of the MovieLens dataset.
- It is mandatory that a participating solution either uses the linked
open data cloud to further extend the feature set or is a rule-based
classifier. Both options simultaneously are preferred.
- A scorer is provided by the organizers so that the participants can
check their progress.
- Challenge submission will consist of the set of additional
recommendations (top-5 movies) for each user from the train dataset
and a file containing the rules that lead to the prediction (rule
based classifiers only, PMML RuleSet model preferred but not
- Recommender Performance
- Relevance scores will be used by an evaluation service to form a Top-5 item recommendation list for each user. This means that for each user only items in the evaluation set are considered to form the Top-5 recommendation list. The evaluation metric for this task is the F-measure@5.
- For each item in the recommendation list, an explanation is expected in terms of rules fired to compute the recommendation. We will consider as good explanations the most compact ones, i.e. those involving small number of easy to understand rules. This metric will be evaluated qualitatively by the PC members.
- By looking at the Top-5 recommendation lists for all the user, the total number of suggested items will be considered in order to evaluate how well the systems perform in terms of recommending diverse items. Also in this case F-measure@5 and aggregate diversity will be combined in an overall score representing a Pareto optimal solution.
Judging and Prize
How to Participate
- Register for the challenge
- Download datasets
- Submit results and check your position on the leader board
- Submit paper describing your approach
- Abstract: no more than 200 words.
- Description: It should contain the details of the system, including why the system is innovative. The paper should cover in detail the rule learning algorithm used and the additional feature expansion if employed. The description should also summarize how participants have addressed the evaluation tasks. Papers must be submitted in PDF format, following the style of the Springer's Lecture Notes in Computer Science (LNCS) series, and not exceeding 2-15 pages in length.
RecSysRules 2015 Deadlines:
|•||Paper and result submission (extended):||June 1st, 2015|
|•||Author Notification (extended)||June 18, 2015|
|•||Camera Ready (extended)||July 19th, 2015|
|•||Challenge:||Aug 5 , 2015|
RecSysRules Challenge Chairs
- Jaroslav Kuchař (Czech Technical University, Prague)
- Tommaso di Noia (Politecnico di Bari, Italy)
- Heiko Paulheim (University Mannheim, Germany)
- Tomas Kliegr (University of Economics, Prague)
Program Committee (to be completed)
- Martin Atzmüller, University of Kassel, Germany
- Johannes Fürnkranz, TU Darmstadt, Germany
- Frederik Janssen, TU Darmstadt, Germany
- Florian Lemmerich, University of Würzburg, Germany
- Václav Zeman, University of Economics, Prague, Czech Republic
- Tomáš Horváth, Pavol Jozef Šafárik University in Košice, Slovakia
- Alan Said, Recorded Future, Sweden