Abstract

Learning from imbalanced datasets is a problem thoroughly studied in binary classification, and to a lesser extent in multiclass classification. Although most multilabel datasets suffer from a high imbalance level, the proposals on how to measure this characteristic and how to deal with this issue are scant.

The purpose of this paper is to present measures aimed to assess the imbalance level in multilabel datasets, as well as to propose several preprocessing algorithms designed to reduce it. Two of the proposed methods are random undersampling algorithms, called LP-RUS and ML-RUS, while the other two accomplish random oversampling, LP-ROS and ML-ROS. All of them are experimentally tested and their effectiveness is statistically evaluated. From the results obtained, a set of guidelines directed to show when these methods should be applied is also provided.

Algorithms proposed in the paper

Four preprocesing algorithms aimed to reduce the imbalance level in multilabel datasets are proposed. Two of them are based on the LP (Label Powerset) transformation, whereas the other two perform individual label imbalance analysis. All of them depend on one parameter P, which establishes the percentage of instances to remove or produce.

• LP-RUS: Random under-sampling taking the labelset of each instance as class, and removing P% of the original number of instances in the dataset.
• LP-ROS: Random over-sampling taking the labelset of each instance as class, and generating P% of the original number of instances in the dataset.
• ML-RUS: Random under-sampling considering each label separately, and reducing a P% of the amount of samples in which majority labels appear.
• ML-ROS: Random over-sampling considering each label separately, and generating a P% of the amount of samples in which minority labels appear.

Datasets

The experimentation was conducted using 13 datasets from the MULAN and MEKA repositories. Each one of these datasets has been partitioned randomly twice in five separate partitions aiming to do a 2x5 folds cross validation, which means 10 runs of every algorithm for each dataset. These partitions are available to download.

Experimentation results

Page created and maintained by Francisco Charte - 2013