Instance-based learning (IBL) methods predict the class label of a new instance based directly on the distance between the new unlabeled instance and each labeled instance in the training set, without constructing a classification model in the training phase. In this paper, we introduce a novel class-based feature weighting technique, in the context of instance-based distance methods, using the Ant Colony Optimization meta-heuristic. We address three different approaches of instance-based classification: k-Nearest Neighbours, distance-based Nearest Neighbours, and Gaussian Kernel Estimator. We present a multi-archive adaptation of the ACOℝ algorithm and apply it to the optimization of the key parameter in each IBL algorithm and of the class-based feature weights. We also propose an ensemble of classifiers approach that makes use of the archived populations of the ACOℝ algorithm. We empirically evaluate the performance of our proposed algorithms on 36 benchmark datasets, and compare them with conventional instance-based classification algorithms, using various parameter settings, as well as with a state-of-the-art coevolutionary algorithm for instance selection and feature weighting for Nearest Neighbours classifiers.
Conference or workshop item
Helal, A. and Otero, F. (2017). Automatic design of ant-miner mixed attributes for classification rule discovery. in:Genetic and Evolutionary Computation (GECCO 2017).ACM, pp. 433-440. Available at: http://dx.doi.org/10.1145/3071178.3071306.
Ant-Miner Mixed Attributes (Ant-MinerMA) was inspired and built based on ACOMV. which uses an archive-based pheromone model to cope with mixed attribute types. On the one hand, the use of an archive-based pheromone model improved significantly the runtime of Ant-MinerMA and helped to eliminate the need for discretisation procedure when dealing with continuous attributes. On the other hand, the graph-based pheromone model showed superiority when dealing with datasets containing a large size of attributes, as the graph helps the algorithm to easily identify good attributes. In this paper, we propose an automatic design framework to incorporate the graph-based model along with the archive-based model in the rule creation process. We compared the automatically designed hybrid algorithm against existing ACO-based algorithms: one using a graph-based pheromone model and one using an archive-based pheromone model. Our results show that the hybrid algorithm improves the predictive quality over both the base archive-based and graph-based algorithms.
Helal, A. and Otero, F. (2016). A Mixed-Attribute Approach in Ant-Miner Classification Rule Discovery Algorithm. in:Genetic and Evolutionary Computation Conference (GECCO 2016).ACM Press, pp. 13-20. Available at: http://dx.doi.org/10.1145/2908812.2908900.
In this paper, we introduce Ant-MinerMA to tackle mixed-attribute classification problems. Most classification problems involve continuous, ordinal and categorical attributes. The majority of Ant Colony Optimization (ACO) classification algorithms have the limitation of being able to handle categorical attributes only, with few exceptions that use a discretisation procedure when handling continuous attributes either in a preprocessing stage or during the rule creation. Using a solution archive as a pheromone model, inspired by the ACO for mixed-variable optimization (ACO-MV), we eliminate the need for a discretisation procedure and attributes can be treated directly as continuous, ordinal, or categorical. We compared the proposed Ant-MinerMA against cAnt-Miner, an ACO-based classification algorithm that uses a discretisation procedure in the rule construction process. Our results show that Ant-MinerMA achieved significant improvements on computational time due to the elimination of the discretisation procedure without affecting the predictive performance.
Helal, A., Brookhouse, J. and Otero, F. (2018). Archive-Based Pheromone Model for Discovering Regression Rules with Ant Colony Optimization. in:2018 IEEE Congress on Evolutionary Computation.
In this paper we introduce a new algorithm, called Ant-Miner-Reg_MA, to tackle the regression problem using an archive-based pheromone model. Existing regression algorithms handle continuous attribute using a discretisation procedure, either in a preprocessing stage or during rule creation. Using an archive as a pheromone model, inspired by the ACO for Mixed-Variable (ACO_MV), we eliminate the need for a discretisation procedure. We compare the proposed Ant-Miner-Reg_MA against Ant-Miner-Reg, an ACO-based regression algorithm that uses a dynamic discretisation procedure, inspired on M5 algorithm, during rule construction process. Our results show that Ant-Miner-Reg_MA achieved a significant improvement in the relative root mean square error of the models created, overcoming the limitations of the dynamic discretisation procedure.