Publications by Dr Fernando Otero

Directional changes (DC) is a recent technique that summarises physical time data (e.g. daily closing prices, hourly data) into events, offering traders a unique perspective of the market to create novel trading strategies. This paper proposes the use of a genetic algorithm (GA) to optimize the recommendations of multiple DC-based trading strategies. Each trading strategy uses a novel framework that combines classification and regression techniques to predict when a trend will reverse. We evaluate the performance of the proposed multiple DC-strategy GA algorithm against nine benchmarks: five single DC-based trading strategies, three technical analysis indicators, as well as buy-and-hold, which is a popular financial benchmark. We perform experiments using 200 monthly physical time datasets from 20 foreign exchange markets—these datasets were created from snapshots of 10 min intervals. Experimental results show that our proposed algorithm is able to statistically significantly outperform all DC and non-DC benchmarks in terms of both return and risk, and establish multi-threshold DCs as an effective algorithmic trading technique.

Data stream mining has recently emerged in response to the rapidly increasing continuous data generation. While the majority of Ant Colony Optimisation (ACO) rule induction algorithms have proved to be successful in producing both accurate and comprehensive classification models in nonstreaming (batch) settings, currently ACO-based algorithms for classification problems are not suited to be applied to data stream mining. One of the main challenges is the iterative nature of ACO algorithms, where many procedures—for example, heuristic calculation, selection of continuous attributes, pruning—require multiple passes through the data to create a model. In this paper, we present a new ACO-based algorithm for data stream classification. The proposed algorithm, called Stream Ant-Miner (sAnt-Miner), uses a novel hybrid pheromone model combining both a traditional construction graph and solution archives models to efficiently handle a large number of mixed-type (nominal and continuous) attributes directly without the need for additional procedures, reducing the computational time required to complete an iteration of the algorithm. Our results show that sAnt-Miner produces statistically significant concise models compared with state-of-the-art rule induction data stream algorithms, without negative effects on their predictive accuracy.

Feature selection is a widespread preprocessing step in the data mining field. One of its purposes is to reduce the number of original dataset features to improve a predictive model's performance. Despite the benefits of feature selection for the classification task, to the best of our knowledge, few studies in the literature address feature selection for the hierarchical classification context. This paper proposes a novel feature selection method based on the general variable neighborhood search metaheuristic, combining a filter and a wrapper step, wherein a global model hierarchical classifier evaluates feature subsets. We used twelve datasets from the proteins and images domains to perform computational experiments to validate the effect of the proposed algorithm on classification performance when using two global hierarchical classifiers proposed in the literature. Statistical tests showed that using our method for feature selection led to predictive performances that were consistently better than or equivalent to that obtained by using all features with the benefit of reducing the number of features needed, which justifies its efficiency for the hierarchical classification scenario.

The majority of forecasting methods use a physical time scale for studying price fluctuations of financial markets. Using physical time scales can make companies oblivious to significant activities in the market as the flow of time is discontinuous, which could translate to missed profitable opportunities or risk exposure. Directional changes (DC) has gained attention in the recent years by translating physical time series to event-based series. Under this framework, trend reversals can be predicted by using the length of events. Having this knowledge allows traders to take an action before such reversals happen and thus increase their profitability. In this paper, we investigate how classification algorithms can be incorporated in the process of predicting trend reversals to create DC-based trading strategies. The effect of the proposed trend reversal estimation is measured on 20 foreign exchange markets over a 10-month period in a total of 1000 data sets. We compare our results across 16 algorithms, both DC and non-DC based, such as technical analysis and buy-and-hold. Our findings show that the introduction of classification leads to return higher profit and statistically outperform all other trading strategies.

Data analysis has become an important field over the last decades. The growing amount of data demands new analytical methodologies in order to extract relevant knowledge. Clustering is one of the most competitive techniques in this context. Using a dataset as a starting point, clustering techniques aim to blindly group the data by similarity. Among the different areas, manifold identification is currently gaining importance. Spectral-based methods, which are one of the main used methodologies in this area, are sensitive to metric parameters and noise. In order to solve these problems, new bio-inspired techniques have been combined with different heuristics to perform the cluster selection, in particular for dense datasets. Dense datasets are featured by areas of higher density, where there are significantly more data instances than in the rest of the search space. This paper presents an extension of a previous algorithm named Spectral-based Ant Colony Optimization Clustering (SACOC), a spectral-based clustering methodology used for manifold identification. This work focuses on improving the SACOC algorithm through the Nystrom extension in order to deal with dense data problems. We evaluated the performance of the proposed approach, called SACON, comparing it against online clustering algorithms and the Nystrom extension of the Spectral Clustering algorithm using several benchmark datasets.

The majority of forecasting methods use a physical time scale for studying price fluctuations of financial markets, making the flow of physical time discontinuous. Therefore, using a physical time scale may expose companies to risks, due to ignorance of some significant activities. In this paper, an alternative and original approach is explored to capture important activities in the market. The main idea is to use an event-based time scale based on a new way of summarising data, called Directional Changes. Combined with a genetic algorithm, the proposed approach aims to find a trading strategy that maximises profitability in foreign exchange markets. In order to evaluate its efficiency and robustness, we run rigorous experiments on 255 datasets from six different currency pairs, consisting of intra-day data from the foreign exchange spot market. The results from these experiments indicate that our proposed approach is able to generate new and profitable trading strategies, significantly outperforming other traditional types of trading strategies, such as technical analysis and buy and hold.

Financial forecasting is an important area in computational finance. Evolutionary Dynamic Data Investment Evaluator (EDDIE) is an established genetic programming (GP) financial forecasting algorithm, which has successfully been applied to a number of international financial datasets. The purpose of this paper is to further improve the algorithm's predictive performance, by incorporating heuristics in the search. We propose the use of two heuristics: a sequential covering strategy to iteratively build a solution in combination with the GP search and the use of an entropy-based dynamic discretisation procedure of numeric values. To examine the effectiveness of the proposed improvements, we test the new EDDIE version (EDDIE 9) across 20 datasets and compare its predictive performance against three previous EDDIE algorithms. In addition, we also compare our new algorithm's performance against C4.5 and RIPPER, two state-of-the-art classification algorithms. Results show that the introduction of heuristics is very successful, allowing the algorithm to outperform all previous EDDIE versions and the well-known C4.5 and RIPPER algorithms. Results also show that the algorithm is able to return significantly high rates of return across the majority of the datasets.

The vast majority of Ant Colony Optimization (ACO) algorithms for inducing classification rules use an ACO-based procedure to create a rule in an one-at-a-time fashion. An improved search strategy has been proposed in the cAnt-MinerPB algorithm, where an ACO-based procedure is used to create a complete list of rules (ordered rules)-i.e., the ACO search is guided by the quality of a list of rules, instead of an individual rule. In this paper we propose an extension of the cAnt-MinerPB algorithm to discover a set of rules (unordered rules). The main motivations for this work are to improve the interpretation of individual rules by discovering a set of rules and to evaluate the impact on the predictive accuracy of the algorithm. We also propose a new measure to evaluate the interpretability of the discovered rules to mitigate the fact that the commonly-used model size measure ignores how the rules are used to make a class prediction. Comparisons with state-of-the-art rule induction algorithms, support vector machines and the cAnt-MinerPB producing ordered rules are also presented.

The application of ACO-based algorithms in data mining has been growing over the last few years, and several supervised and unsupervised learning algorithms have been developed using this bio-inspired approach. Most recent works about unsupervised learning have focused on clustering, showing the potential of ACO-based techniques. However, there are still clustering areas that are almost unexplored using these techniques, such as medoid-based clustering. Medoid-based clustering methods are helpful—compared to classical centroid-based techniques—when centroids cannot be easily defined. This paper proposes two medoid-based ACO clustering algorithms, where the only information needed is the distance between data: one algorithm that uses an ACO procedure to determine an optimal medoid set (METACOC algorithm) and another algorithm that uses an automatic selection of the number of clusters (METACOC-K algorithm). The proposed algorithms are compared against classical clustering approaches using synthetic and real-world datasets.

Semi-supervised learning methods create models from a few labeled instances and a great number of unlabeled instances. They appear as a good option in scenarios where there is a lot of unlabeled data and the process of labeling instances is expensive, such as those where most Web applications stand. This paper proposes a semi-supervised self-training algorithm called Ant-Labeler. Self-training algorithms take advantage of supervised learning algorithms to iteratively learn a model from the labeled instances and then use this model to classify unlabeled instances. The instances that receive labels with high confidence are moved from the unlabeled to the labeled set, and this process is repeated until a stopping criteria is met, such as labeling all unlabeled instances. Ant-Labeler uses an ACO algorithm as the supervised learning method in the self-training procedure to generate interpretable rule-based models—used as an ensemble to ensure accurate predictions. The pheromone matrix is reused across different executions of the ACO algorithm to avoid rebuilding the models from scratch every time the labeled set is updated. Results showed that the proposed algorithm obtains better predictive accuracy than three state-of-the-art algorithms in roughly half of the datasets on which it was tested, and the smaller the number of labeled instances, the better the Ant-Labeler performance.

Ant colony optimization (ACO) algorithms have been successfully applied to discover a list of classification rules. In general, these algorithms follow a sequential covering strategy, where a single rule is discovered at each iteration of the algorithm in order to build a list of rules. The sequential covering strategy has the drawback of not coping with the problem of rule interaction, i.e., the outcome of a rule affects the rules that can be discovered subsequently since the search space is modified due to the removal of examples covered by previous rules. This paper proposes a new sequential covering strategy for ACO classification algorithms to mitigate the problem of rule interaction, where the order of the rules is implicitly encoded as pheromone values and the search is guided by the quality of a candidate list of rules. Our experiments using 18 publicly available data sets show that the predictive accuracy obtained by a new ACO classification algorithm implementing the proposed sequential covering strategy is statistically significantly higher than the predictive accuracy of state-of-the-art rule induction classification algorithms.

The cAnt-Miner algorithm is an Ant Colony Optimization (ACO) based technique for classification rule discovery in problem domains which include continuous attributes. In this paper, we propose several extensions to cAnt- Miner. The main extension is based on the use of multiple pheromone types, one for each class value to be predicted. In the proposed ?cAnt-Miner algorithm, an ant first selects a class value to be the consequent of a rule and the terms in the antecedent are selected based on the pheromone levels of the selected class value; pheromone update occurs on the corresponding pheromone type of the class value. The pre-selection of a class value also allows the use of more precise measures for the heuristic function and the dynamic discretization of continuous attributes, and further allows for the use of a rule quality measure that directly takes into account the confidence of the rule. Experimental results on 20 benchmark datasets show that our proposed extension improves classification accuracy to a statistically significant extent compared to cAnt-Miner, and has classification accuracy similar to the well-known Ripper and PART rule induction algorithms.

Decision trees have been widely used in data mining and machine learning as a comprehensible knowledge representation. While ant colony optimization (ACO) algorithms have been successfully applied to extract classification rules, decision tree induction with ACO algorithms remains an almost unexplored research area. In this paper we propose a novel ACO algorithm to induce decision trees, combining commonly used strategies from both traditional decision tree induction algorithms and ACO. The proposed algorithm is compared against three decision tree induction algorithms, namely C4.5, CART and cACDT, in 22 publicly available data sets. The results show that the predictive accuracy of the proposed algorithm is statistically significantly higher than the accuracy of both C4.5 and CART, which are well-known conventional algorithms for decision tree induction, and the accuracy of the ACO-based cACDT decision tree algorithm.

This paper proposes a novel ant colony optimisation (ACO) algorithm tailored for the hierarchical multi-label classification problem of protein function prediction. This problem is a very active research field, given the large increase in the number of uncharacterised proteins available for analysis and the importance of determining their functions in order to improve the current biological knowledge. Since it is known that a protein can perform more than one function and many protein functional-definition schemes are organised in a hierarchical structure, the classification problem in this case is an instance of a hierarchical multi-label problem. In this type of problem, each example may belong to multiple class labels and class labels are organised in a hierarchical structure—either a tree or a directed acyclic graph structure. It presents a more complex problem than conventional flat classification, given that the classification algorithm has to take into account hierarchical relationships between class labels and be able to predict multiple class labels for the same example. The proposed ACO algorithm discovers an ordered list of hierarchical multi-label classification rules. It is evaluated on sixteen challenging bioinformatics data sets involving hundreds or thousands of class labels to be predicted and compared against state-of-the-art decision tree induction algorithms for hierarchical multi-label classification.

Market timing, one of the core deciding when to buy or sell an asset of interest on a financial market. Market timing strategies can be built by using a collection of components or functions that process market context and return a recommendation on the course of action to take. In this chapter, we revisit the work presented in [20] on the application of Genetic Algorithms (GA) and Particle Swarm Optimization (PSO) to the issue of market timing while using a novel approach for training and testing called Trend Representative Testing. We provide more details on the process of building trend representative datasets, as well as, introduce a new PSO variant with a different approach to pruning. Results show that the new pruning procedure is capable of reducing solution length while not adversely affecting the quality of the solutions in a statistically significant manner.

This chapter proposes a generic framework to build geometric dispersion (GD) operators for Geometric Semantic Genetic Programming in the context of symbolic regression, followed by two concrete instantiations of the framework: a multiplicative geometric dispersion operator and an additive geometric dispersion operator. These operators move individuals in the semantic space in order to balance the population around the target output in each dimension, with the objective of expanding the convex hull defined by the population to include the desired output vector. An experimental analysis was conducted in a testbed composed of sixteen datasets showing that dispersion operators can improve GSGP search and that the multiplicative version of the operator is overall better than the additive version.

This paper introduces MYRA, an open-source Java framework that provides the implementation of several ant colony optimization classification algorithms. The algorithms are ready to be used from the command-line or can be easily called from custom Java code. The framework is implemented using a modular architecture, therefore algorithms can be easily extended to incorporate different procedures and/or use different parameter values. The paper gives particular attention to the common architecture from which the algorithms are built on, highlighting the shared classes among the different implemented algorithms. The source code and documentation of MYRA are available for download at https://github.com/febo/myra.

Ant-Miner Mixed Attributes (Ant-MinerMA) was inspired and built based on ACOMV. which uses an archive-based pheromone model to cope with mixed attribute types. On the one hand, the use of an archive-based pheromone model improved significantly the runtime of Ant-MinerMA and helped to eliminate the need for discretisation procedure when dealing with continuous attributes. On the other hand, the graph-based pheromone model showed superiority when dealing with datasets containing a large size of attributes, as the graph helps the algorithm to easily identify good attributes. In this paper, we propose an automatic design framework to incorporate the graph-based model along with the archive-based model in the rule creation process. We compared the automatically designed hybrid algorithm against existing ACO-based algorithms: one using a graph-based pheromone model and one using an archive-based pheromone model. Our results show that the hybrid algorithm improves the predictive quality over both the base archive-based and graph-based algorithms.

This chapter describes the Sequential Symbolic Regression (SSR) method, a new strategy for function approximation in symbolic regression. The SSR method is inspired by the sequential covering strategy from machine learning, but instead of sequentially reducing the size of the problem being solved, it sequentially transforms the original problem into potentially simpler problems. This transformation is performed according to the semantic distances between the desired and obtained outputs and a geometric semantic operator. The rationale behind SSR is that, after generating a suboptimal function f via symbolic regression, the output errors can be approximated by another function in a subsequent iteration. The method was tested in eight polynomial functions, and compared with canonical genetic programming (GP) and geometric semantic genetic programming (SGP). Results showed that SSR significantly outperforms SGP and presents no statistical difference to GP. More importantly, they show the potential of the proposed strategy: an effective way of applying geometric semantic operators to combine different (partial) solutions, avoiding the exponential growth problem arising from the use of these operators.

Financial forecasting is a vital area in computational finance, where several studies have taken place over the years. One way of viewing financial forecasting is as a classification problem, where the goal is to find a model that represents the predictive relationships between predictor attribute values and class attribute values. In this paper we present a comparative study between two bio-inspired classification algorithms, a genetic programming algorithm especially designed for financial forecasting, and an ant colony optimization one, which is designed for classification problems. In addition, we compare the above algorithms with two other state-of-the-art classification algorithms, namely C4.5 and RIPPER. Results show that the ant colony optimization classification algorithm is very successful, significantly outperforming all other algorithms in the given classification problems, which provides insights for improving the design of specific financial forecasting algorithms.

The application of ACO-based algorithms in data mining is growing over the last few years and several supervised and unsupervised learning algorithms have been developed using this bio-inspired approach. Most recent works concerning unsupervised learning have been focused on clustering, showing great potential of ACO-based techniques. This work presents an ACO-based clustering algorithm inspired by the ACO Clustering (ACOC) algorithm. The proposed approach restructures ACOC from a centroid-based technique to a medoid-based technique, where the properties of the search space are not necessarily known. Instead, it only relies on the information about the distances amongst data. The new algorithm, called MACOC, has been compared against well-known algorithms (K-means and Partition Around Medoids) and with ACOC. The experiments measure the accuracy of the algorithm for both synthetic datasets and real-world datasets extracted from the UCI Machine Learning Repository.

The application of ACO-based algorithms in data mining is growing over the last few years and several supervised and unsupervised learning algorithms have been developed using this bio-inspired approach. Most recent works concerning unsupervised learning have been focused on clustering, where ACO-based techniques have showed a great potential. At the same time, new clustering techniques that seek the continuity of data, specially focused on spectral-based approaches in opposition to classical centroid-based approaches, have attracted an increasing research interest–an area still under study by ACO clustering techniques. This work presents a hybrid spectral-based ACO clustering algorithm inspired by the ACO Clustering (ACOC) algorithm. The proposed approach combines ACOC with the spectral Laplacian to generate a new search space for the algorithm in order to obtain more promising solutions. The new algorithm, called SACOC, has been compared against well-known algorithms (K-means and Spectral Clustering) and with ACOC. The experiments measure the accuracy of the algorithm for both synthetic datasets and real-world datasets extracted from the UCI Machine Learning Repository.

The genetic programming tool EDDIE has been shown to be a successful financial forecasting tool, however it has suffered from an increase in execution time as new features have been added. Speed is an important aspect in financial problems, especially in the field of algorithmic trading, where a delay in taking a decision could cost millions. To offset this performance loss, EDDIE has been modified to take advantage of multi-core CPUs and dedicated GPUs. This has been achieved by modifying the candidate solution evaluation to use an OpenCL kernel, allowing the parallel evaluation of solutions. Our computational results have shown improvements in the running time of EDDIE when the evaluation was delegated to the OpenCL kernel running on a multi-core CPU, with speed ups up to 21 times faster than the original EDDIE algorithm. While most previous works in the literature reported significantly improvements in performance when running an OpenCL kernel on a GPU device, we did not observe this in our results. Further investigation revealed that memory copying overheads and branching code in the kernel are potentially causes of the (under-)performance of the OpenCL kernel when running on the GPU device.

Data mining is a process that supports knowledge discovery by finding hidden patterns, associations and constructing analytical models from databases. Classification is one of the widely studied data mining tasks in which the aim is to discover, from labelled cases, a model that can be used to predict the class of unlabelled cases. Ant-Miner, proposed by Parpinelli et al. [3], is the first ACO algorithm for discovering classification rules. Ant-Miner has been shown to be competitive with well-known classification algorithms, in terms of producing comprehensible model with high predictive accuracy. Therefore, there has been an increasing interest in improving the Ant-Miner algorithm [1]

Many classification schemes for defining protein functions, such as Gene Ontology (GO), are organised in a hierarchical structure. Nodes near the root of the hierarchy represent general functions while nodes near the leaves of the hierarchy represent more specific functions, giving the flexibility to specify at which level the protein will be annotated. In a data mining perspective, hierarchical structures present a more challenging problem, since the relationship between nodes need to be considered. This chapter presents an empirical evaluation of different protein representations for protein function prediction in terms of maximizing predictive accuracy, investigating which type of representation is more suitable for different levels of the GO hierarchy.

This paper proposes a novel Ant Colony Optimisation algorithm for the hierarchical problem of predicting protein functions using the Gene Ontology (GO). The GO structure represents a challenging case of hierarchical classification, since its terms are organised in a direct acyclic graph fashion where a term can have more than one parent in contrast to only one parent in tree structures. The proposed method discovers an ordered list of classification rules which is able to predict all GO terms independently of their level. We have compared the proposed method against a baseline method, which consists of training classifiers for each GO terms individually, in five different ion-channel data sets and the results obtained are promising.

This paper presents an extension to Ant-Miner, named cAnt-Miner (Ant-Miner coping with continuous attributes), which incorporates an entropy-based discretization method in order to cope with continuous attributes during the rule construction process. By having the ability to create discrete intervals for continuous attributes "on-the-fly", cAnt-Miner does not requires a discretization method in a preprocessing step, as Ant-Miner requires. cAnt-Miner has been compared against Ant-Miner in eight public domain datasets with respect to predictive accuracy and simplicity of the discovered rules. Empirical results show that creating discrete intervals during the rule construction process facilitates the discovery of more accurate and significantly simpler classification rules

For a given data set, its set of attributes defines its data space representation. The quality of a data space representation is one of the most important factors influencing the performance of a data mining algorithm. The attributes defining the data space can be inadequate, making it difficult to discover high-quality knowledge. In order to solve this problem, this paper proposes a Genetic Programming algorithm developed for attribute construction. This algorithm constructs new attributes out of the original attributes of the data set, performing an important preprocessing step for the subsequent application of a data mining algorithm.

Market timing is the issue of deciding when to buy or sell a given asset on a financial market. As one of the core issues of algorithmic trading systems, designers of such systems have turned to computational intelligence methods to aid them in this task. In our previous work, we introduced a number of Particle Swarm Optimization (PSO) algorithms to compose strategies for market timing using a novel training and testing methodology that reduced the likelihood of overfitting and tackled market timing as a multiobjective optimization problem. In this paper, we provide a detailed analysis of these multiobjective PSO algorithms and address two limitations in the results presented previously. The first limitation is that the PSO algorithms have not been compared to well-known algorithms or market timing techniques. This is addressed by comparing the results obtained against NSGA-II and MACD, a technique commonly used in market timing strategies. The second limitation is that we have no insight regarding diversity of the Pareto sets returned by the algorithms. We address this by using RadViz to visualize the Pareto sets returned by all the algorithms, including NSGA-II and MACD. The results show that the multiobjective PSO algorithms return statistically significantly better results than NSGA-II and MACD. We also observe that the multiobjective PSOSP algorithm consistently displayed the best spread in its returned Pareto sets despite not having any explicit diversity promoting measures.

Longitudinal datasets contain repeated measurements of the same variables at different points in time. Longitudinal data mining algorithms aim to utilize such datasets to extract interesting knowledge and produce useful models. Many existing longitudinal classification methods either dismiss the longitudinal aspect of the data during model construction or produce complex models that are scarcely interpretable. We propose a new longitudinal classification algorithm based on decision trees, named Nested Trees. It utilizes a unique longitudinal model construction method that is fully aware of the longitudinal aspect of the predictive attributes (variables) and constructs tree nodes that make decisions based on a longitudinal attribute as a whole, considering measurements of that attribute across multiple time points. The algorithm was evaluated using 10 classification tasks based on the English Longitudinal Study of Ageing (ELSA) data.

One of the key elements for businesses to succeed is to get to know their customers. Traditionally this task has been performed through user studies, however, over the last few years clickstream analysis has been proposed as a potential way of conducting automated behavioural studies at scale. In this paper, we explore the use of a recently-proposed unsupervised data-mining technique to identify common behavioural patterns from a clickstream and use them to automatically group users into clusters. In particular, our goal is to validate the potential of behavioural user identification with respect to a key business-level objective. We consider to which extent it is possible to link overall user in-application behaviour to the completion of a particular business-relevant action. Identifying behavior patterns resulting in such business-relevant actions can enable businesses to make changes to their interface, target relevant user groups or trigger actionable insights, all with the objective of maximizing the likelihood of preferable user actions. We analyzed a realworld dataset from a mobile application deployed on both the iOS and Android platforms for this experiment.

The digital processing of electronic documents is widely exploited across many domains to improve the efficiency of information extraction. However, paper documents are still largely being used in practice. In order to process such documents, a manual procedure is used to inspect them and extract the values of interest. As this task is monotonous and time consuming, it is prone to introduce human errors during the process. In this paper, we present an efficient and robust system that automates the aforementioned task by using a combination of machine learning techniques: optical character recognition, object detection and image processing techniques. This not only speeds up the process but also improves the accuracy of extracted information compared to a manual procedure.

In the path planning task for autonomous mobile robots, robots should be able to plan their trajectory to leave the start position and reach the goal, safely. There are several path planning approaches for mobile robots in the literature. Ant Colony Optimization algorithms have been investigated for this problem, giving promising results. In this paper, we propose the Max-Min Ant System for Dynamic Path Planning algorithm for the exploratory path planning task for autonomous mobile robots based on topological maps. A topological map is an environment representation whose focus is the main reference points of the environment and their connections. Based on this representation, the path can be composed by a sequence of state/actions pairs, which facilitates the navigability of the path, with no need to have the information of the complete map. The proposed algorithm was evaluated in static and dynamic envi- ronments, showing promising results in both of them. Experiments in dynamic environments show the adaptability of our proposal.

The introduction of electronic exchanges was a crucial point in history as it heralded the arrival of algorithmic trading. Designers of such systems face a number of issues, one of which is deciding when to buy or sell a given security on a financial market. Although Genetic Algorithms (GA) have been the most widely used to tackle this issue, Particle Swarm Optimization (PSO) has seen much lower adoption within the domain. In two previous works, the authors adapted PSO algorithms to tackle market timing and address the shortcomings of the previous approaches both with GA and PSO. The majority of work done to date on market timing tackled it as a single objective optimization problem, which limits its suitability to live trading as designers of such strategies will realistically pursue multiple objectives such as maximizing profits, minimizing exposure to risk and using the shortest strategies to improve execution speed. In this paper, we adapt both a GA and PSO to tackle market timing as a multiobjective optimization problem and provide an in depth discussion of our results and avenues of future research.

Longitudinal datasets contain repeated measurements of the same variables at different points in time, which can be used by researchers to discover useful knowledge based on the changes of the data over time. Monotonic relations often occur in real-world data and need to be preserved in data mining models in order for the models to be acceptable by users. We propose a new methodology for detecting monotonic relations in longitudinal datasets and applying them in longitudinal classification model construction. Two different approaches were used to detect monotonic relations and include them into the classification task. The proposed approaches are evaluated using data from the English Lon- gitudinal Study of Ageing (ELSA) with 10 different age-related diseases used as class variables to be predicted. A gradient boosting algorithm (XGBoost) is used for constructing classification models in two scenarios: enforcing and not enforcing the constraints. The results show that enforcement of monotonicity constraints can consistently improve the predictive accuracy of the constructed models. The produced models are fully monotonic according to the monotonicity constraints, which can have a positive impact on model acceptance in real world applications.

Market Timing is the capacity of deciding when to buy or sell a given asset on a financial market. Market Timing strategies are usually composed of components that process market context and return a recommendation whether to buy or sell. The main issues with composing market timing strategies are twofold: (i) selecting the signal generating components; and (ii) tuning their parameters. In previous work, researchers usually attempt to either tune the parameters of a set of components or select amongst a number of components with predetermined parameter values. In this paper, we approach market timing as one integrated problem and propose to solve it with two variants of Particle Swarm Optimization (PSO). We compare the performance of PSO against a Genetic Algorithm (GA), the most widely used metaheuristic in the domain of market timing. We also propose the use of trend representative testing to circumvent the issue of overfitting commonly associated with step-forward testing. Results show PSO to be competitive with GA, and that trend representative testing is an effective method of exposing strategies to various market conditions during training and testing.

In this paper we introduce a new algorithm, called Ant-Miner-Reg_MA, to tackle the regression problem using an archive-based pheromone model. Existing regression algorithms handle continuous attribute using a discretisation procedure, either in a preprocessing stage or during rule creation. Using an archive as a pheromone model, inspired by the ACO for Mixed-Variable (ACO_MV), we eliminate the need for a discretisation procedure. We compare the proposed Ant-Miner-Reg_MA against Ant-Miner-Reg, an ACO-based regression algorithm that uses a dynamic discretisation procedure, inspired on M5 algorithm, during rule construction process. Our results show that Ant-Miner-Reg_MA achieved a significant improvement in the relative root mean square error of the models created, overcoming the limitations of the dynamic discretisation procedure.

Most classification algorithms ignore existing domain knowledge during model construction, which can decrease the model's comprehensibility and increase the likelihood of model rejection due to users losing trust in the models they use. One approach to encapsulate this domain knowledge is monotonic constraints. This paper proposes new monotonic pruners to enforce monotonic constraints on models created by an existing ACO algorithm in a post-processing stage. We compare the effectiveness of the new pruners against an existing post-processing approach that also enforce constraints. Additionally, we also compare the effectiveness of both these post-processing procedures in isolation and in conjunction with favouring constraints in the learning phase. Our results show that our proposed pruners outperform the existing post-processing approach and the combination of favouring and enforcing constraints at different stages of the model construction process is the most effective solution.

Genetic programming approaches are moving from analysing the syntax of individual solutions to look into their semantics. One of the common definitions of the semantic space in the context of symbolic regression is a n-dimensional space, where n corresponds to the number of training examples. In problems where this number is high, the search process can became harder as the number of dimensions increase. Geometric semantic genetic programming (GSGP) explores the semantic space by performing geometric semantic operations—the fitness landscape seen by GSGP is guaranteed to be conic by construction. Intuitively, a lower number of dimensions can make search more feasible in this scenario, decreasing the chances of data overfitting and reducing the number of evaluations required to find a suitable solution. This paper proposes two approaches for dimensionality reduction in GSGP: (i) to apply current instance selection methods as a pre-process step before training points are given to GSGP; (ii) to incorporate instance selection to the evolution of GSGP. Experiments in 15 datasets show that GSGP performance is improved by using instance reduction during the evolution.

Classification algorithms generally do not use existing domain knowledge during model construction. The creation of models that conflict with existing knowledge can reduce model acceptance, as users have to trust the models they use. Domain knowledge can be integrated into algorithms using semantic constraints to guide model construction. This paper proposes an extension to an existing ACO-based classification rule learner to create lists of monotonic classification rules. The proposed algorithm was compared to a majority classifier and the Ordinal Learning Model (OLM) monotonic learner. Our results show that the proposed algorithm successfully outperformed OLM's predictive accuracy while still producing monotonic models.

Many data mining algorithms do not make use of existing domain knowledge when constructing their models. This can lead to model rejection as users may not trust models that behave contrary to their expectations. Semantic constraints provide a way to encapsulate this knowledge which can then be used to guide the construction of models. One of the most studied semantic constraints in the literature is monotonicity, however current monotonically-aware algorithms have focused on ordinal classification problems. This paper proposes an extension to an ACO-based regression algorithm in order to extract a list of monotonic regression rules. We compared the proposed algorithm against a greedy regression rule induction algorithm that preserves monotonic constraints and the well-known M5' Rules. Our experiments using eight publicly available data sets show that the proposed algorithm successfully creates monotonic rules while maintaining predictive accuracy.

Recent advances in geometric semantic genetic programming (GSGP) have shown that the results obtained by these methods can outperform those obtained by classical genetic programming algorithms, in particular in the context of symbolic regression. However, there are still many open issues on how to improve their search mechanism. One of these issues is how to get around the fact that the GSGP crossover operator cannot generate solutions that are placed outside the convex hull formed by the individuals of the current population. Although the mutation operator alleviates this problem, we cannot guarantee it will find promising regions of the search space within feasible computational time. In this direction, this paper proposes a new geometric dispersion operator that uses multiplicative factors to move individuals to less dense areas of the search space around the target solution before applying semantic genetic operators. Experiments in sixteen datasets show that the results obtained by the proposed operator are statistically significantly better than those produced by GSGP and that the operator does indeed spread the solutions around the target solution.

In this paper, we introduce Ant-MinerMA to tackle mixed-attribute classification problems. Most classification problems involve continuous, ordinal and categorical attributes. The majority of Ant Colony Optimization (ACO) classification algorithms have the limitation of being able to handle categorical attributes only, with few exceptions that use a discretisation procedure when handling continuous attributes either in a preprocessing stage or during the rule creation. Using a solution archive as a pheromone model, inspired by the ACO for mixed-variable optimization (ACO-MV), we eliminate the need for a discretisation procedure and attributes can be treated directly as continuous, ordinal, or categorical. We compared the proposed Ant-MinerMA against cAnt-Miner, an ACO-based classification algorithm that uses a discretisation procedure in the rule construction process. Our results show that Ant-MinerMA achieved significant improvements on computational time due to the elimination of the discretisation procedure without affecting the predictive performance.

Sequential Symbolic Regression (SSR) is a technique that recursively induces functions over the error of the current solution, concatenating them in an attempt to reduce the error of the resulting model. As proof of concept, the method was previously evaluated in one-dimensional problems and compared with canonical Genetic Programming (GP) and Geometric Semantic Genetic Programming (GSGP). In this paper we revisit SSR exploring the method behaviour in higher dimensional, larger and more heterogeneous datasets. We discuss the difficulties arising from the application of the method to more complex problems, e.g., overfitting, along with suggestions to overcome them. An experimental analysis was conducted comparing SSR to GP and GSGP, showing SSR solutions are smaller than those generated by the GSGP with similar performance and more accurate than those generated by the canonical GP.

In the path planning problem for autonomous mobile robots, robots have to plan their path from the start position to the goal. In this paper, we investigate the application of the MMAS algorithm to the exploratory path planning problem, in which the robots should explore the environment at the same time they plan the path. Max-min ant system is an ant colony optimization algorithm that exploits the best solutions found. In addition, to analyze the quality of solutions obtained, we also analyze the traveled distance spent by robots in the first iteration of the algorithm. The environment is previously unknown to the robots, although it is represented by a topological map, that does not require precise information from the environment and provides a simple way to execute the navigation of the path. Thus, the paths are represented by a sequence of actions that the robots should execute to reach the goal. The navigation of the best solution found was implemented in a realistic robotic simulator. The proposed algorithm provides a very good performance in relation to a genetic algorithm and the well-known A* algorithm that deal with this problem.

Ant-Tree-Miner is a decision tree induction algorithm that is based on the Ant Colony Optimization (ACO) meta- heuristic. Ant-Tree-Miner-M is a recently introduced extension of Ant-Tree-Miner that learns multi-tree classification models. A multi-tree model consists of multiple decision trees, one for each class value, where each class-based decision tree is responsible for discriminating between its class value and all other values present in the class domain (one vs. all). In this paper, we investigate the use of 10 different classification quality evaluation measures in Ant-Tree-Miner-M, which are used for both candidate model evaluation and model pruning. Our experimental results, using 40 popular benchmark datasets, identify several quality functions that substantially improve on the simple Accuracy quality function that was previously used in Ant-Tree-Miner-M.

The majority of Ant Colony Optimization (ACO) algorithms for data mining have dealt with classification or clustering problems. Regression remains an unexplored research area to the best of our knowledge. This paper proposes a new ACO algorithm that generates regression rules for data mining applications. The new algorithm combines components from an existing deterministic (greedy) separate and conquer algorithm—employing the same quality metrics and continuous attribute processing techniques—allowing a comparison of the two. The new algorithm has been shown to decrease the relative root mean square error when compared to the greedy algorithm. Additionally a different approach to handling continuous attributes was investigated showing further improvements were possible.

This paper investigates the impact of geometric semantic crossover operators in a wide range of symbolic regression problems. First, it analyses the impact of using Manhattan and Euclidean distance geometric semantic crossovers in the learning process. Then, it proposes two strategies to numerically optimize the crossover mask based on mathematical properties of these operators, instead of simply generating them randomly. An experimental analysis comparing geometric semantic crossovers using Euclidean and Manhattan distances and the proposed strategies is performed in a test bed of twenty datasets. The results show that the use of different distance functions in the semantic geometric crossover has little impact on the test error, and that our optimized crossover masks yield slightly better results. For SGP practitioners, we suggest the use of the semantic crossover based on the Euclidean distance, as it achieved similar results to those obtained by more complex operators.

Ant Colony Optimization (ACO) is a meta-heuristic for solving combinatorial optimization problems, inspired by the behaviour of biological ant colonies. One of the successful applications of ACO is learning classification models (classifiers). A classifier encodes the relationships between the input attribute values and the values of a class attribute in a given set of labelled cases and it can be used to predict the class value of new unlabelled cases. Decision trees have been widely used as a type of classification model that represent comprehensible knowledge to the user. In this paper, we propose the use of ACO-based algorithms for learning an extended multi-tree classification model, which consists of multiple decision trees, one for each class value. Each class-based decision trees is responsible for discriminating between its class value and all other values available in the class domain. Our proposed algorithms are empirically evaluated against well-known decision trees induction algorithms, as well as the ACO-based Ant-Tree-Miner algorithm. The results show an overall improvement in predictive accuracy over 32 benchmark datasets. We also discuss how the new multi-tree models can provide the user with more understanding and knowledge-interpretability in a given domain.

There are many different genetic programming (GP) frameworks that can be used to implement algorithms to solve a particular optimization problem. In order to use a framework, users need to become familiar with a large numbers of source code before actually implementing the algorithm, adding a learning overhead. In some cases, this can prevent users from trying out different frameworks. This paper discusses the implementation of a code generator in the EpochX framework to facilitate the implementation of GP algorithms. The code generator is based on the GP defini- tion language (GPDL), which is a framework-independent language that can be used to specify GP problems.

Financial forecasting is an important area in computational finance. EDDIE 8 is an established Genetic Programming financial forecasting algorithm, which has successfully been applied to a number of international datasets. The purpose of this paper is to further increase the algorithm's predictive performance, by improving its data space representation. In order to achieve this, we use attribute construction to create new (high-level) attributes from the original (low-level) attributes. To examine the effectiveness of the above method, we test the extended EDDIE's predictive performance across 25 datasets and compare it to the performance of two previous EDDIE algorithms. Results show that the introduction of attribute construction benefits the algorithm, allowing EDDIE to explore the use of new attributes to improve its predictive accuracy.

EpochX is a genetic programming framework with provision for event management – similar to the Java event model – allowing the notification of particular actions during the lifecycle of the evolutionary algorithm. It also provides a flexible Stats system to gather statistics measures. This paper introduces a graphical interface to the EpochX genetic programming framework, taking full advantage of EpochX's event management. A set of representation-independent and tree-dependent GUI components are presented, showing how statistic information can be presented in a rich format using the information provided by EpochX's Stats system.

The vast majority of Ant Colony Optimization (ACO) al- gorithms for inducing classification rules use an ACO-based procedure to create a rule in an one-at-a-time fashion. An improved search strategy has been proposed in the cAnt- MinerPB algorithm, where an ACO-based procedure is used to create a complete list of rules (ordered rules)—i.e., the ACO search is guided by the quality of a list of rules, instead of an individual rule. In this paper we propose an extension of the cAnt-MinerPB algorithm to discover a set of rules (unordered rules). The main motivation for discovering a set of rules is to improve the interpretation of individual rules and evaluate the impact on the predictive accuracy of the algorithm. We also propose a new measure to evaluate the interpretability of the discovered rules to mitigate the fact that the commonly-used model size measure ignores how the rules are used to make a class prediction. Comparisons with state-of-the-art rule induction algorithms and the cAnt-MinerPB producing ordered rules are also presented.

Ant-Miner is a classification rule discovery algorithm that is based on Ant Colony Optimization (ACO) meta-heuristic. cAnt-Miner is the extended version of the algorithm that handles continuous attributes on-the-fly during the rule construction process, while ?Ant-Miner is an extension of the algorithm that selects the rule class prior to its construction, and utilizes multiple pheromone types, one for each permitted rule class. In this paper, we combine these two algorithms to derive a new approach for learning classification rules using ACO. The proposed approach is based on using the measure function for 1) computing the heuristics for rule term selection, 2) a criteria for discretizing continuous attributes, and 3) evaluating the quality of the constructed rule for pheromone update as well. We explore the effect of using different measure functions for on the output model in terms of predictive accuracy and model size. Empirical evaluations found that hypothesis of different functions produce different results are acceptable according to Friedman's statistical test.

Researchers have been interested in exploring the regularities and modularity of the problem space in genetic programming (GP) with the aim of decomposing the original problem into several smaller subproblems. The main motivation is to allow GP to deal with more complex problems. Most previous works on modularity in GP emphasise the structure of modules used to encapsulate code and/or promote code reuse, instead of in the decomposition of the original problem. In this paper we propose a problem decomposition strategy that allows the use of a GP search to find solutions for subproblems and combine the individual solutions into the complete solution to the problem.

Ant Colony Optimisation (ACO) has been successfully applied to the classification task of data mining in the form of Ant-Miner. A new extension of Ant-Miner, called cAnt-MinerPB, uses the ACO procedure in a different fashion. The main difference is that the search in cAnt-MinerPB is optimised to find the best list of rules, whereas in Ant-Miner the search is optimised to find the best individual rule at each step of the sequential covering, producing a list of best rules. We aim to improve cAnt-MinerPB in two ways, firstly by dynamically finding the rule quality function which is used while the rules are being pruned, and secondly improving the rule-list quality function which is used to guide the search. We have found that changing the rule quality function has little effect on the overall performance, but that by improving the rule-list quality function we can positively affect the discovered lists of rules.

Ant colony optimization (ACO) algorithms for classification in general employ a sequential covering strategy to create a list of classification rules. A key component in this strategy is the selection of the rule quality function, since the algorithm aims at creating one rule at a time using an ACO-based procedure to search the best rule. Recently, an improved strategy has been proposed in the cAnt-MinerPB algorithm, where an ACO-based procedure is used to create a complete list of rules instead of individual rules. In the cAnt-MinerPB algorithm, the rule quality function has a smaller role and the search is guided by the quality of a list of rules. This paper sets out to determine the effect of different rule and list quality functions in terms of both predictive accuracy and size of the discovered model in cAnt-MinerPB. The comparative analysis is performed using 12 data sets from the UCI Machine Learning repository and shows that the effect of the rule quality functions in cAnt-MinerPB is different from the results previously presented in the literature.

EpochX is a Genetic Programming (GP) framework written in Java. It allows the creation of tree-based and grammar-based GP systems. It has been created to reflect typical ways in which Java programmers work, for example, borrowing from the Java event model and taking inspiration from the Java collections framework. This paper presents EpochX in general, and gives particular attention to the event model and the statistics analysis framework.

Genetic programming has proven capable of evolving solutions to a wide variety of problems. However, the successes have largely been with programs without iteration or recursion; evolving recursive programs has turned out to be particularly challenging. The main obstacle to evolving recursive programs seems to be that they are particularly fragile to the application of search operators: a small change in a correct recursive program generally produces a completely wrong program. In this paper, we present a simple and general method that allows us to pass back and forth from a recursive program to an associated non-recursive program. Finding a recursive program can be reduced to evolving non-recursive programs followed by converting the optimum non-recursive program found to the associated optimum recursive program. This avoids the fragility problem above, as evolution does not search the space of recursive programs. We present promising experimental results on a test-bed of recursive problems.

Ant Colony Optimization (ACO) differs substantially from other meta-heuristics such as Evolutionary Algorithms (EA). Two of its distinctive features are: (i) it is constructive rather than based on iterative improvements, and (ii) it employs problem knowledge in the construction process via the heuristic function, which is essential for its success. In this paper, we introduce the ACO encoding, which is a self-contained algorithmic component that can be readily used to make available these two particular features of ACO to any search algorithm for continuous spaces based on iterative improvements to solve combinatorial optimization problems.

Refactoring aims at improving the design of ex- isting code by introducing structural modifications without changing its behaviour. It is used to adjust a system's design in order to facilitate its maintenance and extendability. Since deciding which refactoring to apply and where it should be applied is not a straightforward decision, search-based approaches to automating the task of software refactoring have been proposed recently. So far, these approaches have been applied only to human-written code. Despite many years of computer programming experience, certain problems are very difficult for programmers to solve. To address this, researches have developed methods where computers automatically create program code from a description of the problem to be solved. One of the most popular forms of automated program creation is called Genetic Programming (GP). The aim of this work is to make GP more effective by introducing an automated refactoring step, based on the refactoring work in the software engineering community. We believe that the refactoring step will enhance the ability of GP to produce code that solves more complex problems, as well as result in evolved code that is both simpler and more idiomatically structured than that produced by traditional GP methods.

Most real-world classification problems involve continuous (real-valued) attributes, as well as, nominal (discrete) attributes. The majority of Ant Colony Optimisation (ACO) classification algorithms have the limitation of only being able to cope with nominal attributes directly. Extending the approach for coping with continuous attributes presented by cAnt-Miner (Ant-Miner coping with continuous attributes), in this paper we propose two new methods for handling continuous attributes in ACO classification algorithms. The first method allows a more flexible representation of continuous attributes' intervals. The second method explores the problem of attribute interaction, which originates from the way that continuous attributes are handled in cAnt-Miner, in order to implement an improved pheromone updating method. Empirical evaluation on eight publicly available data sets shows that the proposed methods facilitate the discovery of more accurate classification models.

Abstract Ant colony optimisation (ACO) is a metaheuristic to solve optimisation problems inspired by the foraging behaviour of ant colonies. It has been successfully applied to several types of optimisation problems, such as scheduling and routing, and more recently for the discovery of classification rules. The classification task in data mining aims at predicting the value of a given goal attribute for an example, based on the values of a set of predictor attributes for that example. Since real-world classification problems are generally described by nominal (categorical or discrete) and continuous (real-valued) attributes, classification algorithms are required to be able to cope with both nominal and continuous attributes. Current ACO classification algorithms have been designed with the limitation of discovering rules using nominal attributes describing the data. Furthermore, they also have the limitation of not coping with more complex types of classification problems e.g., hierarchical multi-label classification problems. This thesis investigates the extension of ACO classification algorithms to cope with the aforementioned limitations. Firstly, a method is proposed to extend the rule construction process of ACO classification algorithms to cope with continuous attributes directly. Four new ACO classification algorithms are presented, as well as a comparison between them and well-known classification algorithms from the literature. Secondly, an ACO classification algorithm for the hierarchical problem of protein function prediction which is a major type of bioinformatics problem addressed in this thesis is presented. Finally, three different approaches to extend ACO classification algorithms to the more complex case of hierarchical multi-label classification are described, elaborating on the ideas of the proposed hierarchical classification ACO algorithm. These algorithms are compare against state-of-the-art decision tree induction algorithms for hierarchical multi-label classification in the context of protein function prediction. The computational results of experiments with a wide range of data sets including challenging protein function prediction data sets with very large number

Biological and other natural processes have always been a source of inspiration for computer science and information technology. Many emerging problem solving techniques integrate advanced evolution and cooperation strategies, encompassing a range of spatio-temporal scales for visionary conceptualization of evolutionary computation. This book is a collection of research works presented in the VI International Workshop on Nature Inspired Cooperative Strategies for Optimization (NICSO) held in Canterbury, UK. Previous editions of NICSO were held in Granada, Spain (2006 & 2010), Acireale, Italy (2007), Tenerife, Spain (2008), and Cluj-Napoca, Romania (2011). NICSO 2013 and this book provides a place where state-of-the-art research, latest ideas and emerging areas of nature inspired cooperative strategies for problem solving are vigorously discussed and exchanged among the scientific community. The breadth and variety of articles in this book report on nature inspired methods and applications such as Swarm Intelligence, Hyper-heuristics, Evolutionary Algorithms, Cellular Automata, Artificial Bee Colony, Dynamic Optimization, Support Vector Machines, Multi-Agent Systems, Ant Clustering, Evolutionary Design Optimisation, Game Theory and other several Cooperation Models.