Stock Market Analysis and Prediction
Using Artificial Neural Networks
and Machine Learning Techniques
Massimiliano Versace, Rushi Bhatt, Oliver Hinds, Mark Shiffer
Dept. of Cognitive and Neural Systems
Boston University
Boston, MA 02215
Contact us - Read the paper
Abstract: Just as expert technical analysts learn to exploit changing market conditions in deciding which technical indicators to rely on over time, artificial intelligence can utilize adaptive methods to discover hidden patterns in large amounts of dynamic data. Software programs that combine the strengths of traditional statistical fitting techniques, modern machine learning algorithms, and artificial neural networks are adaptively created through the use of genetic algorithms, which determine the optimal combination of techniques to maximize correct prediction of stock market direction. Although the software development techniques are advanced, assurance that proper input parameters are being considered is sought.
Institutions and individual investors are finding a wide range of uses for artificial intelligence (AI) technologies such as expert systems, neural networks, genetic algorithms and fuzzy logic. The main rationale motivating the use of these techniques is to identify and exploit the regularity that is hidden in the apparently chaotic price of a given security. The process of analyzing historical stock prices in an effort to determine probable future prices is also the main goal of Technical analysis (TA). TA attempts to predict the performance of a stock by spotting trends in price, without regard to the financial situation of the underlying company. Similarly to an "Artificial Expert", a TA analyst develops its private set of rules for classifying a given price pattern as leading to a trend. TA is not an exact science, and only a minority of technicians can consistently and accurately determine future prices. These are probably the ones that have synthesized the best "cocktail" of rules that apply to a given security at a given time.
The same argument applies to automated trading systems, in which only a subset of these widely different techniques lead to an optimal strategy at a given time. It is natural to ask, then, if it is possible to find an ideal combination of AI technologies that would optimize the performance over time.
In the last two decades, the field of statistical pattern recognition has undergone major changes with the advances in algorithms usually referred to as Artificial Neural Networks (ANNs). Historically, these algorithms were inspired by investigations into the functioning of the nervous system. Nevertheless, ANNs have been found to have a sound theoretical basis from the perspective of statistical learning theory as well as machine learning, and usually yield good performance when used for real-world data analysis. Apart from that, ANNs have good generalization capabilities and are usually robust against noisy or missing data. These are highly desirable properties for statistical analysis. Importantly, the class of unsupervised neural networks is not hypothesis driven in the way regression techniques usually are. Therefore, unsupervised neural networks have the capability of discovering associations between features that may not have been expected or looked for. This is particularly important when contrasted to classical TA techniques, in which the selection of the rules is often a matter of the subjective taste of the analyst, which may or may not lead to a successful strategy.
Artificial Neural Network methods can be utilized in conjunction with other modern machine learning algorithms (e.g., decision tree learning, Support Vector machines) as well as traditional statistical fitting techniques. It is our belief that an aggregation, or pooling, of such diverse function approximation schemes can yield a machine that will outperform any single technique for predicting the stock market. Extensive theoretical literature exists on pooling learning algorithms with diverse capabilities has been performed in the past under the banner of mixture (or product) of experts, where voting schemes are devised to predict outcomes or responses in systems. Such methods need to be utilized when one attempts to predict the movements in the stock market. Apart from the mixture of expert models described above, meta-level techniques for boosting the correctness of the prediction also exist. One example of such a technique is using Genetic Algorithms (GA), where the attempt is made to combine instances of classifiers (e.g., neural networks with different topologies) in order to build a new classifier (e.g., a network with a different connectivity pattern) that could perform better than its predecessors due to inheritance of the better features from both.
We believe that a gain in performance over traditional statistical techniques can be achieved for stock market prediction using the above techniques. In particular, we believe that it is extremely hard to fit a single uniform multivariate function that can consistently predict the ups and downs of the market. The method of prediction needs to be highly context sensitive (i.e., the moods of the market depend highly on the overall sentiment), and that the traditional parameters for technical analysis of stock data are not sufficient to fully characterize and predict the market response. On the other hand, it may be possible to fit a family of functions (or in ANN terminology, train a group of networks) to predict the market, such that each network may be well suited to perform in a given context. We wish to collaborate with an expert in market behavior who can help us identify any critical "non-traditional" parameters that govern the sentiment of the market. Such "meta-information" can then be used to partition the time series according to these non-quantifiable parameters and one could then use the aforementioned techniques to train a diverse group of algorithms. The pooling techniques mentioned earlier can then be utilized along with any meta-level learning methods to select the best classifier amongst the available ones for the purpose of prediction. Disclaimer
|
