What is a Neural Network?

Ä    Note: It is not necessary to read this section in order to use TradingSolutions. However, understanding the concepts used in the program can help you to get the most out of the intermediate and advanced features.

A neural network is a powerful data modeling tool that is able to capture and represent complex input/output relationships. The motivation for the development of neural network technology stemmed from the desire to develop an artificial system that could perform "intelligent" tasks similar to those performed by the human brain. Neural networks resemble the human brain in the following two ways:

1. A neural network acquires knowledge through learning.

2. A neural network’s knowledge is stored within inter-neuron connection strengths known as synaptic weights.

The true power and advantage of neural networks lies in their ability to represent both linear and non-linear relationships and in their ability to learn these relationships directly from the data being modeled. Traditional linear models are simply inadequate when it comes to modeling data that contains non-linear characteristics.

The most common neural network model is the multilayer perceptron (MLP). This type of neural network is known as a supervised network because it requires a desired output in order to learn. The goal of this type of network is to create a model that correctly maps the input to the output using historical data so that the model can then be used to produce the output when the desired output is unknown. A graphical representation of an MLP is shown below.

Figure 1: Block diagram of a two hidden layer multiplayer perceptron (MLP). The inputs are fed into the input layer and get multiplied by interconnection weights as they are passed from the input layer to the first hidden layer. Within the first hidden layer, they get summed then processed by a nonlinear function (usually the hyperbolic tangent). As the processed data leaves the first hidden layer, again it gets multiplied by interconnection weights, then summed and processed by the second hidden layer. Finally the data is multiplied by interconnection weights then processed one last time within the output layer to produce the neural network output.

The MLP and many other neural networks learn using an algorithm called backpropagation. With backpropagation, the input data is repeatedly presented to the neural network. With each presentation the output of the neural network is compared to the desired output and an error is computed. This error is then fed back (backpropagated) to the neural network and used to adjust the weights such that the error decreases with each iteration and the neural model gets closer and closer to producing the desired output. This process is known as "training".

Figure 2: Demonstration of a neural network learning to model the exclusive-or (Xor) data. The Xor data is repeatedly presented to the neural network. With each presentation, the error between the network output and the desired output is computed and fed back to the neural network. The neural network uses this error to adjust its weights such that the error will be decreased. This sequence of events is usually repeated until an acceptable error has been reached or until the network no longer appears to be learning.

In addition to the multilayer perceptron and other industry-standard neural networks, TradingSolutions includes state-of-the-art time-lag recurrent networks. These dynamic networks are much better than static networks for modeling data that contains information over time, such as financial data. They have locally recurrent connections where they are able to retain past information. Furthermore, the amount of past information stored (memory depth) can be adapted by the neural network to find the optimum. These networks are trained using an extension of backpropagation known as "backpropagation through time", where the errors are not only backpropagated to the neural network for the current instant, but are also backpropagated across time.

&  Continue to the next section, What are Genetic Algorithms?, or return to the Overview for this chapter.