Our live neural network courses held twice per year in Orlando give participants a broad overview of both neural network theory and the NeuroSolutions software. One of the instructors recently recorded a 30-minute video presentation, which covers some of the introductory material presented in these courses. We have made this presentation available for free on our website so that everyone can take advantage of this material.

To view this presentation now, visit: http://www.neurosolutions.com/products/ns/nnandnsvideo.html

Mr. Shramenko uses TradingSolutions to swing-trade the QQQ single-stock futures contract, and day-trade the Dow-Mini and the S&P-Mini futures contracts. With TradingSolutions, he has had an average return of 80% to 120% per month for 2003. About 60% of his trades were winners. However, his average winning trade was 4 times larger than his average losing trade.

More details about this interview are available in our new Customer Interview section, including details on his innovative approach to modeling. Read it now at: http://www.tradingsolutions.com/news/yuri.html

One question we occasionally receive about TradingSolutions is whether it is necessary to normalize your indicators when using them as inputs to neural networks. The short answer is "no." Data normalization is one of the many neural network concepts TradingSolutions handles for you automatically behind the scenes so that you don't have to worry about it.

This question frequently comes from new TradingSolutions users who have read papers or books about neural networks which talk about the importance of data normalization. Here is some background on what it is and why it is used.

The Advantage of Non-Linear Models
Linear models take a set of inputs, scale them, add them together, and provide an output. Any given value of an input will have the same effect on the output, regardless of the other inputs.

Neural networks are able to learn complex principles because they are non-linear models. Non-linear models also take a set of inputs and provide an output. However, any given value of an input can have a different effect on the output depending on the values of the other inputs. As an example in technical analysis, one input may provide information about whether the price is trending or trading (flat). This input could change the type of effect that other inputs have on the output of the equation.

Combining Linear Models
In effect, non-linear models can be viewed as many locally linear models combined into one. Each set of inputs can have a slightly different relationship based on the locally linear model it uses. The result of this is that sometimes increases in an input can cause an increase in the output, and other times they can cause a decrease.

Neural networks typically combine linear models using layers – for example, an input layer and a hidden layer. These two layers are connected through a non-linear function, such as a hyperbolic tangent (tanh) function, which “bends” the output of the input layer and feeds it into the hidden layer. This bending emphasizes some values and de-emphasizes others, resulting in a non-linear relationship between the layers.

Data Normalization
In each layer, the inputs are weighted to increase or decrease their effect on the output. To keep larger inputs from overshadowing smaller inputs in the initial equation, all inputs are scaled to a common range to make them appear equally important. This is called normalization. From this initial point, the training of the neural network adjusts the weights to reproduce the desired output as closely as possible.

Internally, the tanh function performs best with values between -1 and 1. For values larger than 1, the tanh function “saturates” and asymptotically approaches 1. In other words, there is very little difference between the output of the tanh function when the input is 10 or 2000. Similarly, for large negative values, the tanh function saturates toward -1. Therefore, inputs are typically normalized to a range between -1 and 1 so that the values fed into the tanh function begin in a reasonably good range to not be overly saturated, resulting in more efficient training. TradingSolutions neural networks handle this normalization automatically.

In the next newsletter, we'll go into more detail about the effects of saturation and how it can affect your selection of input preprocessing.