Advanced Topology Settings Dialog

Advanced Topology Settings Dialog

The Advanced Topology Settings Dialog allows you to adjust the individual elements of a network topology. It is primarily for people who are familiar with the internal workings of neural networks.

Ä Note: Fields that are currently being used by the Solution Service cannot be modified.

& For help with predictions, see Predicting and Modeling Financial Data.

Viewing a Neural Network Topology

The Advanced Topology Settings Dialog displays a diagram of the currently selected neural network topology. The properties for each component in the topology are available in property pages beneath the diagram. The properties for a component can also be accessed by clicking on the picture of that component.

Ä Note: Users of the NeuroSolutions neural network simulation environment will recognize the diagram as a half-scale display of a NeuroSolutions breadboard. For simplicity, the properties of the back-propagation and gradient search components are displayed as part of the properties for the associated activation layer components.

& For help using property pages, see the help for Property Pages.

Understanding the Topology Diagram

$image\ebd_ebd55.gif$	Axon This component serves as a placeholder for introducing inputs into the network.
$image\ebd_ebd56.gif$	Synapse This component takes each element of the previous component and sends it to each element in the next component with a weight applied.
$image\ebd_ebd57.gif$	TanH Axon This component contains processing elements (PE’s) that introduce non-linear processing into the model.
$image\ebd_ebd58.gif$	Bias Axon This component shifts the results of the processing to have the same mean as the desired outputs.
$image\ebd_ebd59.gif$	L2 Temporal Criterion Component This component analyzes the current results and feeds back an error value used to adjust the weights during the training phase.
$image\ebd_ebd60.gif$	TDNN Memory Axon This component contains memory elements that pass along previous values as inputs to the next component.
$image\ebd_ebd61.gif$	Gamma Memory Axon This component is similar to the TDNN Memory Axon, except that it uses exponents to persist previous values.
$image\ebd_ebd62.gif$	Laguarre Memory Axon This component is similar to the Gamma Memory Axon, except that it uses a more complex equation.
$image\ebd_ebd63.gif$	Delay Synapse This component is similar to the Synapse, except that it delays its output by one sample.
$image\ebd_ebd64.gif$	Integral Axon This component is used in Jordan/Elman networks to apply processing to previous information.
$image\ebd_ebd65.gif$	Context Axon This component is used in Jordan/Elman networks to apply processing to previous information.

Automatically Maintaining a Neural Network Topology

By default, TradingSolutions automatically maintains the best estimates for the settings to use in the neural network topology for a prediction. Adjustments are primarily made in response to changes in the number of inputs.

This automatic updating can be disabled in two ways. One way is to turn it off manually from the Advanced Topology Settings Dialog. The other way is to enable genetic optimization, which turns it off in order to optimize the settings.

þ Automatically Maintain Best Estimates

This option enables you to turn on or off the automatic updating of the settings.

Ä Note: If this option is on, the other settings on this dialog cannot be modified.

Fine-Tuning a Neural Network Topology

Settings are available for each individual component of the neural network topology. In addition, overall settings are available for the entire topology.

0 Overall

The following settings are available for adjusting the overall topology.

¨ Bump Step Sizes

This adjustment allows you to modify the step sizes of all of components with step sizes up or down by a given percentage of their value. Press the Up or Down buttons to increase or decrease the step sizes.

: Example: Setting the value to 25 and pressing the Up button will increase the step sizes of all of the components by 25% of their current value.

p Adjust Seed…

This button displays the Adjust Random Number Seed Dialog, which allows you to set the random number seed used when initializing the weights of the neural networks. The seed is used at the beginning of the first training pass only.

¨ Weight Updates: Batch or Online

This setting indicates whether the weights associated with each component should be updated after all of the data has been presented (batch) or after each piece of data is presented (online). This setting is only available for static networks.

¨ Weight Updates: Samples Per Exemplar

This setting indicates how many samples should be examined each time the neural network updates its weights during the training phase. This setting is only available for dynamic networks. By default, TradingSolutions will set this value to twice the memory depth.

0 Individual Components (non-Criterion)

The individual components of the topology each have some combination of the following settings. The only exceptions are the Criterion component and some components which have no modifiable settings.

¨ Processing Elements (PEs)

This setting indicates the number of processing elements for this component. This setting is only available for the TanH axons in each hidden layer.

Processing elements are the processing portion of a neural network. In theory, more processing elements will be able to learn more complex relationships. However, the network may be harder to train effectively. It may also become overly-specific to the training data, making it poor for predicting values not in the training set.

¨ Memory Taps

This setting indicates the number of memory taps for this component. This setting is only available for memory components, including Input Axons with memory.

The number of memory taps effectively controls the number of samples into the past a memory component can remember.

Ä Note: It is recommend that the input layer have the highest number of memory taps since it has access to the least modified data. The number of memory taps then typically decreases through the hidden layers.

¨ Momentum

This setting indicates the momentum rate for the gradient search portion of this component. This setting is only available for components that are modified during weight updates.

The momentum is used to speed the learning by taking a portion of the most recent weight update and reapplying it. It also helps the network arrive at better solutions by helping it avoid local minima. The momentum value is the amount by which the most recent weight update is multiplied after the step size has been applied. It should have a value less than 1.

¨ Step Size

This setting indicates the step size for the gradient search portion of this component. This setting is only available for components that are modified during weight updates.

The step size is used to determine the amount by which an individual component will react during each weight update. It is the amount by which each weight in the component will be adjusted. Using values close to 0 will cause the component to adjust very slowly. Using values closer to 1 may cause the component to over-react during weight updates and render them incapable or learning.

Ä Note: It is recommended that the first hidden layer have the largest step size, followed by the second hidden layer, then the output layer. In addition, memory components should have significantly lower step sizes than other components on the same layer.

0 Criterion Component

The training of the prediction can be optimized to highlight different aspects of the data. To select individual learning criteria, select the appropriate checkboxes.

Ä Note: These settings are based on research into price prediction. They are not typically used for modeling the optimal signal.

þ Discount Least Recent Values

This optimization adds additional value to successfully learning more recent values. This is useful when predicting time series in which the conditions may be changing over time, such as predictions based on long-term historical data.

þ Optimize for Directional Accuracy

This optimization adds additional value to successfully learning the sign of the desired output. This is useful when basing an entry/exit system on the positive or negative direction of a prediction. It may also be useful for predicting entry/exit signals since it adds additional value to successfully learning the infrequent actual signals, rather than the more frequent non-signals.

Ä Note: The effect of this setting is modified by the Output Data is Pre-Differenced setting.

þ Minimize Number of Large Errors

This optimization adds additional value to successfully reducing the number of large errors, where the error is the difference between the desired output and the predicted value. This is useful for learning infrequent data that has a desired value that is not the same as most of the desired values. However, this may result in less accuracy overall.

þ Output Data is Pre-Differenced

This setting indicates that the values of the desired output data represent change or percent change data. It affects how Optimize for Directional Accuracy operates in that the criterion component will need to calculate the change from the previous value is the data is not pre-differenced. It also affects how the Directional Accuracy analysis is calculated on the Directional Accuracy Sub-page.

Ä Note: This setting is repeated from the Modify Field Dialog: Desired Output page.

What Do I Do Next?

When you are finished modifying the settings, press the Close button. Any changes made to the settings in this dialog will not be saved until the changes to the associated prediction field are saved.

How Did I Get Here?

The Advanced Topology Settings Dialog is displayed when you press the Advanced Topology Settings… button on the Modify Field Dialog: Prediction Model page.