Training Optimization Settings Dialog
The Training Optimization Settings Dialog allows you to set the values associated with the optimization of predictions. It is also used for modifying the default settings for the optimization of predictions.
Ä Note: Fields that are currently being used by the Solution Service cannot be modified.
& For help with predictions, see Predicting and Modeling Financial Data.
& For help with optimizing fields, see Optimizing Signals and Predictions.
Selecting Data Optimizations
The following settings are available for optimizing the data being used in the prediction.
þ Input Selections/Parameters
This option indicates to optimize which inputs are included in the prediction. If optimizable functions are included in the list of inputs, the parameters to these functions will also be optimized.
Ä Note: Prediction inputs which are existing fields cannot have their parameters optimized. These fields can only be included or excluded by the optimization process. Individual fields can be excluded from being optimized from the Modify Prediction Input Dialog for each input.
þ Memory Depth
This option indicates to optimize the memory depth associated with the prediction. Optimizing the memory depth allows the prediction to adjust how many recent values of each input the prediction will use. Integer values from 1 to 20 are attempted.
Ä Note: Memory depth can only be optimized in neural network models containing memory, such as time-lag recurrent networks. You can see the neural network model currently associated with a prediction on the Create a Prediction Wizard: Configure Model page or the Modify Field Dialog: Prediction Model page.
Ä Note: If Maintain Auto Samples-to-Weights Ratio is selected below, the memory depth will be adjusted using a single value for the entire neural network. If another option is selected, the memory depth will be adjusted for the individual neural networks with memory.
Ä Note: This option is not available when using a Custom Solution Wizard (CSW) DLL.
Selecting Model Optimizations
The following settings are available for optimizing the neural network model being used in the prediction.
þ Neural Network Topology
This option optimizes which neural network topology is used. It corresponds to the Neural Model, Hidden Layers and Characteristics settings on the Modify Field Dialog: Prediction Model page.
Ä Note: This option can only be used with Maintain Auto Samples-to-Weights Ratio.
p Select Topologies…
This button will display the Topology Optimization Selection Dialog to select which topologies should be considered during optimization.
Ä Note: By default, brief optimization will only try a common subset of the available topologies.
¨ Samples-to-Weights Ratio Monitoring
The samples-to-weights ratio monitors the number of samples of data available versus the number of weights in the neural network. Weights are added to the neural network based on the number of inputs, the memory depth, and the number of processing elements.
The more weights available in a neural network, the more information it can store about individual patterns it can find. Having more weights allows it to learn more patterns, but can cause the network to become to specific to the data it is using for training and not able to handle new situations. Having too few weights can limit the number of patterns that can be learned. A good rule-of-thumb is to use a samples-to-weights ratio of 10 samples per weight.
The following settings are available for monitoring the samples-to-weights ratio during optimization:
· Maintain Auto Samples-to-Weights Ratio
This option adjusts the number of processing elements to maintain a samples-to-weights ratio as close to the target value as possible. The setting corresponds to the Automatically Maintain Best Estimates setting on the Advanced Topology Settings Dialog. The target value can be set on the Modify Training Settings Dialog.
Ä Note: Selecting this option disables the optimization of the other Model Optimization settings.
· Penalize Low Samples-to-Weights Ratios
This option monitors the samples-to-weights ratio of each test. If the value is less than 75% of the target samples-to-weights ratio, the fitness is significantly decreased as a penalty. The target value can be set on the Modify Training Settings Dialog.
· Do Not Monitor Samples-to-Weights Ratio
This option causes the samples-to-weights ratio to not be monitored during optimization.
Ä Note: This setting can cause the model to become overly specialized on the training data. This is because the most specialized results will typically be found using the most weights. Therefore, not restricting the number of weights can result in overspecialization. This is true even when not optimizing PE’s since other settings can adjust the number of weights.
· Optimize Inputs to Custom Solution Wizard DLL
This option disables the optimization of settings not available when using a Custom Solution Wizard DLL for the model. It is selected automatically if a DLL is selected.
þ Number of Processing Elements
This option optimizes the number of processing elements in each hidden layer of the neural model. This value is normally set from the Advanced Topology Settings Dialog using the sub-page of the Layer 1 Axon and Layer 2 Axon. Genetic optimization will try integer values from 1 to 15.
þ Weight Update Sample Size
This option optimizes the weight update method used by the neural model. This value is normally set from the Advanced Topology Settings Dialog Overview sub-page.
þ Learning Momentum Rate
This option optimizes the momentum associated with the learning rates of the individual components of the neural model. These values are normally set from the Advanced Topology Settings Dialog using the sub-page for each component. Genetic optimization will try all values from 0 to 1.
þ Learning Step Size
This option optimizes the step size associated with the learning rates of the individual components of the neural model. These values are normally set from the Advanced Topology Settings Dialog using the sub-page for each component. Genetic optimization will try all values from 0 to 1.
þ Advanced Learning Criteria
This option optimizes the selection of the settings on the Criterion page of the Advanced Topology Settings Dialog. Specifically, the following settings are considered:
· Discount Least Recent Values
· Optimize for Directional Accuracy
· Minimize Number of Large Errors
Selecting a Fitness Calculation
For each potential group of settings (chromosome), the genetic optimization determines a fitness value to assess how good it is. The settings with the best fitness value are kept.
The following values are available as a basis for the fitness:
· Maximize Profit (based on Trading Style) *
This setting indicates to use the best signal fitness value for a predicted signal. The signal fitness calculation is specified in the trading style associated with the prediction on the Modify Trading Style Dialog: Signal Optimization page.
Ä Note: * This setting is only available for predicted signals. When predicting non-signals, the prediction error will be minimized instead.
· Minimize Prediction Error
This setting indicates to use the best prediction error. The prediction error is defined as the mean squared difference between the predicted value and the desired output.
· Maximize Prediction Correlation
This setting indicates to use the best prediction correlation. The prediction correlation is defined as the correlation between the output and desired output values.
Ä Note: The correlation is examined prior to any postprocessing.
The date range used for calculating the fitness value can be set to the following values:
¨ Date ranges to use for optimization.
This value indicates the training ranges to use for optimization of the postprocessing settings.
· Optimize Training Set, Checking Cross Val.
This setting indicates to optimize the fitness of the training set, but to take the settings from the generation with the best fitness for the cross validation set. It also causes the optimization to stop when no improvement occurs on the cross validation set data over a given number of generations.
· (Optimize Particular Set or Sets)
This setting indicates to optimize the fitness of the indicated set or sets without monitoring the performance of a cross validation set.
Ä Note: * The testing set and the use of multiple sets are not available when optimizing on prediction error. If one of these values are selected, the training error will be used.
Ä Note: You will typically not want to include the testing set in the optimization data. This would result in there not being data set aside for verifying that good general settings are being found.
Other Settings Available
The following options are also available on this dialog.
p Genetic Algorithm Settings…
This button will display the Modify Genetic Algorithm Settings Dialog, which allows you to adjust the settings associated with genetic optimization.
p Restore Defaults
This button will restore the settings to their default values.
What Do I Do Next?
When you are done modifying the optimization settings, press the OK button. If you would prefer to exit this dialog without making modifications, press the Cancel button.
How Did I Get Here?
The Modify Training Settings Dialog is displayed for a single field when you press the Optimization Settings… button on the Predict a Value Wizard: Select Options page or the Modify Field Dialog: Training Settings page.
The Modify Training Settings Dialog is displayed for modifying default values when you press the Optimization Settings… button on the Modify Training Settings Dialog.