Training Analysis Page: Learning Curve

The Modify Field Dialog allows you to analyze and modify the properties of a field. The Training Analysis page allows you to view an analysis of the prediction results versus the desired values.

&  For help with predictions, see Predicting and Modeling Financial Data.

Learning Curve Sub-Page Data

This sub-page displays the learning curve of the best training pass. The learning curve is a graph of the internal error of the neural network over each epoch of training. This is the same graph that is displayed on the Status Dialog during training.

Ä    Note: The internal error of the neural network can be substantially different from the Normalized Errors reported in TradingSolutions. The internal errors being reported are the "cost" associated with the criterion component of the neural network. Typically, this is the Mean Squared Error (MSE) for the normalized values in the neural network. The values in TradingSolutions have been denormalized from their neural network values and the associated errors are then normalized using the "last known values".

The learning curve for the training set and the cross validation set should both decrease over time. Bumps during the initial epochs are acceptable, but both curves should be mostly horizontal when training ends. Over time, the learning curve for the cross validation set will typically begin to increase as the prediction begins to over-specialize on the training data.

Ä    Note: The training weights from the neural network are saved at the point of the best cross validation error.

Ä    Note: If learning criteria options, such as optimizing for directional accuracy, are being used, the learning curves may not be as smooth as those when the options are not used. These options can be enabled or disabled from the Prediction Model page.

In addition to the learning curve, the following statistics are displayed.

p    Total Time

This value indicates the total amount of time that was taken to train this prediction, including all passes. If the training of this prediction is restarted, this value is zeroed before being updated.

p    Epoch of Best

This value indicates which epoch produced the best error for the cross validation set.

p    Total Epochs

This value indicates the number of times the training set was presented to the neural network during the training phase. Training may stop before the maximum number of epochs if the error on the cross validation set does not decrease over a given number of epochs, set on the Modify Training Settings Dialog. Training will also stop if the Stop Training button is pressed on the Status Dialog.

Ä    Note: If the maximum number of epochs is reached and cross validation is being used, more learning may be possible by resuming training with a higher maximum number of epochs. The maximum number of epochs can be increased manually from the Modify Training Settings Dialog.

Adjusting Chart Ranges

The epoch and value axes can be scaled to display specific portions of the data. The options for doing this are similar to those associated with the Chart View. For more information, see Adjusting Chart Ranges.

Learning Curve Sub-Page Analysis

The analysis presented on this page detects common characteristics of learning curves and recommends actions, if appropriate. Some common results include:

·      The best training weights were found in epoch #n.

This is the normal analysis condition. The error on the cross validation was lowest at the reported epoch. Using the typical training values, TradingSolutions will continue training for 50 epochs after this value.

·      The best training weights were found close to the end of training. Continued training may produce better results.

The error on the cross validation was lowest within a few epochs of the end of training. Typically, this indicates that better results can be achieved by resuming the training. The training can be resumed at any time by pressing the Resume Training button on the Overview sub-page.

·      No significant learning appears to have taken place.

This message appears when the best cost occurred on the first epoch.

If a problem occurred during the training or calculation phase, the analysis will be replaced with a description of the error. A summary of these error messages is displayed on the help for the Modify Field Dialog: Training Analysis page.

How Did I Get Here?

This is a sub-page of the Modify Field Dialog: Training Analysis page.