Streaming Intraday Data
Intraday data is typically most effective when it used for trading while it is occurring. The process of bringing in intraday data while it is occurring is called "streaming". In other words, the data is "streaming" from the Internet into your TradingSolutions portfolio.
Streaming Data Basics
In order to stream intraday data into your portfolio, you need to have an account with a TradingSolutions data source that supports streaming data. At this time, eSignal is the only data provider supported by TradingSolutions.
Streaming is turned on and off using the Stream Intraday Data button in the toolbar. When streaming is turned on, any intraday data series that have the option Include in intraday streaming updates selected will be streamed.
Ä Note: Additional information on these topics is available below.
The easiest way to add a symbol to your portfolio so that it will be streamed is to use the Import Data Wizard to import some historical data for that symbol. That historical data can then be used to develop entry/exit signals for this data. These signals can then be tested and traded on new data as it streams in.
Ä Note: If you would like to stream a symbol without importing historical data, use the Create Empty Data Series Dialog.
& For help importing historical data, see Importing Historical Data.
Each streaming symbol has an associated periodicity. The periodicity indicates the number of minutes to include in each High / Low / Open / Close bar. For example, streaming data with a periodicity of five minutes would calculate a new price bar every five minutes and, therefore, update any associated calculations or signals every five minutes. The periodicity also affects how data will be displayed in a chart or spreadsheet.
Ä Note: Only one periodicity can currently be streamed at a time for each symbol.
Making Intraday Data Available for Streaming
In order for an intraday data series to be streamed, the following must be true:
· The data series must have the option Include in intraday streaming updates selected.
· The default data source for the data series must support streaming.
· Your account with that data source must support real-time streaming data.
Ä Note: Some data sources require individual exchanges to be activated. For example, eSignal requires separate activation of NYSE, NASDAQ, and AMEX exchanges within each account.
The data source and update options for individual data series can be viewed and modified from the Modify Data Series Dialog: Properties page These settings can be modified for entire groups from the Modify Group Dialog: Properties page.
Starting and Stopping Streaming Data
Streaming data is started and stopped by selecting Stream Intraday Data in the Data menu or pressing the Stream Intraday Data button in the toolbar.
Streaming will start as long as at least one data series is available for streaming. See above for rules in determining which data series are available for streaming.
Ä Note: Unlike manual updates, streaming cannot be turned on for a single target group. To stream stocks from a specific group, turn the option Include in intraday streaming updates off for the stocks you do not want to stream.
Ä Note: When streaming is started for eSignal, the eSignal Data Manager will start automatically if it has not been started already.
When streaming is first started, one or more "gap fills" may be requested to update the data in the portfolio so that it has data through the current time.
Ä Note: Depending on when a data series was last updated, the associated gap fill may include several hours or days of data. The larger the gap fill, the more processing that will be required to bring a data series up-to-date. Because of this, it may sometimes take several minutes for TradingSolutions to begin processing new data at as fast as speed as possible.
Streaming will continue until it is stopped or until data can no longer be streamed. If streaming is stopped automatically, a message will be displayed in the Messages and Alerts Window indicating what happened.
Processing of Streaming Data
TradingSolutions is not limited in the number of symbols that it can stream at one time. However, it is limited in the amount of data it can process in a timely manner.
When new data is imported or streamed into TradingSolutions, the fields associated with that data are updated in the following order:
1. Alerts
2. Signals in the Portfolio View (Display Fields)
3. Other Values in the Portfolio View (Display Fields)
4. Values in Charts and Spreadsheets
5. Other Calculations
Ä Note: When working with multiple periodicities of data, calculations for the shorter periodicity data are processed first. The one exception to this is "other calculations", which are processed after the other items for all periodicities.
Since most trading decisions are based on the first three items (alerts and display fields), it is important that these values are calculated as quickly as possible. TradingSolutions keeps track of how long it is taking to calculate these fields and warns you if the performance may not be fast enough to make trades effectively.
When streaming is first started, any data and calculations need to be brought up-to-date from the last data in each streaming data series. This process is called a "gap fill" and it is performed automatically for you. All of the data that has occurred since the last data in each streaming data series is downloaded and any calculations are updated. This process can take several minutes (or more) depending on the amount of data that is downloaded and the number of calculations that need to be updated.
Ä Note: Additional information is available below about the messages associated with the status of this processing.
Once processing of the gap fill is complete, TradingSolutions will monitor the amount of time it takes to process each group of new bars for each periodicity. If it cannot calculate the associated alerts and display fields before new bars are arriving, a warning message is displayed indicating that the calculations are not being completed. In addition, if these calculations are taking longer than half of the periodicity, an informational message will be displayed.
Ä Note: Additional information is available below about the messages associated with the status of this processing.
If either of these messages is displayed, you may want to consider reducing the amount of processing associated with streaming. Specifically, consider doing one of the following:
· Reduce the number of data series that are streaming.
· Reduce the number of alerts or calculated display fields associated with streaming data.
Ä Note: The special display fields described below do not have a significant effect on the amount of data that can be streamed.
· Reduce the number of charts and spreadsheets that are open. Updating these displays can take away from the time available for processing values.
· Work with longer periodicity data.
: Example: 1-minute bars require very fast processing and often trade very quickly. Using 2-minute bars (or longer) can make it possible to stream more symbols and perform more calculations.
Delayed Processing During Streaming
While TradingSolutions is streaming, the automatic signal analysis displayed on the Modify Data Dialog: Entry/Exit Signals Page is suspended. Signal analysis fields are still processed and individual signals can still be analyzed by selecting them from this dialog. However, the statistics on this page will not be updated until streaming has stopped.
The optimization or training of new fields can take a significant amount of time. Because of this, these operations are restricted to one of the two available processing tasks. In addition, calculating a new field for an entire data series can also take time. Adding new fields while streaming data can result in streaming results temporarily being delayed while processing being caught up.
Status of Processing of Streaming Data
The following messages will be displayed in the Messages and Alerts Window to notify you what state the streaming bars are in.
· Alerts and display fields for streaming n-minute bars are being brought up-to-date.
This message indicates that TradingSolutions is performing a "gap fill". Specifically, it is importing any data that occurred since the last update. It is also updating any calculations based on that data. Depending on the amount of data that has to be imported, this may take several minutes (or more).
Ä Note: It is valid to trade using signals generated during this period. However, alerts and signals may be delayed more than when they are updated directly from streaming data.
· Alerts and display fields for streaming n-minute bars are now being updated from streaming data.
This message indicates that alerts and display fields associated with streaming data are now being updated from streaming data after TradingSolutions has performed a "gap fill".
· Processing of alerts and display fields for streaming n-minute bars averaging time.
This message appears as a warning that too many symbols may be being streamed or too much high-priority processing may be being performed on streaming data.
Ä Note: This message is displayed if the processing of alerts and calculated display fields takes longer than half of the periodicity.
Ä Note: See above for recommendations of how to handle this message.
· Processing of alerts and display fields for streaming n-minute bars not completing before new data arrives.
This message appears as a warning that too many symbols may be being streamed or too much high-priority processing may be being performed on streaming data.
Ä Note: See above for recommendations of how to handle this message.
Special Display Fields for Streaming Data
Special display fields are available for displaying information in the Portfolio View while data is streaming. These include the following:
· Tick Price, Volume, and Time
· Bid/Ask Price, Size, and Time
Ä Note: These tick values are used for display purposes only. They are not stored in the portfolio. Calculated fields are only updated based on complete bars.
The tick price can be added as a display field automatically by selecting Add the streaming tick price to the portfolio display for the target group on the Import Data Wizard: Select Data to Download page. The display fields can also by modified by selecting Modify Display Fields… from the context menu for any group. This will display the Modify Group Dialog: Display Fields page.
Ä Note: These display fields are only available while streaming is turned on and only for data series for which streaming is available.
Streaming Charts
Charts that are displayed for data that is currently being streamed are updated automatically with the most recent tick price and volume.
Ä Note: As with the special display fields, it is important to understand that these tick values are used for display purposes only. They are not stored in the portfolio. Calculated fields are only updated based on complete bars.
To display a streaming chart, simply display a chart for streaming data and include price or volume information in the fields being displayed. This can be the price fields themselves or composite fields, such as HLOC and Japanese Candlesticks. Fields calculated from these values will not be streamed with the price and will only be updated when the bar has completed.