Click here to Login

How to Download Trading Data for Certain Securities Only

Updated on 2013-05-29

Downloaders implemented in QuantShare can retrieve any kind of data. The downloader tool is powerful and can create any sort of data retriever. You can for example, create an item to download data for specified symbols or an item that downloads a big compressed file that contains historical EOD data. That file will be automatically decompressed, read, parsed then its data added to a custom or built-in database.

An example of such downloader is the Daily US Stocks Data, which downloads historical EOD data for U.S. stocks. This item gets data by date, which means that on each request it gets trading data that occurred on that specific date. These kinds of items usually get data for all securities supported by the data provider. This means that you run the downloader; it will add and update trading data for all U.S. stocks.

But, let us say you want to update only S&P 500 stocks. How to do that?

In this post, I will show you how to update the downloader so that it downloads data only for the securities you choose. Of course, you can apply the same technique to other downloaders.

There are two ways to accomplish this.

Basic Technique

This technique consists of telling the downloader to download data only for stocks available in the database.
In our case, we must remove all securities except S&P 500 stocks. You can do that using "Symbol -> Manage Symbols".

After removing these securities, follow the next steps:
- Select "Download -> Download Manager"
- Select "Daily US Stocks Data"
- Click on "Update"
- Click on "Parser" button under "Parser" column
- Select one URL to debug (These are the URLs used to get data)
- Click on "Next"
- In the "CSV Settings" control, uncheck "Automatically add new symbols"
- Click on "Next" until the form closes
- Click ok "OK" to save the download item

Advanced Technique

The advanced technique consists of passing a list of ticker symbols to the downloader and asking it to download data only for symbols within this list.

- Select "Download -> Download Manager"
- Select "Daily US Stocks Data"
- Click on "Update"

How to bring the Symbols tab to the downloader

Click on the button under "Fields" column then add a new field by clicking on the "Add Field" button.
Set the new field's name to "symbol" then update its type and set it to "Symbol".
By defining a "Symbol" field, you have just instructed QuantShare to add a symbols tab to the downloader.

How to avoid running a request for each symbol

Now that we have defined a symbols tab, the URL-Script, which is the script the downloader uses to get URLS dynamically, will be executed for each symbol. Since our downloader do not requires symbol names to create URLs, the same request (For example download 20-05-2013 data) will be executed several times, once for each symbol.

We don't want that and we must tell the downloader that the URL-Script should be executed only once.
- Click on "Settings" button under "Settings" control
- Check "Load the script only once"
- Click on "OK"

Saving the list of symbols

The main idea of the advanced technique consists of saving the list of symbols so that we can pass it to the "Pre-Script", which is responsible for updating/modifying data just before it is analyzed.

- Click on "Settings" button under "Settings" control
- Click on "Create a URL-Script"

There you will find the code source used by this download item to dynamically generate URLs.

Add the following line at the end of the script:
Global.SetVariable("DownloaderSL", Functions.GetAllValues("symbol"));

// Functions.GetAllValues: Gets a list of all symbols defined in the "Symbols" tab (You see this tab when you open a downloader)
// Global.SetVariable: Creates a global variable and stores data in it. The global variable can be used any QuantShare script.

Reading the list of symbols

Let us open the parser settings control
- Click on "Parser" button under "Parser" column
- Select a URL then click on "OK"
- Click on "Next"
- Click on "Pre-Script" button

Add the following code at the bottom of the script:

string[] list = (string[])Global.GetVariable("DownloaderSL");
for(int i=0;i < list.Length;i++)
      list[i] = list[i].ToUpperInvariant();
for(int i=0;i < Content.Rows.Length;i++)
      string symbol = Content.Rows[i].Data[0].ToUpperInvariant();
      if(Array.IndexOf(list, symbol) < 0)
            Content.Rows[i].IsIgnoreLine = true;

- Click on "OK", "Next" then "Finish"
- Click on "OK" again to save the download item

Now, select your download item, click on "Open Selected Downloader", select "Symbols" tab, choose your symbols then click on "Start Downloading" to get end-of-day data for the selected symbols only.

no comments (Log in)

QuantShare Blog
Search Posts

Recent Posts

Create Graphs using the Grid Tool
Posted 1298 days ago

Profile Graphs
Posted 1403 days ago

Previous Posts

More Posts


Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright 2024
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.