Click here to Login









How to Automatically Import Local Data Into QuantShare

Updated on 2013-07-13





The downloader tool, one of the best features of QuantShare, let you retrieve data from Internet and add it to the quote or any custom database. But did you know that you can use the downloader tool to automate importing data files from your hard drive?

By simply replacing a URL by a file path, you can use the downloader tool to read files data and import that data into your database.

Here is what you will learn in this blog post:

- Using the downloader to parse local files
- Import all files in a specific directory
- Automate data import


How to create a downloader to parse local files

- Select "Download -> Download Manager"
- Click on "Add"
- Click on "Add URL"
- Under "URL" column, type the file you want to parse. Example: c:\data\appl.csv
- Under "Content Type", select "CSV" or "Zip" if the files you want to parse are compressed
- Click on "Parser" button under "Parser" column
- In "Parser Settings", specify the correct settings to successfully parse the file

If the symbol name of each security is specified in the file content then you simply have to reference it within parser settings ("Quotes -> Symbol" column). Otherwise, you must update the downloader so that the symbol name is picked from the file name.

In the latter case, set the file location to:
C:\data\[SYMBOL].csv

Click on "0 Field(s)" under "Fields" column then add a new field. Set the field type to "Symbol" then type a symbol to test under "Value" column. Also, uncheck the option under the last column.

Now, that we have parsed a single file, let me show you how to parse several files using a single downloader.


Import all files in a specific directory

- Select the downloader we have previously created then click on "Update" in the download manager.
- Click on "Settings" button under "Settings" column
- Check "Load the URL-Script only once"
- Click on "Create URL-Script"

This script allows you to dynamically create URLs or file paths to parse.
- Type the following script then click on "OK"

string[] files = System.IO.Directory.GetFiles(@"C:\data\", "*.csv");
int count = files.Length;
for(int i=0;i < files.Length;i++)
{
string file = files[i];
Functions.AddURL(file, "Parsing: " + i + "/" + count);
}


As you can see from the above code, the directory location is specified in the first line. The "GetFiles" function returns all files with the "csv" extension. If your files have another extension then simply update the "*.csv" value with "*.[extension]". To read all files, type "*.*".

In case the symbol name is not specified within the file content, we must get it from the file path.
In "CSV Settings" (click on "Parser" to bring this control), add a new "Quotes -> Symbol" column, click on "Pre-Script" then type the following code:

int symbolColumnIndex = 0;

System.IO.FileInfo path = new System.IO.FileInfo(Content.GetURLORFileName());
string name = path.Name.Replace(path.Extension, "");
for(int i=0;i < Content.Rows.Length;i++)
{
Content.Rows[i].Data[symbolColumnIndex] = name;
}


Make sure you update the "symbolColumnIndex" variable so its value correspond to the position of the "Quotes -> Symbol" column. If that column is set first then you should set a value of 0 to "symbolColumnIndex".


Automate data import

If you want to import data every day (or every few minutes/hours) then you will be probably interested in automating this process. The script editor is a QuantShare tool that allows you to create scripts to control QuantShare and automate things. The script we will implement now, simply starts a specific downloader. If you mix that tool with the task manager then you will be able to create a task that automatically starts any downloader at a specific time.

- Create a new script using "Tools -> Script Editor"
- Add the following code to your script:

Downloader.StartDownloader("Name");

// Replace "Name" by the name of your download item

- Now, open the task manager by selecting "Tools -> Task Manager"
- Click on "Click here to add a task"
- Select the "Minute/Hour/Week/Month" settings
- Click on "Add" under "Scripts to be executed"
- Select the previously created script then click on "Add" under "Task" panel to create your new task

If you have any questions then please add them below.












2 comments (Log in)

QuantShare Blog
QuantShare
Search Posts




QuantShare
Recent Posts

Create Graphs using the Grid Tool
Posted 1400 days ago

Profile Graphs
Posted 1505 days ago

QuantShare
Previous Posts

More Posts

Back







QuantShare
Product
QuantShare
Features
Create an account
Affiliate Program
Support
Contact Us
Trading Forum
How-to Lessons
Manual
Company
About Us
Privacy
Terms of Use

Copyright © 2024 QuantShare.com
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Google+
Follow us on Google+
RSS Trading Items



Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.